Year: 2020

Abstract:
The Propositional Satisfiability Problem (SAT), and more generally, the Constraint Satisfaction Problem (CSP), are mathematical questions defined as finding an assignment to a set of variables such that all the constraints are satisfied. The modern approach is trending to solve CSP through neural symbolic methods. Most recent works are sequential model-based, and adopt neural embedding, i.e., reinforcement learning with graph neural networks, and graph recurrent neural networks. In this work, we propose Heterogeneous Graph Transformer (HGT), a one-shot model derived from the eminent Transformer architecture for factor graph structure to solve the CSP problem. We define the heterogeneous attention mechanism based on meta-paths for the self-attention between literals, the cross-attention based on the bipartite graph links between literals and clauses. Exploiting highlevel parallelism, our model is able to achieve exceptional speed and accuracy on the factor graph for CSPs with arbitrary size. The experimental results have demonstrated the competitive performance and generality of HGT compared to the most recent baseline approaches.

heading: 1 INTRODUCTION
text: The Constraint Satisfaction Problems (CSP) is of central importance in several aspects of computer science, including theoretical computer science, complexity theory, algorithmics, cryptography, and artificial intelligence. CSP aims at finding a consistent assignment of values to variables such that all constraints, which are typically defined over a finite domain, are satisfied. In particular, there is an assortment of problems arising from artificial intelligence and circuit design that can be reduced to CSP subtypes, including map coloring, vertex cover, independent set, dominating set, and clique detection.
Solving a CSP on a finite domain is often an NP-complete problem with respect to the domain size. The conventional CSP-solvers rely on handcrafted heuristics that guide the search for satisfying assignments. These algorithms are focused on solving CSP via backtracking or local search. Hence, the resulted model is bounded by the greedy strategy, which is generally sub-optimal. With the advent of Graph Neural Networks (Scarselli et al. (2009)), the geometric deep learning (Bronstein et al. (2017)) for Non-Euclidean data has become one of the most emerging fields of machine learning. In particular, it brought deep learning solutions to one of the most dominant combinatorial optimization problems, the Constraint Satisfaction Problem (CSP) (Khalil et al. (2017)). Works including NeuroSAT (Selsam et al. (2018)) and Circuit-SAT (Amizadeh et al. (2018)) commenced the study of neural methods targeted at CSP. Later works, such as Yolcu & Póczos (2019) and You et al. (2019), attempted to solve CSP through different deep learning approaches. However, most pioneering works, such as neural approaches utilizing RNN or Reinforcement Learning, are still restricted to sequential algorithms, while clauses are parallelizable even though they are strongly correlated through shared variables.
In this work, we propose a hybrid model of the Transformer architecture (Vaswani et al. (2017)) and the Graph Neural Network for solving combinatorial problems, especially CSP. Our main contributions in this work are: (a) We derived meta-paths adopted from Sun et al. (2011) to formulate the message passing mechanism between homogeneous nodes (i.e., variable to variable, or clause to clause), which enable us to perform self-attention and let message pass through either variables sharing the same clauses, or clauses that include the same variables. We apply the cross-attention mechanism to optimize message exchanges between heterogeneous nodes(i.e., clause to variable, or variable to clause). (b) With the combination of homogeneous attention and heterogeneous atten-
tion mechanisms on bipartite graph structure, we then combine Transformer with Neuro-Symbolic methods to resolve combinatorial optimization on graphs. (c) We proposed Heterogeneous Graph Transformer (HGT), a general framework for graphs with heterogeneous nodes. In this work, we trained the HGT framework to approximate the solutions of CSP (but not limited to CSP). Our model is able to achieve competitive accuracy, parallelism, and generality on CSP problems with arbitrary sizes

heading: 2 RELATED WORK
text: Recently, the Machine Learning community has seen an increasing interest in applications and optimizations related to constraint satisfaction problem solving. Various frameworks utilising diverse methodologies have been proposed, offering new insights into developing CSP solvers and classifiers. For example, Bello et al. (2016) adopts Reinforcement Learning in their Neural Combinatorial Optimization, with an approach based on policy gradients. On the other hand, works such as Evans et al. (2018) and Arabshahi et al. (2018) have demonstrated the effectiveness of recursive neural networks in modeling symbolic expressions. Meanwhile, Prates et al. (2019) proposed an embeddingbased message-passing algorithm for solving Traveling Salesman Problem (TSP), a highly relevant CSP problem.
NeuroSAT (Selsam et al. (2018)) is a graph neural network model that aims at solving the Boolean Satisfiability Problem (SAT) without leveraging the greedy search paradigm. It approaches SAT as a binary classification problem during training and finds an SAT assignment from the latent representations during inference. NeuroSAT is able to search for solutions to problems of various difficulties despite training for relatively small number of iterations. As an extension to this line of work, Selsam & Bjørner (2019) proposes a neural network that facilitates variable branching decision making within high-performance SAT solvers on real problems.
PDP Amizadeh et al. (2019) is a generic neural framework for learning CSP solvers based on the idea of Propagation, Decimation, and Prediction. PDP provides a completely unsupervised training mechanism for solving SAT via energy minimization, and can be seen as learning optimal message passing strategy on probabilistic graphical models.
G2SAT (You et al. (2019)) is a deep generative framework that learns to generate SAT formulas from a given set of input formulas while preserving the graph statistics. Even though G2SAT lacks the ability to derive solutions, it provides synthetic formulas for hyperparameter optimization.
RLSAT (Yolcu & Póczos (2019)) learns SAT solvers through deep reinforcement learning and iterative refinement. It incorporates a graph neural network into a Stochastic Local Search(SLS) algorithm to act as the variable selection heuristic during training. However, since SLS begins with randomly initialized parameters, and a non-zero terminal reward is given only when a satisfying assignment is found, RLSAT requires a curriculum learning process for performance improvement. As a result, RLSAT becomes inefficient when its learning process starts with large complex graphs, in which satisfying assignments are hard to obtain. In our HGT, each possible state of assignment corresponds to a likelihood, which can be minimized to train the model. In particular, HGT is able to achieve optimal performance in efficiency and accuracy regardless of input graph sizes.

heading: 3 BACKGROUND

heading: 3.1 CONSTRAINT SATISFACTION PROBLEMS
text: Constraint Satisfaction Problems (CSP) (Kumar (1992)) is a fundamental problem in logic study that constitutes the cornerstone of combinatorial optimization. It provides feasible models to real world applications and is intensively involved in the design of artificial intelligence. An instance of CSP problem, CSP(V,U), is constituted of two main components: a set of N variables V = {vi ∈ D : i ∈ 1...N}, defined over a discrete domain D; and a set of M constraint functions or factors, U = {uj(qj) : j ∈ 1...M}, where qj is a subset of V subject to uj . For each uj ∈ U , uj : D
|qj | → {0, 1} outputs 1 if the input qj satisfies constraint uj , and 0 otherwise. A CSP problem can be formulated in Conjunctive Normal Form (CNF) (Pfahringer (2010)) with the goal of finding an assignment of variables that satisfies all constraints. For a given assignment to V , the measure of
a CSP problem, φ : {0, 1}N → {0, 1}, is a function that returns 1 if all the clauses in the problem are satisfied, and 0 otherwise. Such a measure function is in CNF if
1. For all uj ∈ U , uj is a disjunction of literals. Here literals refer to vi or ¬vi, for vi ∈ qj ; 2. φ(V ) = ∧uj , j = {1, ...,M}.

heading: 3.2 GRAPH NEURAL NETWORKS AND GRAPH ATTENTION NETWORKS
text: Graph Neural Network (GNN) (Scarselli et al. (2009)) provides a basic architecture to operate neural network on graphs. Suppose V is a set of nodes, where each vertex vi ∈ V carries a node feature hi ∈ RdV , and E ⊂ V × V is a set of undirected edges, where each edge ei,j ∈ E that connects vi to vj ∈ V carries an edge feature hi,j ∈ RdE . For time step t ∈ {1, ..., T}, a GNN on the graph G(V, E) passes messages between direct neighbors, and iteratively updates the feature vector of each vi ∈ V following:
htvi = Q ht−1vi , ⋃ vj∈Ni M t(ht−1vi , h t−1 vj , hi,j)  (1) where Ni = {vj ∈ V : ∃ei,j ∈ E}, and M , Q are the message function and the update function to be learned, respectively.

heading: 3.3 TRANSFORMER AND THE RELATION WITH GNN
text: Transformer (Vaswani et al. (2017)) is the state-of-the-art approach for sequential data and transduction problems that relies on self-attention mechanism. When looking into the architecture of Transformer, the computation of similarity between queries and keys highly resembles finding a correlation between two types of nodes in fully connected bipartite graphs, while the self-attention mechanism among queries or keys (or tokens in NLP terminology) can be seen as calculating the weighted contribution in a half-complete connected graph, where an output token does not depend on future words. With these resemblance, we want to extend the Transformer architecture to heterogeneous graph structures, such as bipartite graphs.

heading: 4 METHODOLOGY

heading: 4.1 FACTOR GRAPHS AND META-PATHS
text: The value of each constraint is binary, which makes it possible to express the measure of a CNF as:
φ(V,U) = M∏ j=1 uj(qj) (2)
The expression can be properly presented as an undirected bipartite factor graph. We construct such a factor graph G((V,U), E) by defining the set of variables V = {v1, . . . , vn}, the set of clauses U = {u1, . . . , um}, and edges E by: ei,j ∈ E iff variable vi is involved in constraint uj either in positive or negative relation. In Yolcu & Póczos (2019), each edge is assigned with a type depending on the polarity of the variable it connects to. The positive occurrence of a variable vi in a clause (or factor) uj is represented with the positive sign (+), whereas its negative occurrence ¬vi in uj is represented with the negative sign (−). Hence, a pair of n×m bi-adjacency matrix A = (A+, A−) is used to store two types of edges such that A+(i, j) = 1⇔ vi ∈ uj and A−(i, j) = 1⇔ ¬vi ∈ uj . Here vi ∈ uj implies that vi instead of its negation ¬vi is directly involved in uj . Each edge ei,j ∈ E is then assigned a value equals to 1 for edges in A+ and −1 for edges in A−. With the factor graph representation, graph neural network can be applied as a CSP-solver (Selsam et al. (2018); Yolcu & Póczos (2019)). However, due to the fact that factor graph is bipartite in which variables are only connected to clauses, clauses are only connected to variables. Furthermore, traditional GNN only passes messages to local neighbors, and is thus inefficient at passing messages between variables or between clauses. Consequently, variable nodes update their state solely based on their current states and the states of the clauses they affect, vice versa. In this work, we propose to pass message through length-2 meta-paths in addition to existing edges, which enables: 1) variables
to incorporate the information from variables that share the same clauses during state updates; 2) clauses to incorporate the information from clauses that share some variables when update their states. In a CSP factor graph, we define that a meta-path mi,j = (vi, vk, vj) between nodes vi and vj exists if there exists some vk ∈ V s.t. ∃ei,k ∈ E and ∃ek,j ∈ E . Since self-attention mechanism is not symmetric, our meta-path is directed. As a result, we get four types of meta-paths in total (i.e., {(+,+), (+,−), (−,+), (−,−)} ), as illustrated in Figure 1 (c). The adjacent matrix of such a meta-path can be easily computed by matrix multiplication of A+ and A− or their transposes. Take A(+,+), A(+,−) as examples, the adjacency matrix A(+,+) = A+AT+ stores all (+,+) meta-paths, and A(+,−) = A+AT− stores all (+,−) meta-paths. A diagonal entry A(+,+)[i, i] indicates the number of positive edges that vi has, and an off-diagonal entry A(+,+)[i, j] indicates the existence of (+,+) meta-path from vi to vj .

heading: 4.2 HOMOGENEOUS AND HETEROGENEOUS GRAPH ATTENTION CONVOLUTION
text: In order to incorporate attention mechanism into the bipartite graph, we extend the original Graph Attention Network (GATConv) (Veličković et al. (2017)) with heterogeneous graph attention, while we consider the original attention in GATConv as homogeneous graph attention. Heterogeneous attention is used for nodes of different kinds. Nodes in different partitions are normally of different features or characteristics, e.g., literals forming a partition different from clauses in their partition. Suppose literal and clause feature matrices are V ∈ RN×Fv and U ∈ RM×Fu , respectively; and their adjacency matrix A ∈ RM×N . The heterogeneous attention is define as:
evi,uj = σ([av||au]T [Wv−→vi ||Wu−→uj ]), euj ,vi = σ([au||av]T [Wu−→uj ||Wv−→vi ])
αvi,uj = softmaxuj (evi,uj ) = exp(evi,uj )∑
uk∈N(vi) exp(evi,uk)
αuj ,vi = softmaxvi(euj ,vi) = exp(euj ,vi)∑
vk∈N(uj) exp(eui,vk)
V ′ = ReLU(VWv +Au→vUWu), U ′ = ReLU(UWu +Av→uVWv)
(3)
where Wv ∈ RFv×F and Wu ∈ RFu×F are embedding weights for variables and clauses that transform input feature into a common high-level feature of dimension F . av ∈ RF and au ∈ RF are shared attention mechanism on variables and clauses. Attention matrix Au→v ∈ RN×M stores edge scores from clauses to variables as Au→v[i, j] = αuj ,vi ; and Av→u ∈ RM×N stores edge scores from variables to clauses asAv→u[j, i] = αvi,uj . σ is non-linearity function, i.e., LeakyReLU used in GATConv (Veličković et al. (2017)).

heading: 4.3 HETEROGENEOUS GRAPH TRANSFORMER (HGT)
text: Our model HGT adopts an encoder-decoder structure similar to Britz et al. (2017), with four encoder-layers in the encoder, and three decoder-layers in the decoder. It is flexible to adjust the number of encoder and decoder layers.
Encoder. Prior to entering the stack of encoder-layers, the input node features are embedded into high-dimensional space with MLP. Within each encoder-layer, every graph node first aggregates the message (or information) from nodes of its kind through meta-paths. Note that a node (variable or clause) of bipartite graph has no direct connection within homogeneous nodes. Messages can only get passed among homogeneous nodes through meta-paths. We emphasize such type of communication between nodes of the same kind as self-attention, which is implemented with homogeneous attention mechanism. The weighted messages are passed between variables and clauses through the cross-attention mechanism, implemented with the heterogeneous attention mechanism. The two types of attention are connected through the residual block and layer normalization (Ba et al. (2016)), as shown in Figure 2 (a).
Decoder. Decoder consists of several decoder-layers. Inside each decoder-layer, there is one crossattention module followed by residual connection and layer normalization, as in Figure 2 (c). The attention-weighted node features are then fed into the MLP for dimension reduction.

heading: 4.3.1 LOSS EVALUATION
text: For a given CSP(V,U), each combination of variable assignments corresponds to a probability. The original measure φ(V,U) is a non-differentiable staircase function defined on a discrete domain. φ(V,U) evaluates to 0 if any uj ∈ U is unsatisfied, which disguises all other information including the number of satisfied clauses. For training purpose, a differentiable approximate function is desirable. Therefore, the proposed model generates a continuous scalar output xi ∈ [0, 1] for each variable, and the assignment of each vi can be acquired through:
vi = ⌊ xi 0.5 + ⌋ (4)
where is a small value to keep the generated vi in {0, 1}. With continuous xi, i = 1, ..., N , we can approximate disjunction with max(·) function and define φ(·) as
φ(x1, ..., xN ) = M∏ j=1 max({l(xi) : vi ∈ qj}) (5)
Here, the literal function l(xi, eia) = 1−eia2 + eiaxi is applied to specify the polarity of each variable. Inspired by Amizadeh et al. (2019), we replace the max function with a differentiable smoothmax(·)
Sτ (x1, ..., xN ) =
∑n i=1 xie
τxi∑n i=1 e τxi (6)
Mathematically, Sτ (x1, ..., xN ) converges to max(x1, ..., xN ) as τ → ∞. However, since the assignment of each xi is continuous and the threshold is 0.5, HGT doesn’t need a strict max function to judge if a clause contains a literal with value 1. We note that τ = 5 is enough for our model in practice. By maximizing the modified φ, HGT is trained to find the optimal assignment for each CSP problem. For numerical stability and computational efficiency, we train HGT by minimizing the negative log-loss
L(xi, ..., xN ) = −log(φ) = − M∑ j=1 log(Sτ ({l(xi) : vi ∈ qj})) (7)
During the training process, we realize that the value for each xi is trending towards 0.5 to minimize the loss function. We then append a regularizer
r(ui, ..., uM ) = − M∑ j=1 ReLU(uj − 0.5) (8)
to punish those clauses with values near 0.5, while allowing clauses with small values to increase. In this way, the regularizer encourages the value of each xi to approach 0 or 1, without disturbing the maximization of the values of clauses that xi is connected to.

heading: 5 EXPERIMENTAL EVALUATION

heading: 5.1 DATASET
text: In order to learn a CSP solver that can be applied to various classes of satisfiability problems, we drew our training set from four classes of problems with distinct distributions: random 3-SAT, graph coloring, vertex cover, and clique detection.
For the random 3-SAT problems, we drew a total of 1200 synthetic SAT formulas from the SATLIB benchmark library (Hoos & Stützle (2000)). These graphs consist of variables and clauses of various sizes, and should reflect a wide range of difficulties. For the latter three graph problems, we sampled 4000 instances from each of the distributions that are generated according to the scheme proposed in Yolcu & Póczos (2019). In particular, we tuned the problem-specific parameters for each set of instances to assess our model’s ability to generalize to other CSP problems. All formulas in our dataset, regardless of distribution, are satisfiable, and are encoded in DIMACS Conjunctive Normal Form (CNF), the natural format for SAT problems. Models trained on these graphs generalizes to other CSP problems, datasets, and larger graphs.

heading: 5.2 BASELINES
text: Baseline models. To assess the validity of our model, we compared our framework against three categories of baselines: (a) the classic stochastic local search algorithm for CSP solving - WalkSAT (Selman et al.), (b) the reinforcement learning-based CSP solver - RLSAT (Yolcu & Póczos (2019)), (c) the generic but innovative neural framework for learning CSP solvers - PDP (Amizadeh et al. (2019)). Among these baselines, PDP falls into the same category as the proposed model, which utilizes the one-shot algorithm.
Baseline setting. For the RLSAT and PDP baseline models, we strictly preserved their default parameters in order to reproduce the original performances. Specifically, the RLSAT model was
trained on their self-generated datasets via curriculum learning. For the PDP framework, we tested a fully neural CSP solver with weight decay = 10−10 and dropout rate = 0.2.

heading: 5.3 EXPERIMENTAL SETTING
text: Software. The PyTorch implementation of our model is open-sourced, and the code is available at https://git.io/JUybf. Our model employs PyTorch Geometric (Fey & Lenssen (2019)), and is able to achieve high efficiency in both training and testing by taking full advantage of GPU computation resources via parallelism.
General setup. Our model for the experiments discussed in this section is configured as follows. Structures are implemented according to the architecture presented in Figure 2. For the encoder, we adopted four encoder-layers with the number of channels set to {16, 32, 32, 32}; for the decoder, we adopted three decoder-layers with the number of channels set to {32, 16, 16}. Optimizer Adam(Kingma & Ba (2017)) with β1 = 0.9, β2 = 0.98, = 10−9 was applied to train the model. Our learning rate varies with each step taken, and follows a pattern that is similar to the one adopted by Noam (Vaswani et al. (2017)). learning rate = β ∗min(step num−0.5, step num× warmup steps−1.5) (9) Specifically, our learning rate increases linearly for the initial warm-up steps, then gradually decay at a rate that is inversely proportional to the square root of the step number. β is the coefficient used to adjust the scale of learning rate. To ensure fairness of comparison, we adopted a dropout rate of 0.2 for regularization, which is the same value as Amizadeh et al. (2019).

heading: 5.4 RESULTS AND EVALUATION
text: Validity of training. Table 2 summarizes the performance of our model, as compared to that of RLSAT. For the training process of each model, we strictly followed the setting as explained in Section 5.2 and 5.3. Since our model adopts a semi-supervised training strategy, and is capable of processing graphs of arbitrary size, we were able to combine numerous distributions of the same problem class into one single dataset during training, regardless of the problem-specific parameters. We observed that our model’s average accuracy is higher than that of RLSAT in all six distributions. For further analysis, we present the holistic learning curves in Figure 3, where the shaded areas visualize the standard deviations of each model’s validation scores. From the figure, we noticed that for the latter 100 epochs, RL-KC and RL-KV’s validation performance start to oscillate significantly. Investigating the characteristics of Reinforcement Learning, we discovered that RLSAT, upon encountering graphs with new scales, performs a whole new process of exploration. Therefore, RLSAT fails to generalize its learnt experience to subsequent larger graphs, which results in an unstable validation score during training. In comparison, our model adopts a highly paralleled message-passing mechanism, which updates all nodes of all graphs simultaneously at each epoch. As a result, we are able to reach a high score rapidly, while maintaining the accuracy with little fluctuation.
We understand that the graph problems generated by RLSAT may not be sufficient in size and difficulty to fully assess the capability of our model. Hence, in addition to testing on a diversified distribution of graph problems, we also experimented the validity of our model on the classic random 3-SAT dataset, and compared our results with that of PDP. As seen in Table 3, our model retains the ability to achieve high accuracy with little time required for processing each graph. In comparison, PDP takes a significantly longer time for validation, while reaching an average accuracy that fails to exceed ours.
Efficiency of testing. Noting the discrepancy between the validation time, we further analyzed the efficiency of our model by measuring the time taken for each trained model to test on an individual graph. To diversify our test sets, we chose 2000 instances from each of the following distributions: {color3(20, 0.5), cover3(9, 0.5), clique3(20, 0.05) }, as well as 100 instances from each of the random 3-SAT distributions: {rand3(100, 430), rand3(150, 645), rand3(200, 860)}. Summarized in Figure 4 are the results we obtained from comparing the average test speeds of our model against that of PDP, with the performance of WalkSAT as the metric. As demonstrated in the figure, our model is capable of achieving higher average test speeds regardless of the graph structure. This observation can be explained by the fact that our model allows communication within homogeneous nodes, which provide all nodes with abundant semantic information when updating their states. Therefore, our model requires fewer iterations of message passing, and achieves greater efficiency.

heading: 6 CONCLUSION
text: In this paper, we proposed Heterogeneous Graph Transformer (HGT), a one-shot model derived from the eminent Transformer architecture for factor graph structures, to solve the CSP problem. We defined the homogeneous attention mechanism based on meta-paths for the self-attention between literals or clauses, as well as the heterogeneous cross-attention based on the bipartite graph links from literals to clauses, or vice versa. Our model achieved exceptional parallelism and accuracy on the factor graph form of CSPs with arbitrary sizes. The experimental results have demonstrated the competitive performance and generality of HGT in several aspects. Future efforts in this direction would include the extraction of high-level embeddings from HGT as well as the application of our proposed algorithm to further aid classic CSP solvers on solving combinatorial optimization problems. Our work also suggests the possibility of transforming classic techniques based on Conflict-Driven Clause Learning solvers to neural frameworks.

Review summaries:
The paper presents a model for inferring the solution of a constraint satisfaction problem over boolean variables expressed in *Conjunctive Normal Form (CNF)*. The proposed model builds on existing works which represent the factor graph of the CNF as a bipartite graph, and use a graph neural network for the learning task. It re-defines the graph to facilitate the message passing among the nodes which are in the same part of the graph (i.e. CNF variables/clauses). It also introduces a cross-attention mechanism between the nodes belonging to different parts. 

This paper presents Heterogeneous Graph Transformer (HGT), a new architecture that combines useful properties of GNNs and Transformers to design HGT for combinatorial reasoning problems, particularly Boolean Satisfiability (SAT). The idea of combining these two powerful models is appealing. The motivation behind considering both homogenous attention (between two literals, or between two clauses) and heterogenous attention (between a literal and a clause) seems new and interesting too. However, there are several missing pieces in the paper in the present form, as outlined below.