N EURAL M ODELS FOR O UTPUT -S PACE I NVARIANCE IN C OMBINATORIAL P ROBLEMS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recently many neural models have been proposed to solve combinatorial puzzles by implicitly learning underlying constraints using their solved instances, such as sudoku or graph coloring (GCP).
One drawback of the proposed architectures, which are often based on Graph Neural Networks (GNN) (Zhou et al., 2020), is that they cannot generalize across the size of the output space from which variables are assigned a value, for example, set of colors in a GCP, or board-size in sudoku.
We call the output space for the variables as ‘value-set’.
While many works have demonstrated generalization of GNNs across graph size, there has been no study on how to design a GNN for achieving value-set invariance for problems that come from the same domain.
For example, learning to solve 16 × 16 sudoku after being trained on only 9 × 9 sudokus, or coloring a 7 colorable graph after training on 4 colorable graphs.
In this work, we propose novel methods to extend GNN based architectures to achieve value-set invariance.
Speciﬁcally, our model builds on recently proposed Recurrent Relational Networks (RRN) (Palm et al., 2018).
Our ﬁrst approach exploits the graph-size invariance of GNNs by converting a multi-class node classiﬁcation problem into a binary node classiﬁcation problem.
Our second approach works directly with multiple classes by adding multiple nodes corresponding to the values in the value-set, and then connecting variable nodes to value nodes depending on the problem initialization.
Our experimental evaluation on three different combinatorial problems demonstrates that both our models perform well on our novel problem, compared to a generic neural reasoner.
Between two of our models, we observe an inherent trade-off: while the binarized model gives better performance when trained on smaller value-sets, multi-valued model is much more memory efﬁcient, resulting in improved performance when trained on larger value-sets, where binarized model fails to train.
I NTRODUCTION
The capability of neural models to perform symbolic reasoning is often seen as a step towards the framework for uniﬁed AI, i.e. , building end-to-end trainable system for tasks, which need to combine low level perception with high level cognitive reasoning (Kahneman, 2011).
While neural networks are naturally excellent at perception, they are increasingly being developed for high-level reasoning tasks, e.g. , solving SAT (Selsam et al., 2019;
Amizadeh et al., 2019a;b), neural theorem proving (Rocktäschel et al., 2015), differentiable ILP ( ∂ ILP) (Evans & Grefenstette, 2018), playing blocks world (Dong et al., 2019), solving sudoku (Palm et al., 2018; Wang et al., 2019).
Our work follows this literature for solving combinatorial puzzles – in particular, the methods that do not assume an explicit knowledge of rules, but instead learn to directly solve the problem using some of its solved instances, e.g. Recurrent Relational Networks (RRN) (Palm et al., 2018).
Such models assume a ﬁxed value-set, i.e. , the set from which variables are assigned values is assumed to be constant during training and testing.
This is a signiﬁcant limitation, since it may not always be possible to generate sufﬁcient training data for similar large problems in which variables take values from a bigger value-set (Najaﬁan et al., 2018).
Moreover, it is also a desirable goal since as humans, we often ﬁnd it natural to generalize to problems of unseen variable and value sizes, once we know how to solve similar problems of a different size, e.g. , we may solve a 12 × 12 sudoku after learning to solve a 9 × 9 sudoku.
We note that graph based models have been shown to generalize well on varying graph sizes, e.g. , ﬁnding a satisfying solution of a CNF encoding of a CSP with
Boolean-variables, after training on CNF encodings of CSPs with only 40 Boolean-variables (Selsam et al., 2019).
However, the model trained using CNF encoding of Boolean-CSPs cannot be used directly for a non-Boolean CSP in which variables take value from a different (larger) value-set.
In response, we study value-set invariance in combinatorial puzzles from the same domain.
To formally deﬁne a similar puzzle with variables taking values from a different value-set, we make use of Lifted CSP (Joslin & Roy, 1997), a (ﬁnite) ﬁrst-order representation that can be ground to CSPs of varying variable and value-set sizes.
We note that even though we use Lifted CSPs to deﬁne value-set invariance, its complete speciﬁcation is assumed to be unknown.
Speciﬁcally, we do not have access to the constraints of the CSP, and thus neural SAT solvers like NeuroSAT (Selsam et al., 2019) can not be used.
While training, we only assume access to solved instances along with their constraint graph.
We deﬁne our problem as: given solved instances and corresponding constraint graph of an unknown ground CSP with a value-set of size k , can we learn neural models that generalize to instances of the same lifted CSP, but with a different value-set of size k (cid:48) (typically k (cid:48) > k )?
An example task includes training a model using data of 9 × 9 Sudoku, but testing on a 12 × 12 or a 16 × 16 Sudoku.
We build our solution using RRNs as the base architecture.
They run GNN on the constraint graph, and employ iterative message passing in a recurrent fashion – the nodes (variables) are then decoded to obtain a solution.
We present two ways to enhance RRNs for value-set invariance.
Binarized Model: Our ﬁrst model converts a multi-class classiﬁcation problem into a binary classiﬁcation problem by converting a multi-valued variable into multiple Boolean variables, one for each value in the value-set.
The binarized constraint graph gets deﬁned as: if there is an edge between two variables in original constraint graph, there are k edges between Boolean nodes corresponding to the same value and the same two variables in the new graph.
In addition, all k Boolean variables, corresponding to a multi-valued variable, are connected with each other.
This model naturally achieves value-set invariance.
At test time, a larger value-set just results in a larger graph size.
All
GNN weights are tied, and because all the variables in the binarized model are Boolean, embeddings for binary values ‘0’ and ‘1’, trained at training time, are directly applicable at test time.
Multi-valued Model: Our sec ond model directly operates on the given multi-valued variables and the corresponding constraint graph, but introduces a value node for every value in the value-set.
Each pre-assigned (unassigned) variable node is connected to that (respectively, every possible) value node.
The challenge in this model is initializing value nodes at test time when k (cid:48) >
k .
We circumvent this problem by training upfront k (cid:48) or more value embeddings by randomly sub-selecting a k sized subset during each learning iteration.
This random sub-selection exploits the symmetry of value-set elements across instances.
We can also address the issue of non-symmetric values where additional relations between values may be known.
During test time, k (cid:48) of the learned embeddings are used.
We perform extensive experimental evaluation on puzzles generated from three different structured
CSPs: Graph Coloring (GCP), Futoshiki, and Sudoku.
We compare two of our models with an
NLM (Dong et al., 2019) baseline – a generic neural reasoner, which either fails to scale or performs signiﬁcantly worse for most test sizes used in our experiments.
We also compare our two models along the axes of performance and scalability and discuss their strengths and weaknesses.
R ELATED W ORK
This paper belongs to the broad research area of neural reasoning models, in which neural models learn to solve pure reasoning tasks in a data-driven fashion.
Some example tasks include theorem proving (Rocktäschel et al., 2015; Evans & Grefenstette, 2018), logical reasoning (Cingillioglu &
Russo, 2019), probabilistic logic reasoning (Manhaeve et al., 2018), classical planning (Dong et al., 2019), probabilistic planning in a known MDP (Tamar et al., 2017; Bajpai et al., 2018), and our focus – combinatorial problems that are instances of an unknown constraint satisfaction problem.
There are two main research threads within neural CSPs and SAT.
First thread builds neural models for problems where the CSP constraints or SAT clauses are explicitly provided to the model.
For example, NeuroSAT (Selsam et al., 2019) and PDP (Amizadeh et al., 2019b) assume that the CSP is expressed in a Conjunctive (or Disjunctive) Normal Form.
Similarly, Circuit-SAT (Amizadeh et al., 2019a) uses the knowledge of exact constraints to convert a CSP into a Boolean Circuit.
This research has similarities with logical reasoning models like DeepProbLog (Manhaeve et al., 2018), and DeepLogic (Cingillioglu & Russo, 2019), which require human designed rules for reasoning.
Our work belongs to the second thread where the constraints or clauses are not provided explicitly, and only some underlying structure (e.g., Sudoku grid cell connectivity) is given along with training data.
The intention is that the model not only learns to reason for the task, but also needs to learn the implicit semantics of each constraint.
SATNET (Wang et al., 2019) falls in this category – it formulates a learnable low-rank Semi-deﬁnite Program (SDP) relaxation for a given MAXSAT problem trained via solved SAT problems.
Similarly, Recurrent Relational Networks (RRN) (Palm et al., 2018) use recurrent message passing graph neural network to embed the variables of the unknown CSP, and the relationship between them, in a latent vector space and ﬁnally assign a value to each variable based on its embedding.
Both these works assume a ﬁxed number of variables that remains unchanged across training and test.
While we build on RRNs, we substantially extend the formalism to study value-set invariance.
Formally, our work can be seen as solving a (ﬁnite) ﬁrst-order formulation of the CSP, called Lifted CSP (Joslin & Roy, 1997), which can be ground to CSPs with varying number of variables and values.
To our knowledge, there is relatively limited prior work on neural models that can generalize to variable-sized instances of an underlying ﬁrst order reasoning task – one related approach builds neural models for First-order MDPs (Garg et al., 2020).
Finally, there has been a long history of work dedicated to learning rules or constraints from training data using Inductive Logic Programming (Lavrac & Raedt, 1995; Friedman et al., 1999).
Evans &
Grefenstette (2018) propose differentiable neural relaxation of ILP ( ∂ ILP).
Neural Logic Machines (NLM) (Dong et al., 2019) is another framework that learns lifted rules, shown to be more scalable than ∂ ILP.
It allows learning of ﬁrst-order logic rules expressed as Horn Clauses over a set of predicates.
It consists of a stack of logic machines that learn predicate relationships between objects using training data.
Learning of ﬁrst-order rules makes NLM amenable to transfer over different CSP sizes (Nandwani et al., 2021), and are thus directly comparable to our work.
The main challenge of such approaches is that they fail to scale to the size of the problems considered in this work.
In our experiments, we compare our methods against both deep and shallow versions of NLM.
P RELIMINARIES AND P ROBLEM D EFINITION
A combinatorial puzzle can be thought of as a grounded CSP and to formally deﬁne a puzzle from the same domain but a larger value-set, we resort to the notion of ‘ Lifted CSPs ’ that represent an abstraction over multiple ground CSPs of the same type.
A lifted CSP does not include a speciﬁc set of variables and values; instead, it operates in terms of variable and value references that can be instantiated with all ground variables and values in a ground CSP.
This makes them amenable to instantiate CSPs or puzzles with varying number of variables as well as values.
We deﬁne a
Lifted CSP L C as a three tuple (cid:104)P , R , C(cid:105) .
P is a set of predicates: a predicate p ∈ P represents a Boolean function from the set of its arguments, which are variable references.
Similarly, R is a set of relations over value space – a r ∈ R reprents a Boolean function over arguments that are value references.
A predicate (or a relation) with its arguments is called an atom.
C is a set of lifted constraints, constructed by applying logical operators to atoms – they are interpreted as universally quantiﬁed over all instantiations of variable and value references.
Finally, Lifted CSP uses a special unary function Value , whose argument is a variable reference and evaluates to a value reference.
As an example, a lifted CSP for Sudoku may have a P
= { Nbr } for whether two cells are in same row, column or box, R = { Neq } , representing two values are unequal, and a lifted constraint:
Nbr ( c 1 , c 2 ) →
Neq ( Value ( c 1 ) , Value ( c 2 )) .
A lifted CSP L C yields a ground CSP C , given a set of variables O , and a set of values V , and a complete instantiation of all predicates and relations over this set (e.g., in Sudoku, the number of cells, possible values, and which cells are neighbors and which are not).
The ground constraints are constructed by instantiating lifted constraints over all variables and values.
A (satisfying) solution, y , of a CSP refers to a complete speciﬁcation of Value : O → V function, such that all the constraints are satisﬁed.
We are often given a partial (satisfying) solution, x – an assignment of values to a subset of variables ˜ O ⊆ O and the goal is to output y , such that y agrees with x for the subset ˜ O .
Given a ground CSP C , the Constraint Graph , G C
=
( N C , E C ) , is constructed by having each variable in the CSP represent a node in the graph and introducing an edge between two nodes n C 1 , n Ciff the corresponding variables appear together in some constraint.
The edges in the constraint graph are typed based on the identity of the lifted constraint from which it comes.
Note that there could be multiple edges between nodes n C 1 , n C 2 in G C , if these nodes appear together in more than one constraint.
We embed the knowledge about relations between values in V in the form of another graph, called Relation Graph , G R =
( N R , E R ) , where there is a node for every value in the set V , and there is a (directed) edge between nodes corresponding to v l , v (cid:48)
l depending on whether r ( v l , v l (cid:48) ) is true or not, for every r ∈ R .
Similar to G C , this graph can also have multi-edges between two pairs of nodes, if more than one relationship holds between the corresponding values.
Problem Deﬁnition: To achieve value-set invariance, our goal is to train a model M Θ on training data from an unknown ground CSP C (with variables O and value-set V ) obtained from an unknown lifted CSP L C , and test it on an arbitrary ground CSP C (cid:48) from the same lifted CSP (with variables
O (cid:48) and value-set V (cid:48) ), where |V| (cid:54) =
|V (cid:48) | .
Formally, we are given training data D as a set of tuples { (( x i , G C i ) , y i ) } M i =1 , along with a relationship graph G R encoding relations between values in the value-set V .
Here, i th instance denotes a partial and corresponding complete solution for C i .
We note that explicit form of the constraints in C i or L C are not available, only the graphs are given to the model.
Our goal is to learn model M Θ , such that given graphs G C (cid:48) and G R (cid:48) , and a partial solution x (cid:48) (for CSP C (cid:48) ): M Θ ( x (cid:48) ) =
y (cid:48) , only if y (cid:48) is a corresponding complete solution for x (cid:48) .
Note that in one of our models, we will additionally assume that max |V (cid:48) | is known to us at training time, which we argue is a benign assumption for most practical applications.
M ODELS D ESCRIPTION
An example Futoshiki Puzzle of size 3 × 3 and the corresponding graphs.
A value of 0 in the input puzzle represents an unassigned variable.
Black and red edges are
Constraint and Relation edges respectively.
We propose two models for value-set invariance: the
Binarized Model , and the Multi-valued Model .
In each case, we assume the training data is provided in the form D
= ( { ( x i , G C i ) , y i } M i =1 , G R ) as described in Section 3.
Let V and V (cid:48) denote the value-sets at train and test time, with cardinality k, k (cid:48) , respectively.
For each model, we divide the description in the following parts (a) Construction of Message Passing
Graph (b) Message Passing Rules (c) Loss Computation, and ﬁnally (d) Prediction on a problem with larger value-set.
We describe the details below.
4. B INARIZED M ODEL
Construction of Message Passing Graph: We denote the Message Passing Graph (MPG) by G =
( N, E ) with the set of nodes N and set of edges E , constructed as follows:
Nodes: For each node n C j ∈ N C in the constraint graph (yellow nodes in
Constraint Graph in Figure 1), we construct k binary valued nodes, denoted as n j , 1 , n j , 2 · · · n j ,k in N (blue nodes in Binary Graph).
Therefore, the total number of nodes constructed is | N C | k .
Edges:
We construct two categories of edges in G .
The ﬁrst type of edges are deﬁned for the edges that exist in the constraint graph G C (black edges).
For every edge, e C ( j , j (cid:48) )
∈ E C , where e C ( j , j (cid:48) )
.type
= q , we introduce k edges denoted as e q ( jl , j (cid:48) l ) , i.e., there is an edge between every pair of nodes, n j , l and n j (cid:48) , l , 1 ≤ l ≤ k .
These edges are directly inherited from the constraint graph, with k copies created for each edge due to binarization.
Edge type is same as in the original constraint graph and is denoted by q .
We refer to these as Constraint Edges .
We denote the set of all constraint edge types as Q .
The second type of edges are deﬁned for relations that exist between values in the value-set.
For every pair of binary nodes, n j , l and n j , l (cid:48) , we introduce an edge e r ( jl , jl (cid:48) )
if r ( v l , v l (cid:48) ) holds based on the relationship graph G R , i.e., e R ( l , l (cid:48) )
∈
E R with edge type r ( e.g. red edges encoding less-than relationship between value pairs (1 , 2) , (2 , 3) and (1 , 3) ).
For each relation r , there is a total of | N C || r
| edges introduced in the graph, where | r | denotes the number of (ordered) value pairs from
V which are related through r .
The edge type is the same as the relation r .
We refer to these as
Relational Edges .
We denote the set of all relational edge types as R .
Recurrent Message Passing: Once the message passing graph G has been constructed, we follow message passing rules similar to those used in RRNs (Palm et al., 2018), with some subtle differences.
For each node n j , l in the graph, we maintain a hidden state which is updated at each step based on the messages received from its neighbors.
Initialization: The initial
, of
, is based on the initial content of the variable
[ j ] , j .
If it is unassigned (yellow nodes with 0 in it), then
0 ( n
) =
[ − 1] ,
∈ V ( e.g. three blue nodes in the third row of the Binary Graph).
On the other hand, if x [ j ] is preassigned a value v l ( e.g. 1st variable assigned a value of
), then g and
and
Here,
− 1]
, and
k
-set V .
The hidden
Edge Dependent Message Passing: The nodes communicate their current hidden state via the messages sent to their neighbouring nodes across the edges.
The message depends not only on the current state of the sender and receiver, but also on the edge type across which the message is sent.
Speciﬁcally, for each edge type, z , there is a separate message passing function, f z , with z ∈ ( Q ∪ R ) where Q and R are the set of all constraint edge types and relation edge types respectively.
We compute the message for each edge e z ( j 1 l 1 , j 2 l 2 ) ∈ E as: m t (cid:104)
e z ( j 1 l 1 , j 2 l 2 ) (cid:105) =
f z ( h t ( n j 1 , l 1 ) , h t ( n j 2 , l 2 )) , ∀ e z ( j 1 l 1 , j
[Equation]
Hidden State Update: For each node, the incoming messages on the edges of the same type are aggregated by taking their weighted average.
The weights, a t , are computed using Bahdanau
Attention (Bahdanau et al., 2015) over constraint edges, whereas messages across relation edges are simply averaged: m t,z [ n j , l 1 ] =
(cid:80) e z ( jl 1 , j
2 l 2) ∈
E a t [ e z ( jl 1 , j 2 l 2 ) ]
m t [ e z ( jl 1 , j 2 l 2 ) ] , ∀ z ∈ ( Q ∪ R ) .
Finally, all messages, m t,z [ n j , l ] ∀ z ∈ ( Q ∪ R ) , are concatenated to create the input, m t [ n j , l ] for each node, n j , l .
The hidden state at step t is updated by the following state update function to generate the state h t +1 ( n j , l ) : h t +1 ( n j , l )
=
g ( h t ( n j , l ) , m t [ n j , l ] , u 0 ( n j , l )) , ∀ n j , l ∈ N .
Loss Computation
At each step t ∈ { 1 . . .
T } , we can compute the probability Pr ( n j , l .v = 1; Θ) of classifying a node n j , l as 1 by passing its hidden state through a scoring function s , i.e. , Pr t ( n j , l .v = 1; Θ)
=
σ ( s ( h t ( n j , l ))) , where σ is the standard Sigmoid function.
Here, n j , l .v denotes the value that node n j , l can take and belongs to the set { 0 , 1 } .
The Binary Cross Entropy (BCE) loss for each node n j , l is computed w.r.t.
its target, ˜y [ j , l ] , which is deﬁned as 1 whenever y
[ j ] = l and 0 otherwise.
Total loss at step t is the average BCE loss across all the nodes: 1 | N | (cid:80) n j , l ∈ N ˜y [ j , l ] log Pr t ( n j , l .v = 1; Θ) + (1 − ˜y [ j , l ]) log Pr t ( n j , l .v = 0; Θ) .
Like Palm et al.
(2018), we back-propagate through the loss at every step t ∈ { 1 . . .
T } as it helps in learning a convergent message passing algorithm.
During training, the objective is to learn the 3 initial embeddings u [ − 1] , u [0] , u [1] , message passing functions f z , ∀ z ∈ Q ∪ R , attention function a , state update function g , and the scoring function s .
Prediction on a problem with larger size of value-set: While testing, let the constraint and relation graph be G C (cid:48) and G R (cid:48) with n (cid:48) and k (cid:48) nodes respectively.
Let x (cid:48) be a partial solution, with n (cid:48)
variables x (cid:48)
[ j ] , each taking a value from value-set V (cid:48) of size k (cid:48) .
As described above, we create a graph G (cid:48) with n (cid:48) k (cid:48) nodes, run message passing for T steps, and for each variable x (cid:48)
[ j ] , compute the k (cid:48) probabilities, one for each of the k (cid:48) nodes n j , l ∀ l ∈ V (cid:48) corresponding to the variable x (cid:48)
[ j ] , which is assigned the value corresponding to maximum probability, i.e. , ˆ y
[ j ] = arg max l ∈V (cid:48)
Pr T ( n j , l .v = 1; Θ) .
M
ULTI - VALUED M ODEL
Construction of Message Passing Graph(s): Multi-valued model differs from the binarized model by avoiding binarization of nodes, and instead explicitly adding nodes in the message passing graph, one for each value in the value-set.
The message graph consists of two components: (a) A Graph
G = ( N, E ) to represent constraints inherited from the constraint graph G C =
( N C , E C ) (b) A
Graph ˜ G =
( ˜ N, ˜ E ) to represent relations inherited from the relationship graph G R =
( N R , E R ) .
We refer to G as Constraint Message Passing Graph , and ˜ G as Relationship Message Passing Graph .
Construction of Constraint Message-Passing Graph (CMPG)
Nodes: For each node n C j ∈ N C in the constraint graph, we construct a k -valued node, denoted as n j ∈ N .
Total number of such nodes constructed is | N C | .
We refer to these as Variable Nodes (yellow nodes in Multi-Valued Graph in ﬁg. 1).
Additionally, for each value v l ∈ V in the value-set, we create a node, denoted as n v l ∈ N .
Total number of such nodes constructed is |V| .
We refer to these as
Value Nodes (orange nodes).
Edges:
For every edge, e C ( j , j (cid:48) )
∈ E C , where e C ( j , j (cid:48) )
.type
= q , we introduce an edge denoted as e q ( j , j (cid:48) ) with type q .
These edges are directly inherited from the constraint graph.
Edge type is same as in the original constraint graph and is denoted by q .
We refer to these as
Constraint Edges (black edges) and the set of all constraint edge types as Q .
Additionally, to indicate the pre-assignment of values to the variables in x , we introduce new edges connecting value nodes to appropriate variable nodes.
Whenever x [ j ] = v l , add an edge, e a ( j , l ) between variable node n j and value node n v l (blue edges).
If x [ j ] is NULL , i.e. , unassigned, then add k edges, e ¯ a ( j , l ) , ∀ v l ∈ V , connecting the variable node n j with all k value nodes n v l ( e.g. , green edges connecting orange value node ‘2’ to all ‘0’ yellow variable nodes).
We refer to them as assignment edges .
Construction of Relationship Message-Passing Graph (RMPG)
Nodes: For each value v l ∈ V , create a node denoted as ˜ n v l ∈ ˜ N (purple nodes in Relation Graph in ﬁg. 1).
Total number of such nodes constructed is |V| .
We refer to these as Value Nodes .
Edges:
For every pair of value nodes, ˜ n v l and ˜ n v l (cid:48) , introduce an edge ˜ e r ( l , l (cid:48) ) if r ( v l , v l (cid:48) ) holds based on the relationship graph G R , i.e., e R ( l , l (cid:48) )
∈
E R with edge label r (red edges).
The edge type is the same as relation r .
These types of edges are deﬁned for relations that exist between values in the value-set and we refer to them as Relational Edges and denote the set of all relational edge types as R .
Achieving Value-set Invariance: A key question arises here: why do we need to construct a separate
RMPG ( ˜ G ) ?
Why not embed relevant edges in CMPG ( G ), as done for the binarized model?
The answer lies in realizing that we represent each value in the value-set explicitly in the multi-valued model, unlike the binarized model.
Hence, our model needs to learn representation for each of them in the form of value node embeddings.
Further, to generalize we need to learn as many embeddings as there are values in the largest test value-set, i.e., k max = max |V (cid:48) | .
We achieve this by randomly sub-selecting a k -sized set from { 1 . . .
k max } and permuting the chosen subset for each training example in a given mini-batch, and then computing the ‘relationship-aware’ embeddings from this permuted subset through message passing in RMPG.
The ‘relationship-aware’ embeddings are then used to initialize the value nodes (orange nodes) during message passing in CMPG.
For instance, if the permutation obtained is { w 1 , · · · , w l , · · · , w k } , where ∀ l , 1 ≤ w l ≤ k max , then embedding for the value node ˜ n v l in ˜ G is initialized by w th l learnable embedding ( e.g. , purple nodes for values ‘1’, ‘2’, and ‘3’ are initialized by the 5th, 7th, and 1st learnable embedding, respectively).
After message passing on ˜ G , the ‘relationship-aware’ embedding of ˜ n v l (purple node) is used to initialize the embedding for value node n v l (orange node) in G .
This elegant process is able to train all the k max embeddings by simply using the training data corresponding to V , and the corresponding relationship information.
Since these relationship aware embeddings need to be pre-computed before they can be passed to the downstream constraint processing, we construct two different message passing graphs, one for computing relationship-aware embeddings and one for constraint handling.
Recurrent Message Passing on RMPG
Initialization:
For each query x , select a k − permutation, P x , of k max ( e.g. , (5 , 7 , 1) , shown in square boxes in ﬁg. 1).
Initialize the embedding of ˜ n v l (purple nodes) in ˜ G with ˜ u
[ P x [ l ]] , ∀ l ∈ { 1 . . .
k } ( e.g. , ˜ u
[5] , ˜ u
[7] , ˜ u
[1] for initializing purple value nodes ‘1’,‘2’, and ‘3’ respectively).
Here, ˜ u
[ l (cid:48) ] , l (cid:48) ∈ { 1 . . .
k max } , are k max learnable embeddings.
Initialize the hidden state, ˜ h 0 (˜ n v l ) , ∀ ˜ n v l ∈ ˜ N with a 0 vector.
Computation of messages and hidden state update at step t is similar to the one described for Binarized Model in section 4.1.
Exact equations are in the appendix.
Hidden states are updated for ˜ T steps and the ﬁnal embeddings, ˜ h ˜ T (˜ n v l ) ∀ v l ∈ V , are used as ‘relationship-aware’ embeddings for initializing the input features (embeddings) of the nodes in CMPG G .
Recurrent Message Passing on CMPG
Initialization: Initialize the embedding of value nodes (orange nodes), n v l in G , using the ﬁnal ‘relationship-aware’ embeddings,
˜ h ˜ T (˜ n v l ) , of ˜ n v l (purple nodes) in ˜ G .
The variable nodes that are preassigned a value (non-zero yellow nodes) in x , are initialized by the embedding of the corresponding value node, i.e. , if x [ j ] = l , then n j is initialized with the ‘relationship-aware’ embedding, ˜ h ˜ T (˜ n v l ) , of ˜ n v l .
On the other hand, the embedding of nodes corresponding to the unassigned variables (‘0’ yellow nodes) are initialized by the average, (1 /k ) (cid:80) v l ∈V ˜ h ˜ T (˜ n v l ) , of all ‘relationship-aware’ embeddings.
For a variable node n j , we denote its initial embedding as: u 0 ( n j ) .
[Table] [Table]
[Table]
Initialize the hidden state, h 0 ( n j ) , for each variable node n j in G , with a 0 vector.
Message passing and state update is similar to the one described for Binarized Model.
See appendix for details.
Loss Computation:
At step t , a scoring function, s , computes a score, s ( h t ( n j ) , h t ( n v l )) , for assigning a value v l to a variable n j based on the hidden state of corresponding value and variable nodes.
For each variable node, a Softmax converts these scores into probabilities over the values v l ∈ V , i.e. , Pr ( n j .v = v l ) =
Softmax ( s ( h t ( n j ) , h t ( n v l ))) , ∀ v l ∈ V , where, n j
.v ∈ V denotes the value that node n j can take.
Total loss at step t is nothing but the average of standard Cross Entropy over variable nodes: L t = − 1 | N | (cid:80) n j ∈ N log Pr ( n j .v =
y
[ j ]) .
To ensure that the multi-valued model learns different embeddings for each value in the value-set, we add an auxiliary loss term, corresponding to the total pairwise dot product (similarity) of any two embeddings, before and after message passing in ˜ G .
We call it Orthogonality Loss .
Its weight, α , is a hyper-parameter.
Prediction on a problem with larger size of value-set: For a puzzle with larger value-set, V (cid:48) , a bigger RMPG is created, whose k (cid:48) nodes are initialized with the (learnt) ﬁrst k (cid:48) embeddings.
Unlike training, we always choose ﬁrst k (cid:48) embeddings to avoid randomness during testing.
Prediction is made using the probabilities at the last step T , i.e. , ˆ y
[ j ] = arg max v l ∈V (cid:48)
Pr ( n j .v = v l ) .
Relative Comparison: In the binarized model, the constructed graph G has k | N C | nodes and at least k | E C | + | N C | k
( k − 1) / 2 edges due to binarization.
This increases the graph size by a factor of at least k .
As a result, we soon hit the memory limits of a GPU while training the binarized model with bigger problems.
The model also needs signiﬁcantly more inference time due to its bigger size.
On the other hand, multi-valued model, while being compact in terms of its representation, needs to learn additional embeddings, for a speculative size of value-set during testing.
This poses additional requirement on the model both in terms of representation, and learning, possibly affecting the quality of generalization.
While this is a simple analytical understanding of the possible merits of the two models, we examine experimentally the impact of these issues on real datasets.
E XPERIMENTS
The goal of our experiments is to evaluate the effectiveness and relative strengths and weaknesses of the two proposed methods, binarized model and multi-valued model, for achieving value-set invariance.
We also compare our models with a generic constraint learner, NLM (Dong et al., 2019).
We experiment on datasets generated from Lifted CSPs of three different puzzles, viz.
, Sudoku,
Futoshiki, and Graph Coloring (see Table 5 in appendix for data statistics).
We train each of the models on data generated from a ﬁxed value-set, and test on instances generated from larger value-sets.
5. T ASK D ESCRIPTION AND D ATASETS
Futoshiki: This is a number puzzle in which we have to place numbers { 1 . . .
k } on a k × k grid, such that no two cells in a row or column contain the same number.
In addition, there may be an ordering constraint between two cells, which needs to be honored in the ﬁnal solution.
The input has some of the grid cells already ﬁlled with a number and the task is to complete the grid, respecting the additional ordering constraint where ever it exists.
We train our model on 6 × 6 puzzles, with the percentage of missing cells varying uniformly between 28 − 70% .
We test our models on puzzles with board size ranging between 6 × 6 to 12 × 12 , with the same percentage of missing cells.
[Table]
Graph Coloring (GCP): In this task we are given a partially colored graph along with the number of colors k , and the objective is to color rest of the nodes using k colors such that no two adjacent nodes have the same color.
We train our model on randomly generated 4 − colorable graphs, and test on k (cid:48)
− colorable graphs, with k (cid:48) ∈ { 4 , 5 , 6 , 7 } .
Training data has graphs with graph order varying uniformly between 40 − 120 , and percentage of masked nodes vary uniformly between 28 − 70% .
Sudoku: We randomly select 10 , 000 training queries from the 9 × 9 dataset introduced in Palm et al.
Our test set has k (cid:48)
× k (cid:48) puzzles, with k (cid:48) ∈ { 10 , 12 , 15 , 16 } .
Data generation process is similar to Futoshiki, with the distribution of missing cells varying between 30 − 68% depending on the board size.
Instead of backtracking, solution validity is checked through the GSS library (Pieters, 2019).
Please see appendix for more details on data generation process for all three tasks.
5. E XPERIMENTAL S ETUP & B ASELINES
[Table]
In both our models, nodes are initialized with learnable 96 dimensional embeddings.
In multi-valued model, k max = 12 , 7 , and 16 embeddings are learnt for Futoshiki, GCP, and Sudoku respectively.
Message passing on G in binarized model runs for 32 steps.
Message passing on RMPG, ˜ G and CMPG, G in the multi-valued model runs for ˜ T = 1 and
T = 32 steps respectively.
The message passing functions, f z , in both the models, are 3 layer
MLPs, similar to those in RRN, with a difference that there is a separate function for each edge type z .
In both the models, a layer normalized LSTM cell with hidden dimension 96 acts as state update functions g and ˜ g .
For the orthogonality loss in multi-valued model, we choose α = 0 .
To minimize the loss, we use Adam optimizer with learning rate 0 . 0002 .
All models are trained on K
GPU nodes with 12GB memory.
Each model takes less than 75 hours to train.
See appendix for more details.
We take simple average of model weights stored at multiple points (Izmailov et al., 2018).
All checkpoints obtained after ﬂattening of the learning curve are selected for computing the average.
Baseline: For Futoshiki, we train two versions of NLM by varying depth : the number of Logic
Machines that are stacked on top of each other.
Like (Nandwani et al., 2021), we train one 30 layer deep NLM model with residual connections for Futoshiki, but unlike them, we assume access to constraint graph, which we provide as a binary predicate input to the model.
NLM with 30 depth could not ﬁt puzzles with board-size greater than 10 within 12GB memory of K40 GPU.
Hence, we train another version by reducing the depth to 15 .
For GCP, we train a model with depth 24 .
For
Sudoku, on increasing depth beyond 14 , we could not ﬁt even one 9 × 9 train sample within GPU memory.
Note that the maximum depth chosen for the graph experiments reported in (Dong et al., 2019) is 8 .
This is because they work with much smaller graphs (up to maximum 50 nodes), whereas smallest graph in Futoshiki has 6 3 = 216 binary nodes, warranting creation of much deeper models.
Evaluation Metrics: We report two metrics: board accuracy and point-wise accuracy .
In the former, we consider output of the model as correct only if it satisﬁes the underlying CSP, whereas in the later, we give partial credit even for assigning some of the variables correctly. See Appendix for details.
For each setting, we report the mean and standard deviation over three runs by varying random seed.
5. R ESULTS AND D ISCUSSION
[Table]
We report the accuracies over different sizes of value-set for Futoshiki,
GCP and Sudoku in Table 1, Table 2, and Table 3, respectively.
We ﬁrst observe that NLM fails to train on Sudoku, and its performance is worse than one or both of our models for all experimental settings in Futoshiki and
GCP.
As expected, in Futoshiki, NLM model with depth 30 fails to run on board sizes 11 and 12 and depth 15 model fails to run on size 12 .
Note that both NLM and our binarized model work by binarizing the underlying puzzle, but we observe that binarized model shows signiﬁcantly better generalization across value-sets.
We note that NLM performs decently well for GCP even for the test graphs with chromatic number k (cid:48)
= 7 .
We attribute this to the fact that in our test data for k (cid:48) = 7 , graphs are relatively small, with maximum 80 graph nodes, resulting in total 560 binary objects in NLM, which is similar to the maximum 400 binary objects that it trains over ( k =4, max 100 nodes).
Comparison between binarized model and multi-valued model: We ﬁrst observe that both our models achieve similar performance on the value-set over which they are trained.
We observe that the standard deviation of the board accuracy increases signiﬁcantly as the size of value-set increases, whereas the pointwise accuracy is relatively stable.
This is due to the high sensitivity of the board accuracy to pointwise accuracy: even if a single variable is incorrectly assigned in a puzzle, its contribution towards board accuracy goes to 0, whereas it still contributes positively towards pointwise accuracy.
When trained on small sizes, binarized model shows better generalization.
But as the problem size increases, the computational graph for binarized model fails to ﬁt in the available
GPU memory and thus its performance degrades.
On the other hand, multi-valued model being memory efﬁcient, scales much better.
To demonstrate this, Table 4 reports the performance of multivalued model further ﬁnetuned on sudoku puzzles of board-size 24, and tested on board-sizes varying between 15 and 25.
We couldn’t ﬁnetune the binarized model as its computational graph doesn’t ﬁt in the GPU.
The binarized model trained on puzzles of board-size 9 gives 0 .
0 board accuracy on sizeand 25.
The performance of multi-valued model is better than binarized model not only on board-size 25, but also on board-sizes smaller than 24.
This also demonstrates that the poor performance of the same multi-valued model trained on smaller board-size is not due to any lack of representation power , but due to difﬁculty in learning additional embeddings: when training k (cid:48) embeddings from puzzles of board-size k , multi-valued model never gets to see all k (cid:48) value embeddings together.
Moreover, the different combinations of k out of k (cid:48) embeddings increase exponentially with ( k (cid:48) − k ) , making it further difﬁcult to train.
To validate this, we train a multi-valued model with only 7 learnable embeddings for Futoshiki and observe that the board accuracy on 7 board-size increases to 97 .
82% (at par with binarized model) from 90 .
18% which is achieved when trained with 12 embeddings.
[Figure]
2: Resource utilization for Futoshiki
Computational complexity: In ﬁg. 2, for the two models, we compare the average inference time and the GPU memory occupied by a batch of 32 Futoshiki puzzles over value-sets of varying sizes.
As expected, the multi-valued model is much more efﬁcient, both in terms of time and memory.
C ONCLUSION AND F UTURE W ORK
We have looked at the novel problem of value-set invariance in combinatorial puzzles, formally deﬁned using lifted CSPs and proposed two different neural solutions, extending an existing architecture (RRN).
Our experimental evaluation demonstrates the superior performance of our models compared to an existing neural baseline.
We discuss the relative strengths and weaknesses of our proposed models.
Directions for future work include solving puzzles generated from more complicated CSPs, and scaling to even larger sizes.
E THICS S TATEMENT
In its current form, our work is primarily a technical contribution, with no immediate ethical consequences.
Our work develops the line of recent research in which constraint reasoning is carried out through neural architectures.
We believe that neural approaches for symbolic reasoning will go a long way in creating an integrated AI system.
This is because an integrated system requires not only perceptual, but also high-level reasoning.
Neural approaches will provide a uniform vocabulary so that both these forms of reasoning can interact with each other, improving performance of the overall system.
As more AI systems start to be used in critical applications such as healthcare, law, and disaster management, it is important that they honor the safety and accountability constraints set up by domain experts.
Their ability to perform high-level reasoning enables them to honor such constraints more effectively.
Thus, our line of work, in the long run, could have signiﬁcant positive ethical implications.
We see no obvious negative implications of our work.
R EPRODUCIBILITY S TATEMENT
To ensure reproducibility, we have discussed the dataset creation process and provided model architecture details in Section 5.1 and Section 5.2, respectively.
We provide the details of the exact hyper-parameters, computational resources used, and additional experimental details in the appendix.
We will make all the data, and source code for training and data generation, publicly available on paper’s acceptance.
A A PPENDIX 4 M ODELS D ESCRIPTION 4.2 M ULTI - VALUED M ODEL Recurrent Message Passing ( on RMPG)
Message Passing Update: At step t , update the hidden state, ˜ h t (˜ n v l ) , of each of the value nodes in ˜ G , by the concatenation, m t [˜ n v l ] , of average messages, m rt [˜ n v l ] , received across edges of type r ∈ R :
˜ h t +1 (˜ n v l ) =
˜ g
(˜ h t (˜ n v l ) , m t [˜ n v l ] , ˜ u
[ P x [ l ]]) , where ˜ g is the hidden state update function.
Like (Palm et al., 2018), it always takes the initial embedding, ˜ u
[ P x [ l ]] , of the value node ˜ n v l as one of the inputs.
Notice that the message, m rt [˜ n v l ] , is the average of the messages, f r (˜ h t (˜ n v l ) , ˜ h t (˜ n v l (cid:48) ))
∀
˜ e r ( l , l (cid:48) )
∈
˜ E , where f r is the message passing function for edge type r ∈ R .
The hidden states are updated for ˜
T steps and the ﬁnal embeddings, ˜ h ˜ T (˜ n v l ) ∀ v l ∈ V , are used as ‘relationship-aware’ embeddings for initializing the input features (embeddings) of both variable nodes, n j , and value nodes, n v l in G (orange and yellow nodes respectively in Multi-valued Graph).
Recurrent Message Passing ( on CMPG)
Message Passing Update: At step t , similar to binarized model, each node receives messages from its neighbors, that are aggregated based on the edge type.
For each node, the aggregated messages, m t,z [ n j ] , for different edge types, z ∈ ( Q ∪ A ) , are stacked to create, m t [ n j ] , which updates the hidden state as: h t +1 ( n j ) =
g ( h t ( n j ) , m t [ n j ] , u 0 ( n j )) , ∀ n j ∈ N .
Here A denotes the set of all assignment edge types (edges from orange value nodes to yellow variable nodes).
5 E XPERIMENTS
We experiment on datasets generated from Lifted CSPs of three different puzzles, viz.
, Sudoku,
Futoshiki, and Graph Coloring.
In addition, we ﬁne-tune our multi-valued model for Sudoku onpuzzles of size 24 and test it on puzzles of different board sizes.
Table 5 contains the details of both train and test data for the different experiments.
[Table]
5.1 T ASK D ESCRIPTION AND D ATASETS
Futoshiki: We train our model on 6 × 6 puzzles, with the percentage of missing cells varying uniformly between 28 − 70% .
We test our models on puzzles with board size ranging between 6 ×to 12 × 12 , with the same percentage of missing cells.
The number of ordering constraints is twice the board size.
To generate data, we ﬁrst randomly generate a completely ﬁlled k × k board and then
[Table]
randomly mask m % of the cells.
We search for its all possible completions using backtracking to ensure that it has only one solution.
Finally, we insert ordering constraints between 2 k randomly picked pairs of adjacent cells.
The entire process is repeated by varying k and m to generate both the test and train dataset.
The training data consists of 12 , 300 puzzles on 6 x 6 board size with the percentage of missing variables varying between 28 − 70% .
The exact details of the testing data for different board sizes are provided in Table 6.
We note that it becomes increasingly difﬁcult to ﬁnd puzzles with unique solution as the board size increases.
Therefore, we are forced to reduce the maximum percentage of masked (unassigned) cells with increasing board size.
GCP:
The training data for Graph Coloring Problem consists of around 25 thousand 4-colorable graphs with graph order varying uniformly between 40 − 120 , and the percentage of unassigned (masked) variables varying uniformly between 28 − 70% depending on the graph order.
The exact details of the testing data for different chromatic number (value-set size) are provided in Table 6.
To create non-trivial problems for the dataset, we always attempt to color graphs with the smallest possible number of colors, i.e., the chromatic number of the graph.
We follow Erd ˝ os–Rényi (ER) model to generate random graphs.
It takes number of nodes, n , and an edge probability, p , as input, and adds an edge independent of the other edges with probability p .
We note that to sample a graph with n nodes and a given chromatic number k , we need to carefully adjust the range from which edge probability p is sampled.
The exact range from which p is sampled uniformly for each chromatic number k and a range of nodes n is given in Table 7.
We use a CSP solver (Perron & Furnon, 2019) to determine the chromatic number of a given graph, which becomes a bottleneck while generating graphs with higher chromatic number.
As a result, we were not able to generate graphs with more than 80 nodes for chromatic number 7 , in a reasonable amount of time.
Sudoku: The training data consists of 10 thousand 9 x 9 puzzles randomly selected from the dataset introduced in (Palm et al., 2018).
For standard 9 × 9 board, we use the same test data as used in
RRN (Palm et al., 2018) 1 .
The test data for the board-sizes between 10 and 16 is generated using the methodology similar to Futoshiki.
Instead of backtracking, solution validity and uniqueness is checked through the GSS library (Pieters, 2019).
The exact details of the testing data for different board sizes are provided in Table 6.
For the experiment where we ﬁne-tune our models on 24 ×puzzles, both the train and test data for board size 24 and 25 are generated following the methodology
[Table]
similar to Futoshiki.
In this setup, we were not able to verify the uniqueness of solution through GSS library as it doesn’t scale to such large sizes.
Solution Multiplicity in GCP Dataset and larger board-size puzzles of Sudoku: An input query in GCP may have more than one solution, out of which only one is given at train time.
But the network may discover a new valid solution, and computing loss of the discovered solution w.r
[Table]
t. the given solution may unnecessarily penalize the model.
To avoid this, we algorithmically verify if a prediction is a valid coloring or not, and compute loss w.r [Table]
t. to this discovered solution, instead of the given one.
This is equivalent to solving a One-of-Many Learning problem (Nandwani et al., 2021) with all possible colorings given at training time.
The same phenomenon of solution multiplicity exists for Sudoku puzzles of size 24 and 25 , as verifying the uniqueness of puzzles on such large board-size became computationally infeasible.
5.2 E XPERIMENTAL S ETUP AND B ASELINES
Evaluation Metrics: We report two metrics: board accuracy and point-wise accuracy for all our experiments.
In the former, we consider output of the model as correct only if it satisﬁes the underlying CSP, whereas in the later, we give partial credit even for assigning some of the variables correctly.
We formally deﬁne it below:
Pointwise accuracy:
Let Y x be the set of possible solutions for a query x with k variables, and let ˆ y be the model’s prediction. Pointwise accuracy of the prediction ˆ y with respect to the solution y ∈ Y x , denoted as, PointAcc ( y , ˆ y ) , is deﬁned to be the fraction of variables that match between the y and ˆ y : PointAcc ( y , ˆ y ) = 1 k (cid:80) ki =1 1 { y [ i ] =
= ˆ y
[ i ] } , where 1 { . } is the indicator function.
Given above, we deﬁne pointwise accuracy for a prediction ˆ y of a query x with respect to a solution set Y x to be the maximum among the pointwise accuracy with respect to each of the solutions in the set Y x .
Mathematically, PointAcc ( Y x , ˆ y ) =
max y ∈ Y x PointAcc ( y , ˆ y ) .
For Sudoku and Futoshiki, since there is a unique solution, we can easily compute pointwise accuracy as the target set Y x is singleton.
For the GCP task, whenever the model returns a valid coloring, pointwise accuracy is 1 , otherwise, in the absence of access to complete Y x , we report a lower bound by performing a local search, using Google’s OR-Tools 2 , for a valid coloring closest to the model prediction.
Same is true for sudoku puzzles on 24 and 25 board size.
Why care about point-wise accuracy?
In our settings, the generalization problem can be hard, especially when there is a large difference between the sizes of the value-sets for the train and test domains.
Given that we are deﬁning a novel task, and it is important to measure progress even when problems are hard, we compare the two models using a simpler metric (pointwise accuracy) as well, in addition to board accuracy.
This additional metric can help us detect progress, and also compare the relative performance of underlying models.
Computational resources: All models are trained on K40 GPUs with 12 GB memory, available on an HPC cluster.
Hyperparameters: The list below enumerates the various hyperparameters with a brief description and the values chosen.
Batch Size: For each task, we selected the maximum batch size that can be accommodated in 12GB GPU memory.
Refer to Table 8 for details. [Table]
Weight Decay in Optimizer
: As in the original RRN paper, we chose a weight decay factor of 1E-
Orthogonality Loss Factor: To ensure that the multi-valued model learns different embeddings for each value in the value-set, we add an auxiliary loss term, corresponding to the total pairwise dot product of any two embeddings, before and after message passing on the Relation Message Passing Graph (RMPG), ˜ G .
Its weight, α , was chosen amongst { 0 . 01 , 0 . 1 , 0 . 5 } by cross validating on a devset for Futoshiki, and then ﬁxed afterwards for all our experiments.
Edge Dropout : While collating the messages from the edges of the same type we drop 10% of the messages, as done in RRN.
Dropout is used in the Message Passing Graph (MPG) of the binarized model, and the Constraint Message Passing Graph (CMPG) of the multi-valued model
Model Averaging: As suggested in (Izmailov et al., 2018), to reduce the variance of our model performance, we take simple average of model weights stored at multiple points.
All checkpoints beyond a point when the learning curve ﬂattened are selected for computing the average.
[Table] Training Time: Table 9 enumerates the exact training cost, in terms of total training epochs, number of gradient updates [Table]
and clock time, for all three tasks and for both our models as well as the baseline NLM model. Note that while a multi-valued model may have fewer parameters, and results in a much smaller graph and inference time for a given problem, its training time could still be higher, especially in terms of total training epochs and number of gradient updates. However, because of its memory efﬁciency, we can keep a much larger batch size during training, and because of its speed efﬁciency, each update is much faster.
As a result, the overall clock time comes out to be comparable to the binary model for the two of our tasks, i.e. Futoshiki and Sudoku, and it is within 2x for GCP, even though the number of epochs is much higher.