Neural Models for Output-Space Invariance in Combinatorial Problems

Recently many neural models have been proposed to solve combinatorial puzzles1 by implicitly learning underlying constraints using their solved instances , such2 as sudoku or graph coloring ( GCP ) . One drawback of the proposed architectures,3 which are often based on Graph Neural Networks ( GNN ) ( Zhou et al. , 2020 ) , is4 that they can not generalize across the size of the output space from which variables5 are assigned a value , for example , set of colors in a GCP , or board-size in sudoku.6 We call the output space for the variables as ‘ value-set ’ . While many works have7 demonstrated generalization of GNNs across graph size , there has been no study8 on how to design a GNN for achieving value-set invariance for problems that9 come from the same domain . For example , learning to solve 16× 16 sudoku after10 being trained on only 9× 9 sudokus , or coloring a 7 colorable graph after training11 on 4 colorable graphs . In this work , we propose novel methods to extend GNN12 based architectures to achieve value-set invariance . Specifically , our model builds13 on recently proposed Recurrent Relational Networks ( RRN ) ( Palm et al. , 2018 ) .14 Our first approach exploits the graph-size invariance of GNNs by converting a15 multi-class node classification problem into a binary node classification problem.16 Our second approach works directly with multiple classes by adding multiple17 nodes corresponding to the values in the value-set , and then connecting variable18 nodes to value nodes depending on the problem initialization . Our experimental19 evaluation on three different combinatorial problems demonstrates that both our20 models perform well on our novel problem , compared to a generic neural reasoner.21 Between two of our models , we observe an inherent trade-off : while the binarized22 model gives better performance when trained on smaller value-sets , multi-valued23 model is much more memory efficient , resulting in improved performance when24 trained on larger value-sets , where binarized model fails to train.25 1 INTRODUCTION26 The capability of neural models to perform symbolic reasoning is often seen as a step towards the27 framework for unified AI , i.e. , building end-to-end trainable system for tasks , which need to combine28 low level perception with high level cognitive reasoning ( Kahneman , 2011 ) . While neural networks29 are naturally excellent at perception , they are increasingly being developed for high-level reasoning30 tasks , e.g. , solving SAT ( Selsam et al. , 2019 ; Amizadeh et al. , 2019a ; b ) , neural theorem proving31 ( Rocktäschel et al. , 2015 ) , differentiable ILP ( ∂ILP ) ( Evans & Grefenstette , 2018 ) , playing blocks32 world ( Dong et al. , 2019 ) , solving sudoku ( Palm et al. , 2018 ; Wang et al. , 2019 ) . Our work follows33 this literature for solving combinatorial puzzles – in particular , the methods that do not assume an34 explicit knowledge of rules , but instead learn to directly solve the problem using some of its solved35 instances , e.g . Recurrent Relational Networks ( RRN ) ( Palm et al. , 2018 ) . Such models assume a36 fixed value-set , i.e. , the set from which variables are assigned values is assumed to be constant37 during training and testing . This is a significant limitation , since it may not always be possible to38 generate sufficient training data for similar large problems in which variables take values from a39 bigger value-set ( Najafian et al. , 2018 ) . Moreover , it is also a desirable goal since as humans , we40 often find it natural to generalize to problems of unseen variable and value sizes , once we know how41 to solve similar problems of a different size , e.g. , we may solve a 12 × 12 sudoku after learning42 to solve a 9 × 9 sudoku . We note that graph based models have been shown to generalize well43 on varying graph sizes , e.g. , finding a satisfying solution of a CNF encoding of a CSP with 10044 Boolean-variables , after training on CNF encodings of CSPs with only 40 Boolean-variables ( Selsam45 et al. , 2019 ) . However , the model trained using CNF encoding of Boolean-CSPs can not be used46 directly for a non-Boolean CSP in which variables take value from a different ( larger ) value-set.47 In response , we study value-set invariance in combinatorial puzzles from the same domain . To48 formally define a similar puzzle with variables taking values from a different value-set , we make use49 of Lifted CSP ( Joslin & Roy , 1997 ) , a ( finite ) first-order representation that can be ground to CSPs of50 varying variable and value-set sizes . We note that even though we use Lifted CSPs to define value-set51 invariance , its complete specification is assumed to be unknown . Specifically , we do not have access52 to the constraints of the CSP , and thus neural SAT solvers like NeuroSAT ( Selsam et al. , 2019 ) can not53 be used . While training , we only assume access to solved instances along with their constraint graph.54 We define our problem as : given solved instances and corresponding constraint graph of an unknown55 ground CSP with a value-set of size k , can we learn neural models that generalize to instances of56 the same lifted CSP , but with a different value-set of size k′ ( typically k′ > k ) ? An example task57 includes training a model using data of 9× 9 Sudoku , but testing on a 12× 12 or a 16× 16 Sudoku.58 We build our solution using RRNs as the base architecture . They run GNN on the constraint graph,59 and employ iterative message passing in a recurrent fashion – the nodes ( variables ) are then decoded60 to obtain a solution . We present two ways to enhance RRNs for value-set invariance.61 Binarized Model : Our first model converts a multi-class classification problem into a binary classifi-62 cation problem by converting a multi-valued variable into multiple Boolean variables , one for each63 value in the value-set . The binarized constraint graph gets defined as : if there is an edge between two64 variables in original constraint graph , there are k edges between Boolean nodes corresponding to65 the same value and the same two variables in the new graph . In addition , all k Boolean variables,66 corresponding to a multi-valued variable , are connected with each other . This model naturally67 achieves value-set invariance . At test time , a larger value-set just results in a larger graph size . All68 GNN weights are tied , and because all the variables in the binarized model are Boolean , embeddings69 for binary values ‘ 0 ’ and ‘ 1 ’ , trained at training time , are directly applicable at test time.70 Multi-valued Model : Our second model directly operates on the given multi-valued variables and the71 corresponding constraint graph , but introduces a value node for every value in the value-set . Each72 pre-assigned ( unassigned ) variable node is connected to that ( respectively , every possible ) value node.73 The challenge in this model is initializing value nodes at test time when k′ > k. We circumvent74 this problem by training upfront k′ or more value embeddings by randomly sub-selecting a k sized75 subset during each learning iteration . This random sub-selection exploits the symmetry of value-set76 elements across instances . During test time , k′ of the learned embeddings are used.77 We perform extensive experimental evaluation on puzzles generated from three different structured78 CSPs : Graph Coloring ( GCP ) , Futoshiki , and Sudoku . We compare two of our models with an79 NLM ( Dong et al. , 2019 ) baseline – a generic neural reasoner , which either fails to scale or performs80 significantly worse for most test sizes used in our experiments . We also compare our two models81 along the axes of performance and scalability and discuss their strengths and weaknesses.82 2 RELATED WORK83 This paper belongs to the broad research area of neural reasoning models , in which neural models84 learn to solve pure reasoning tasks in a data-driven fashion . Some example tasks include theorem85 proving ( Rocktäschel et al. , 2015 ; Evans & Grefenstette , 2018 ) , logical reasoning ( Cingillioglu & 86 Russo , 2019 ) , probabilistic logic reasoning ( Manhaeve et al. , 2018 ) , classical planning ( Dong et al.,87 2019 ) , probabilistic planning in a known MDP ( Tamar et al. , 2017 ; Bajpai et al. , 2018 ) , and our focus88 – combinatorial problems that are instances of an unknown constraint satisfaction problem.89 There are two main research threads within neural CSPs and SAT . First thread builds neural models90 for problems where the CSP constraints or SAT clauses are explicitly provided to the model . For91 example , NeuroSAT ( Selsam et al. , 2019 ) and PDP ( Amizadeh et al. , 2019b ) assume that the CSP92 is expressed in a Conjunctive ( or Disjunctive ) Normal Form . Similarly , Circuit-SAT ( Amizadeh93 et al. , 2019a ) uses the knowledge of exact constraints to convert a CSP into a Boolean Circuit . This94 research has similarities with logical reasoning models like DeepProbLog ( Manhaeve et al. , 2018 ) ,95 and DeepLogic ( Cingillioglu & Russo , 2019 ) , which require human designed rules for reasoning . Our96 work belongs to the second thread where the constraints or clauses are not provided explicitly , and97 only some underlying structure ( e.g. , Sudoku grid cell connectivity ) is given along with training data.98 The intention is that the model not only learns to reason for the task , but also needs to learn the implicit99 semantics of each constraint . SATNET ( Wang et al. , 2019 ) falls in this category – it formulates a100 learnable low-rank Semi-definite Program ( SDP ) relaxation for a given MAXSAT problem trained101 via solved SAT problems . Similarly , Recurrent Relational Networks ( RRN ) ( Palm et al. , 2018 ) use102 recurrent message passing graph neural network to embed the variables of the unknown CSP , and the103 relationship between them , in a latent vector space and finally assign a value to each variable based104 on its embedding . Both these works assume a fixed number of variables that remains unchanged105 across training and test . While we build on RRNs , we substantially extend the formalism to study106 value-set invariance . Formally , our work can be seen as solving a ( finite ) first-order formulation of107 the CSP , called Lifted CSP ( Joslin & Roy , 1997 ) , which can be ground to CSPs with varying number108 of variables and values . To our knowledge , there is relatively limited prior work on neural models109 that can generalize to variable-sized instances of an underlying first order reasoning task – one related110 approach builds neural models for First-order MDPs ( Garg et al. , 2020 ) .111 Finally , there has been a long history of work dedicated to learning rules or constraints from training112 data using Inductive Logic Programming ( Lavrac & Raedt , 1995 ; Friedman et al. , 1999 ) . Evans & 113 Grefenstette ( 2018 ) propose differentiable neural relaxation of ILP ( ∂ILP ) . Neural Logic Machines114 ( NLM ) ( Dong et al. , 2019 ) is another framework that learns lifted rules , shown to be more scalable115 than ∂ILP . It allows learning of first-order logic rules expressed as Horn Clauses over a set of116 predicates . It consists of a stack of logic machines that learn predicate relationships between objects117 using training data . Learning of first-order rules makes NLM amenable to transfer over different CSP118 sizes ( Nandwani et al. , 2021 ) , and are thus directly comparable to our work . The main challenge of119 such approaches is that they fail to scale to the size of the problems considered in this work . In our120 experiments , we compare our methods against both deep and shallow versions of NLM . Note that our121 work relies on the assumption that GNNs generalize across graph sizes . Yehudai et al . ( 2021 ) study122 the scenarios under which this assumption may not hold . We discuss the details in the appendix.123 3 PRELIMINARIES AND PROBLEM DEFINITION124 A combinatorial puzzle can be thought of as a grounded CSP and to formally define a puzzle from125 the same domain but a larger value-set , we resort to the notion of ‘ Lifted CSPs ’ that represent an126 abstraction over multiple ground CSPs of the same type . A lifted CSP does not include a specific127 set of variables and values ; instead , it operates in terms of variable and value references that can128 be instantiated with all ground variables and values in a ground CSP . This makes them amenable129 to instantiate CSPs or puzzles with varying number of variables as well as values . We define a130 Lifted CSP LC as a three tuple 〈P , R , C〉 . P is a set of predicates : a predicate p ∈ P represents131 a Boolean function from the set of its arguments , which are variable references . Similarly , R is132 a set of relations over value space – a r ∈ R reprents a Boolean function over arguments that133 are value references . A predicate ( or a relation ) with its arguments is called an atom . C is a set134 of lifted constraints , constructed by applying logical operators to atoms – they are interpreted as135 universally quantified over all instantiations of variable and value references . Finally , Lifted CSP136 uses a special unary function Value , whose argument is a variable reference and evaluates to a value137 reference . As an example , a lifted CSP for Sudoku may have a P = { Nbr } for whether two cells are138 in same row , column or box , R = { Neq } , representing two values are unequal , and a lifted constraint:139 Nbr ( c1 , c2 ) → Neq ( Value ( c1 ) , Value ( c2 ) ) .140 A lifted CSP LC yields a ground CSP C , given a set of variables O , and a set of values V , and a141 complete instantiation of all predicates and relations over this set ( e.g. , in Sudoku , the number of142 cells , possible values , and which cells are neighbors and which are not ) . The ground constraints are143 constructed by instantiating lifted constraints over all variables and values . A ( satisfying ) solution , y,144 of a CSP refers to a complete specification of Value : O → V function , such that all the constraints145 are satisfied . We are often given a partial ( satisfying ) solution , x – an assignment of values to a subset146 of variables Õ ⊆ O and the goal is to output y , such that y agrees with x for the subset Õ.147 Given a ground CSP C , the Constraint Graph , GC = ( NC , EC ) , is constructed by having each148 variable in the CSP represent a node in the graph and introducing an edge between two nodes nC1 , n C 2149 iff the corresponding variables appear together in some constraint . The edges in the constraint graph150 are typed based on the identity of the lifted constraint from which it comes . Note that there could151 be multiple edges between nodes nC1 , n C 2 in GC , if these nodes appear together in more than one152 constraint . We embed the knowledge about relations between values in V in the form of another153 graph , called Relation Graph , GR = ( NR , ER ) , where there is a node for every value in the set V ,154 and there is a ( directed ) edge between nodes corresponding to vl , v′l depending on whether r ( vl , vl′ ) 155 is true or not , for every r ∈ R. Similar to GC , this graph can also have multi-edges between two156 pairs of nodes , if more than one relationship holds between the corresponding values.157 Problem Definition : To achieve value-set invariance , our goal is to train a model MΘ on training158 data from an unknown ground CSP C ( with variables O and value-set V ) obtained from an unknown159 lifted CSP LC , and test it on an arbitrary ground CSP C ′ from the same lifted CSP ( with variables160 O′ and value-set V ′ ) , where |V| 6= |V ′| . Formally , we are given training data D as a set of tuples161 { ( ( xi , GCi ) , yi ) } Mi=1 , along with a relationship graph GR encoding relations between values in the162 value-set V . Here , ith instance denotes a partial and corresponding complete solution for Ci . We note163 that explicit form of the constraints in Ci or LC are not available , only the graphs are given to the164 model . Our goal is to learn model MΘ , such that given graphs GC′ and GR′ , and a partial solution165 x′ ( for CSP C ′ ) : MΘ ( x′ ) = y′ , only if y′ is a corresponding complete solution for x′ . Note that in166 one of our models , we will additionally assume that max |V ′| , denoted as kmax , is known to us at167 training time , which we argue is a benign assumption for most practical applications.168 4 MODELS DESCRIPTION169 We propose two models for value-set invariance : the170 Binarized Model , and the Multi-valued Model . In171 each case , we assume the training data is provided in172 the formD = ( { ( xi , GCi ) , yi } Mi=1 , GR ) as described173 in Section 3 . Let V and V ′ denote the value-sets at174 train and test time , with cardinality k , k′ , respectively.175 For each model , we first present a high level intu-176 ition , followed by description of : ( a ) Construction of177 Message Passing Graph ( b ) Message Passing Rules178 ( c ) Loss Computation , and finally ( d ) Prediction on179 a problem with larger value-set.180 4.1 BINARIZED MODEL181 Intuition behind our Binarized Model comes directly182 from the ‘ sparse encoding ’ of a discrete CSP into a183 SAT formula ( de Kleer , 1989 ; Walsh , 2000 ) , in which184 assignment of a value v ∈ V to any variable x [ j ] ∈185 O is encoded by a Boolean variable that represents186 x [ j ] == v. Such an encoding converts a single multi-187 valued variable into multiple Boolean valued variables.1 We convert a Constraint Graph ( fig . 1 ) 188 with nodes representing multi-valued variables ( yellow nodes ) , into a Binary Graph ( fig . 1 ) with189 Boolean nodes ( blue nodes ) . This creates a |NC | × k grid of Boolean nodes , with a row representing190 a variable , a column representing a value and a grid cell ( a Boolean node ) representing assignment of191 a particular value to a particular variable . Such a graph can easily represent relationship between the192 values as well ( horizontal red edges ) , thereby encapsulating the information present in the Relation193 Graph ( fig . 1 ) . We use this Binary Graph for message passing.194 Construction of Message Passing Graph : We denote the Message Passing Graph ( MPG ) by195 G = ( N , E ) with the set of nodes N and set of edges E , constructed as follows : Nodes : For each196 node nCj ∈ NC in the Constraint Graph ( fig . 1 , yellow nodes ) , we construct k binary valued nodes,197 denoted as nj,1 , nj,2 · · ·nj , k in N ( blue nodes in Binary Graph ) . Edges : We construct two categories198 of edges in G. The first category of edges are directly inherited from the edges of the constraint199 graph GC ( black vertical edges ) , with k copies created due to binarization . Edge type is same as in200 the original constraint graph and is denoted by q . Formally , for every edge , eC ( j , j′ ) ∈ EC , where201 eC ( j , j′ ) .type = q , we introduce k edges denoted as e q ( jl , j′l ) , i.e. , there is an edge between every pair202 1There is an alternative encoding scheme called ‘ compact encoding ’ . It is discussed in the appendix of nodes , nj , l and nj′ , l , 1 ≤ l ≤ k. We refer to them as Constraint Edges . The second category of203 edges encode the information from the Relationship Graph GR into the MPG , with |NC | copies of it204 created , one for each variable . For every edge eR ( l , l′ ) ∈ ER with edge type r , create an edge er ( jl , jl′ ) 205 with type r between every pair of binary nodes nj , l and nj , l′ , 1 ≤ j ≤ |NC | ( e.g. , red edges encoding206 less-than relation between value pairs ( 1 , 2 ) , ( 2 , 3 ) and ( 1 , 3 ) ) . We refer to them as Relational Edges.207 Recurrent Message Passing : Once MPG has been constructed , we follow recurrent message208 passing rules , with weights shared across layers , similar to RRNs ( Palm et al. , 2018 ) with some209 differences . For each node nj , l in the graph , we maintain a hidden state ht ( nj , l ) , which is updated at210 each step t based on the messages received from its neighbors . This hidden state is used to compute211 the probability of a binary node taking a value of 1 . Since we use sparse encoding , only the node with212 maximum probability amongst the k binary nodes nj , l ; 1 ≤ l ≤ k , corresponding to multi-valued213 variable x [ j ] , is assigned a value 1 , at the end of message passing . We give the details of message214 passing and state update function in appendix . Next , we discuss how the nodes are initialized before215 message passing starts , followed by the details of loss computation.216 Initialization : Irrespective of the size of value-set V or verticesNC , there are 3 learnable embeddings217 ( u [ 0 ] , u [ 1 ] and u [ −1 ] ) for initialization : two for binary values 0 and 1 , and one for value −1 repre-218 senting unassigned nodes . All k nodes corresponding to an unassigned variable x [ j ] are initialized219 with u [ −1 ] , i.e. , whenever x [ j ] is NULL ( yellow nodes with −1 ) , u0 ( nj , l ) = u [ −1 ] , ∀vl ∈ V , where220 u0 represents initial embedding function . On the other hand , if x [ j ] is preassigned a value vl̂ , then221 u0 ( nj , l ) = u [ 0 ] , ∀vl 6= vl̂ , and u0 ( nj , ̂l ) = u [ 1 ] . E.g. , variable corresponding to the binary nodes in222 1st row has a preassigned value of ‘ 3 ’ , consequently , binary nodes in 1st and 2nd column of the 1st223 row are initialized with u [ 0 ] , and binary node in the 3rd column of 1st row , which corresponds to224 assignment ‘ x [ 1 ] = 3 ’ , is initialized with u [ 1 ] . Lastly , the hidden state , h0 ( nj , l ) , of each node , nj , l,225 is initialized as a 0 vector , ∀j , ∀vl.226 Loss Computation : The Binary Cross Entropy ( BCE ) loss for each node nj , l is computed w.r.t . its227 target , ỹ [ j , l ] , which is defined as 1 whenever y [ j ] = l and 0 otherwise . At each step t ∈ { 1 . . . T } ,228 we can compute the probability Pr ( nj , l.v = 1 ; Θ ) of classifying a node nj , l as 1 by passing its229 hidden state through a learnable scoring function s , i.e. , Prt ( nj , l.v = 1 ; Θ ) = σ ( s ( ht ( nj , l ) ) ) ,230 where σ is the standard Sigmoid function . Here , nj , l.v denotes the value that node nj , l can231 take and belongs to the set { 0 , 1 } . Loss at step t is the average BCE loss across all the nodes:232 1 |N | ∑ nj , l∈N ỹ [ j , l ] logPrt ( nj , l.v = 1 ; Θ ) + ( 1− ỹ [ j , l ] ) logPrt ( nj , l.v = 0 ; Θ ) . Like Palm et al.233 ( 2018 ) , we back-propagate through the loss at every step t ∈ { 1 . . . T } as it helps in learning a con-234 vergent message passing algorithm . During training , the objective is to learn the 3 initial embeddings235 u [ −1 ] , u [ 0 ] , u [ 1 ] , functions used in message passing and state update , and the scoring function s.236 Prediction on a problem with larger size of value-set : While testing , let the constraint and relation237 graph beGC′ andGR′ with n′ and k′ nodes respectively . Let x′ be a partial solution , with n′ variables238 x′ [ j ] , each taking a value from value-set V ′ of size k′ . As described above , we create a graph G′ with239 n′k′ nodes , run message passing for T steps , and for each variable x′ [ j ] , compute the k′ probabilities,240 one for each of the k′ nodes nj , l∀l ∈ V ′ corresponding to the variable x′ [ j ] , which is assigned the241 value corresponding to maximum probability , i.e. , ŷ [ j ] = arg maxl∈V′ PrT ( nj , l.v = 1 ; Θ ) .242 4.2 MULTI-VALUED MODEL243 Multi-valued model differs from the binarized model by avoiding binarization of nodes , and instead244 explicitly adding Value Nodes in the message passing graph , one for each value in the value-set.245 The message graph consists of two components : ( a ) A Graph G = ( N , E ) to represent constraints246 inherited from the constraint graph GC = ( NC , EC ) ( b ) A Graph G̃ = ( Ñ , Ẽ ) to represent relations247 inherited from the relationship graph GR = ( NR , ER ) . We refer to G as Constraint Message Passing248 Graph ( CMPG ) , and G̃ as Relationship Message Passing Graph ( RMPG ) . Message passing on249 RMPG first generates desired number of embeddings ( upto kmax ) , one for each of the value nodes.250 This is followed by message passing on CMPG which uses the embeddings of the value nodes251 generated by RMPG and computes embeddings for each variable node . Finally , the variable nodes252 are classified based on the similarity of their embedding with the embeddings of the value nodes253 computed by RMPG . Learning to generate upto kmax embeddings from training samples with only254 k ( < kmax ) values in the value-set is the main technical challenge that we address in this model.255 Construction of CMPG : Nodes : For each node nCj ∈ NC in the constraint graph , we construct256 a k-valued node , denoted as nj ∈ N . Total number of such nodes constructed is |NC | . We refer257 to these as Variable Nodes ( yellow nodes in Multi-Valued Graph in fig . 1 ) . Additionally , for each258 value vl ∈ V in the value-set , we create a node , denoted as nvl ∈ N . Total number of such nodes259 constructed is |V| . We refer to these as Value Nodes ( orange nodes ) . Edges : For every edge,260 eC ( j , j′ ) ∈ EC , where eC ( j , j′ ) .type = q , we introduce an edge denoted as eq ( j , j′ ) with type q. These261 edges are directly inherited from the constraint graph . We refer to these as Constraint Edges ( black262 edges ) . Additionally , to indicate the pre-assignment of values to the variables in x , we introduce new263 edges connecting value nodes to appropriate variable nodes . Whenever x [ j ] = vl , add an edge , ea ( j , l ) 264 between variable node nj and value node nvl ( blue edges ) . If x [ j ] is NULL , i.e. , unassigned , then add265 k edges , eā ( j , l ) , ∀vl ∈ V , connecting the variable node nj with all k value nodes n v l ( e.g. , green edges266 connecting orange value node ‘ 2 ’ to all ‘ -1 ’ variable nodes ) . We refer to them as Assignment Edges.267 Construction of RMPG : Nodes : For each value vl ∈ V , create a node denoted as ñvl ∈ Ñ ( purple268 nodes in Relation Graph in fig . 1 ) . Total number of such nodes constructed is |V| . We refer to these269 as Value Nodes . Edges : For every pair of value nodes , ñvl and ñvl′ , introduce an edge ẽ r ( l , l′ ) with type270 r if r ( vl , vl′ ) holds based on the relationship graph GR , i.e. , eR ( l , l′ ) ∈ ER with edge label r ( red271 edges ) . These edges are defined for relations that exist between values in the value-set.272 Achieving Value-set Invariance : A key question arises here : why do we need to construct a separate273 RMPG ( G̃ ) ? Why not embed relevant edges in CMPG ( G ) , as done for the binarized model ? The274 answer lies in realizing that we represent each value in the value-set explicitly in the multi-valued275 model , unlike the binarized model . Hence , our model needs to learn representation for each of them276 in the form of value node embeddings . Further , to generalize we need to learn as many embeddings277 as there are values in the largest test value-set , i.e. , kmax = max |V ′| . We achieve this by randomly278 sub-selecting a k-sized set from { 1 . . . kmax } and permuting the chosen subset for each training279 example in a given mini-batch , and then computing the ‘ relationship-aware ’ embeddings from this280 permuted subset through message passing in RMPG . The ‘ relationship-aware ’ embeddings are then281 used to initialize the value nodes ( orange nodes ) during message passing in CMPG . For instance,282 if the permutation obtained is { w1 , · · · , wl , · · · , wk } , where ∀l , 1 ≤ wl ≤ kmax , then embedding283 for the value node ñvl in G̃ is initialized by w th l learnable embedding ( e.g. , purple nodes for values284 ‘ 1 ’ , ‘ 2 ’ , and ‘ 3 ’ are initialized by the 5th , 7th , and 1st learnable embedding , respectively ) . After285 message passing on G̃ , the ‘ relationship-aware ’ embedding of ñvl ( purple node ) is used to initialize286 the embedding for value node nvl ( orange node ) in G. This elegant process is able to train all the kmax287 embeddings by simply using the training data corresponding to V , and the corresponding relationship288 information . Since these relationship aware embeddings need to be pre-computed before they can be289 passed to the downstream constraint processing , we construct two different message passing graphs,290 one for computing relationship-aware embeddings and one for constraint handling.291 Recurrent Message Passing on RMPG : Rules of message passing and hidden state updates at292 every step t are similar to RRN in Palm et al . ( 2018 ) and defined in detail in the appendix . After293 updating the hidden states for total T̃ steps , the final embeddings , h̃T̃ ( ñ v l ) ∀vl ∈ V , are used as294 ‘ relationship-aware ’ embeddings for initializing the input features ( embeddings ) of the nodes in295 CMPG G. We now discuss the initialization of the value nodes before message passing in RMPG.296 Initialization : There are a total of kmax learnable embeddings , ũ [ l′ ] , 1 ≤ l′ ≤ kmax , out of which any297 k are randomly chosen for initializing the nodes in RMPG . e.g. , ũ [ 5 ] , ũ [ 7 ] , ũ [ 1 ] are chosen to initialize298 the purple value nodes ‘ 1 ’ , ‘ 2 ’ , and ‘ 3 ’ in Relation Graph in fig . 1 . Formally , for each input x , select299 a k-permutation , Px , of kmax . Initialize the embedding of ñvl in G̃ with ũ [ Px [ l ] ] , ∀l ∈ { 1 . . . k } .300 Initialize the hidden state , h̃0 ( ñvl ) , ∀ñvl ∈ Ñ with a 0 vector.301 Recurrent Message Passing on CMPG : Message passing on CMPG updates the hidden state,302 ht ( nj ) , of each variable node nj for a total of T ( t ≤ T ) steps using the messages received from its303 neighbors . The details are similar to message passing in binarized model and are discussed in the304 appendix . Below we describe the initialization of node embeddings followed by computation of loss.305 Initialization : We initialize the embedding of value nodes ( orange nodes ) , nvl in G , using the306 final ‘ relationship-aware ’ embeddings , h̃T̃ ( ñ v l ) , of ñ v l ( purple nodes ) in G̃ . The variable nodes307 that are preassigned a value ( non-zero yellow nodes ) in x , are initialized by the embedding of308 the corresponding value node , i.e. , if x [ j ] = l , then nj is initialized with the ‘ relationship-aware ’ 309 embedding , h̃T̃ ( ñ v l ) , of ñ v l . The embedding of nodes corresponding to the unassigned variables310 ( ‘ -1 ’ yellow nodes ) are initialized by the average , ( 1/k ) ∑ vl∈V h̃T̃ ( ñ v l ) , of all ‘ relationship-aware ’ 311 embeddings . Initialize hidden state h0 ( nj ) of each variable node nj with a 0 vector.312 Loss Computation : For each variable represented by node nj , the ground truth value y [ j ] acts313 as the target for computing standard Cross Entropy Loss . The probabilities over V are computed314 as follows : At step t , a scoring function , s , computes a score , s ( ht ( nj ) , ht ( nvl ) ) , for assigning315 a value vl ∈ V to a variable nj based on the hidden state of corresponding value and variable316 nodes . For each variable node , a Softmax converts these scores into probabilities over the values317 vl ∈ V , i.e. , Pr ( nj.v = vl ) = Softmax ( s ( ht ( nj ) , ht ( nvl ) ) ) , ∀vl ∈ V , where , nj.v ∈ V denotes318 the value that node nj can take . Loss at step t is nothing but the average over variable nodes:319 Lt = − 1|N | ∑ nj∈N logPr ( nj.v = y [ j ] ) . To ensure that the multi-valued model learns different320 embeddings for each value in the value-set , we add an auxiliary loss term , corresponding to the total321 pairwise dot product ( similarity ) of any two embeddings , before and after message passing in G̃ . We322 call it Orthogonality Loss . Its weight , α , is a hyper-parameter.323 Prediction on a problem with larger size of value-set : For a puzzle with larger value-set , V ′ , a324 bigger RMPG is created , whose k′ nodes are initialized with the ( learnt ) first k′ embeddings . Unlike325 training , we always choose first k′ embeddings to avoid randomness during testing . Prediction is326 made using the probabilities at the last step T , i.e. , ŷ [ j ] = arg maxvl∈V′ Pr ( nj.v = vl ) .327 Relative Comparison : In the binarized model , the constructed graph G has k|NC | nodes and at328 least k|EC |+ |NC |k ( k − 1 ) /2 edges due to binarization . This increases the graph size by a factor of329 at least k. As a result , we soon hit the memory limits of a GPU while training the binarized model330 with bigger problems . The model also needs significantly more inference time due to its bigger size.331 On the other hand , multi-valued model , while being compact in terms of its representation , needs to332 learn additional embeddings , for a speculative size of value-set during testing . This poses additional333 requirement on the model both in terms of representation , and learning , possibly affecting the quality334 of generalization . While this is a simple analytical understanding of the possible merits of the two335 models , we examine experimentally the impact of these issues on real datasets.336 5 EXPERIMENTS337 The goal of our experiments is to evaluate the effectiveness of our two proposed methods for achieving338 value-set invariance . We compare our models with a generic neural constraint learner , NLM ( Dong339 et al. , 2019 ) .2 We experiment on datasets generated from Lifted CSPs of three different puzzles , viz.,340 Sudoku , Futoshiki , and Graph Coloring ( ref . Table 5 in appendix for details ) . We train each model on341 data generated from a fixed value-set , and test on instances generated from larger value-sets.342 5.1 TASK DESCRIPTION AND DATASETS343 Futoshiki : This is a number puzzle in which we have to place numbers { 1 . . . k } on a k × k grid,344 such that no two cells in a row or column contain the same number . In addition , there may be an345 2Our aim is not to directly compete with SOTA SAT solvers , which are much more scalable than neural methods . Refer to appendix for a discussion on comparison with them as well as neural SAT solvers . ordering constraint between two cells , which needs to be honored in the final solution . The input has346 some of the grid cells already filled with a number and the task is to complete the grid , respecting the347 additional ordering constraint where ever it exists . We train our model on 6 × 6 puzzles , with the348 percentage of missing cells varying uniformly between 28− 70 % . We test our models on puzzles349 with board size ranging between 6 × 6 to 12 × 12 , with the same percentage of missing cells.350 Graph Coloring ( GCP ) : In this task351 we are given a partially colored graph352 along with the number of colors k,353 and the objective is to color rest of354 the nodes using k colors such that355 no two adjacent nodes have the same356 color . We train our model on ran-357 domly generated 4−colorable graphs,358 and test on k′−colorable graphs , with359 k′ ∈ { 4 , 5 , 6 , 7 } . Training data has360 graphs with graph order varying uni-361 formly between 40−120 , and percent-362 age of masked nodes vary uniformly between 28− 70 % .363 Sudoku : We randomly select 10 , 000 training queries from the 9 × 9 dataset introduced in Palm364 et al . ( 2018 ) . Our test set has k′ × k′ puzzles , with k′ ∈ { 10 , 12 , 15 , 16 } . Data generation process is365 similar to Futoshiki , with the distribution of missing cells varying between 30− 68 % depending on366 the board size . Instead of backtracking , solution validity is checked through the GSS library ( Pieters,367 2019 ) . Please see appendix for more details on data generation process for all three tasks.368 5.2 EXPERIMENTAL SETUP & BASELINES369 In both our models , nodes370 are initialized with learn-371 able 96 dimensional em-372 beddings . In multi-valued373 model , kmax = 12 , 7 , and374 16 embeddings are learnt375 for Futoshiki , GCP , and Su-376 doku respectively . Message377 passing on G in binarized378 model runs for 32 steps.379 Message passing on RMPG , G̃ and CMPG , G in the multi-valued model runs for T̃ = 1 and380 T = 32 steps respectively . The message passing functions in both the models are 3 layer MLPs,381 similar to those in RRN , with a difference that there is a separate function for each edge type . In both382 the models , a layer normalized LSTM cell with hidden dimension 96 acts as state update functions.383 All models are trained on K40 GPU nodes with 12GB memory . We take simple average of model384 weights stored at multiple points ( Izmailov et al. , 2018 ) . All checkpoints obtained after flattening of385 the learning curve are selected for computing the average . See appendix for more details.386 Baseline : For Futoshiki , we train two versions of NLM by varying depth : the number of Logic387 Machines that are stacked on top of each other . Like ( Nandwani et al. , 2021 ) , we train one 30 layer388 deep NLM model with residual connections for Futoshiki , but unlike them , we assume access to389 constraint graph , which we provide as a binary predicate input to the model . NLM with 30 depth390 could not fit puzzles with board-size greater than 10 within 12GB memory of K40 GPU . Hence , we391 train another version by reducing the depth to 15 . For GCP , we train a model with depth 24 . For392 Sudoku , on increasing depth beyond 14 , we could not fit even one 9× 9 train sample within GPU393 memory . Note that the maximum depth chosen for the graph experiments reported in ( Dong et al.,394 2019 ) is 8 . This is because they work with much smaller graphs ( up to maximum 50 nodes ) , whereas395 smallest graph in Futoshiki has 63 = 216 binary nodes , warranting creation of much deeper models.396 Evaluation Metrics : We report two metrics : board accuracy and point-wise accuracy . In the former,397 we consider output of the model as correct only if it satisfies the underlying CSP , whereas in the later,398 we give partial credit even for assigning some of the variables correctly . See Appendix for details.399 For each setting , we report the mean and standard deviation over three runs by varying random seed.400 5.3 RESULTS AND DISCUSSION401 We report the accuracies over differ-402 ent sizes of value-set for Futoshiki,403 GCP and Sudoku in Table 1 , Table 2,404 and Table 3 , respectively . We first ob-405 serve that NLM fails to train on Su-406 doku , and its performance is worse407 than one or both of our models for all408 experimental settings in Futoshiki and409 GCP . As expected , in Futoshiki , NLM410 model with depth 30 fails to run on board sizes 11 and 12 and depth 15 model fails to run on size411 12 . Note that both NLM and our binarized model work by binarizing the underlying puzzle , but we412 observe that binarized model shows significantly better generalization across value-sets . We note that413 NLM performs decently well for GCP even for the test graphs with chromatic number k′ = 7 . We414 attribute this to the fact that in our test data for k′ = 7 , graphs are relatively small , with maximum415 80 graph nodes , resulting in total 560 binary objects in NLM , which is similar to the maximum 400416 binary objects that it trains over ( k=4 , max 100 nodes ) .417 Comparison between binarized model and multi-valued model : We first observe that both our418 models achieve similar performance on the value-set over which they are trained . We observe419 that the standard deviation of the board accuracy increases significantly as the size of value-set420 increases , whereas the pointwise accuracy is relatively stable . This is due to the high sensitivity of421 the board accuracy to pointwise accuracy : even if a single variable is incorrectly assigned in a puzzle,422 its contribution towards board accuracy goes to 0 , whereas it still contributes positively towards423 pointwise accuracy . When trained on small sizes , binarized model shows better generalization . But424 as the problem size increases , the computational graph for binarized model fails to fit in the available425 GPU memory and thus its performance degrades . On the other hand , multi-valued model being426 memory efficient , scales much better . To demonstrate this , Table 4 reports the performance of multi-427 valued model further finetuned on sudoku puzzles of board-size 24 , and tested on board-sizes varying428 between 15 and 25 . We couldn ’ t finetune the binarized model as its computational graph doesn ’ t fit in429 the GPU . The binarized model trained on puzzles of board-size 9 gives 0.0 board accuracy on size 24430 and 25 . The performance of multi-valued model is better than binarized model not only on board-size431 25 , but also on board-sizes smaller than 24 . This also demonstrates that the poor performance of the432 same multi-valued model trained on smaller board-size is not due to any lack of representation power,433 but due to difficulty in learning additional embeddings : when training k′ embeddings from puzzles434 of board-size k , multi-valued model never gets to see all k′ value embeddings together . Moreover,435 the different combinations of k out of k′ embeddings increase exponentially with ( k′ − k ) , making436 it further difficult to train . To validate this , we train a multi-valued model with only 7 learnable437 embeddings for Futoshiki and observe that the board accuracy on 7 board-size increases to 97.82 % 438 ( at par with binarized model ) from 90.18 % which is achieved when trained with 12 embeddings.439 Computational complexity : In fig . 2 , for the two models , we com-440 pare the average inference time and the GPU memory occupied by441 a batch of 32 Futoshiki puzzles over value-sets of varying sizes . As442 expected , the multi-valued model is much more efficient , both in443 terms of time and memory.444 6 CONCLUSION AND FUTURE WORK445 We have looked at the novel problem of value-set invariance in com-446 binatorial puzzles , formally defined using lifted CSPs and proposed447 two different neural solutions , extending an existing architecture448 ( RRN ) . Our experimental evaluation demonstrates the superior per-449 formance of our models compared to an existing neural baseline . We450 discuss the relative strengths and weaknesses of our proposed mod-451 els . Directions for future work include solving puzzles generated452 from more complicated CSPs , and scaling to even larger sizes.453 ETHICS STATEMENT454 In its current form , our work is primarily a technical contribution , with no immediate ethical455 consequences . Our work develops the line of recent research in which constraint reasoning is carried456 out through neural architectures . We believe that neural approaches for symbolic reasoning will go a457 long way in creating an integrated AI system . This is because an integrated system requires not only458 perceptual , but also high-level reasoning . Neural approaches will provide a uniform vocabulary so459 that both these forms of reasoning can interact with each other , improving performance of the overall460 system.461 As more AI systems start to be used in critical applications such as healthcare , law , and disaster462 management , it is important that they honor the safety and accountability constraints set up by domain463 experts . Their ability to perform high-level reasoning enables them to honor such constraints more464 effectively . Thus , our line of work , in the long run , could have significant positive ethical implications.465 We see no obvious negative implications of our work.466 REPRODUCIBILITY STATEMENT467 To ensure reproducibility , we have discussed the dataset creation process and provided model468 architecture details in Section 5.1 and Section 5.2 , respectively . We provide the details of the exact469 hyper-parameters , computational resources used , and additional experimental details in the appendix.470 We will make all the data , and source code for training and data generation , publicly available on471 paper ’ s acceptance.472 REFERENCES473 Saeed Amizadeh , Sergiy Matusevych , and Markus Weimer . Learning to solve circuit-sat : An474 unsupervised differentiable approach . In 7th International Conference on Learning Representations,475 ICLR 2019 , New Orleans , LA , USA , May 6-9 , 2019 . OpenReview.net , 2019a . URL https:476 //openreview.net/forum ? id=BJxgz2R9t7.477 Saeed Amizadeh , Sergiy Matusevych , and Markus Weimer . PDP : A general neural framework for478 learning constraint satisfaction solvers . CoRR , abs/1903.01969 , 2019b . URL http : //arxiv.479 org/abs/1903.01969.480 Dzmitry Bahdanau , Kyunghyun Cho , and Yoshua Bengio . Neural machine translation by jointly481 learning to align and translate . In 3rd International Conference on Learning Representations,482 ICLR 2015 , San Diego , CA , USA , May 7-9 , 2015 , Conference Track Proceedings , 2015 . URL483 http : //arxiv.org/abs/1409.0473.484 Aniket Bajpai , Sankalp Garg , and Mausam . Transfer of deep reactive policies for MDP planning . In485 Advances in Neural Information Processing Systems 31 : Annual Conference on Neural Information486 Processing Systems 2018 , NeurIPS 2018 , December 3-8 , 2018 , Montréal , Canada , pp . 10988–487 10998 , 2018.488 Beatrice Bevilacqua , Yangze Zhou , and Bruno Ribeiro . Size-invariant graph representations for graph489 classification extrapolations . In Proceedings of the 38th International Conference on Machine490 Learning , ICML 2021 , 18-24 July 2021 , Virtual Event , volume 139 , pp . 837–851 , 2021 . URL491 http : //proceedings.mlr.press/v139/bevilacqua21a.html.492 Nuri Cingillioglu and Alessandra Russo . Deeplogic : Towards end-to-end differentiable logical493 reasoning . In Proceedings of the AAAI 2019 Spring Symposium on Combining Machine Learning494 with Knowledge Engineering ( AAAI-MAKE 2019 ) Stanford University , Palo Alto , California , USA,495 March 25-27 , 2019. , Stanford University , Palo Alto , California , USA , March 25-27 , 2019 , volume496 2350 of CEUR Workshop Proceedings . CEUR-WS.org , 2019 . URL http : //ceur-ws.org/497 Vol-2350/paper21.pdf.498 Johan de Kleer . A comparison of ATMS and CSP techniques . In Proceedings of the 11th International499 Joint Conference on Artificial Intelligence . Detroit , MI , USA , August 1989 , pp . 290–296 , 1989.500 URL http : //ijcai.org/Proceedings/89-1/Papers/046.pdf.501 Honghua Dong , Jiayuan Mao , Tian Lin , Chong Wang , Lihong Li , and Denny Zhou . Neural logic502 machines . In 7th International Conference on Learning Representations , ICLR 2019 , New Orleans,503 LA , USA , May 6-9 , 2019 . OpenReview.net , 2019 . URL https : //openreview.net/forum ? 504 id=B1xY-hRctX.505 Michael D. Ernst , Todd D. Millstein , and Daniel S. Weld . Automatic sat-compilation of planning506 problems . In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence,507 IJCAI 97 , Nagoya , Japan , August 23-29 , 1997 , 2 Volumes , pp . 1169–1177 , 1997 . URL http:508 //ijcai.org/Proceedings/97-2/Papers/055.pdf.509 Richard Evans and Edward Grefenstette . Learning explanatory rules from noisy data . J. Artif . Intell.510 Res. , 61:1–64 , 2018. doi : 10.1613/jair.5714 . URL https : //doi.org/10.1613/jair.511 5714.512 Nir Friedman , Lise Getoor , Daphne Koller , and Avi Pfeffer . Learning probabilistic relational models.513 In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence , IJCAI 99,514 Stockholm , Sweden , July 31 - August 6 , 1999 . 2 Volumes , 1450 pages , pp . 1300–1309 , 1999 . URL515 http : //ijcai.org/Proceedings/99-2/Papers/090.pdf.516 Sankalp Garg , Aniket Bajpai , and Mausam . Symbolic network : Generalized neural policies for517 relational mdps . In Proceedings of the 37th International Conference on Machine Learning , ICML518 2020 , 13-18 July 2020 , Virtual Event , volume 119 of Proceedings of Machine Learning Research,519 pp . 3397–3407 . PMLR , 2020.520 Kazuo Iwama and Shuichi Miyazaki . Sat-variable complexity of hard combinatorial problems . In In521 Proceedings of the World Computer Congress of the IFIP , pp . 253–258 , 1994.522 Pavel Izmailov , Dmitrii Podoprikhin , Timur Garipov , Dmitry P. Vetrov , and Andrew Gordon Wil-523 son . Averaging weights leads to wider optima and better generalization . In Proceedings of the524 Thirty-Fourth Conference on Uncertainty in Artificial Intelligence , UAI 2018 , Monterey , Cali-525 fornia , USA , August 6-10 , 2018 , pp . 876–885 , 2018 . URL http : //auai.org/uai2018/526 proceedings/papers/313.pdf.527 David Joslin and Amitabha Roy . Exploiting symmetries in lifted csps . In The 14th National528 Conference on Artificial Intelligence ( AAAI ) , pp . 197–202 . AAAI Press , 1997.529 Daniel Kahneman . Thinking , fast and slow . Macmillan , 2011.530 Nada Lavrac and Luc De Raedt . Inductive logic programming : A survey of european research.531 AI Commun. , 8 ( 1 ) :3–19 , 1995. doi : 10.3233/AIC-1995-8101 . URL https : //doi.org/10.532 3233/AIC-1995-8101.533 Robin Manhaeve , Sebastijan Dumancic , Angelika Kimmig , Thomas Demeester , and Luc De534 Raedt . Deepproblog : Neural probabilistic logic programming . In Advances in Neu-535 ral Information Processing Systems 31 : Annual Conference on Neural Information Pro-536 cessing Systems 2018 , NeurIPS 2018 , December 3-8 , 2018 , Montréal , Canada , pp.537 3753–3763 , 2018 . URL https : //proceedings.neurips.cc/paper/2018/hash/538 dc5d637ed5e62c36ecb73b654b05ba2a-Abstract.html.539 Mehrab Najafian , Mohammad Hesam Tadayon , and Morteza Esmaeili . Construction of strongly540 mutually distinct sudoku tables and solid sudoku cubes by cyclotomic cosets . IEEE Transactions541 on Games , PP:1–1 , 11 2018. doi : 10.1109/TG.2018.2880953.542 Yatin Nandwani , Deepanshu Jindal , Mausam , and Parag Singla . Neural learning of one-of-many543 solutions for combinatorial problems in structured output spaces . In International Conference on544 Learning Representations ( ICLR ) , 2021.545 Rasmus Berg Palm , Ulrich Paquet , and Ole Winther . Recurrent relational networks . In Advances in546 Neural Information Processing Systems 31 : Annual Conference on Neural Information Processing547 Systems 2018 , NeurIPS 2018 , 3-8 December 2018 , Montréal , Canada , pp . 3372–3382 , 2018 . URL548 http : //papers.nips.cc/paper/7597-recurrent-relational-networks.549 Laurent Perron and Vincent Furnon . Or-tools , 2019 . URL https : //developers.google.550 com/optimization/.551 Bart Pieters . Generic sudoku solver . https : //github.com/bartp5/gss , 2019.552 Tim Rocktäschel , Sameer Singh , and Sebastian Riedel . Injecting logical background knowledge553 into embeddings for relation extraction . In NAACL HLT 2015 , The 2015 Conference of the554 North American Chapter of the Association for Computational Linguistics : Human Language555 Technologies , Denver , Colorado , USA , May 31 - June 5 , 2015 , pp . 1119–1129 , 2015 . URL556 http : //aclweb.org/anthology/N/N15/N15-1118.pdf.557 Daniel Selsam , Matthew Lamm , Benedikt Bünz , Percy Liang , Leonardo de Moura , and David L. Dill.558 Learning a SAT solver from single-bit supervision . In 7th International Conference on Learning559 Representations , ICLR 2019 , New Orleans , LA , USA , May 6-9 , 2019 . OpenReview.net , 2019 . URL560 https : //openreview.net/forum ? id=HJMC_iA5tm.561 Aviv Tamar , Yi Wu , Garrett Thomas , Sergey Levine , and Pieter Abbeel . Value iteration networks . In562 Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence , IJCAI563 2017 , Melbourne , Australia , August 19-25 , 2017 , pp . 4949–4953 , 2017.564 Hao Tang , Zhiao Huang , Jiayuan Gu , Bao-Liang Lu , and Hao Su . Towards565 scale-invariant graph-related problem solving by iterative homogeneous gnns . In566 NeurIPS , 2020 . URL https : //proceedings.neurips.cc/paper/2020/hash/567 b64a70760bb75e3ecfd1ad86d8f10c88-Abstract.html.568 Toby Walsh . SAT v CSP . In Principles and Practice of Constraint Programming - CP 2000 , 6th569 International Conference , Singapore , September 18-21 , 2000 , Proceedings , volume 1894 of Lecture570 Notes in Computer Science , pp . 441–456 . Springer , 2000. doi : 10.1007/3-540-45349-0\_32 . URL571 https : //doi.org/10.1007/3-540-45349-0_32.572 Po-Wei Wang , Priya L. Donti , Bryan Wilder , and J. Zico Kolter . Satnet : Bridging deep learning and573 logical reasoning using a differentiable satisfiability solver . In Proceedings of the 36th International574 Conference on Machine Learning , ICML 2019 , 9-15 June 2019 , Long Beach , California , USA,575 volume 97 of Proceedings of Machine Learning Research , pp . 6545–6554 . PMLR , 2019 . URL576 http : //proceedings.mlr.press/v97/wang19e.html.577 Gilad Yehudai , Ethan Fetaya , Eli A. Meirom , Gal Chechik , and Haggai Maron . From local structures578 to size generalization in graph neural networks . In Proceedings of the 38th International Conference579 on Machine Learning , ICML 2021 , 18-24 July 2021 , Virtual Event , volume 139 , pp . 11975–11986,580 2021 . URL http : //proceedings.mlr.press/v139/yehudai21a.html.581 Jie Zhou , Ganqu Cui , Shengding Hu , Zhengyan Zhang , Cheng Yang , Zhiyuan Liu , Lifeng Wang,582 Changcheng Li , and Maosong Sun . Graph neural networks : A review of methods and applications.583 AI Open , 1:57–81 , 2020.584 A APPENDIX585 2 RELATED WORKS586 Generalization of GNNs across graph size : Our work relies heavily on the assumption that GNNs587 generalize across size . Here we briefly discuss the works that question the same . The existing set of588 papers ( and results ) in this line of research can be broadly divided into two sub-classes . The first set589 of results talk about the representation power of GNNs to handle various graph sizes . The second set590 of results talk about learnability issues with GNNs under varying train/test distributions . We look at591 some of the results below and try to explain why GNNs in our case are able to generalize well , both592 in terms of representation power , as well as learnability.593 Representation Power : We hypothesize that there are two design choices that are helping us gain594 good representation power : 1 . Ability to create a deep network without blowing up the number of595 parameters because of weight tying across layers , and 2 . Preassigned class labels to some of the596 variables which act as node features and help in breaking the symmetry . We argue it on the basis of597 Theorem 4.2 in Yehudai et al . ( 2021 ) , which proves that there exists a ( d+ 3 ) layered GNN that can598 distinguish between nodes having different local structure , which is quantified via d-patterns that599 can be thought of as a generalization of node degree to d-hop neighborhood . Hence , to be able to600 distinguish between nodes on the basis of a GNN , all we need to do is ensure that different nodes601 have different d-patterns . This can be achieved in 2 ways : 1 . By increasing d , e.g . two nodes may602 have the same degree and hence the same 1-pattern , but their neighbors may have different degrees,603 which will lead to different 2-pattern for these two nodes . 2 . By assigning node features , e.g . two604 nodes may have the same degree , but their neighbors may have different node features , leading to605 a different 1-pattern for them as d-pattern also takes initial node features into account . In addition,606 Tang et al . ( 2020 ) also argue that one way of increasing the representation power of GNNs is by607 increasing their depth , and it achieves the same by proposing IterGNN that applies the same GNN608 layer for an adaptive number of iterations depending on the input graph . This is equivalent to tying609 the weights of different layers as in RRNs , as well as in our models.610 Learnability : With respect to learnability , Yehudai et al . ( 2021 ) prove the existence of a ‘ bad ’ local611 minima that overfits on train data but fails on test samples that have unseen d-patterns . Our test612 dataset clearly has unseen d-patterns ( e.g . nodes in 16 x 16 sudoku have different degrees than nodes613 in 9 x 9 sudoku ) , but our models still generalize . We note that Yehudai et al . ( 2021 ) only talks about614 the existence of some bad local minima , but does not rule out the possibility of the existence of other615 good local minima , which could generalize well , despite differences in local structure between train616 and test sets . This goes into the whole learnability argument , and whether we can find such not-so-bad617 local minimas ( which presumably exist since the possibility has not been ruled out ) . One aspect that618 possibly comes to our rescue is that , unlike most GNN architectures , our design is recurrent in nature,619 i.e. , parameters are tied across different GNN layers as inspired by Palm et al . ( 2018 ) . Parameter620 tying assumption , possibly helps us in learnability , since the recurrence can be seen as a form of621 regularization , avoiding overfitting ( or getting stuck in bad local minima ) . Exploring this further is a622 direction for future work.623 In addition to Yehudai et al . ( 2021 ) , Bevilacqua et al . ( 2021 ) deal with varying train/test distributions624 by proposing a size invariant representation of graphs . Their approach focuses on graph classification625 tasks , and is limited to creating size invariant representations for the entire graph . The theoretical626 claims presented in their paper primarily focus on the limitation of standard GNN based formulations627 for generalizing across sizes for graph classification tasks . On the other hand , we are interested in628 learning representations for each node in the graph for node classification , and it is not clear how the629 claims , as well as techniques proposed in the paper , extend to our setting.630 4 MODELS DESCRIPTION631 4.1 BINARIZED MODEL632 Recurrent Message Passing633 There are two categories of edges in the Message Passing Graph : Constraint Edges and Relation634 Edges . Each edge inherits an edge type , either from Constraint Graph , or Relation Graph . We denote635 the set of all constraint edge types as Q , and the set of all relational edge types as R. We now describe636 the details of message passing and hidden state update equations.637 Edge Dependent Message Passing : The nodes communicate their current hidden state via the638 messages sent to their neighbouring nodes across the edges . The message depends not only on639 the current state of the sender and receiver , but also on the edge type across which the mes-640 sage is sent . Specifically , for each edge type , z , there is a separate message passing func-641 tion , fz , with z ∈ ( Q ∪ R ) where Q and R are the set of all constraint edge types and re-642 lation edge types respectively . We compute the message for each edge ez ( j1l1 , j2l2 ) ∈ E as:643 mt [ ez ( j1l1 , j2l2 ) ] = fz ( ht ( nj1 , l1 ) , ht ( nj2 , l2 ) ) , ∀ez ( j1l1 , j2l2 ) ∈ E , z ∈ ( Q ∪R ) .644 Hidden State Update : For each node , the incoming messages on the edges of the same type645 are aggregated by taking their weighted average . The weights , at , are computed using Bahdanau646 Attention ( Bahdanau et al. , 2015 ) over constraint edges , whereas messages across relation edges are647 simply averaged : mt , z [ nj , l1 ] = ∑ ez ( jl1 , j2l2 ) ∈E at [ e z ( jl1 , j2l2 ) ] mt [ e z ( jl1 , j2l2 ) ] , ∀z ∈ ( Q ∪R ) 648 Finally , all messages , mt , z [ nj , l ] ∀z ∈ ( Q ∪R ) , are concatenated to create the input , mt [ nj , l ] for each649 node , nj , l . The hidden state at step t is updated by the following state update function to generate650 the state ht+1 ( nj , l ) : ht+1 ( nj , l ) = g ( ht ( nj , l ) , mt [ nj , l ] , u0 ( nj , l ) ) , ∀nj , l ∈ N . See Figure 3 for an651 illustration of edge dependent message passing and state update at a given step t.652 4.2 MULTI-VALUED MODEL653 There are two separate message passing graphs in multi-valued model : RMPG and CMPG . RMPG654 contains edges encoding the relationship between the values . Each edge has an associated edge type655 representing the relationship it encodes . We denote the set of all edge types in RMPG as R. In656 CMPG , there are two categories of edges : Constraint Edges and Assignment Edges . Further , each657 edge may have an associated edge type . The set of all constraint edge types is denoted as Q , and the658 set of assignment edge types ( edges from orange value nodes to yellow variable nodes in fig . 1 ) is659 denoted as A . Finally , the initial embedding of a variable node nj is denoted as u0 ( nj ) .660 We now describe the message passing rules and hidden state update equations.661 Recurrent Message Passing ( on RMPG ) 662 Message Passing Update : At step t , update the hidden state , h̃t ( ñvl ) , of each of the value nodes in663 G̃ , by the concatenation , mt [ ñvl ] , of average messages , m r t [ ñ v l ] , received across edges of type r ∈ R:664 h̃t+1 ( ñ v l ) = g̃ ( h̃t ( ñ v l ) , mt [ ñ v l ] , ũ [ Px [ l ] ] ) , where g̃ is the hidden state update function . Like ( Palm665 et al. , 2018 ) , it always takes the initial embedding , ũ [ Px [ l ] ] , of the value node ñvl as one of the inputs.666 Notice that the message , mrt [ ñ v l ] , is the average of the messages , fr ( h̃t ( ñ v l ) , h̃t ( ñ v l′ ) ) ∀ ẽ r ( l , l′ ) ∈ Ẽ,667 where fr is the message passing function for edge type r ∈ R. The hidden states are updated for668 T̃ steps and the final embeddings , h̃T̃ ( ñ v l ) ∀vl ∈ V , are used as ‘ relationship-aware ’ embeddings669 for initializing the input features ( embeddings ) of both variable nodes , nj , and value nodes , nvl in G670 ( orange and yellow nodes respectively in Multi-Valued Graph in fig . 1 ) .671 Recurrent Message Passing ( on CMPG ) 672 Message Passing Update : At step t , similar to binarized model , each variable node receives673 messages from its neighbors , that are aggregated based on the edge type . For each node , the674 aggregated messages , mt , z [ nj ] , for different edge types , z ∈ ( Q ∪A ) , are stacked to create , mt [ nj ] ,675 which updates the hidden state as : ht+1 ( nj ) = g ( ht ( nj ) , mt [ nj ] , u0 ( nj ) ) , ∀nj ∈ N .676 677 678 Discussion on an alternate Encoding Scheme679 As discussed in section 4.1 , the main intuition for our binarized modelcomes from ‘ sparse encoding ’ 680 of an integer CSP to a SAT . In addition to ‘ sparse encoding ’ , there is another way of converting681 integer CSP into SAT , called ‘ compact encoding ’ ( Ernst et al. , 1997 ; Iwama & Miyazaki , 1994 ) , in682 which each Boolean SAT variable represents a single bit of the integer value that a CSP variable can683 take . The final assignment of a CSP variable is given by the integer represented by the log k Boolean684 SAT variables corresponding to that variable . Motivated by the ‘ compact encoding ’ , one can construct685 another model : instead of a one-hot encoding which requires k nodes ( one for each value in V ) for686 each variable , create log k binary valued nodes for each variable and assign a value v ∈ V to the687 variable based on the integer represented by log k bits corresponding to it . This results in a graph with688 |NC | log k nodes for a CSP with |NC | variables and k classes , instead of |NC |k nodes in the binarized689 model , and brings it closer to the graph size of |NC |+ k created in multi-valued model . However,690 such an approach failed to generalize across the size of the value-set in our experiments . In addition,691 such an encoding has a limitation in its representational capability . It can not encode the relationship692 between the values effectively . For example , in k × k Futoshiki , we have an ordinal relationship693 between the k values representing the numerical numbers 1 to k. In our proposed approaches , we694 encode this by adding appropriate relational edges between nodes representing different values in V .695 In the binarized model , it is done for each variable separately , whereas , in the multi-valued model , it696 is done in the RMPG . In absence of an explicit node for a value in this encoding scheme , it is not697 clear how to represent such a relationship.698 5 EXPERIMENTS699 Discussion on comparison with SAT Solvers : In this work , we are interested in creating ( and700 learning ) a neural solver for symbolic tasks , instead of using a symbolic algorithm like SAT solver.701 Such an approach has many benefits , e.g. , because of being differentiable , it can be used seamlessly702 in a unified framework requiring both perception as well as reasoning , e.g . visual sudoku ; neural703 models have been shown to be resistant to varying amount of noise in the data as well , which purely704 logical ( SAT style ) solvers may not be able to handle . As is the case with other papers in this line of705 work e.g ( Selsam et al. , 2019 ; Amizadeh et al. , 2019a ) , at this point our main motivation is scientific.706 We are interested in understanding to what extent neural reasoners can generalize across varying707 sizes of the value-set in train and test domains . Instead of comparing with an industrial SAT Solver,708 perhaps a fair comparison would be with a generic state-of-the-art neural SAT solver e.g. , CircuitSAT709 ( Amizadeh et al. , 2019a ) , NeuroSAT ( Selsam et al. , 2019 ) . Both of these papers observe that there is a710 long way to go before they can compete with industrial SAT solvers . In fact , both of these approaches711 experiment with much smaller problem instances . CircuitSAT uses a model trained on k-SAT-3-10712 problems ( k-SAT with 3 to 10 Boolean variables ) for coloring graphs with number of nodes ranging713 between 6 to 10 , and achieves a meager 27 % performance and NeuroSAT fails to solve any of the714 problems in the coloring dataset used by CircuitSAT ( section 5.2 in Amizadeh et al . ( 2019a ) ) . On715 the other hand , the smallest of the problems in our dataset has 40 nodes ( GCP ) and the largest has716 253 ( = 25 , 625 ) nodes ( in 25 × 25 Sudoku ) , and hence we do not expect the generic neural SAT717 solvers to scale up to our problem sizes.718 5.1 TASK DESCRIPTION AND DATASETS719 We experiment on datasets generated from Lifted CSPs of three different puzzles , viz. , Sudoku,720 Futoshiki , and Graph Coloring . In addition , we fine-tune our multi-valued model for Sudoku on 8000721 puzzles of size 24 and test it on puzzles of different board sizes . Table 5 contains the details of both722 train and test data for the different experiments . Below we describe the three tasks and their datasets723 in detail.724 Futoshiki : We train our model on 6 × 6 puzzles , with the percentage of missing cells varying725 uniformly between 28− 70 % . We test our models on puzzles with board size ranging between 6× 6726 to 12× 12 , with the same percentage of missing cells . The number of ordering constraints is twice727 the board size . To generate data , we first randomly generate a completely filled k × k board and then728 randomly mask m % of the cells . We search for its all possible completions using backtracking to729 ensure that it has only one solution . Finally , we insert ordering constraints between 2k randomly730 picked pairs of adjacent cells . The entire process is repeated by varying k and m to generate both the731 test and train dataset.732 The training data consists of 12 , 300 puzzles on 6 x 6 board size with the percentage of missing733 variables varying between 28− 70 % . The exact details of the testing data for different board sizes734 are provided in Table 6 . We note that it becomes increasingly difficult to find puzzles with unique735 solution as the board size increases . Therefore , we are forced to reduce the maximum percentage of736 masked ( unassigned ) cells with increasing board size.737 GCP : The training data for Graph Coloring Problem consists of around 25 thousand 4-colorable738 graphs with graph order varying uniformly between 40 − 120 , and the percentage of unassigned739 ( masked ) variables varying uniformly between 28− 70 % depending on the graph order . The exact740 details of the testing data for different chromatic number ( value-set size ) are provided in Table 6.741 To create non-trivial problems for the dataset , we always attempt to color graphs with the smallest742 possible number of colors , i.e. , the chromatic number of the graph . We follow Erdős–Rényi ( ER ) 743 model to generate random graphs . It takes number of nodes , n , and an edge probability , p , as input,744 and adds an edge independent of the other edges with probability p. We note that to sample a graph745 with n nodes and a given chromatic number k , we need to carefully adjust the range from which edge746 probability p is sampled . The exact range from which p is sampled uniformly for each chromatic747 number k and a range of nodes n is given in Table 7 . We use a CSP solver ( Perron & Furnon , 2019 ) 748 to determine the chromatic number of a given graph , which becomes a bottleneck while generating749 graphs with higher chromatic number . As a result , we were not able to generate graphs with more750 than 80 nodes for chromatic number 7 , in a reasonable amount of time.751 Sudoku : The training data consists of 10 thousand 9 x 9 puzzles randomly selected from the dataset752 introduced in ( Palm et al. , 2018 ) . For standard 9 × 9 board , we use the same test data as used in753 RRN ( Palm et al. , 2018 ) 3 . The test data for the board-sizes between 10 and 16 is generated using754 the methodology similar to Futoshiki . Instead of backtracking , solution validity and uniqueness is755 checked through the GSS library ( Pieters , 2019 ) . The exact details of the testing data for different756 board sizes are provided in Table 6 . For the experiment where we fine-tune our models on 24× 24757 puzzles , both the train and test data for board size 24 and 25 are generated following the methodology758 similar to Futoshiki . In this setup , we were not able to verify the uniqueness of solution through GSS759 library as it doesn ’ t scale to such large sizes.760 Solution Multiplicity in GCP Dataset and larger board-size puzzles of Sudoku : An input query761 in GCP may have more than one solution , out of which only one is given at train time . But the762 network may discover a new valid solution , and computing loss of the discovered solution w.r.t . the763 given solution may unnecessarily penalize the model . To avoid this , we algorithmically verify if a764 prediction is a valid coloring or not , and compute loss w.r.t . to this discovered solution , instead of the765 given one . This is equivalent to solving a One-of-Many Learning problem ( Nandwani et al. , 2021 ) 766 with all possible colorings given at training time . The same phenomenon of solution multiplicity767 exists for Sudoku puzzles of size 24 and 25 , as verifying the uniqueness of puzzles on such large768 board-size became computationally infeasible.769 3Available at : https : //data.dgl.ai/dataset/sudoku-hard.zip 5.2 EXPERIMENTAL SETUP AND BASELINES770 Evaluation Metrics : We report two metrics : board accuracy and point-wise accuracy for all our771 experiments . In the former , we consider output of the model as correct only if it satisfies the772 underlying CSP , whereas in the later , we give partial credit even for assigning some of the variables773 correctly . We formally define it below:774 Pointwise accuracy : Let Yx be the set of possible solutions for an input x with k variables , and let ŷ775 be the model ’ s prediction . Pointwise accuracy of the prediction ŷ with respect to the solution y ∈ Yx,776 denoted as , PointAcc ( y , ŷ ) , is defined to be the fraction of variables that match between the y and777 ŷ : PointAcc ( y , ŷ ) = 1k ∑k i=1 1 { y [ i ] == ŷ [ i ] } , where 1 { . } is the indicator function.778 Given above , we define pointwise accuracy for a prediction ŷ of an input x with respect to a solution779 set Yx to be the maximum among the pointwise accuracy with respect to each of the solutions in the780 set Yx . Mathematically , PointAcc ( Yx , ŷ ) = maxy∈YxPointAcc ( y , ŷ ) .781 For Sudoku and Futoshiki , since there is a unique solution , we can easily compute pointwise accuracy782 as the target set Yx is singleton . For the GCP task , whenever the model returns a valid coloring,783 pointwise accuracy is 1 , otherwise , in the absence of access to complete Yx , we report a lower bound784 by performing a local search , using Google ’ s OR-Tools 4 , for a valid coloring closest to the model785 prediction . Same is true for sudoku puzzles on 24 and 25 board size.786 Why care about point-wise accuracy ? In our settings , the generalization problem can be hard,787 especially when there is a large difference between the sizes of the value-sets for the train and test788 domains . Given that we are defining a novel task , and it is important to measure progress even when789 problems are hard , we compare the two models using a simpler metric ( pointwise accuracy ) as well,790 in addition to board accuracy . This additional metric can help us detect progress , and also compare791 the relative performance of underlying models.792 Computational resources : All models are trained on K40 GPUs with 12 GB memory , available on793 an HPC cluster.794 Hyperparameters : The list below enumerates the various hyperparameters with a brief description795 and the values chosen.796 1 . Batch Size : For each task , we selected the maximum batch size that can be accommodated797 in 12GB GPU memory . Refer to Table 8 for details.798 2 . Optimizer : To minimize the loss , we use Adam optimizer with learning rate 0.0002 . As in799 the original RRN paper , we chose a weight decay factor of 1E-4.800 4https : //developers.google.com/optimization 3 . Orthogonality Loss Factor : To ensure that the multi-valued model learns different em-801 beddings for each value in the value-set , we add an auxiliary loss term , corresponding to802 the total pairwise dot product of any two embeddings , before and after message passing803 on the Relation Message Passing Graph ( RMPG ) , G̃ . Its weight , α , was chosen amongst804 { 0.01 , 0.1 , 0.5 } by cross validating on a devset for Futoshiki , and then fixed afterwards for805 all our experiments.806 4 . Edge Dropout : While collating the messages from the edges of the same type , we drop807 10 % of the messages , as done in RRN . Dropout is used in the Message Passing Graph808 ( MPG ) of the binarized model , and the Constraint Message Passing Graph ( CMPG ) of the809 multi-valued model.810 Model Averaging : As suggested in ( Izmailov et al. , 2018 ) , to reduce the variance of our model811 performance , we take simple average of model weights stored at multiple points . All checkpoints812 beyond a point when the learning curve flattened are selected for computing the average.813 Training Time : Table 9 enumerates the exact training cost , in terms of total training epochs , number814 of gradient updates , and clock time , for all three tasks and for both our models as well as the baseline815 NLM model . Note that while a multi-valued model may have fewer parameters , and results in a816 much smaller graph and inference time for a given problem , its training time could still be higher,817 especially in terms of total training epochs and number of gradient updates . However , because of its818 memory efficiency , we can keep a much larger batch size during training , and because of its speed819 efficiency , each update is much faster . As a result , the overall clock time comes out to be comparable820 to the binary model for the two of our tasks , i.e . Futoshiki and Sudoku , and it is within 2x for GCP,821 even though the number of epochs is much higher.822

Summary: This paper intends to develop a learning framework (based on the recurrent relational neural network (RRN) ) that can handle output-space invariance issue.  Output-space invariance means that the model trained on smaller size data (like 9 by 9 Sudoku) can be generalized to larger size data (16 by 16) without any further modification. The entire problem formulation is done by lifted CSP.   Based on RRN, the authors propose two methods: The binarized method and the Multi-value method.   For the binarized method: the output value space is binarized, i.e., the model will score each candidate individually and the parameters are shared.  For the multi-value model, the value nodes are added to the architecture and the nodes connected to value k mean that the node can be assigned value k or it is already assigned value k.  A decent method is proposed to help the model learn a meaningful representation for all the possible values.