Combining Learned Representations for Combinatorial Optimization

1 INTRODUCTION . The Ising Problem has long been known to be in the class of NP-Hard problems , with no exact polynomial solution existing . Because of this , a large class of combinatorial optimization problems can be reformulated as Ising problems and solved by finding the ground state of that system ( Barahona , 1982 ; Kirkpatrick et al. , 1983 ; Lucas , 2014 ) . The Boltzmann Machine ( Ackley et al. , 1987 ) was originally introduced as a constraint satisfaction network based on the Ising model problem , where the weights would encode some global constraints , and stochastic units were used to escape local minima . The original Boltzmann Machine found favor as a method to solve various combinatorial optimization problems ( Korst & Aarts , 1989 ) . However , learning was very slow with this model due to the difficulties with sampling and convergence , as well as the inability to exactly calculate the partition function . More recently , the Restricted Boltzmann Machine ( RBM ) has experienced a resurgence as a generative model that is able to fully approximate a probability distribution over binary variables due to its ease of computation and training via the contrastive divergence method ( Hinton , 2002 ) . The success of the RBM as a generative model has been limited due to the difficulties in running the Markov chain Monte Carlo ( MCMC ) algorithm to convergence ( Tieleman , 2008 ; Tieleman & Hinton , 2009 ) . In this work , we propose a generative model composed of multiple learned modules that is able to solve a larger problem than the individually trained parts . This allows us to circumvent the problem of training large modules ( which is equivalent to solving the optimization problem in the first place , as we are simply providing the correct answers to the module as training data ) , thus minimizing training time . As RBMs have the ability to fill in partial values and solutions , this approach is very flexible to the broad class of combinatorial optimization problems which can be composed of individual atomic operations or parts . Most notably , we show that our approach of using “ invertible boolean logic ” is a method of solving the boolean satisfiability problem , which can be mapped to a large class of combinatorial optimization problems directly ( Cook , 1971 ; Karp , 1972 ) . The ability of RBMs to fully model a probability distribution ensures model convergence and gives ideas about the shape of the underlying distribution . Many other generative models , such as Generative Adverserial Neural Networks ( GANs ) , ( Goodfellow et al. , 2014 ) Generative Stochastic Networks ( Alain et al. , 2015 ) and others do not explicitly model the probability distribution in question , but rather train a generative machine to draw samples from the desired distribution . Although these can be useful for modeling high dimensional data , they do not provide the same guarantees that RBMs do . Because we use an RBM as our generative model we can perform a full Bayesian analysis , and condition on any subset of the variables to solve a variety of problems . This leads to increased model flexibility , and the ability to generalize the learned models further . 2 RELATED WORK . People have shown that learned and trained features have the ability to outperform hand calculated features in a variety of tasks , from image classification to speech detection and many others . In addition , other network architectures , such as Recurrent Neural Networks ( RNNs ) have been shown to be Turing complete , and posses the ability to be used as a conventional Turing machine ( Graves et al. , 2014 ; Zaremba & Sutskever , 2014 ) . However , these architectures have mostly been used as generalized computers , rather than to solve specific problems . Deterministic , feedforward neural networks have also been used to solve factorization problems , but their approach does not solve the factors for as high bit numbers as presented here , and is not flexible enough to be used in problems outside of prime factorization ( Jansen & Nakayama , 2005 ) . In addition , these models are not generative or reversible . Many attempts have also been made to use quantum computers to solve such Ising model based problems . Large scale realizations of quantum computers , however , are still far from completion ( Whitfield et al. , 2012 ; Lucas , 2014 ; Roland & Cerf , 2002 ; Xu et al. , 2012 ) . Therefore , methods than can exploit a classical hardware are of critical importance . In this regard , one recent work has proposed the use of “ p-bits ” , to realize a form of Boltzmann Machine ( Camsari et al. , 2017 ) . The work by ( Traversa & Di Ventra , 2017 ) similarly tries to create an “ invertible boolean logic ” , but does so in a deterministic manner , rather than the probabilistic one presented in this work . Our approach falls within the broad category of transfer learning and compositional learning . There have been other works on using combinations of RBMs for object recognition and computer vision tasks ( Ranzato et al. , 2010 ) , and on combinations of RBMs with weight sharing for collaborative filtering ( Salakhutdinov et al. , 2007 ) . Nonetheless , the specific method presented here , as described in the following sections , to the best of our knowledge has not been used before . 3 APPROACH An RBM is a binary stochastic neural network , which can be understood as a Markov random field of binary variables divided into a bipartite graph structure , where all of the visible units are conditionally independent of each other given the hidden unit states , and all hidden units are conditionally independent given the visible unit states . Here we denote v as the visible state , h as the hidden states , and E ( v , h ) as the energy associated with those states . The probability assigned to a given state p ( v , h ) = 1Z e −E ( v , h ) whereZ = ∑ v , h e −E ( v , h ) is the normalizing constant of the distribution . The general approach to solving these combinatorial optimization problems is to recognize the atomic unit necessary to solve the problem , and use this as a base trained unit ( this is discussed further later in the approach section ) . We combine these units by performing a merge operation of different RBMs . If we know two different units share a connection ( such as the output of one logical unit being the input of another , or two cities being identical in a TSP ) , then these units would be merged to create a larger unit ( see Figure 1 and explanation with weight matrices and biases below ) . The probabilities and energies of the two RBMs are dictated by their weight matrices , WA ∈ Rn×r and WB ∈ Rm×s , visible biases bA ∈ Rn and bB ∈ Rm , and hidden biases aA ∈ Rr and aB ∈ Rs . The energies and probabilities of these are as follows : EA ( v , h ) = −vTWAh− aTAh− bTAv ; pA ( v , h ) = 1 ZA e−EA ( v , h ) EB ( v , h ) = −vTWBh− aTBh− bTBv ; pB ( v , h ) = 1 ZB e−EB ( v , h ) ( 1 ) We can write the weight matrices as a series of row vectors corresponding to one visible units connections to a set of hidden units . WA = w A 1 ... wAn , WB = w B 1 ... wBm , ( 2 ) With this definition , the merge operation is shown below . If units k of RBM A is merged with unit l of RBM B the associated weight matrix WA+B ∈ R ( n+m−1 ) × ( r+s ) , visible bias vA+B ∈ Rn+m−1 and hidden bias hA+B ∈ Rr+s dictate the probabilities and energies for the merged RBM . Merging multiple units between these two RBMs corresponds to moving multiple row vectors from WB to WA , which creates the associated decrease in dimensionality of WA+B and bA+B ( where WA+B ∈ R ( n+m−d ) × ( r+s ) and vA+B ∈ Rn+m−d where d is the number of merged units . WA+B = WA 0 wBl 0 0 WB\l = wA1 ... wAk ... wAn 0 wBl 0 0 wB1 ... wBl−1 wBl+1 ... wBm bA+B = bA1 ... bAk + b B l ... bAn bB1 ... bBl−1 bBl+1 ... bBm aA+B = aA1 ... aAr aB1 ... aBs ( 3 ) Below , we show how this relates to the original energies and probabilities . The vectors v and h corresponds to the visible vector put into the combined RBM , while vA , vB , hA and hB correspond to the equivalent state vectors that would be inputted into the single RBMs . Using these equations , we can see that the combined RBM energy factorizes into a sum of the original RBM energies and the probability is the product of the original probabilities . v = v1 ... vn+m−1 h = h1 ... hr+s vA = v1 ... vl ... vn , vB = vn+1 ... vn+l−1 vl vn+l ... vn+m−1 , hA = h1 ... hr hB = hr+1 ... hr+s ( 4 ) EA+B ( v , h ) = EA ( vA , hA ) + EB ( vB , hB ) , ( 5 ) pA+B ( v , h ) = 1 ZA+B e−EA+B ( v , h ) ∝ pA ( vA , hA ) pB ( vB , hB ) ( 6 ) An alternative way of viewing the merged probability distribution is to multiply the distributions of the original RBMs , and marginalize over all states where the merged units are different . Because of the probabilities approximately multiplying , we can also say that if each of the distributions differed from the “ ideal ” distribution ( denoted here by q ) , then we can expect the error ( as measured by the KL divergence ) to increase approximately linearly with the number of connections . This shows that the error should increase with the number of connected units , rather than directly with the number of bits . This is especially important with combinatorial optimization problems where the deterministic algorithm run time increases exponentially with the number of bits . However , we note that as the number of connections between the units increases , the KL divergence increasingly diverges from linear . DKL ( q‖p ) ≈ DKL ( qA‖pA ) +DKL ( qB‖pB ) ; p = pA ( vA , hA ) pB ( vB , hB ) , q = qA ( vA , hA ) qB ( vB , hB ) Many combinatorial optimization problems can be broken down into associated sub-problems , and solved using a greedy approach ( i.e . using the nearest neighbor approach in the Travelling Salesman Problem , or multiplying single digits in a larger multiplication , or evaluating one Boolean logic statement in a Boolean satisfiability problem ) . Using a greedy approach ( such as used in the travelling salesman problem with nearest neighbors ) can produce non-optimal results , and evaluating all permutations of possible solutions to find the most optimal can be computationally intractable in a large problem space . By combining these sub-problems using the method proposed here , we bypass the problems associated with those two approaches . We encode possible solutions as a probability indicating its local optimality , and combine these sub problems by merging the visible units of their RBMs as shown . The approach is that each sub-problem has overlapping units that share the same value , allowing them to be combined . This combination mechanism multiplies the probabilities such that the solution with global optimality is encoded as the mode of the distribution modeled by the larger RBM . As the combination method shown here still keeps the less optimal local solutions as a possible part of a global solution , we effectively bypass the problem of locally optimal solutions not solving the globally optimal problem . In addition , as the phase space of the problem space is 2v where v is the number of visible units , we can also encode a large problem space using minimal units and decrease the amount of computation . In this work we show a usage of this structure to solve a variant of the Boolean satisfiability problem by combining multiple trained logic gates , and use this to perform integer factorization and other arithmetic tasks . As the Boolean satisfiability problem can be mapped to solving a variety of NP-Hard problems ( Cook , 1971 ; Karp , 1972 ) , we believe variants of this approach can be used to solve other tasks . As an example , a usage for this approach to solve the Travelling Salesman Problem ( TSP ) might be to have an atomic unit that is the one step ( or few ) transitions between cities , and the global problem solved by combining these one ( or few ) step transition RBMs to create a full tour . Solving a given optimization problem is equivalent to sampling from the model distribution of the RBM and finding the mode of that distribution . The RBM is partially clamped to values given in the problem statement , and samples are generated via MCMC as explained and examined below .

Summary: The paper introduces a new approach to combine small RBMs that are pretrained in order to obtain a large RBM with good performance. This will bypass the need of training large RBMs and suggests to break them into smaller ones. The paper then provides experimental evidence by applying the method on "invertible boolean logic". MCMC is used to find the the solution to large RBM and compare it against the combined solutions of smaller RBMs.