Combining Learned Representations for Combinatorial Optimization

1 INTRODUCTION . The Ising Problem has long been known to be in the class of NP-Hard problems , with no exact polynomial solution existing . Because of this , a large class of combinatorial optimization problems can be reformulated as Ising problems and solved by finding the ground state of that system ( Barahona , 1982 ; Kirkpatrick et al. , 1983 ; Lucas , 2014 ) . The Boltzmann Machine ( Ackley et al. , 1987 ) was originally introduced as a constraint satisfaction network based on the Ising model problem , where the weights would encode some global constraints , and stochastic units were used to escape local minima . The original Boltzmann Machine found favor as a method to solve various combinatorial optimization problems ( Korst & Aarts , 1989 ) . However , learning was very slow with this model due to the difficulties with sampling and convergence , as well as the inability to exactly calculate the partition function . More recently , the Restricted Boltzmann Machine ( RBM ) has experienced a resurgence as a generative model that is able to fully approximate a probability distribution over binary variables due to its ease of computation and training via the contrastive divergence method ( Hinton , 2002 ) . The success of the RBM as a generative model has been limited due to the difficulties in running the Markov chain Monte Carlo ( MCMC ) algorithm to convergence ( Tieleman , 2008 ; Tieleman & Hinton , 2009 ) . In this work , we propose a generative model composed of multiple learned modules that is able to solve a larger problem than the individually trained parts . This allows us to circumvent the problem of training large modules ( which is equivalent to solving the optimization problem in the first place , as we are simply providing the correct answers to the module as training data ) , thus minimizing training time . As RBMs have the ability to fill in partial values and solutions , this approach is very flexible to the broad class of combinatorial optimization problems which can be composed of individual atomic operations or parts . Most notably , we show that our approach of using “ invertible boolean logic ” is a method of solving the boolean satisfiability problem , which can be mapped to a large class of combinatorial optimization problems directly ( Cook , 1971 ; Karp , 1972 ) . The ability of RBMs to fully model a probability distribution ensures model convergence and gives ideas about the shape of the underlying distribution . Many other generative models , such as Generative Adverserial Neural Networks ( GANs ) , ( Goodfellow et al. , 2014 ) Generative Stochastic Networks ( Alain et al. , 2015 ) and others do not explicitly model the probability distribution in question , but rather train a generative machine to draw samples from the desired distribution . Although these can be useful for modeling high dimensional data , they do not provide the same guarantees that RBMs do . Because we use an RBM as our generative model we can perform a full Bayesian analysis , and condition on any subset of the variables to solve a variety of problems . This leads to increased model flexibility , and the ability to generalize the learned models further . 2 RELATED WORK . People have shown that learned and trained features have the ability to outperform hand calculated features in a variety of tasks , from image classification to speech detection and many others . In addition , other network architectures , such as Recurrent Neural Networks ( RNNs ) have been shown to be Turing complete , and posses the ability to be used as a conventional Turing machine ( Graves et al. , 2014 ; Zaremba & Sutskever , 2014 ) . However , these architectures have mostly been used as generalized computers , rather than to solve specific problems . Deterministic , feedforward neural networks have also been used to solve factorization problems , but their approach does not solve the factors for as high bit numbers as presented here , and is not flexible enough to be used in problems outside of prime factorization ( Jansen & Nakayama , 2005 ) . In addition , these models are not generative or reversible . Many attempts have also been made to use quantum computers to solve such Ising model based problems . Large scale realizations of quantum computers , however , are still far from completion ( Whitfield et al. , 2012 ; Lucas , 2014 ; Roland & Cerf , 2002 ; Xu et al. , 2012 ) . Therefore , methods than can exploit a classical hardware are of critical importance . In this regard , one recent work has proposed the use of “ p-bits ” , to realize a form of Boltzmann Machine ( Camsari et al. , 2017 ) . The work by ( Traversa & Di Ventra , 2017 ) similarly tries to create an “ invertible boolean logic ” , but does so in a deterministic manner , rather than the probabilistic one presented in this work . Our approach falls within the broad category of transfer learning and compositional learning . There have been other works on using combinations of RBMs for object recognition and computer vision tasks ( Ranzato et al. , 2010 ) , and on combinations of RBMs with weight sharing for collaborative filtering ( Salakhutdinov et al. , 2007 ) . Nonetheless , the specific method presented here , as described in the following sections , to the best of our knowledge has not been used before . 3 APPROACH An RBM is a binary stochastic neural network , which can be understood as a Markov random field of binary variables divided into a bipartite graph structure , where all of the visible units are conditionally independent of each other given the hidden unit states , and all hidden units are conditionally independent given the visible unit states . Here we denote v as the visible state , h as the hidden states , and E ( v , h ) as the energy associated with those states . The probability assigned to a given state p ( v , h ) = 1Z e −E ( v , h ) whereZ = ∑ v , h e −E ( v , h ) is the normalizing constant of the distribution . The general approach to solving these combinatorial optimization problems is to recognize the atomic unit necessary to solve the problem , and use this as a base trained unit ( this is discussed further later in the approach section ) . We combine these units by performing a merge operation of different RBMs . If we know two different units share a connection ( such as the output of one logical unit being the input of another , or two cities being identical in a TSP ) , then these units would be merged to create a larger unit ( see Figure 1 and explanation with weight matrices and biases below ) . The probabilities and energies of the two RBMs are dictated by their weight matrices , WA ∈ Rn×r and WB ∈ Rm×s , visible biases bA ∈ Rn and bB ∈ Rm , and hidden biases aA ∈ Rr and aB ∈ Rs . The energies and probabilities of these are as follows : EA ( v , h ) = −vTWAh− aTAh− bTAv ; pA ( v , h ) = 1 ZA e−EA ( v , h ) EB ( v , h ) = −vTWBh− aTBh− bTBv ; pB ( v , h ) = 1 ZB e−EB ( v , h ) ( 1 ) We can write the weight matrices as a series of row vectors corresponding to one visible units connections to a set of hidden units . WA = w A 1 ... wAn , WB = w B 1 ... wBm , ( 2 ) With this definition , the merge operation is shown below . If units k of RBM A is merged with unit l of RBM B the associated weight matrix WA+B ∈ R ( n+m−1 ) × ( r+s ) , visible bias vA+B ∈ Rn+m−1 and hidden bias hA+B ∈ Rr+s dictate the probabilities and energies for the merged RBM . Merging multiple units between these two RBMs corresponds to moving multiple row vectors from WB to WA , which creates the associated decrease in dimensionality of WA+B and bA+B ( where WA+B ∈ R ( n+m−d ) × ( r+s ) and vA+B ∈ Rn+m−d where d is the number of merged units . WA+B = WA 0 wBl 0 0 WB\l = wA1 ... wAk ... wAn 0 wBl 0 0 wB1 ... wBl−1 wBl+1 ... wBm bA+B = bA1 ... bAk + b B l ... bAn bB1 ... bBl−1 bBl+1 ... bBm aA+B = aA1 ... aAr aB1 ... aBs ( 3 ) Below , we show how this relates to the original energies and probabilities . The vectors v and h corresponds to the visible vector put into the combined RBM , while vA , vB , hA and hB correspond to the equivalent state vectors that would be inputted into the single RBMs . Using these equations , we can see that the combined RBM energy factorizes into a sum of the original RBM energies and the probability is the product of the original probabilities . v = v1 ... vn+m−1 h = h1 ... hr+s vA = v1 ... vl ... vn , vB = vn+1 ... vn+l−1 vl vn+l ... vn+m−1 , hA = h1 ... hr hB = hr+1 ... hr+s ( 4 ) EA+B ( v , h ) = EA ( vA , hA ) + EB ( vB , hB ) , ( 5 ) pA+B ( v , h ) = 1 ZA+B e−EA+B ( v , h ) ∝ pA ( vA , hA ) pB ( vB , hB ) ( 6 ) An alternative way of viewing the merged probability distribution is to multiply the distributions of the original RBMs , and marginalize over all states where the merged units are different . Because of the probabilities approximately multiplying , we can also say that if each of the distributions differed from the “ ideal ” distribution ( denoted here by q ) , then we can expect the error ( as measured by the KL divergence ) to increase approximately linearly with the number of connections . This shows that the error should increase with the number of connected units , rather than directly with the number of bits . This is especially important with combinatorial optimization problems where the deterministic algorithm run time increases exponentially with the number of bits . However , we note that as the number of connections between the units increases , the KL divergence increasingly diverges from linear . DKL ( q‖p ) ≈ DKL ( qA‖pA ) +DKL ( qB‖pB ) ; p = pA ( vA , hA ) pB ( vB , hB ) , q = qA ( vA , hA ) qB ( vB , hB ) Many combinatorial optimization problems can be broken down into associated sub-problems , and solved using a greedy approach ( i.e . using the nearest neighbor approach in the Travelling Salesman Problem , or multiplying single digits in a larger multiplication , or evaluating one Boolean logic statement in a Boolean satisfiability problem ) . Using a greedy approach ( such as used in the travelling salesman problem with nearest neighbors ) can produce non-optimal results , and evaluating all permutations of possible solutions to find the most optimal can be computationally intractable in a large problem space . By combining these sub-problems using the method proposed here , we bypass the problems associated with those two approaches . We encode possible solutions as a probability indicating its local optimality , and combine these sub problems by merging the visible units of their RBMs as shown . The approach is that each sub-problem has overlapping units that share the same value , allowing them to be combined . This combination mechanism multiplies the probabilities such that the solution with global optimality is encoded as the mode of the distribution modeled by the larger RBM . As the combination method shown here still keeps the less optimal local solutions as a possible part of a global solution , we effectively bypass the problem of locally optimal solutions not solving the globally optimal problem . In addition , as the phase space of the problem space is 2v where v is the number of visible units , we can also encode a large problem space using minimal units and decrease the amount of computation . In this work we show a usage of this structure to solve a variant of the Boolean satisfiability problem by combining multiple trained logic gates , and use this to perform integer factorization and other arithmetic tasks . As the Boolean satisfiability problem can be mapped to solving a variety of NP-Hard problems ( Cook , 1971 ; Karp , 1972 ) , we believe variants of this approach can be used to solve other tasks . As an example , a usage for this approach to solve the Travelling Salesman Problem ( TSP ) might be to have an atomic unit that is the one step ( or few ) transitions between cities , and the global problem solved by combining these one ( or few ) step transition RBMs to create a full tour . Solving a given optimization problem is equivalent to sampling from the model distribution of the RBM and finding the mode of that distribution . The RBM is partially clamped to values given in the problem statement , and samples are generated via MCMC as explained and examined below .

Summary: The paper proposes learning Restricted Boltzmann Machines for solving small computational tasks (e.g., 1-bit addition) and composing those RBMs to form a more complex computational module (e.g., 16-bit addition). The claim is that such an approach can be more data efficient than learning a single network to directly learn the more complex module. Results are shown for addition and factoring tasks.