Matrix encoding networks for neural combinatorial optimization

1 Introduction . Many combinatorial optimization ( CO ) problems of industrial importance are NP-hard , leaving them intractable to solve optimally at a large scale . Fortunately , researchers in operations research ( OR ) have developed ways to tackle these NP-hard problems in practice , mixed integer programming ( MIP ) and meta-heuristics being two of the most general and popular approaches . With the rapid progress in deep learning techniques over the last several years , a new approach based on machine learning ( ML ) has emerged . ML has been applied in both ways successfully [ 1 ] , as a helper for the traditional OR methods aforementioned or as an independent CO problem solver trained in an end-to-end fashion . One way to leverage ML for solving CO problems is to employ a “ front-end ” neural net , which directly takes in the data specifying each problem . The neural net plays a critical role of analyzing all input data as a whole , from which it extracts useful global information . In an end-to-end ML-based approach , such global information may be encoded onto the representations of the entities making up the problem . Greedy selection strategies based on these representations are then no longer too near-sighted , allowing globally ( near- ) optimal solutions to be found quickly . Existing OR methods can also benefit from incorporating the global information . Many hybrid approaches already exist that leverage information extracted by a neural net , both in ML-MIP forms [ 2 , 3 ] and ML-heuritic forms [ 4 , 5 , 6 ] . The literature on neural combinatorial optimization contains many different types of the front-end models , conforming to the variety of data types upon which CO problems are defined . Yet , there has been no research for a model that encodes matrix-type data . This puts a serious limitation on the range of CO problems that an ML engineer can engage . Take , for example , the traveling salesman problem ( TSP ) , the most intensely studied topic by the research community of neural combinatorial 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . optimization . Once we lift the Euclidean distance restriction ( the very first step towards making the problem more realistic ) , the problem instantly becomes a formidable challenge that has never been tackled before because it requires a neural net that can process the distance matrix.1 The list of other classical CO problems based on data matrices includes job-shop/flow-shop scheduling problems and linear/quadratic assignment problems , just to name a few . All of these examples are fundamental CO problems with critical industrial applications . To put it in general terms , imagine a CO problem made up of two different classes of items , Ã = { ã1 , . . . , ãM } and B̃ = { b̃1 , . . . , b̃N } , where M and N are the sizes of Ã and B̃ respectively . We are interested in the type where features of each item are defined by its relationships with those in the other class . The data matrix we want to encode , D ∈ RM×N , would be given by the problem,2 where its ( i , j ) th element represents a quantitative relationship of some sort between a pair of items ( ãi , b̃j ) . The sets Ã and B̃ are unordered lists , meaning that the orderings of the rows and the columns of D are arbitrary . Such permutation invariance built into our data matrices is what sets them apart from other types of matrices representing stacked vector-lists or a 2D image . In this paper , we propose Matrix Encoding Network ( MatNet ) that computes good representations for all items in Ã and B̃ within which the matrix D containing the relationship information is encoded . To demonstrate its performance , we have implemented MatNet as the front-end models for two end-to-end reinforcement learning ( RL ) algorithms that solve the asymmetric traveling salesman ( ATSP ) and the flexible flow shop ( FFSP ) problems . These classical CO problems have not been solved using deep neural networks . From our experiments , we have confirmed that MatNet achieves near-optimal solutions . Especially , for the specific FFSP instances we have investigated , our MatNet-based approach performs substantially better than the conventional methods used in operations research including mixed integer programming and meta-heuristics . 2 Related work . A neural net can accommodate variable-length inputs as is needed for encoding the parameters of a CO problem . Vinyals et al . [ 9 ] , one of the earliest neural CO approaches , have introduced PtrNets that use a recurrent neural network ( RNN ) [ 10 ] as the front-end model . Points on the plane are arranged into a sequence , and RNN processes the Cartesian coordinates of each point one by one . Bello et al . [ 11 ] continue the RNN approach , enhancing the Ptr-Net method through combination with RL . Nazari et al . [ 12 ] further improve this approach by discarding the RNN encoder , but keeping the RNN decoder to handle the variable input size . The graph neural network ( GNN ) [ 13 , 14 ] is another important class of encoding networks used for neural CO , particularly on ( but not limited to ) graph problems . GNN learns the message passing policy between nodes that can be applied to an arbitrary graph with any number of nodes . Khalil et al . [ 7 ] are among the firsts to show that a GNN-based framework can solve many different types of CO problems on graph in a uniform fashion . Li et al . [ 15 ] extend the work by switching to a more advanced structure of graph convolutional networks ( GCNs ) [ 16 ] . Manchanda et al . [ 17 ] demonstrate efficient use of GCN that can solve CO problems on graphs with billions of edges . Karalias and Loukas [ 18 ] use unsupervised learning on GNN to improve the quality of the CO solutions . Encoding networks most relevant to our work are those based on the encoder of the Transformer [ 19 ] . Note that this encoder structure can be viewed as a graph attention network ( GAT ) [ 20 ] operating on fully connected graphs . Using this type of encoder , Kool et al . [ 21 ] encode Cartesian coordinates of all nodes given by the problem simultaneously and solve the TSP and many other related routing problems . Kwon et al . [ 8 ] use the same encoder as Kool et al . but produce solutions of significantly improved quality with the use of the RL training and the inference algorithms that are more suitable for combinatorial optimizations . 1One could , in principle , consider a general GNN approach such as Khalil et al . [ 7 ] that can encode any arbitrary graphs . However , as can be seen from the comparison of the TSP results of Khalil et al . ( > 8 % error on random 100-city instances ) with some of the state-of-the-art ML results ( e.g. , Kwon et al . [ 8 ] , < 0.2 % error ) , a general GNN is most suitable for problems that deal with diverse graph structures , not those with fixed , dense graph structures ( such as matrices ) . 2The relationship can have multiple ( f ) features , in which case we are given f number of different data matrices D1 , D2 , . . . , Df . For simplicity we only use f=1 in this paper , but our model covers f > 1 cases as well . ( See Appendix A.1 ) . MatNet architecture is designed to encode a bipartite graph , as will be explained in detail in the following section . Although none support embedding of a single matrix like MatNet , many interesting deep learning works exist that rely on bipartite graph embeddings or matrix embeddings . Gasse et al . [ 22 ] learn effective branch-and-bound variable selection policies for solving MIP instances , where constraints and variables of an MIP instance form a bipartite graph weighted by constraint coefficients . Gibbons et al . [ 23 ] solve weapon-target assignment problem , a classic CO problem that is presented with a bipartite graph . Duetting et al . [ 24 ] find good auction policies given a matrix that describes the value that each bidder assigns to each auctioning item . 3 Model architecture MatNet can be considered a particular type of GNN that operates on a complete bipartite graph with weighted edges . The graph has two sets of nodes , A = { a1 , . . . , aM } and B = { b1 , . . . , bN } , and the edge between ai and bj has the weight e ( ai , bj ) ≡ Dij , the ( i , j ) -th element of D , as illustrated in Figure 1 . MatNet is inspired by the encoder architecture of the Attention Model ( AM ) by Kool et al . [ 21 ] , but because the AM deals with a quite different type of a graph ( fully connected nodes and no edge weights ) , we have made significant modifications to it . The encoder of the AM follows the node-embedding framework of graph attention networks ( GATs ) [ 20 ] , where a model is constructed by stacking multiple ( L ) graph attentional layers . Within each layer , a node v ’ s vector representation ĥv is updated to ĥ′v using aggregated representations of its neighbors as in ĥ′v = F ( ĥv , { ĥw | w ∈ Nv } ) . ( 3.1 ) Here , Nv is the set of all neighboring nodes of v. The ( learnable ) update function F is composed of multiple attention heads , and their aggregation process utilizes the attention score for each pair of nodes ( v , w ) , which is a function of ĥv and ĥw . MatNet also generally follows the GATs framework , but it differs in two major points : ( 1 ) It has two independent update functions that separately apply to nodes in A and B . ( 2 ) The attention score for a pair of nodes ( ai , bj ) is not just a function of ĥai and ĥbj , but the edge weight e ( ai , bj ) as well . The update function in each layer of MatNet can be described as ĥ′ai = FA ( ĥai , { ( ĥbj , e ( ai , bj ) ) | bj ∈ B } ) for all ai ∈ A , ĥ′bj = FB ( ĥbj , { ( ĥai , e ( ai , bj ) ) | ai ∈ A } ) for all bj ∈ B . ( 3.2 ) 3.1 Dual graph attentional layer . The use of two update functions FA and FB in Eq . ( 3.2 ) is somewhat unorthodox . The GATs framework requires that the update rule F be applied uniformly to all nodes because it loses its universality otherwise . In our case , however , we focus on a particular type of a problem , where the items in Ã and B̃ belong to two qualitatively different classes . Having two separate functions allows MatNet to develop a customized representation strategy for items in each class . The two update functions are visualized as a dual structure of a graph attentional layer depicted in Figure 2 ( a ) . The MatNet architecture is a stack of L graph attentional layers , and each layer is made of two sub-blocks that implement FA and FB . The two sub-blocks are structurally identical , where each one is similar to the encoder layer of the AM , which in turn is based on that of the Transformer model [ 19 ] . Note , however , that while both the AM and the Transformer model use the simple self-attention for encoding , MatNet requires a clear separation of queries ( nodes in one set ) and the key-value pairs ( nodes in the other set ) at the input stage to perform cross-attentions . For detailed descriptions of computing blocks used in Figure 2 ( a ) , we refer the readers to the Transformer architecture [ 19 ] .

Summary: The paper introduces a neural net model that takes relationship matrices as input for solving combinatorial optimization problems. The proposed network can be considered as a special GNN fed by a complete bipartite graph with weighted edges. The experiment results show that their end-to-end RL framework can achieve a similar performance as LKH3 for asymmetric traveling salesman problems and significantly outperform conventional OR methods for flexible flow shop problems.