Learning a Latent Search Space for Routing Problems using Variational Autoencoders

1 INTRODUCTION . Significant progress has been made in learning to solve optimization problems via machine learning ( ML ) . Especially for practical applications , learning-based approaches are of great interest because of the high labor costs associated with the development of completely hand-crafted solution approaches . For routing problems such as the traveling salesperson problem ( TSP ) and the capacitated vehicle routing problem ( CVRP ) , recent ML-based approaches are able to generate good solutions for small problem instances in a fraction of a second ( e.g. , Kool et al . ( 2019 ) ) . However , in many real-world applications of these problems users gladly accept more computation time for solutions of even higher quality . Recently proposed approaches ( e.g. , Hottung & Tierney ( 2020 ) ) address this demand and integrate learning-based components with high-level search procedures . While these approaches offer improved performance over non-search-based methods , they rely on domain knowledge encapsulated in the high-level search procedures . In this work , we present a learning-based optimization approach for routing problems that is able to perform an extensive search for high-quality solutions . In contrast to other approaches , our method does not rely on domain-specific high-level search procedures . Our approach learns an instancespecific mapping of solutions to a continuous search space that can then be searched via any existing continuous optimization method . We use a conditional variational autoencoder ( CVAE ) that learns to encode a solution to a given instance as a numerical vector and vice versa . Some genetic algorithm variants ( e.g. , Gonçalves & Resende ( 2012 ) ) use numerical vectors to represent solutions to combinatorial optimization problems . However , these approaches rely on decoding schemes that are carefully handcrafted by domain experts . In contrast , our approach learns the problem-specific decoding schema on its own , requiring no domain or optimization knowledge on the side of the user . The performance of an optimization algorithm heavily depends on the structure of the fitness landscape of the search space , such as its smoothness . If solutions close to each other in the search space are semantically similar , resulting in a smooth landscape , the employed search algorithm can iteratively move towards the more promising areas of the search space . It has been observed for some problems that variational autoencoders ( VAEs ) are capable of learning a latent space in which semantically similar inputs are placed in the same region . This allows , for example , a semantically meaningful interpolation between two points in the latent space ( see e.g . Berthelot et al . ( 2018 ) ) . However , it is unclear if this property upholds for a conditional latent space that encodes routing problems . We show experimentally that our CVAE-based approach is indeed capable of learning a latent search space in which neighboring solutions have a similar objective function value . Furthermore , we introduce a novel technique that addresses the issue of symmetries in the latent space and show that it enables our method to match and surpass state-of-the-art ML-based methods . We train our method using high-quality solutions because we aim to learn a latent search space that contains mostly high-quality solutions . Hence , our method usually requires a long offline phase ( e.g. , to generate solutions using a slow , domain-independent , generic solver ) . However , this offline phase is offset by fast , online solution generation . We focus on the TSP and the CVRP , which are two of the most well-researched problems in the optimization literature . The TSP is concerned with finding the shortest tour between a set of cities that visits each city exactly once and returns to the starting city . The CVRP describes a routing problem where the routes for multiple vehicles to a set of customers must be planned . All customers have a certain demand of goods and all vehicles have a maximum capacity that they can carry . All routes must start and end at the depot . The task is to find a set of routes with minimal cost so that the demand of all customers is fulfilled and each customer is visited by exactly one vehicle . We consider the versions of the TSP and CVRP where the distance matrix obeys the triangle inequality . The contributions of this work are as follows : • We propose a novel approach that learns a continuous , latent search space for routing problems based on CVAEs . • We show that our approach is able to learn a well-structured latent search space . • We show that the learned search space enables a standard differential evolution search strategy to outperform state-of-the-art ML methods . 2 RELATED WORK . In Hopfield & Tank ( 1985 ) , it was first proposed to use an ML-based method to solve a routing problem . The authors use a Hopfield network to solve small TSP instances with up to 30 cities . In Vinyals et al . ( 2015 ) , pointer networks are proposed and trained to solve TSP instances with up to 50 cities using supervised learning . Bello et al . ( 2016 ) extend this idea and train a pointer network via actor-critic reinforcement learning . More recently , graph neural networks have been used to solve the TSP , e.g. , a graph embedding network in Khalil et al . ( 2017 ) , a graph attention network in Deudon et al . ( 2018 ) , or a graph convolutional network in Joshi et al . ( 2019 ) . The significantly more complex CVRP has first been addressed in Nazari et al . ( 2018 ) and Kool et al . ( 2019 ) , in which a recurrent neural network decoder coupled with an attention mechanism and a graph attention network are used , respectively . While some of these methods use a high-level search procedure ( such as beam search ) , all of them are focused on finding solutions quickly ( in under one second ) . In contrast , our approach is able to exploit a longer runtime ( more than one minute for larger instances ) to find solutions of better quality . A couple of approaches use local search like algorithms combined with ML techniques to solve routing problems . Chen & Tian ( 2019 ) propose to learn an improvement operator that makes small changes to an existing solution . The operator is applied to a solution iteratively to find a highquality solutions for the CVRP . However , with a reported runtime of under half a second for the CVRP with 100 nodes , the method is not focused on performing an extensive search . In Hottung & Tierney ( 2020 ) , another iterative improvement method for the CVRP is proposed that integrates learned heuristics into a large neighborhood search framework . The method is used to perform an extensive search with reported runtimes of over one minute for larger instances . In contrast to our method , the high-level large neighborhood search framework contains domain specific components and is known to perform exceptionally well on routing problems ( Ropke & Pisinger , 2006 ) . Perhaps most similar to our work is the line of research based on Gómez-Bombarelli et al . ( 2018 ) , in which the authors use a VAE to learn a continuous latent search space for discovering molecules . They use an additional Gaussian process model that is trained to predict the the quality of molecules given their latent search space representation to allow for a gradient-based search . Kusner et al . ( 2017 ) and Jin et al . ( 2018 ) use a similar setup , but use Bayesian optimization for the search . Winter et al . ( 2019 ) propose to use particle swarm optimization to search a learned latent space for new molecules . To a more limited degree , the idea of optimizing in a continuous learned space has also been used for neural architecture optimization ( Luo et al. , 2018 ) . In contrast to the aforementioned methods , we do not use a separate model to predict the solution quality based on their latent representation , because decoding and evaluating solutions in our setting is cheap ( compared to molecules or neural network architectures ) . Furthermore , our approach addresses a fundamentally different problem , because routing problems must be solved with respect to a given context ( i.e. , a problem instance that describes location coordinates that must be visited ) and we hence use a CVAE in this work . Learning a latent space conditioned on a problem instance ( with the number of possible instances being basically infinite ) is significantly more challenging . Ichter et al . ( 2018 ) propose to use CAVEs to learn a latent space conditioned on problem instances to represent solutions to robot motion planing problems . However , they only sample solutions at random from the learned distribution and do not perform a guided search . We show that the learned structured latent space of our approach enables a guided search that significantly outperforms random sampling . Different generative models have been used to sample new population members in probabilistic evolutionary algorithms known as estimation of distribution algorithms ( e.g. , a Helmholtz machine ( Zhang & Shin , 2000 ) , a restricted Boltzmann machine ( Tang et al. , 2010 ; Shim et al. , 2010 ; Probst et al. , 2017 ) , or a VAE ( Garciarena et al. , 2018 ; Bhattacharjee & Gras , 2019 ) ) . All these methods are focused on how to explore an existing search space using generative models . In contrast , our method is focused on learning the search space itself , leaving the actual search to a generic optimizer . 3 METHOD . Our novel approach , called CVAE-Opt , learns a continuous ( latent ) search space for routing problems that can be searched by any continuous optimization method . It is based on a CVAE that learns to map solutions to routing problem instances to a continuous , n-dimensional space . In contrast to conventional search spaces , the learned latent search is trained to contain only high-quality solutions . Autoencoders are neural networks that are used to learn an efficient encoding of data . They consist of an encoder and a decoder network . The encoder learns to reduce an input x to a point z in a low dimensional space and the decoder tries to reconstruct the input x based on z . The objective of the training is to minimize the difference between the input x and the output of the decoder , requiring the network to learn an efficient encoding of x . In contrast , VAEs are generative models that do not use a deterministic encoder , but instead an encoder that parameterizes an approximate posterior distribution over z . In our context , we do not want to train the decoder to generate solutions for only a single instance ( e.g. , a given set of coordinates for the TSP ) , but instead for all instances of a certain instance type ( e.g. , all TSP instances with 50 cities ) . We thus use a CVAE ( Sohn et al. , 2015 ) , which enables us to learn a latent search space conditioned on the problem instances . 3.1 VARIATIONAL AUTOENCODER-BASED COMBINATORIAL OPTIMIZATION . The overall training process of CVAE-Opt is shown in Figure 1a . The stochastic encoder q ( z|l , s ) receives a problem instance l and a high-quality solution s and outputs an n-dimensional vector z . The decoder p ( s|l , z ) is given z together with the instance l and outputs a solution s′ . One objective of the training is to minimize the difference between the original high-quality solution s and the solution s′ generated by the decoder . While the decoder is powerful enough to construct a good solution based on the instance l alone , it is also given the latent variable z that describes the aspects of the solution s that the decoder can not reliably infer on its own . The second objective during training is to ensure that high-quality solutions can be generated for values of the latent variable that have not been seen during training . This objective is explained in more detail below . Figure 1b shows the iterative search process , in which the decoder p ( s|l , z ) is used together with any unconstrained continuous optimizer to search for solutions to a problem instance l. The unconstrained continuous optimizer navigates the search through the learned latent search space . At each iteration , the optimizer outputs a vector z describing a point in the latent search space . The decoder generates a solution s′ based on z and the objective function value of s′ is returned to the optimizer . With an effective optimizer and the learned search space , high-quality solutions to l can be found . Routing problem representation We describe a routing problem instance by a graphG = ( V , E ) , with V = { v0 , ... , vn } . The representation of a problem instance l consists of a set of n feature vectors x0 , . . . , xi , . . . , xn , where xi describes node vi . For the TSP , each node represents a location ( e.g. , a city ) with each two-dimensional feature vector describing the location ’ s coordinates . For the CVRP , the node v0 represents the depot , and all other nodes represent the customers . As in Nazari et al . ( 2018 ) , each feature vector is four-dimensional and describes the unfulfilled demand of a location , the remaining capacity of the vehicle , and the coordinates of the location . For both problems , a solution s describes a sequence of locations vs0 , . . . , vsT ( for the TSP , T = n ) in which the first location is the starting city ( for the TSP ) or the depot ( for the CVRP ) . We note that our formalism focuses on routing problems on a Euclidean plane . While we anticipate that our approach will work for other types of combinatorial optimization problems ( with adjustment of the input layers ) , we save showing this for future work .

Summary: This paper proposes a method to learn a continuous latent space via CVAE to represent solutions to routing problems. Combined with differentiable evolution search algorithms, one can search in the learned latent space for solutions to new problem instances at test time. The proposed method is evaluated on two classes of routing problems: TSP and CVRP. Results show better performance in terms of objective values and runtime. They are also competitive with established expert-designed algorithms such as LKH3.