Learning to delegate for large-scale vehicle routing

1 Introduction . Vehicle routing problems ( VRPs ) have enjoyed ample applications in logistics and ride-hailing services [ 23 ] around the world for decades . While determining the optimal solution to a VRP is NP-hard [ 25 ] , there have been numerous attempts to solve VRPs both exactly and approximately : provable algorithms have been designed for specific problem instances up to size 130 [ 28 ] , and powerful heuristic solvers such as LKH-3 [ 13 ] and HGS [ 49 , 48 ] find good solutions in practice for problems of size more than 1000 . However , heuristics methods often suffer from inflexibility due to extensive hand-crafting and the heavy computational burden from lengthy iterative procedures , as in the case of LKH-3 . For example , LKH-3 takes more than an hour to solve a size 2000 CVRP instance ( Table 1 ) , which is impractical for applications such as large-scale courier or municipal services . More recently , machine learning methods inspired by the Pointer Network [ 50 ] provide alternatives to traditional solvers : the learning-based methods greatly reduce computation time while maintaining decent solution quality on small problem instances ( less than 100 cities ) , by training on diverse sets of problem distributions either via supervised [ 50 ] or reinforcement learning [ 29 , 20 ] . However , these methods remain difficult to scale , and few report results on problems of size more than 200 . Our work aims to address scalability by learning to identify smaller subproblems which can be readily solved by existing methods . Our learned subproblem selector guides the problem-solving process to focus local improvement on promising subregions . While there exists a combinatorial number ⇤Equal Contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . of subproblems that can be selected at each step , we leverage spatial locality commonly found in combinatorial optimization problems to restrict the selection space size to be linear in the problem size . Intuitively , objects far away from each other generally have very small influence on each other ’ s solution and are likely to be in different routes , so they should not be part of the same subproblem . The greatly reduced search space enables us to feasibly train an attention-based subproblem selector . Our framework combines the advantages of learning and heuristics : our network identifies promising subproblems to improve upon , dramatically speeding up solution times . Using a competitive subsolver on subproblems , we achieve good solution quality without the high computational costs of running the subsolver on large problem instances . In summary , our contributions are : • We propose learning-to-delegate , a learning-based framework for solving large-scale VRPs by iteratively identifying and solving smaller subproblems . • Despite the high dimensionality and NP-hardness of subproblems , we design a Transformer architecture that effectively predicts the subsolver ’ s solution quality for a subproblem . • With extensive validation , we show that learning-to-delegate offers significant speedups and/or objective improvements over both its base subsolver and random ( or heuristic ) subproblem selection , for a variety of VRP variants , distributions , and solvers . 2 Preliminary : Capacitated Vehicle Routing Problems ( CVRP ) . In CVRP , there is a depot node 0 and city nodes { 1 , ... , N } . Each city node i has demand di to fulfill . A vehicle with capacity C starts and ends at the depot and visits a route of city nodes such that the sum of city demands along the route does not exceed C , after which the vehicle starts a new route again . The objective is to find a valid solution minimizing the solution cost . We define the following : • Route : a sequence of nodes , where the first and last node are the depot 0 , and the rest are city nodes . In a valid route , the sum of demands of the city nodes does not exceed C. • Route cost : the sum of edge costs for the sequence of nodes . For an edge from node i to node j , the edge cost is the Euclidean distance between node i and j . • Solution : a feasible solution consists of a set of valid routes visiting each city exactly once . • Solution cost : the sum of route costs for all routes in the solution . An optimal solution is a feasible solution with the minimum solution cost . • Subproblem : a CVRP consisting of the depot , a subset of cities , and corresponding demands . 3 Related Work . Classical methods . Heuristics for solving combinatorial optimization problems have been studied for decades . The most powerful methods , such as local search [ 1 ] , genetic algorithms [ 36 ] , and ant colony methods [ 10 ] , involve iteratively improving the solution in a hand-designed neighborhood . For example , move , swap [ 52 ] , and 2-opt [ 9 ] are well-known heuristics for the traveling salesman and vehicle routing problems . The competitive VRP solver LKH-3 [ 13 ] uses the Lin–Kernighan heuristic [ 26 ] as a backbone , which involves swapping pairs of sub-routes to create new routes , whereas the CVRP solver HGS [ 49 , 48 ] uses a hybrid genetic and local search procedure to achieve state-of-the-art solution qualities on problems up to size 1000 . While LKH-32 tackles a variety of VRP variants , the publicly available implementation of HGS 3 only solves CVRP . For large problems , low-level heuristics are combined with meta-heuristics , including Tabu Search with Adaptive Memory [ 40 ] , guided local search [ 51 ] , and Large Neighborhood Search [ 35 ] . The inspiration for our work derives from Partial OPtimization Metaheuristic Under Special Intensification Conditions ( POPMUSIC ) , which iteratively optimizes problem subparts and has been used to solve problems as diverse as map labeling [ 24 ] , berth allocation [ 22 ] , and p-median clustering [ 39 ] . Despite the promise of the POPMUSIC framework for large-scale combinatorial optimization , the impact of certain design choices is not well understood , such as the subproblem selection ordering 2We refer to the LKH-3 code at http : //webhotel4.ruc.dk/~keld/research/LKH-3/ . 3We refer to the HGS code at https : //github.com/vidalt/HGS-CVRP . and size [ 43 ] . For example , it is not clear how to meaningfully order the subproblems . One early work in this direction demonstrated that a last-in-first-out stack order performs better than random [ 3 ] . Our work provides a natural approach to order the subproblems based on predicted improvement . Deep learning methods . Recently , there has been a surge of interest in training deep neural networks ( DNN ) to solve combinatorial problems . As observed by Kwon et al . [ 21 ] , most methods fall into one of the following two categories : 1 ) construction methods [ 29 , 20 ] , where an autoregressive model such as the Pointer Network [ 50 ] directly outputs a solution , and 2 ) improvement methods [ 7 , 15 ] , where the DNN iteratively performs local updates to the solution , resembling local search . These methods are approaching the solution quality of LKH-3 [ 13 ] on small problem instances . For example , Kwon et al . [ 21 ] extends a construction approach [ 20 ] to encourage more diverse output samples on N 100 VRPs . Among improvement methods , Lu et al . [ 27 ] learn a meta-controller to select among a set of local search heuristics , marking the first learning-based approach to outperform LKH-3 on N 100 VRPs . Despite these successes , learning-based approaches for large-scale VRPs are poorly understood . In addition , all of the aforementioned methods are trained using deep reinforcement learning ; for large problems , trajectory collection becomes prohibitively expensive . Scaling up learning methods . A few recent works have begun to investigate scaling of learned networks for NP-hard graph problems . For example , Ahn et al . [ 2 ] propose an iterative scheme called learning-what-to-defer for maximum independent set . At each stage , for each node in the graph , the network either outputs the solution for the node or defers the determination to later stages . Song et al . [ 38 ] proposes an imitation-learning-based pipeline for Integer Linear Programs ( ILP ) , where at each stage they partition all variables into disjoint subsets , and use the Gurobi [ 12 ] ILP solver to solve the partitions . Due to differences in graph structure , our work presents a more natural scheme to handle VRP constraints and structures . A few works attempt to incorporate learning to decompose large-scale VRPs [ 6 , 47 , 31 ] . However , the decomposition approaches proposed appear to be experimentally less effective than ours . 4 An Iterative Framework for VRPs . While CVRPs are NP-hard and thus require worst-case exponential time in the problem size to solve optimally , typical CVRPs exhibit structure that practical solvers may exploit . We hypothesize that in such situations , the larger problem can be efficiently approximately solved as a sequence of smaller subproblems , which can be delegated to efficient subsolvers . To test this hypothesis , we propose a learning-based , iterative framework with two components : 1 ) a subsolver capable of solving a small problem instance ( exactly or approximately ) , and 2 ) a learned model for identifying suitable subproblems within a larger problem to delegate to the subsolver . Algorithm 1 : Learning to Delegate Input : Problem instance P , initialized solution X , subproblem selector f✓ , subsolver Subsolver , number of steps T , parameter k denoting the size of subproblems 1 for Step t = 1 : T do 2 Sk , local ConstructSubproblems ( P , X , k ) 3 S f✓ ( Sk , local ) 4 X 0S Subsolver ( S ) 5 X X 0S [ XP\S 6 end for We illustrate our iterative framework in Figure 1 , which takes in a problem instance P with a feasible initial solution X0 . At each step , given the current solution X , we select a smaller subproblem S ⇢ P with our learned model f✓ then apply the subsolver to solve S ; we then update X with the new solution for S. To maintain feasibility after an update , we restrict S to be the set S of visited cities of a subset of routes from X . Since routes with cities in P \ S remain valid routes , we obtain a new feasible solution X 0 = X 0S [ XP\S , where X 0S is the subsolution for S and XP\S consists of unselected routes from X . Intuitively , a strong f✓ should identify a subproblem S such that the subsolver solution X 0S results in a large improvement in objective from X to X 0 . 4.1 The Restricted Subproblem Selection Space . As the number of routes in P is R = O ( N ) , the cardinality of the selection space S is O ( 2R ) , which is exponential in the problem size N and difficult for learned models to consider . If we restrict each subproblem to cities from exactly k routes , there are still R k = O ( Rk ) subproblems to consider . Therefore we further restrict selection to subproblems with spatial locality . As shown in Figure 1 ( d ) , we only consider subproblems from Sk , local , where each subproblem is centered around a particular route r and contains the k routes whose centroids have the smallest Euclidean distance to the centroid of r. In this way , we reduce the selection space to |Sk , local| = R = O ( N ) from |S| = O ( 2N ) . In Algorithm 1 , we refer to this restriction as ConstructSubproblems . Our restriction to a local selection space is motivated by the fact that many combinatorial optimizations problems have inherent spatial locality , i.e . problem entities are more strongly affected by nearby entities than faraway entities . Earlier heuristical methods such as POPMUSIC [ 42 ] leverage similar spatial locality .

Summary: The paper proposes new ways to learn good decomposition steps in a heuristic search algorithm for the capacitated vehicle routing problem. The learning algorithm can select good seed points for the decomposition, often permitting significant speed-ups (1.5x) over other baseline decomposition algorithms (random and other simple variations). Additional experiments are conducted to evaluate the impact of the client distribution on the performance of the proposed approach.