S UPERVISED P ERMUTATION I NVARIANT N ETWORKS FOR SOLVING THE CVRP WITH BOUNDED FLEET SIZE
Anonymous authors
Paper under double-blind review
A BSTRACT
Learning to solve combinatorial optimization problems, such as the vehicle routing problem, offers great computational advantages over classical operations research solvers and heuristics.
The recently developed deep reinforcement learning approaches either improve an initially given solution iteratively or sequentially construct a set of individual tours.
However, most of the existing learning-based approaches are not able to work for a ﬁxed number of vehicles and thus bypass the complex assignment problem of the customers onto an apriori given number of available vehicles.
On the other hand, this makes them less suitable for real applications, as many logistic service providers rely on solutions provided for a speciﬁc bounded ﬂeet size and cannot accommodate short term changes to the number of vehicles.
In contrast we propose a powerful supervised deep learning framework that constructs a complete tour plan from scratch while respecting an apriori ﬁxed number of available vehicles.
In combination with an efﬁcient post-processing scheme, our supervised approach is not only much faster and easier to train but also achieves competitive results that incorporate the practical aspect of vehicle costs.
In thorough controlled experiments we compare our method to multiple state-of-the-art approaches where we demonstrate stable performance, while utilizing less vehicles and shed some light on existent inconsistencies in the experimentation protocols of the related work.
I NTRODUCTION
The recent progress in deep learning approaches for solving combinatorial optimization problems (COPs) has shown that speciﬁc subclasses of these problems can now be solved efﬁciently through training a parameterized model on a distribution of problem instances.
This has most prominently been demonstrated for the vehicle routing problem (VRP) and its variants (Kool et al.
; Chen & Tian (2019); Xin et al.
The current leading approaches model the VRP either as a local search problem, where an initial solution is iteratively improved or as a sequential construction process successively adding customer nodes to individual tours until a solution is achieved.
Both types of approaches bypass the implicit bin-packing problem that assigns packages (the customers and their demands) to a pre-deﬁned maximum number of bins (the vehicles).
We show that this assignment can be learned explicitly while also minimizing the main component of the vehicle routing objective, i.e. the total tour length.
Furthermore, ﬁnding a tour plan for a ﬁxed ﬂeet size constitutes an essential requirement in many practical applications.
To this end, many small or medium-sized service providers cannot accommodate ﬂeet size adjustments where they require additional drivers on short notice or where very high acquisition costs prohibit the dynamic extension of the ﬂeet.
Figure 1 shows the variation of ﬂeet sizes for different problem sizes.
For all problem sizes the baseline approaches (green, red, purple) employ more vehicles than our approach (blue, orange) and thereby inquire potentially more costs by requiring more vehicles.
In contrast our vanilla approach (blue) guarantees to solve the respective problems with exactly the apriori available number of vehicles (4, 7, 11 for the VRP20, VRP50, VRP100 respectively) or less.
Our approach of learning to construct a complete, valid tour plan that assigns all vehicles a set of customers at once has so far not been applied to the VRP.
We amend and extend the Permutation Invariant Pooling Model (Kaempfer & Wolf (2018)) that has been applied solely to the simpler multiple Traveling Salesmen Problem (mTSP), for which also multiple tours need to be constructed, but are albeit not subject to any capacity constraints.
In this work we propose an end-to-end learning framework that takes customer-coordinates, their associated demands and the maximal number of available vehicles as inputs to output a bounded number of complete tours and show that the model not only learns how to optimally conjunct customers and adhere to capacity constraints, but that it also outperforms the learning-based reinforcement learning (RL) baselines when taking into account the total routing costs.
By adopting a full-ﬂedged supervised learning strategy, we contribute the ﬁrst framework of this kind for the capacitated VRP that is not only faster and less cumbersome to train, but also demonstrates that supervised methods are able, in particular under practical circumstances, to outperform state-of-the-art RL models.
[Figure]
The contributions of this work can be summarized as follows:
• Tour-plan construction approach for the capacitated VRP with ﬁxed Vehicle Costs : We propose a deep learning framework that learns to solve the NP-hard capacitated VRP and constructs a complete tour-plan for an speciﬁed ﬂeet size.
In this way it ensures to solve the problem for an apriori ﬁxed number of vehicles.
• Supervised learning for discrete optimization : The proposed approach is the ﬁrst ML-based construction approach for the VRP that relies entirely on supervised learning to output feasible tours for all available vehicles.
• Competitiveness : We compare our model against prominent state-of-the-art deep learning and OR solvers through a thorough and uniﬁed experimental evaluation protocol that ensures comparability amongst different ML-based approaches.
We show that our approach delivers competitive results in comparison to approaches that work only for unbounded ﬂeet sizes and outperforms these methods when considering ﬁxed vehicle costs.
R ELATED W ORK
Learned Construction Heuristics.
The work by Vinyals et al. (2015) introducing Pointer Networks (PtrNet) revived the idea of developing end-to-end deep learning methods to construct solutions for
COPs, particularly for the TSP, by leveraging an encoder-decoder architecture trained in a supervised fashion.
Bello et al.
(2016) and Khalil et al.
(2017) proposed RL to train the Pointer Network for the TSP, while Nazari et al.
(2018) extended the PtrNet in order to make the model invariant to the order of the input sequence and applied it to the CVRP.
The Attention Model (AM) described in
Kool et al.
(2019) features an attention based encoder-decoder model that follows the Transformer architecture (Vaswani et al.
to output solution sequences for the TSP and two variants of the CVRP.
Xin et al.
(2021) extend the AM by replacing the single decoder with multiple, identical decoders with no shared parameters to improve diversity among the generated solutions.
Besides these sequential approaches, "heatmap" construction approaches that learn to assign probabilities to given edges in a graph representation through supervised learning have recently been applied to routing problems.
Joshi et al.
(2019) use a Graph Convolutional Network to construct TSP tours and show that by utilizing a parallelized beam search, auto-regressive construction approaches for the
TSP can be outperformed.
Kool et al.
(2021) extend the proposed model by Joshi et al.
for the
CVRP while creating a hybrid approach that initiates partial solutions using a heatmap representation as a preprocessing step, before training a policy to create partial solutions and reﬁning these through dynamic programming.
Kaempfer & Wolf (2018) extend the learned heatmap approach to the number of tours to be constructed; their Permutation Invariant Pooling Network addresses the mTSP (a TSP involving multiple tours but no additional capacity constraints), where feasible solutions are obtained via a beam search and have been proven to outperform a meta-heuristic mTSP solver.
Learned Search Heuristics.
In contrast to construction heuristics that build solutions to routing problems from scratch, learned search heuristics utilize well-known local search frameworks commonly adopted in the ﬁeld of operations research (OR) to learn to improve an initial given solution in an iterative fashion through RL.
The NeuRewriter model (Chen & Tian (2019)) learns a policy that is composed of a "rule-picking" and a "region-picking" part in order to iteratively reﬁne a VRP solution and demonstrates superior performance over certain auto-regressive approaches (Nazari et al.
(2018); Kool et al.
Similarly, the Learning to Improve approach by Lu et al.
(2020) learns a policy that iteratively selects a local search operator to apply to the current solution and delivers new state-of-the-art results amongst existing machine learning heuristics.
However, it is computationally infeasible during inference in some cases taking more than thirty minutes to solve a single instance, which makes this approach incomparable to most other methods.
In contrast,
Neural Large Neighborhood Search (NLNS) (Hottung & Tierney (2020)) is based on an attention mechanism and integrates learned OR-typical operators into the search procedure, demonstrating a fair balance between solution quality and computational efﬁciency, but can only be evaluated in batches of thousand instances in order to provide competitive results.
P ROBLEM F ORMULATION
This section introduces the capacitated vehicle routing problem which implicitly encompasses a bin packing problem through ﬁnding a feasible assignment of customer nodes to a given set of vehicles.
We showcase how existing ML-based approaches circumvent this property and solve a simpler variant of the problem.
We then demonstrate how to cast the VRP into a supervised machine learning task and ﬁnally propose an evaluation metric that respects the utilization of vehicles in terms of ﬁxed vehicle costs.
3. T HE C APACITATED V EHICLE R OUTING P ROBLEM
Following Baldacci et al.
(2007) the Capacitated Vehicle Routing Problem (CVRP) can be deﬁned in terms of a graph theoretical problem.
It involves a graph G =
( V, E ) with vertex set V = { 0 , ..., N } and edge set E .
Each customer i ∈ V c
= { 1 , ..., N } has a demand q i which needs to be satisﬁed and each weighted edge { i, j } ∈ E represents a non-negative travel cost d ij .
The vertex 0 ∈ V represents the depot.
A set M of identical (homogeneous) vehicles with same capacity Q is available at the depot.
The general CVRP formulation sets forth that "all available vehicles must be used" (Baldacci et al.
(2007, p. 271))
and M cannot be smaller than M min , corresponding to the minimal ﬂeet size needed to solve the problem.
Accordingly, the CVRP consists of ﬁnding exactly M simple cycles with minimal total cost corresponding to the sum of edges belonging to the tours substitute to the following constraints: (i) each tour starts and ends at the depot, (ii) each customer is served exactly once and (iii) the sum of customer demands on each tour cannot be larger than Q .
A formal mathematical deﬁnition of the problem can be found in Appendix A.
Therefore the solution requires exactly M cycles to be found, which requires in turn the value of M to be set in an apriori fashion.
However, already the task of ﬁnding a feasible solution with exactly
M tours is NP-complete, thus many methods choose to work with an unbounded ﬂeet size (Cordeau et al.
(2007, p. 375)) which guarantees a feasible solution by simply increasing M if there are any unvisited customers left.
In contrast, our method tackles the difﬁculty of the assignment problem by jointly learning the optimal sequence of customer nodes and the corresponding optimal allocation of customers to a ﬁxed number of vehicles.
In order to do so, we rely on generated near-optimal targets that determine M min for each problem instance in the training set.
3. T HE VRP AS A S UPERVISED M ACHINE L EARNING P ROBLEM
The goal of the supervised problem is to ﬁnd an assignment ˆ Y that allocates optimally ordered sequences of customers to vehicles.
Respectively each VRP instance X which deﬁnes a single graph theoretical problem, is characterised by the following three entities :
• A set of N customers.
Each customer i , i ∈ { 1 , ..., N } is represented by the features x cus i .
• M vehicles, where each vehicle k ∈ { 1 , ..., M } is characterised by the features x veh k .
• The depot, the 0 -index of the N customers, is represented by a feature vector x dep .
Consequently, a single VRP instance is represented as the set X = ( x dep ; x cus 1 ,...,n ; x veh 1 ,...M ) .
The corresponding ground truth target reﬂects the near-optimal tour plan and is represented by a binary tensor Y M × N × N , where Y k,i,j = 1 if the k th vehicle travels from customer node i to customer node j in the solution and else Y k,i,j = 0 .
Let X represent the predictor space populated by VRP instances X and let Y deﬁne the target space comprising a population of
target instances
Y .
The proposed model is trained to solve the following assignment problem: Given a sample D ∈ ( X × Y ) ∗ from an unknown distribution p and a loss function (cid:96) :
Y × Y → R , ﬁnd a model
ˆ y : X → Y which minimizes the expected loss, i.e.
[Equation]
For a sampled predictor set x , the model ˆ y ( x ) outputs a stochastic adjacency
tensor ˆ Y M × N × N that consists of the approximate probability distribution deﬁning a tour plan.
3. F IXED V EHICLE C OSTS
In order to highlight the importance of the assignment problem incorporated in the CVRP and the possible impact of an unlimited ﬂeet sizes in real world use cases, we introduce ﬁxed vehicle costs in the evaluation.
As mentioned above, a problem instance is theoretically always "solvable" in terms of feasibilty by any method that operates on the basis of unbounded ﬂeet sizes.
Nevertheless, the increase in the number of tours is undisputedly also a cost driver besides the mere minimization of route lengths.
Thus, we compare our approach to state-of-the-art RL approaches that do not limit the number of tours in a more holistic and realistic setting via the metric that we denote Cost v :
[Equation]
where c v is the ﬁxed cost per vehicle used , such that if a vehicle k leaves the depot, we have
Y k, 0 , 0 = 0 , else we indicate that this vehicle remains at the depot with Y k, 0 ,
= 1 .
Concerning the value of c v we rely on data used in the Fleet Size and Mix VRP in Golden et al.
and ﬁnd matching values corresponding to the capacities used in our experimental setting.
To this end, we incorporate ﬁxed costs of c v = 35 , c v = 50 and c v = 100 for the VRP with 20, 50 and 100 customers respectively (a summarizing Table can be found in Appendix C.2).
We want to note that even though the RL baselines are not speciﬁcally trained to minimize this more realistic cost function, our model is neither, as will be shown in section 4.
This realistic cost setting functions as mere statistical evaluation metric to showcase the implications of ﬁxed vehicle costs.
P ERMUTATION I NVARIANT VRP M ODEL
The proposed model outputs a complete feasible tour-plan for the CVRP in the form of a stochastic adjacency tensor ˆ Y .
Inspired by the Transformer-based architecture in Kaempfer & Wolf (2018) that tackles the mTSP, we extend this architecture to additionally capture vehicle capacity constraints.
The framework encompasses three components: (i) an embedding layer ( Encoder ), (ii) an information extraction mechanism ( Pooling ) and (iii) an output construction block ( Decoder ).
Figure 2 displays an overview of the model architecture.
[Figure]
Max Pooling Layer
LayerNorm
Shared FC Layer
LayerNorm
Encoder
Information Extraction
Decoder h dep ( L ) h cus ( L ) ; E =
E i :.
E j :.
; E' =
V' k 1D Conv + ReLU
E' V' k ;
A k
Softmax
Linear Projection g ... h veh ,k( L ) V k
= | i, j = 1 : N'
Embedding .
The three entities that make up a VRP instance X are embedded separately, such that each encoding is embedded into a shared dimensional space d m by a linear projection:
[Equation]
where h (0) g ∈ R l g × d m is the initial embedding of each entity that will remain of different lengths l cus =
N, l veh =
M, l dep = 1 .
The separate embedding for each entity and the architecture of the network ensure that the order of the elements in the entities is not relevant for the operations performed in the network, which establishes the permutation invariance.
Information Extraction .
The information extraction component iteratively pools contexts across entities and linearly transforms these context vectors c g, 1 , . .
, c g,l g with the entity embeddings of the current layer h lg .
For each element r ∈ { 1 , . . .
, l g } in an entity the contexts are pooled:
[Equation]
For the next layer’s entity representation, the contexts and previous embeddings are linearly projected:
[Equation]
This pooling mechanism proceeds until a ﬁnal representation of each entity is retrieved; h ( L ) dep , h ( L ) cus and h ( L ) veh .
These are then passed to the decoding procedure.
A more detailed description of the operations performed in the Pooling Layer is described in Appendix C.3.
Decoder .
The decoding step in the model constructs the output tensor ˆ Y ∈ R M × N (cid:48)
× N (cid:48) where
N (cid:48)
=
N + 1 indicates the size of the full adjacency tensor including the depot.
A preliminary step consists of forming a feature representation of all edges (potential paths) between all pairs of vertices in the graph, denoted as E (cid:48) :
[Equation]
The ﬁnal construction procedure combines the edge ( E (cid:48) ) and ﬂeet ( V = h ( L ) veh ) representations where each vehicle representation V k
= h ( L ) veh ,k , k ∈ M enters the combination process twice: 1.
In the linear transformation with the edges E (cid:48) , where W o and bias b o are the learned weights and V (cid:48) k is an expanded version of V k to match the dimensionality of E (cid:48) :
[Equation]
In a scaled dot-product which returns compatibility scores for each path potentially travelled by vehicle k to emphasize a direct interaction between vehicles and convolved edges:
[Equation]
Finally a softmax is applied to transform the compatibility scores of vehicles and edges into probabilities ˆ Y ∈ R M × N (cid:48)
× N (cid:48) .
4. S OLUTION D ECODING
In order to transform the doubly-stochastic probability tensor ˆ Y into discrete tours in terms of a binary assignment, we use a greedy decoding strategy.
In training , a pseudo-greedy decoding (see Appendix
C.4) renders potentially infeasible solutions by transforming ˆ Y into a binary assignment tensor ˆ Y ∗ , where ˆ Y ∗ = 1 for the predicted path of vehicle k between vertices i and j and 0 otherwise.
Input
[Equation]
Output: Routes ˆ Y ∗ ∈ { 0 , 1 } M × N (cid:48) × N (cid:48) 1: for i ∈ U do 2: if Q (cid:48) − q i <
M then 3:
M M
1 ; (cid:46) Add new tour if guarantee solution is set 4: ˆ Y
M,.,.
← 0 ; (cid:46) Update binary solution tensor 5:
Q
M ← Q ; (cid:46) Update capacity tensor 6: end if 7: v
; (cid:46) Vehicle with max capacity left 8:
Select all possible vertices for insertion 9:
ˆ Y v,V a ,i )] ; (cid:46) Assign incoming edge of inserted customer 10: j after ← arg max( ˆ Y ∗ v,j before , : ) ; (cid:46)
Assign outgoing edge of inserted customer 11: ˆ Y ∗ v,j before , : ← 0 .
; (cid:46) Update Solution Tensor 12: ˆ Y ∗ v,j before ,i ← 1 .13:
ˆ Y ∗ v,i,j before ← 1 .14:
Q (cid:48) v ← Q (cid:48) v − q i ; (cid:46) v ’s capacity after insertion 15: end for 16: return ˆ Y ∗
During inference , the pseudo-greedy decoding is substituted by a strictly greedy decoding, and thereby accepts only tour-plans, which do not violate the respective capacity constraint.
This is done by tracking the remaining capacity of all vehicles and masking nodes that would surpass the capacity.
Any un-assigned customers violating capacity constraints in the ﬁnal state of the algorithm are recovered in a list U and passed as input to a repair procedure for ˆ Y ∗ (Algorithm 1).
For each unassigned customer in list U , Algorithm 1 assigns customer i to the vehicle with most remaining capacity and positions the missing customer between j before and j after according to the distribution ˆ Y , before updating the predicted binary solution tensor ˆ Y ∗ and the remaining capacity Q (cid:48) accordingly.
4. I NFERENCE S OLUTION P OST -P ROCESSING
Unfortunately the repair operation of Algorithm 1 leads to a situation where the resulting tours during inference are not necessarily local optima anymore.
To improve these tours we add a heuristic post processing procedure doing several iterations of local search via the google OR-tools solver (Perron & Furnon (2019)).
Given a valid solution for a particular number of vehicles, the solver runs a (potentially time-limited) search that improves the initial solution ˆ Y ∗ .
During inference, the method optionally can relax the bound on the ﬂeet size to guarantee to ﬁnd a solution which is important in cases where instances are particularly difﬁcult to solve due to a very tight bound on capacity.
This is done by initially adding an artiﬁcial tour to the plan for the remaining missing customers before running the post-processing scheme.
Thus, this mechanism enables our method to also solve instances it is not initially trained for to provide maximum ﬂexibility.
4. T RAINING THE P ERMUTATION I NVARIANT VRP M ODEL
In order to train our model to learn the assignment of customer nodes to the available vehicles and adhere to capacity constraints, we extend the original negative log-likelihood loss by a penalty formulation ( L over ) and a auxiliary load loss, controlled via weights α over and α load .
The model’s characteristic of permutation invariance induces the necessity for the training loss to be agnostic to the vehicle-order as well as the travel direction of the tours.
Concerning the route-direction, we denote Y
k,i,j = Y
k,i,j and Y 1 k,i,j = Y k,j,i
Therefore, the loss to be minimized is the minimum (penalized) normalized negative log-likelihood of ˆ Y with respect to the sampled target Y :
L ( ˆ Y ) :=
[Equation]
(9) with the load of a tour T =
Y k,.,.
∈ { 0 , 1 } N (cid:48)
× N (cid:48) load ( T ) :
=
N (cid:48) (cid:88) i,j =
T i,j q i (10) and a shifted quadratic error for overloading
L over ( q ) :
= (cid:26) 0 , if q ≤ Q (1 + q − Q ) 2 , else (11)
The calculation of the loss formulation in Equation 9 would induce considerable computational overhead, therefore a less memory-intensive formulation of the same loss is implemented (see
Appendix C.5).
The extensions to the original Permutation Invariant Pooling Network are summarized in Appendix C.6.
E XPERIMENTS
In the following experiments we want to showcase and validate our main contributions:
Our method works for problems with bounded ﬂeet sizes and outperforms state-of-the-art RL models when accounting for ﬁxed vehicle costs.
Our supervised learning model is fast to train and delivers competitive results in comparison to the state-of-the-art, while utilizing considerably less vehicles.
We consider three different problem sizes, with 20, 50 and 100 customer vertices respectively and generate solvable training instances following a rejection-sampled version of the data generating process in Kool et al.
The near-optimal target instances are generated with google’s OR-Tools guided local search solver (Perron & Furnon (2019)), where we set M = 4 , 7 , 11 and Q = 30 , 40 ,for each problem size respectively.
In total we generate roughly 100,000 training samples per problem size.
The hyperparameter settings are described in Appendix D.1.
For the evaluation, we work with two versions of the test dataset in Kool et al.
(2019), the originally provided one and a rejection sampled version of it, as technically our model is trained to solve only the instances for which the lower bound (cid:80) i ∈ n q i ≤ MQ holds.
We compare our model to recent RL methods that comprise leading autoregressive (AM (Kool et al.
, MDAM (Xin et al. (2021))) as well as a search based (NLNS (Hottung & Tierney (2020))) approaches.
The NeurRewriter (Chen & Tian (2019)) and L2I model (Lu et al.
(2020)) are not re-evaluated due to extensive training- and inference times.
Details concerning the re-evaluated (and not re-evaluated) baselines are found in Appendix D.2.
Furthermore, we revisit different published results in a per-instance re-evaluation to elevate comparability and demonstrate strengths and weaknesses of the methods.
We note that most methods were originally evaluated in a per-batch fashion where they can leverage the massive parallelization capabilities of current graphical processing units, while that setting is of less practical relevance.
5. F IXED V EHICLE C OSTS
To validate the ﬁrst point of our main contribution
[Table]
we evaluate the performance of different RL methods and our own model on the metric deﬁned in Equation 2.
Table 1 shows the results for evaluating the methods on the total tour length (Cost), as well as the measure incorporating ﬁxed costs for each vehicle (Cost v ).
The results in Table 1 shows that method outperforms state-of-the-art
[Table]
RL methods on the VRP with ﬁxed vehicle costs across all sizes of the problem and even outperforms them on the plain total tour length minimization cost for the VRP with 20 customers, while delivering the shortest inference times per instance together with the AM Model.
Furthermore, our method’s results do not greatly differ whether we set the option of guaranteeing a solution (last row) to True versus accepting that there are unsolved samples, speaking for the strength of our model for being able to solve both problems incorporated in the VRP at once, the feasible assignment problem to a particular number vehicles and the minimization of the total tour length.
This is also represented in
Figure 1; while the amount of solutions solved with more than the apriori ﬁxed ﬂeet size is vanishingly small for the VRP20 and VRP50 it is exactly 0 for the VRP100.
For the VRP of size 50 the RL-based methods only outperform our greedily evaluated method on the routing length minimization when employing sampling or a beam search during inference, while being worse when used greedily.
Concerning the problem with 100 customers, our method is outperformed by the RL approaches on the vanilla total route length cost, but solves the majority of the problems with a signiﬁcantly smaller ﬂeet size, reﬂected by the considerably smaller total costs (Cost v ).
This is strongly supported by Figure 1, where we see that especially for the VRP with 100 customers, the RL-based methods consistently require more vehicles than needed.
5. C OMPARATIVE R ESULTS ON THE B ENCHMARK D ATASET
We want to assess the competitiveness of our method also on the literature’s bench-marking test set provided in Kool et al.
Technically, these instances are not all solvable by our model, as it is not trained to solve problems where (cid:80) i ∈ n q i ≤ MQ .
In Table 2 we therefore indicate the percentage of solved instances by our plain model in brackets.
Looking at Table 2, our method’s performance is generally on par with or slightly better than RL-based construction methods when employing a greedy decoding.
Concerning the per-instance run times, MDAM and the NLNS take orders of magnitude longer than the AM model and our method.
We acknowledge that exploiting parallelism to enhance computational efﬁciency is a justiﬁed methodological choice, but in terms of comparability, we argue for the pragmatic approach of comparing per-instance runtimes to remedy problematic comparison of models using different architectures and mini batch sizes.
For completeness Tablein Appendix D.3 illustrates the per-batch evaluation of the baselines.
Appendix D.2 also discusses the discrepancies in performances of NLNS and MDAM with respect to their published results.
Even though the solution quality and competitiveness of our method decrease when considering larger problem sizes, the per-instance inference time remains among the best.
Nevertheless, we want to emphasize that the baselines in general require more tours than our method potentially leading to higher costs when employed in real-world scenarios
[Table] [Table]
5. T RAINING T IMES
We showcase the fast training of our model against the NeuRewriter model (Chen & Tian (2019)) as a representative for RL methods.
Table 3 shows the results from runs on an A100-SXM4 machine with 40GB GPU RAM.
To train the NeuRewriter for the VRP50 and VRP100 with the proposed batch size of 64, 40GB GPU RAM are not sufﬁcient.
Instead, we used a batch size of 32 and 16 respectively, where the total runtime is estimated for completing 10 epochs.
Accounting for our dataset generation, we note that with 2 Xeon Gold 6230 CPU cores, it takes roughly 11 days to generate one dataset.
That said, we emphasize that the OR Tools is straightforwardly parallelizable (emberrassingly parallel!), such that doubling the amount of cores, roughly halves the runtime.
[Table]
C ONCLUSION AND F UTURE W ORK
With the proposed supervised approach for solving the VRP, we investigate a new road to tackle routing problems with machine learning that comes with the beneﬁt of being fast to train and which is able to produce feasible solutions for a apriori ﬁxed ﬂeet size.
We show that our model is able to jointly learn to solve the VRP assignment problem and the route length minimization.
Our work focuses on and showcases practical aspects for solving the VRP that are important for decision makers in the planning industry and shows that our method outperforms existing models when accounting for ﬁxed vehicle costs.
In future work, we aim to alleviate the computational shortcomings of the train loss calculation, such that the model’s fast training capability can be extended to problems with larger ﬂeet sizes.
R EPRODUCIBILITY S TATEMENT
Considering the current pace at which new approaches and methods are emerging not only but also for the research area of learning based combinatorial optimization, fast and easy access to source codes with rigorous documentation needs to be established in order to ensure comparability and veriﬁed state-of-the-arts.
To this end we will provide the code for our fast supervised learning method on a github repository which will be publicly available.
During the review process the code will be made available to he reviewers.
On a broader scope since the ﬁndings in this paper comprise not only our own results but also those of re-evaluated existing approaches in the ﬁeld we plan to develop a benchmark suite on learning-based routing methods for which the re-evaluations and produced codes will build the basis for.
A F ORMAL D EFINITIONS A. M ATHEMATICAL F ORMULATION OF THE CVRP
Following the notation in section 3.1 of the paper the CVRP can be formalized as the following two-index vehicle ﬂow formulation ﬁrst introduced by Laporte et al
Given a binary decision variable x ij taking either a value of 1 or 0 ∀{ i, j } ∈ E {{ 0 , j } : j ∈ V c } where x ij = 1 if the path { i, j } is traversed and x 0 j = 2 if a tour including the customer j is selected in the solution solve the following integer program:
[Equation]
where a subset of customers is denoted by S ⊆ V c , δ ( S )
= { i, j } ∈ E : i ∈ S, j (cid:54)∈ S or i (cid:54)∈ S, j ∈
S is the cutset deﬁned by S and S =
{ S : S ⊆ V c , | S | ≥ 2 } .
Equations (12b) lists the degree constraints.
Equation (12c) indicates that for any subset S of customers, excluding the depot, at least [ q ( S ) /Q ]
vehicles enter and leave it.
Constraint (12d) ensures that M vehicles leave and return to the depot.
Equations (12e) and (12f) constitute the integrality constraints.
A. B IN P ACKING P ROBLEM
The Bin Packing Problem can be formulated in words as follows:
Given a set of n items { 1 , . . . , n } , where w j is the weight of the item j ∈ { 1 , . . . , n } and n bins, each with capacity C .
Assign each item to one bin so that:
• The total weight of items in each bin does not exceed C
• The total number of utilized bins is minimized.
In mathematical terms, this results in solving the following minimization problem, where x ij ∈ { 0 , 1 } is equal to 1 if item j is assigned to bin i , and z i ∈ { 0 , 1 } is equal to 1 if bin i is used:
[Equation]
B A BLATIONS
This section demonstrates that the changes in our approach and notably to the loss structure of the approach, compared to the original model in Kaempfer & Wolf (2018), are improving the model’s performance and efﬁciency.
The results summarized in Table 2 and 3 are retrieved form an evaluation on the rejection sampled Kool et al.
Dataset with the option of a guaranteed solution.
B. A DDITIONAL L OSS S TRUCTURES
This section demonstrates that the additional loss structures (Penalty and auxiliary Load Loss) imposed on the pure assignment Loss help reduce the capacity violation during training and guide the model towards a better performance.
The right hand side of Figure 3 shows that with the two additional loss structures, the model achieves an average capacity violation of 0.017 per vehicle, while the model, when trained with the plain assignment loss, still exhibit an average capacity violation of 0.021 and even starts increasing again after 40 epochs.
While an average capacity violation of 0.
per vehicle, still means that the model cannot control the capacity constraint to a hundred percent, this still signiﬁcantly helps the Repair Algorithm 1 to generate feasible solutions.
[Figure]
T r a i n / V a li d a t i o n L o ss
[Figure]
Loss
VRP
Cost
Cost v
Plain 6.
Penalty 6.
Ours 6.
The left sub-ﬁgure illustrates the train (solid line) and validation (dashed line) losses over epochs across the three loss conﬁgurations for the VRP with 20 customers; "Plain", i.e. no additional loss structures (green), "Penalty" (blue) and both "Penalty" and Load Loss structures (red).
The right sub-ﬁgure displays the average capacity constraint violation of vehicles as training proceeds.
Table
The loss functions on the left hand side of Figure 3 show that the losses including the auxiliary loss structures are overall higher, but can exhibit a higher delta between starting losses and ﬁnal losses, which translates in better performance illustrated in Table 5.
B. S OFT -A SSIGN L AYER
In this section we want to demonstrate the empirical ﬁndings that a simple Softmax Normalization function is competitive to the SoftAssign Score Normalization Algorithm by Kaempfer & Wolf (2018).
[Table]
B. G ENERALIZATION
This section documents how the supervised VRP models trained on graph sizes of 20, 50 andgeneralize to the respective other problem sizes, as well as to more realistic problem instances (Uchoa100) published in Kool et al.
(2021) that are generated for graph-sizes of 100 following the data distribution documented in Uchoa et al.
The underlying dataset in Table 6 is the rejection-sampled version of the dataset used in Kool et al.
[Table]
Table 6 shows that in general and as expected, the models trained on a particular distribution perform best with regards to the objective cost, when tested on the same distribution.
B.
[Table]
In this section we provide additional head-to-head comparative results against the underlying targetgenerating method OR Tools (Perron & Furnon (2019)).
The results are generated on the rejection sampled version of the Test-set used in Kool et al.
We compare the OR-Tools method with our results that guarantee a solution to be found and set the search limit for OR-Tools Guided Local
Search (GLS) to 10 seconds.
It is worth mentioning that with the indicated setting of the GLS
[Table] for 18, 7 and of the test instance [Table] for the VRP20, VRP50 and VRP100 no solution could be found.
[Table]
B. E FFECTIVENESS OF P OST - PROCESSING
The Table 8 compares the effects of post-processing on both our model and the AM baseline model.
The AM model is chosen as for this baseline, the resulting tours for each of the test instances can be easily retrieved and hence the postprocessing scheme can be run on top of those initially retrieved tours.
However, we note that the additional post-processing procedure took an additional 3.5 minutes for the VRP20, 20 minutes for the VRP50 and 1 hour for the VRP100 in terms of runtime.
[Table]
D ECODING P ROCEDURE C OMPARISON TO K AEMPFER & W OLF (2018)
The most effective methodological contribution is the decoding process of solutions. In this section, we provide comparative results to the original decoding procedure in Kaempfer & Wolf (2018).
Kaempfer & Wolf (2018) employ the "SoftAssign" normalization (mentioned in sub-section B.2) to obtain the probability distribution of tours and decode the discrete tours by a Backtracking or "Beam Search" procedure. In contrast to the Beam Search procedure we decode the solution greedily followed by a repair operation and a ﬁnal postprocessing scheme. Table 9 shows that the Beams Search decoding in Kaempfer & Wolf (2018), even though originally employed to solve the mTSP, is not capable of ﬁnding valid VRP solutions reliably. A simple greedy decoding followed by the proposed repair procedure increases the percentage of solved instances already by roughly 30%.
[Table]
A VRP instance X , which deﬁnes a single graph theoretical problem, is characterised by the following three entities :
• A set of N customers.
Each customer i , i ∈ { 1 , ..., N } has a demand q i to be satisﬁed and is represented by a feature vector
x cus i , which consists of its two-dimensional coordinates and the normalized demand q i /Q .
• A collection of M vehicles, each vehicle k ∈ { 1 , ..., M } disposing a capacity Q and characterised by the features x veh k .
The corresponding feature vector incorporates a local ( 1 /k ) and global ( k ) numbering, capacity Q and the aggregated load that must be distributed.
• The depot node at the 0 -index of the N customer vertices which is represented by a feature vector x dep comprising the depot coordinates as well as a centrality measure indicating the relative positioning of the depot w.r.t.
the customer locations.
C. V EHICLE C OSTS
Concerning the vehicle costs, Table 10 summarizes the different ﬁxed costs, while accounting for the different vehicles capacities.
[Table]
In real-world use cases it is indeed justiﬁed that larger vehicles, disposing more capacity, also inquire more costs if we assume that not only buying or leasing these vehicles plays a role, but also the required space (e.g. adequate garage) or maintenance costs.
C. L EAVE - ONE - OUT P OOLING
The max-pooling operation on the one hand enforces the interaction of learnable information between entities and on the other hand mitigates self-pooling, i.e. each entity element’s context vector consists only of other inter- and intra-group information, but not its own information.
When the contexts to be pooled for element r ∈ { 1 , . .
, l g } stem from its own entity, the leave-one-out pooling is performed in the Max Pooling Layer:
[Equation]
When on the other hand, the contexts for element r ∈ { 1 , . .
, l g } are pooled from another entity g (cid:48) that r does not belong to, regular pooling is performed:
[Equation]
C. P SEUDO -G REEDY D ECODING
The pseudo-greedy decoding described in section 4.2 transforms the doubly-stochastic probability tensor
ˆ Y into a binary assignment tensor ˆ Y ∗ , from which the predicted vehicle loads (required for the train loss) can be deduced.
Algorithm 2 below shows the decoding procedure.
Input: ˆ Y ∈ R M × N (cid:48) × N (cid:48)
Output: ˆ Y ∗ ∈ { 0 , 1 } M × N (cid:48) × N (cid:48) 1: ˆ Y ∗ ← { 0 } M × N (cid:48)
× N (cid:48) ; (cid:46) Initialize ˆ Y ∗ 2: ˆ Y c ← copy( ˆ Y ) ; (cid:46) Create a copy of ˆ Y for masking 3: for k ∈ M do 4: i curr ← arg max( ˆ Y k, 0 , : ) ; (cid:46) Assign "from-depot" customer for vehicle k 5:
ˆ Y ∗ k, 0 ,i curr ← 1 . 0 ; (cid:46)
Fill ˆ Y ∗ with the assigned start path 6:
ˆ Y c : , : ,i curr ← −∞ ; (cid:46) Mask assigned path for all vehicles 7: while i curr (cid:54) =
do 8:
i next ← arg max( ˆ Y ck,i curr , : ) ; (cid:46) Select outgoing node from i curr 9: ˆ Y ∗ k,i curr ,i next ← 1 . 0 ; (cid:46)
Fill ˆ Y ∗ with the assigned path 10: ˆ Y c : , : ,i curr ← −∞ ; (cid:46) Mask assigned path for all vehicles 11: ˆ Y c : , : , 0 ← ˆ Y : , : , 0 ; (cid:46) Reset "to depot" paths to original probabilities 12: i curr ← i next ; (cid:46) Reset index 13: end while 14: end for 15: return ˆ Y ∗
First the predicted binary assignment is initialised and a copy of the original probability tensor is created that can be used to mask the already visited customer nodes across vehicles and outgoing ﬂows.
After the starting customer of a vehicle k is determined ( i curr ) and stored in the output tensor ˆ
Y ∗ , we loop through vehicle k ’s path until the reset current vertex is again the depot, i.e. i curr = 0 .
This is successively done for all vehicles and in batches of training instances, such that the algorithm proceeds reasonably fast.
C.
[Equation]
RAIN L OSS
[Equation]
In a second step, after the optimal route direction has been determined, the vehicle permutation that renders the minimal loss is determined:
[Equation]
C. D ELINEATION FROM THE P ERMUTATION I NVARIANT P OOLING N ETWORK
There are ﬁve main components of the original Permutation Invariant Pooling Network that were extended considerably;
• The encodings are extentended; The customer entity is complemented by their demands, the depot encoding captures now the relative centrality of its location and the ﬂeet encoding is enriched by the total demand to be distributed.
• Priming weights for the depot and customer groups in the pooling mechanism have been extended to incorporate the outer sum of demands in conjunction with the distance matrix to reﬂect that not only locations but also the demands of customers plays a role.
• Most changes were introduced in the decoding process.
Instead of layering two simple 1D convolutions, we introduce an attention dot product to enhance the interaction of each vehicle and the edges to be allocated.
• The SoftAssign layers introduced by Kaempfer & Wolf (2018) seemed to be not as expressive as initially intended.
Empirically we show in the experiments that even though the SoftAssign layers are technically effective, the same result can be achieved with a simple softmax normalization instead.
• Lastly we amended the loss for training the model in order to train the model also to learn feasible assignments of customers on to the tours.
D E XPERIMENT D ETAILS D. H YPERPARAMETERS
The embedding size d m , the hidden dimension of the shared fully-connected layer and the number of pooling layers L is set to 256, 1024 and 8 respectively for the problem sizes of 20 and 50, due to GPU memory constraints we set d m = 128 for the problem size of 100.
The model is trained in roughlyepochs with a batch size of 128, 64 and 64 for the problem sizes 20, 50 and 100 respectively, using the Adam Optimizer (Kingma & Ba (2014)) with a learning rate of 10 − 4 .
D. N OTE ON B ASELINE R E - EVALUATION
We want to note some particularities across the re-evaluations to accompany the ﬁndings in Table 2.
First we note that the MDAM Model (Xin et al.
(2021)) did not yield the same performances when evaluated in batches of thousand instances and in a single-instance based setting.
This is due to the fact that the authors decided to evaluate their model in "training" mode during inference and consequently carried over batch-wise normalization effects 1 .
Albeit, this might signal wanted transductive effects during inference, we found it more appropriate to document the results without transduction effects, as also the incorporation of these effects were not states in the publication.
Concerning the NeuRewriter model (Chen & Tian (2019)) we only provide published results without the per-instance re-evaluation in Table 11, because even though we retrained the model according to the information in their supplementary materials, we were not able to reproduce the published results in a tractable amount of time.
In order to re-evaluate the NLNS (Hottung & Tierney (2020)) and reproduce the published results, a GPU-backed machine with at least 10 processing cores was needed (as was indicated and conﬁrmed by the authors).
For the per-instance evaluations, we employed the batch-evaluation version of NLNS while setting the batch size to 2.
The provided single instance evaluation method in Hottung & Tierney (2020) would require 191 seconds per instance, which again would be intractable to reproduce for the given test set of 10,000 instances.
The L2I approach (Lu et al.
(2020)) was not considered in this comparison as a single instance for the VRP100 requires 24 minutes (Tesla T4), which would lead to a total runtime of more than 160 days for the complete test set.
Generally, the re-evaluation of the baseline models and the evaluation of our own approach were done on a Tesla P100 or V100 machine (except for the NLNS approach, since we requiredprocessing cores - we re-evaluated this method on an A100 machine).
D. F ULL R ESULTS ON B ENCHMARK D ATASET
[Table]
D. E VALUATION ON MORE REALISTIC T EST I NSTANCES
In this section
[Table]
we provide generalization results for the test set provided in Kool et al. We compare the LKH method and DPDP (Kool et al.
to the generalization results that were achieved after training our model on uniformly -distributed data of graph size 50 only.
Table 12 shows that, even though our approach is trained on uniformly distributed VRP50 data
[Table]
it yields competitive results on the 10000 out of sample instances while being among the fastest methods
[Table]
[Table]