NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem
Authors: Liang Xin
Wen Song
Zhiguang Cao
Jie Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Deep Learning
Vehicle Routing Problems

Abstract: We present NeuroLKH, a novel algorithm that combines deep learning with the strong traditional heuristic Lin-Kernighan-Helsgaun (LKH) for solving Traveling Salesman Problem. Specifically, we train a Sparse Graph Network (SGN) with supervised learning for edge scores and unsupervised learning for node penalties, both of which are critical for improving the performance of LKH. Based on the output of SGN, NeuroLKH creates the edge candidate set and transforms edge distances to guide the searching process of LKH. Extensive experiments firmly demonstrate that, by training one model on a wide range of problem sizes, NeuroLKH significantly outperforms LKH and generalizes well to much larger sizes. Also, we show that NeuroLKH can be applied to other routing problems such as Capacitated Vehicle Routing Problem (CVRP), Pickup and Delivery Problem (PDP), and CVRP with Time Windows (CVRPTW).

NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsganr Heuristic for Solving the Traveling Salesman Problem
Liang Xin Nanyang Technological University Singapore XINL00030e wwt.tt..ee. edu. sg
Wen Song Shandong Unviersity Qingdao, China wensongdemail sdu. edu. cn
Zhiguang Cao"
Jie Zhang
Singapore Institute of Manufacturing Technology Singapore zhiguuaggaa@@otllook.comm com
Nanyang Technological University Singapore Schang@ntu.edu..gg
Abstract
We present NeuroLKH, novel algorithm that combines deep learning with the strong traditional heuristic Lin Sin--eemighhn--eee Helsgaun (LKH) for solving Traveling Salesman Problem. Specifically, we train Sparse Graph Network (SGN) with supervised learning for edge scores and unsupervised learning for node penalties, both of which critical for improving the performance of LKH. Based the output of SGN, NeuroLKH creates the edge candidate set and transforms edge distances guide the searching process of LKH. Extensive experiments firmly demonstrate that, by training one model on a wide range of problem sizes, NeuroLKH significantly outperforms LKH and generalizes well much larger sizes. Also, we show that KeuroLKH can be applied to other routing problems such as Capacitated Vehicle Routing Problem (CVRP). Pickup and Delivery Problem (PDP), and CVRP with Time Windows (CVRPTW).
Introduction
Traveling Salesman Problem (TSP) an important NP-hard Combinatorial Optimization Problem with extensive industrial applications in various domains. Exact methods have the exponential worst- case computational complexity, which renders them impractical for solving large-scat scale problems in reality, even for highly optimized solvers such Concorde. In contrast, although lacking optimality guarantees and non-trivial theoretical analysis, heuristic solvers search for near-optimal solutions with much lower complexity. They are usually desirable for real-life applications where statistically better performance is the goal.
Traditional heuristic methods are manually designed based on expert knowledge which is usually human- iinnaanniterpeetable. However, supported by the recent development of deep learning technology, modern methods train powerful deep neural networks to learn the complex patterns from the TSP instances generated from some specific distributions B2 of deep learning models for solving TSP are constantly improved by these works, which unfortunately are still worse than the strong traditional heuristic solver and generally limited to relatively small problem sizes.
33 35). The performances
"Zhiguang Cao is the corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

We believe that learning- based methods should be combined with strong traditional heuristic algo- rithms, which is also suggested by (2). In such a way, while learning the complex patterns from data samples, the efficient heuristics highly optimized by researchers for decades can be effectively utilized, especially for problems such as TSP which are well-studied due to their importance.
The Lin-Kernige Mhermighan-Helggaae (LKH) algorithm [12 13 is generally considered as very strong heuristic for solving TSP, which developed based on the Lin- Ker-Kenng (LK) heuristic (25). LKH iteratively searches for X-opt moves to improve the existing solution where À edges of the tour are exchanged for another edges to form a shorter tour. To save the searching time, the edges add are limited to a small edge candidate set, which is created before search. One of the most significant contributions of LKH is to generate the edge candidate set based on Minimum Spanning Tree, rather than using the nearest neighbor method in the LK heuristic. Furthermore. LKH applies penalty values to the nodes which are iteratively optimized using subgradient optimization (will be detailed in Section B). The optimized node penalties are used by LKH to transform the edge distances for the X-opt searching process and improve the quality of edge candidate sets, both of which help find better solutions.
However, the edge candidate set generation in LKH is still guided by hand-crafted rules, which could limit the quality of edge candidates and hence the search performance. Moreover, the iterative optimization of node penalties is time-come timeconsuminn, especially for large-scale problems. To address these limitations, we propose NeuroLKH, a novel eeerning-based method featuring Sparse Graph Network (SGN) combined with the highly efficient X-opt local search of LKH. SGN outputs the edge scores and node penalties simultaneously, which are trained by supervised learning and unsupervised learning, respectively. NeuroL transforms the edge distances based on the node penalties learned inductively from training instances, instead of performing iterative optimization for each instance, therefore saving a significant amount of time. More importantly, at the same time the edge scores are used to create the edge candidate set, leading to substantially better sets than those created by LKH. NeuroLKH trains one single network on TSP instances across a wide range of sizes and generalizes well to substantially larger problems with minutes of unsupervised offline fine-tuning to adjust the node penalty scales for different sizes.
Same as existing works on deep learning models for solving TSP, NeuroLKH aims to learn complex patterns from data samples to find better solutions for instances following specific distributions. Following the evaluation process in these works, we perform extensive experiments. Results show that NeuroLKH improves the baseline algorithms by large margins, not only across the wide range of training problem sizes, but also on much larger problem sizes not used in training. Furthermore, NeuroLKH trained with instances of relatively simple distributions generalizes well to traditional benchmark with various node distributions such the TSPLIB 277 Also, we show that NeuroL can be applied to guide the extension of LKH 14] for more complicated routing problems such as the Capacitated Vehicle Routing Problem (CVRP), Pickup and Delivery Problem (PDP) and CVRP with Time Windows (CVRPTW), using generated test datasets and traditional benchmarks [28] 30
2 Related works
Till now, for routing problems such as TSP, most works focus on learning construction heuristics, where deep neural networks are trained to sequentially select the nodes to visit with supervised
learning B2 15
or reinforcement learning [ 26
21 22]. Similarly, networks are trained to pick
5 researchers employ deep learning models to learn the actions for improving existing solutions, such picking regions and rules selecting nodes for the 2-opt heuristic. However, the performance of these works is still quite far from the strong non-learning heuristics such as LKH. In addition, they focus only on relatively small-sized problems (up to hundreds of nodes).
edges in 18 20
In another line of works 4 33
16
A recent work [8] generalizes network pre-trained fixed-size small graphs to solve larger size problems by sampling small -raphs to infer and merging the results. This interesting idea can be applied to very large graphs, however, the performance is still inferior to LKH and deteriorates rapidly with the increase of problem size.
In a concurrent work 36 VSR-LKH method is proposed which also applies learning method in combination with LKH. However, very different from our method, VSR-LKH applies traditional reinforcement learning during the searching process for each instance, instead of learning patterns for

---

TSP Ineww9o
Repest for Tries
TSP Inelarce
Kiarse
SON Enooder
Grigh c
Tarta Beander
Enge Booado
r85a Eostion
Optinsstioo
WhEW
Proeees for Aop Exchange
Swesshil Mirenann Scarning Tree
Kort Peralies
Frlye Storee
Pranatar Disaanee
Create Dage
Conargence
Cantidure Set
Edja Desance
Figure 1: NeuroLKH algorithm and the original LKH algorithm.
a class of instances. Moreover, VSR-LKH aims to guide the decision on edge selections within the edge candidate set, which is generated using the original procedure of LKH. NeuroLKH significantly outperforms VSR-LKH by large margins in all the settings of our experiments on testing instances following the training distributions, especially when the time limits are short. Even more impressively, NeuroLKH achieves performance similar to VSR-LKH on traditional benchmark TSPLIB (27 with various node distributions, which are very different from the training distributions for Ke.roLKH.
3 Preliminaries: LKH algorithm
The Lin-Keenighan-Heiee
(LKH) algorithm 13 is local optimization algorithm developed based on the X-opt move [24. where edges in the current tour are exchanged by another set of edges to achieve a shorter tour. While solving one instance, the LKH algorithm can conduct multiple trials to find better solutions. In each trial, starting from randomly initialized tour, iteratively searches for X-opt exchanges that improve the tour, until no such exchanges can be found. In each iteration, the X-opt exchanges are searched in the ascending order of variable and the tour will be replaced once exchange is found to reduce the tour distance.
One central rule that the X-opt searching process restricted and directed by an edge candidate set, which is created before search based on the a-measure using sensitivity analysis of the Minimum Spanning Tree. Here we briefly introduce the related concepts. A TSP graph can be viewed as an undirected graph G (V. E) with V as the set of [V] nodes and E as the set of edges weighted by distances. A spanning tree of G is connected graph with V
edges from G' and no cycles where any pair of nodes is connected by path. A 1-tree of G is spanning tree for the graph of node set V((1) combined with two edges in connected to node 1, an arbitrary special node in V. A minimum 1-tree is the 1-tree with minimum length. The a-measure of an edge (i,j) E for graph G is defined as ji IL(T (i, ()) L(T), where L((T) is the length of Minimum 1-Tree T and L(T't )) is the length of Minimum 1-Tree I (i. required to include the edge (i. The a-measure of an edge can be viewed as the extra length of the Minimum 1-Tree to include this edge.
The edge candidate set consists of the k edges with the smallest a-measures connected to each node (k as default). During the X-opt searching process, the edges to be included into the new tour are limited to the edges in this candidate set, and edges with smaller a-measures will have higher priorities to be searched over. Therefore this candidate set not only restricts but also directs the search.
Moreover, the quality of a-measures can be improved significantly by a subgradient optimization method. If we add penalty m, each node and transform the original distance si.j of the edge (i,j) to new distance C, as Crj = = + ग, + Ty, the optimal tour for the TSP will stay the same but the Minimum 1-Tree usually will change. Because by definition, a Minimum 1-Tree with node degrees all equal to 2 is an optimal solution for the corresponding TSP instance. With the length of Minimum 1-Tree resulting from the penalty π = (TI, ) as L(T,), w(x) LLTT), 22I, is a lower bound of the optimal tour distance for the original TSP instance. LKH applies subgradient optimization [I] to iteratively maximize this lower bound for multiple steps until convergence by applying x π + t (d' - 2) at step T, where t" is the scalar step size, d' is the vector of node degrees the Minimum 1-Tree with penalty π². Therefore, the node degrees are pushed towards 2. The a-measures after this optimization will substantially improve the quality of edge candidate set. Furthermore, the transformed edge distance 0,j after this optimization helps find better solutions when used during the searching process for X-opt exchanges.
3

---

4
The proposed NeuroLKH algorithm
The subgradient optimization in LKH can substantially improve the quality of edge candidate sets based on the a-measures, and transform the edge distances effectively to achieve reasonably good performance. However, it still has major limitations as the optimization process is over one instance iteratively until convergence, which costs a large amount of time, especially for large-scale problems. Moreover, even after subgradient optimization, some critical patterns could be missed by the relatively straightforward sensitivity analysis of spanning tree. Therefore, the quality of edge candidate set could be further improved by large margins, which will in turn improve the overall performance.
We propose the Ke algorithm, which employs a Sparse Graph Network to learn the complex patterns associated with the TSP instances generated from distribution. Concretely, the network will learn the edge scores and node penalties simultaneously with a multi-task training process. The edge scores are trained with supervised learning for creating the edge candidate set, while the node penalties are trained with unsupervised learning for transforming the edge distances. The architecture of KeuroLKH is presented in Figure] I along with the original LKH algorithm. We will detail the Sparse Graph Network, the training process and the proposed NeuroLKH algorithm in the following.
4.1 Sparse Graph Network
For the Sparse Graph Network (SGN), we format the TSP instance as a sparse directed graph G* = -V, (V) containing the node set V and a sparse edge set F which only includes the y shortest edges pointed from each node, as shown in the leftmost green box in Figure [ where the circles represent the nodes and the diamonds represent the directed edges. Sparsification of the graph is crucial for effectively training the deep learning model on large TSP instances and generalizing to even larger sizes. Note that edge (i.j) belongs to E* does not necessarily mean that the opposite- direction edge (j. belongs to E*. The node inputs Ze RR2 are the node coordinates and the edge inputs xe E R are the edge distances. Though we focus on 2-dimensional TSP with Euclidean distance as the other deep learning literature like (2I]. the model can be applied to other kinds of TSP.
The SGN consists of 1) one encoder embedding the edge and node inputs into the corresponding feature vectors, and 2) two decoders for the edge scores and node penalties, respectively.
Encoder. The encoder first linearly projects the node inputs zy and the edge inputs x. into feature vectors v E IRD and eh, E ID respectively, where D is the feature dimension, i E V and (1,5) E E Then the node and edge features are embedded with L Sparse Graph Convolutional Layers, which are defined formally as follows:
attn's exp(WW.
Σ
(1)
(..))E.
of = o.-1 + ReLU(BN
Σ
attm 2 oWi
(2)
(.EE
[W WWp,
(f, E E otherwise
1/.
(3)
< = + ReLU W/o + W + +r,,)),
(4)
where @ and @ represent the element-wise multiplication and the element-wise division, respectively; I == 2..,... L is the layer index; W W W W W/. W W e RDD and p' E D are trainable parameters; Eqs. (2) and 4 consist of a Skip-Connection layer [0] and a Batch Normalization layer
17 in each; and the idea of element-wine attention in Eq. (1) is adopted from (3) As the input graph G* is directed and sparse, edges with different directions are embedded separately. But obviously the embedding of edge (i, should benefit from knowing whether its opposite-drrection counterpart (j. i) is also in the graph and the information of (j. i). which motivates our design of Eqs. (3) and (4).
Decoders. The edge decoder takes the edge embeddings et' from the encoder and embeds them with two layers of linear projection followed by ReLU activation into c/.-- Then the edge scores Br calculated follows: Bi.s Ejameer- exp(Weef.))
exp(Waef.))
(5)
4

---

Algorithm NeuroLKH Algorithm
Input: TSP instance, number of trials K
Output: TSP solution BestTow our
1: Convert the TSP instance to SGN imput G", Tv. Ze
2: Calculate the edge scores B,, and the node penalties Ti with Eqs. (5) and
3: BigggDDDstonn---TansoomEgggeeee
4: PeggCCnnttitSSeeeerrraeeCeenaned
5: BestT TerTTour=KHHSS TiaarLLKKSaaccchiinriiitaaaaaa Distancce, EdgeCandidateSet, K)
6: return BestT our
Similarly, the node decoder first embeds the node embeddings uf with two layers of linear projection and ReLU activation into uf- Then the node penalties H, are calculated as follows:
H C Cttanh(==':]
(6)
where Wg. W E Dx1 trainable parameters; C 10 is used to keep the node penalties in the range of -00 10].
4.2 Training process
We train the network to learn the edge scores with supervised learning. And the edge loss La is detailed as follows:
Lg = VIVI
E (I((1,)) E E3) log + +1((1,) $ log(1 -1,5)), (KJ)EE
where E {(i,)) 5i) in the optimal tour). Effectively, increase the edge scores 3 if the edge (i.5) belongs to the optimal tour and decrease them otherwise.
The node penalties are trained by unsupervised learning. Similar to the goal of subgradient opti- mization LKH, we are trying to transform the Minimum 1-Tree generated from the TSP graph G closer to tour where all nodes have a degree of 2. An important distinction from LKH is that we are learning the patterns for a class of TSP instances following a distribution, instead of optimizing the penalties for a specific TSP instance. The node loss C is detailed follows:
1 Σ (di(T)) (d)( REV
IV -
Cx -
- 2)
(8)
where (TT) is the degree of node in the Minimum 1-Tree Tx induced with penalty π (T....., T The penalties are increased for nodes with degrees larger than and decreased for nodes with smaller degrees. The SGN is trained for the task of outputting the edge scores and node penalties simultaneously with the loss function L La + T/,,,, where 1/= is the coefficient for balancing the two losses.
4.3 NeuroLKH algorithm
The process of using NeuroLKH to solve instance is shown in Algorithm I Firstly, the TSP instance is converted to sparse directed graph G Then the SGN encoder embeds the nodes and edges in G into feature embeddings, based on which the decoders output the node penalties and edge scores B. Afterwards, Neurol. creates powerful edge candidate set and transforms the distance of each edge effectively, which further guides NeuroLKH to conduct multiple LKH trials to find good solutions. We detail each part as follows.
Transform Edge Distance. Based on the node penalties т;, the original edge distances sty transformed into new distances C,, = Si. + म; + Tj, which will be used in the search process. With such transformation, the optimal solution tour will stay the same. And the tour distance calculated with the transformed edge distances will be subtracted by 2 Eiev क; to restore the tour distance for the original TSP.
Create Edge Candidate Set. For each node V. the edge scores Bi,, are sorted for (i, E F* and the edges with the top-k largest scores are included in the edge candidate set. Edges with larger
5

---

scores have higher priorities in the candidate set, which will be tried first for adding in the exchange during the LKH search process. Note that neither the original LKH nor NeuroLKH can guarantee all the edges in the optimal tour to be included in the edge candidate set. However, optimal solutions are still likely to be found during the multiple trials.
LKH Searching Trials. To solve one TSP instance, LKH conducts multiple trials to find better solutions. In each trial, one tour is initialized randomly, and iterations of LKH search are conducted for the A-opt exchanges until the tour can no longer be improved by such exchanges. In each iteration, LKH searches in the ascending order of X for X-opt exchanges reduce tour length, which will be applied once found.
Based the trained SGN network, NeuroLKH infers the edge distance transformation and candidate set to guide the L trials, which is done by performing forward calculation through the model. This is much faster than the corresponding procedure in the original LKH, which employs subgradi- ent optimization on each instance iteratively until convergence and is apparently time-consuming especially for large-scale problems. More importantly, rather than using the hand-crafted rules based on sensitivity analysis in the original LKH, NeuroL KH learns to create edge candidate set of much higher quality with the powerful deep model, leading to significantly better performance.
5 Experiments
In this section, we conduct extensive experiments on TSP with various sizes and show the effective performance of KeuroLKH compared to the baseline algorithms. Our code is publicly available
Dataset distribution. Closely following the existing works such as [21], we experiment with the 2 2-dimensional TSP instances in the Euclidean distance space where both coordinates of each node are generated independently from a unit uniform distribution. We train only one network using TSP instances ranging from 101 to 500 nodes. Since the amount of supervision and feedback during training is linearly related to the number of nodes. We generate 500000/[V] instances for each size [V] in the training dataset, resulting in approximately 780000 instances in total. Therefore the amounts of supervision and feedback are kept similar across different sizes. We use Concorde to get the optimal edges E for the supervised training of edge scores. For testing, we generate 1000 instances for each testing problem size.
Hyperparameters. We choose the number of directed edges pointed from one node in the sparse edge set E- as y = 20, which results in only 0.01% % of the edges in the optimal tours missed in E* for the training dataset. We also conduct experiments to justify this choice in Appendix Section A. The hidden dimension is set D 128 in the network with L 30 Sparse Graph Convolutional Layers. The node penalty coefficient in the loss function is 7 = 1. The network is trained by Adam Optimizer [19] with learning rate of 0.0001 for 16 epochs, which takes approximately days. The deep learning models are trained and evaluated with one RTX-20801 GPU. The other parts of experiments without deep models for NeuroL and other baselines are conducted with random seed 1234 on an Intel(R) Core(T'M) 10-10940X CPU unless stated otherwise. Hyperparameters for the LKH searching process are consistent with the example script for TSP given by LKH available online and those used in [36].
5.1 Comparative study TSP
Here, we compare NeuroLKH with the original LKH algorithm 11 and the recently proposed VSR-LKH algorithm [36]. We do not compare with other deep learning based methods here because their performances are rather inferior to LKH, and most of them can hardly generalize to problems with more than 100 nodes. One exception the method in g which tested on large problems but the performances are still far worse than LKH.
All algorithms are run once for each testing instance as we find running multiple times only provides very marginal improvement. For each testing problem size, we run the original LKH for 1, 10, 100, and 1000 trials, and record the total amounts of time in solving the 1000 instances. Then we impose
"https//gitthh.comm// hwtp:///ww.....oooocc//////oo//e
P hhtps://www.math http:://www.mnahuuuuvarrrooooooe www.tteelooo..ce/p//ooooooomoooe http://kkraa.n TEeeee/euueeeeeec///HHHHHHLLLLL

---

Table 1: Comparative results on training sizes
[V]
V
200
[
500
Method
Time(s)
Obj
Gap(Xsco)
Time(s)
Obj
Gap(%oc)
Timels)
Obj
Gap(%cc)
Concorde
207
*7.753246
0.000
1072
+10.701303
0,000
17022
*16.541830
0,000
LKH trial)
7.755071
2.353
10.707043
5.364
16.556733
9.009
VSR-LKH
33
7.754980
2.236
80
10.706739
5,080
338
16.557297
9.350
KH
7.753332
0.111
10.701873
0.533
16.543197
0.826
LKII (10 trials)
7.754177
1.200
10.703724
2.263
16.548017
3,740
VSR-IKH
43
7.754184
1.209
10.703997
2.518
445
16.549591
4.692
NeuroLKH
7,753311
0.083
10.701623
0.299
16,542880
0.634
LKH (100 trials)
7,753450
0.263
10.701755
0.423
16.543707
1.134
VSR-LKII
127
7.753407
0.207
368
10.701687
0.359
1147
16.543085
0.759
Ne
7.753270
0.030
10.701381
0.073
16,542163
0.201
LKH (1000 mials)
7.753254
0.010
10.701351
0,045
16.542125
0.178
VSR-LKH
938
7.753322
0.097
2805
10,701336
0.031
7527
16.541934
0.063
Ne
7.753247
0.000
10.701303
0.000
16.541847
0.010
the same amounts of time as time limits NeuroLKH and VSR-LKH for solving the same 1000 instances for fair comparison. Note that for NeuroLKH, the solving time is the summation of the inference time of SGN on GPU and LKH searching time on CPU. In the following tables, for each size and time limit, we report the average performance (tour distance) and the total solving time for the 1000 testing instances.
Comparison on training sizes. In Table [ we report the performances of LKH, VSR-LKH and NeuroLKH on three testing datasets with 100, 200 and 500 nodes, which are within the size range of instances used in training. Note that we train only one SGN Network on a wide range of problem sizes and here we use these three sizes to demonstrate the testing performances. We also use the exact solver Concorde on these instances to obtain the optimal solutions and compute the optimality gap for each method. As shown in this table, it is clear that KeuroLKH outperforms both LKH and VSR-LKH significantly and consistently across different problem sizes and with different time limits. Notably, the optimality gaps are reduced by at least an order of magnitude for most of the cases, which is a significant improvement.
Generalization analysis on larger sizes. We further show the generalization ability of NeuroLKH on much larger graph sizes of 1000, 2000 and 5000 nodes. Note that while the edge scores in SGN generalize well without any modification, it is hard for the node penalties to directly generalize. This is because they are trained unsupervisedly and SGN does not have any knowledge about how to penalize the nodes for larger TSP instances. Nevertheless, this could be resolved by a simple fine-tuning step. As the learned node embeddings are very powerful, we only fine-tune the very small amount of parameters in the SGN node decoder and keep the other parameters fixed. Specifically, for each of the large sizes, we fine-tune the node decoder for 100 iterations with batch size of 5000/ VI, which only takes less than one minute for each size of 1000, 2000 and 5000. This fast fine-tuning process is for TSPs of one size generated from the distribution instead of specific instances, and may be viewed as adjusting the scale of penalties for large sizes. The generalization results are summarized in Table 2 Note that we do not run Concorde here due to the prohibitively long running time, and the gaps are with respect to the best value found by all methods. Clearly, Ne generalizes well to substantially larger problem sizes and the improvement of NeuroLKH over baselines is significant and consistent across all the settings.
Further discussion. The inference time of SGN in NeuroI.Kt for the 1000 instances of 100, 200, 500, 1000, 2000 and 5000 nodes 3s, 6s, 16s, 33s, 63s and 208s, which is approximately linear with the number of nodes [V]. In contrast, the subgradient optimization in LKH and VSR-LKH needs 20s. 51s, 266s, 1028s, 4501s and 38970s, which grows superlinearly with [V| and is much longer than SGN inference, especially for large-scale problems. For NeuroLKH, the saved time is used to conduct more trials, which effectively helps to find better solutions. This effect is more salient with short time limit. Meanwhile, the number of trials is also small for short time limit and the algorithm only searches a small number of solutions, in which case the guidance of edge candidate set is more important. Due to these two reasons, the improvement of Neurol.KI over baselines is particularly substantial for short time limit. This is a desirable property especially for time-critical applications and solving large-scale problems, for which large numbers of trials are not feasible.
7

---

Table 2: Comparative results on generalization sizes
[V]
1000
|
2000
[
5000
Method
Timets)
Obj
Gap(%)se)
Time(s)
Obj
Gap(%
Time(s)
Obj
Gap(Xees)
LKH (1 trial)
23.155916
10:593
32,483851
11.264
51,025519
12.284
VSR-LKH
1183
23.154946
10.173
4843
32.485551
11,788
40048
$1.025539
12.288
Neurol KH
23.133494
0.899
32,449752
0.755
50.965382
0.484
LKH trials)
23.143435
5.197
32,466953
6.056
50.998721
7.026
VSR-I.KH
1414
23. 23.143347
5.159
5322
32.467997
6.377
41523
51 51.000093
7.295
NH
23.133066
0.714
32.449519
0.683
50,965219
0.482
LKH (100 trials)
23.135427
1.735
32,455454
2.512
50.976677
2.700
VSR-LKH
2567
23.134426
1.302
7371
32,454427
2.195
47884
50.979317
3.218
NH
23.132258
0.365
32.448666
0.420
51,.944777
0.345
LKH (1000 trials)
23.132216
0.347
32.448954
0.509
50.965233
0,455
VSR-LKH
12884
23.1316588
0.105
25613
32,447953
0.200
103885
50.965300
0.468
NH
23.131414
0.000
32.447300
0.000
50.962916
0.000
In Figure 2 we plot the performance of the LKH, VSR-LKH and NeuroLKH algorithms for solving the testing datasets with different numbers of nodes against different running time to visualize the improvement process (the resulting objective values after each trial). The time limits are set to the longest ones used Table D and Table 2 which are the running time of LKH with 1000 trials. Clearly, NeuroLKH outperforms both LKH and VSR-LKH significantly and consistently across different problem sizes and with different time limits. In particular, NeuroLKH is superior as not only reaches good solutions fast but also converges to better solutions eventually. With the same performance (i.e. objective value), NeuroL considerably reduces the computational time. We can also conclude that when the time limit is short, the improvement of NeuroLKH over baselines is particularly substantial. In addition, we show that the subgradient optimization is necessary for LKH and VSR-LKH. As exhibited in Figure 2 the performances of both LKH and VSR-LKH are much worse without subgradient optimization (w/o SO). More impressively, even ignoring the preprocessing time (IPT) used for subgradient optimization (pertaining to LKH and VSR-LKH) and Sparse Graph Network inferring (pertaining to Neurol HH), NeuroL still outstrips both LKH and VSR-LKH. Note that this comparison is unfair for NeuroLKH LKH and VSR-LKH consume much longer preprocessing time which is unavoidable.
For the results reported in Table I and Table 2 almost all the improvements of NeuroLKH over LKH and VSR-LKH on different sizes and with different time limits are statistically significant with confidence levels larger than 99%. The only one exception is the performance TSP with 100 nodes and the running time of LKH with 1000 trials where the confidence levels are 90.5% and 97.6% for the improvements, respectively.
In the Appendix Section A, we also show that NeuroL.HH substantially outperforms other deep 33
learning based methods 21 20
I8
15
g 22
Generalization to TSPLIB benchmark. Besides generalization larger sizes, generalization to different distributions remains a crucial challenge for deep learning based methods in existing works. The TSPLIB benchmark contains instances with various node distributions, making it extremely hard for such methods. We test on all the 72 TSPLIB instances with Euclidean distances and less than 10000 nodes. The number of trials is set to be the number of nodes and the algorithms are run 10 times for each instance following the convention for TSPLIB in 112 36). With the various unknown node distributions, we do not fine-tune the model for the node penalties and only use the edge scores in NeuroI.KH. For the 24 instances labeled as hard in 36). which the original LKH fails to solve optimally during at least one of the 10 runs, NeuroLKH trained with uniformly distributed data is able to find optimal solutions 6. 13 times average, which is much better than LKH (3.75 times). As an active learning method, VSR-LKH finds optimal solutions 6.42 times on average, slightly better than KeuroLKH. While NeuroLKH improves the results on most hard instances, it could generalize poorly on instances with certain special patterns such as where most nodes are located along several horizontal lines, making it fail to solve 11 of the 48 easy instances optimally for some runs.
With the same training dataset size, we trained another model NeuroLLH_M, using a mixture of instances with uniformly distributed nodes, clustered nodes with 3-8 clusters, half uniform and half clustered nodes following 30) KeuroL.H M finds optimal solutions 6.79 times on average for the hard instances and fails to solve only 5 easy instances optimally for some runs, better than the
8

---

7,350
VOHLUH
UK4
16.556
10.706
10.554
NeurLUH
Oojectev Vita
77745
10.705
10.350
7,7422
NeurSLLOI IFT
10.704
Neurr KG1
VSR UKH
10.350
10.340
VSR NeursLIOI IFT
2,7538
19:546
19.564
7:7335
19.542
7.7533
200
400
600
8000
563
1000
-500
2000
2500
19.540
1050
2000
3000
4000 500D 9C00 7010
(a) TSP with 100 nodes
(b) TSP with 200 nodes
(c) TSP with 500 nodes
23.155
USH
22.450
VSR-UH
51.020
UKH
Obeect Valle
21,150
32.475
$1.910
23.145
UK4
$2.41
UK+
31.000
(PT
VSR-UI PT
NouISLEH IPT
32.405
NoursLEH IFT
50.990
VSR-UII PT
NourSLEH IFT
23.140
12.400
50,900
23.335
50.800
2000
1000
6200
1000
10000 12000
5000
10000
15000
20000
2500
20000
60000
C0020
00200
(d) TSP with 1000 nodes
(e) TSP with 2000 nodes
(i) TSP with 5000 nodes
Figure 2: Performances of LKH, VSR-LKH and NeuroLKH for solving TSP with different sizes
against different running time
NeuroLKH trained with only uniformly distributed instances. For all the 72 instances, NeuroLKHL KH_M finds optimal solutions 8. 7. times on average, which is much better than LKH (7.92 times) but slightly worse than VSR-LKH (8.78 times). Detailed results of each instance are listed in the Appendix Section B.
5.2 Experiments on other routing problems
Finally, we show that NeuroLKH can be easily extended solve much more complicated routing problems such as the Capacitated Vehicle Routing Problem (CVRP), the Pickup and Delivery Problem (PDP) and CVRP with Time Windows (CVRPTW). We briefly introduce the problems in the Appendix Section C. Different from TSP, the node penalties do not apply to these problems. Therefore, KE only learns the edge candidate set. As these three problems are very hard to solve and the optimal solutions are not available in reasonable amount of time, we use LKH with 10000 trials to get solutions as training labels. The demands, capacities, starts and ends in the time windows taken node inputs along with the coordinates. For PDP, we add connections between each pair of pickup and delivery nodes and assign weight matrices for these connections in Eq, (2). For PDP and CVRPTW, the edge directions affect the tour feasibility therefore the model learns the in-direction edge scores and out-direction edge scores for each node with Eq. (5).
The node coordinates are also generated uniformly from the unit square for all three problems, following [2] 23 For CVRP, the demands of customers are generated uniformly from integers [1.9) with the capacity fixed as 40 + 0.1 x [V], compatible with the largest CVRP (100 nodes) studied 21). For CVRPTW, we use the same way to generate demands, capacity, serving time and time windows as (7). A training dataset for CVRP with 101-500 nodes and 120000/IV/ | instances for each size (about 180000 in total) is used to train the SGN for 10 epochs. PDP and CVRPTW are harder to solve therefore we each size. The other hyperparameters in SGN are the same as TSP and those for LKH searching process are consistent with example scripts given by LKH for CVRP, PDP and CVRPTW (with the SPECIAL hyperparameter).
training dataset with 41-200 nodes and 240000/ V instances for
In Table 3. we show the performance of KE and the original LKH on testing datasets with 1000 instances for the smallest and largest graph sizes (number of customers) used in training as well as much larger generalization size. We use the solving time of LKH with 100, 1000, 10000 trials as the time limits. For 100 trials, both methods fail to find feasible solutions for less than 1% of the PDP and CVRPTW test instances with 300 nodes. Whenever this happens, push the infeasible visits to

---

Table Comparative results for other routing problems
Method
Time(s)
Obj
Gap(S)
Time(s)
Obj
Gap(%)
Time(s)
Obj
Gap(%)
[V| 100
IV - 500
Generalization [V] 1000
LKH (100 trials)
15.8363
1.675
42.1621
5,394
58.1372
9.750
NeuroLKH
485
15.7770
1,295
2043
41.7311
4.316
4607
56.6469
6.937
CVRP
I.KH (1000 trials)
15.6483
0.468
40,6103
1.515
54.3412
2.584
KeuroLKH
4520
15.6295
0.348
15812
40.4974
1.233
30133
54.0499
2.034
I.KH (10000 trials)
15.5823
0.044
40.0670
0.157
53, 13.1093
0.255
45435
166875
319368
NeuroLKH
15.5754
0.000
40.0043
0.000
$2.9723
0.000
[V] 40
[V 200
Generalization [V]
300
LKH (100 trials)
6.2495
0.819
13.8390
5.535
17.0913
6.916
115
2832
7939
NeuroLKH
6,2241
0.409
13.6246
3,899
16.7867
5,011
PDP
LKH (1000 trials)
6.2088
0.163
13.2850
1.310
16.2447
1.620
KH
845
6,2041
0.087
21216
13.2443
0.999
55643
16.1857
1.251
LKH (10000 trials)
6.1998
0.018
13.1387
0.194
16.0119
0.163
7989
195220
515377
NeuroLKH
6,1988
0.000
13.1132
0.000
15.9857
0.000
V 40
IV = 2000
Gineralization [V]
300
LKH (100 triuls)
9.3051
1.081
26.1757
7.124
34.2301
8.798
CVRPTW
Neurol KH
147
9.26006
0.597
813
25.4000
3.949
1746
32,9676
4.786
LKH (1000 trials)
9.2276
0.239
24.9770
2.218
32.2671
2.559
1017
4525
7820
NeuroLKH
9.2207
0.164
24.7857
1.435
32.0224
1.781
LKH (10000 trials)
9.2073
0.018
24.5338
0,405
31.5719
0.350
NeuroLKH
9624
9.2056
0.000
45509
24.4350
0.000
75481
31.4620
0.000
the end to get feasible solutions. The inferring time of SGN is 1s, 3s, 7s 10s, 19s and 40s in total for the 1000 instances in the testing datasets with 40, 100, 200, 300, 500 and 1000 nodes, which is a tiny fraction compared the LKH searching process. As shown in Table NeuroLKH significantly improves the solution quality compared with the original LKH which is very strong heuristic solver for all three problems, showing its potential in handling various types of routing problems.
Performance on traditional benchmarks. To show the effectiveness of Neurol KH on complicated routing problems with various distributions, we perform experiments CVRPLIB 30 and Solomon 28 benchmark datasets. CVRPLIB [30 contains various sized CVRP instances with a combination of depot positioning, 3 customer positioning and 7 demand distributions. Solomon benchmark
28 contains CVRPTW instances with 100 customers and various distributions of time windows.
We detail the benchmarks, the training datasets and the results for each instance in the Appendix Section D. In summary, tested on the 43 instances with 100-300 nodes in CVRPLIB (30], NeuroLKH improves the average performances on 38, 38 and 31 instances when the time limits are set to the time of LKH with 100, 1000 and 10000 trials, respectively. On the 11 Solomon R2-type instances, NeuroLKH outperforms LKH almost consistently with all the settings (32 out of the 33).
6 Conclusion
In this paper, we propose an algorithm utilizing the great power of deep learning models combine with strong heuristic for TSP. Specifically, one Sparse Graph Network is trained to predict the edge scores and the node penalties for generating the edge candidate set and transforming the edge distances, respectively. As shown in the extensive experiments, the improvement of Neurol.KI over baseline algorithms within different time limits is consistent and significant. And NeuroLKH generalizes well to instances with much larger graph sizes than training sizes and traditional benchmarks with various node distributions. Also, we use CVRP, PDP and CVRPTW demonstrate that KeuroLKH effectively applies to other routing problems. NeuroL can effectively learn the routing patterns for TSP which generalize well to much larger sizes and different distributions of nodes. However, for other complicated routing problems such as CVRP and CVRPTW, although NeuroLKH generalizes well to larger sizes, it is hard to directly generalize to other distributions of demands and time windows without training, which is limitation of NeuroI.HH and is left for future research. In addition, NeuroL.HH can be further combined with other learning based techniques such as sparsifying the TSP graph [29] and other strong traditional algorithms such the Hybrid Genetic Search [I].
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the A*STAR Cyber-Physicee Production System (CPPS)- Towards Con- textual and Intelligent Response Research Program, under the RIE2020 IAF-PP Grant A 119C1a0018 88 and Model Factory SIM'Tech, in part by the National Natural Science Foundation of China under Grant 61 61803 104 and Grant 62 62102228, and in part by the Young Scholar Future Plan of Shandong University under Grant 62420089964188.
References
[1] Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio. Neural combinatorial optimiza- tion with reinforcement learning. In Proceedings of International Conference on Learning Representations (ICLR)., 2016.
[2] Y. Bengio, A. Lodi, and A. Prouvost. Machine learning for combinatorial optimization: a methodological tour d' horizon. European Journal of Operational Research, 2020.
[3] X. Bresson and T. Laurent. Residual gated graph convnets. arXiv preprint arXiv: 17
2017.
[4] X. Chen and Y. Tian. Learning to perform local rewriting for combinatorial optimization. In Advances Neural Information Processing Systems, pages
2019.
[5] R. d. O. da Costa, J. Rhuggenaath, Y. Zhang, and A. Akcay. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In Asian Conference Machine Learning, pages 465-480. PMLR, 2020.
[6] H. Dai, E. Khalil, Y. Zhang, B. Dilkina, and L. Song. Learning combinatorial optimization algorithms over graphs. In Advances in Neural Information Processing Systems, pages 6348- 6358, 2017.
[7] J. K. Falkner and L. Schmidt-Thiema Learning to solve vehicle routing problems with time windows through joint attention. arXiv preprint arXiv:2006. 0000000999/00000 09/00, 2020.
[8] Z. Fu, K.-B. Qiu, and H. Zha. Generalize a small pre-trained model to arbitrarily large tsp instances. In Proceedings of the AAAI Conference Artificial Intelligence, 2021.
[9] Y. Hao Lu, Xingwen Zhang. A learning-based iterative method for solving vehicle routing problems. In Proceedings of International Conference on Learning Representations (ICLR)., 2020.
[10] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778, 2016.
[11] M. Held and R. M. Karp. The traveling-salssme salesman problem and minimum spanning trees: Part Mathematical programminng 1(1):6-25, 1971.
[12] K. Helsgaun. An effective implementation of the lin-kernighan traveling salesman heuristic. European Journal of Operational Research, 126(1):106-12) 130, 2000.
[13] K. Helsgaun. General k-opt submoves for the lin-kernighan tsp heuristic. Mathematical Programming Computation, 1(2-3): 19-163, 2009.
[14] K. Helsgaun. An extension of the fnnKkernighan-helsaaun tsp solver for constrained traveling salesman and vehicle routing problems. Roskilde: Roskilde University, 2017.
[15] A. Hottung, B. Bhandari, and K. Tierney. Learning a latent search space for routing problems using variational autoencoders. In Proceedings of International Conference on Learning Representations (ICLR)., 2021.
[16] A. Hottung and K. Tierney, Neural large neighborhood search the capacitated vehicle routing problem. In European Conference on Artificial Intelligence, 2020.

---

[17] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448-456, 2015.
[18] C. K. Joshi, T. Laurent, and X. Bresson. An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv: 022200 2019.
[19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations (ICIR)., 2014.
[20] W. Kool, H. van Hoof, Gromicho, and M. Welling. Deep policy dynamic programming for vehicle routing problems. arXiv preprini arXiv:2102.11756, 2021.
[21] W. Kool, H. van Hoof, and M. Welling. Attention, learn to solve routing problems! In Proceedings of International Conference on Learning Representations (ICLR)., 2019.
[22] Y.-D. Kwon, Choo, B. Kim, I. Yoon, Y. Gwon, and S. Min. Pomo: Policy optimization with multiple optima for reinforcement learning. In Advances in Neural Information Processing Systems, volume 33, 2020.
[23] J. Li, L. Xin, Z. Cao, A. Lim, W. Song, and J. Zhang. Heterogeneous attentions for solving pickup and delivery problem via deep reinforcement learning, IEEE Transactions on Intelligets Trunsportation Systems, 2021.
[24] S. Lin. Computer solutions of the traveling salesman problem. Bell System Technical Journal, 44(10):2245-2269, 1965.
[25] S. Lin and B. W. Kermighan. An effective heuristic algorithm for the traveling-salesman problem. Operations research, 21(2):498-5 516, 1973.
[26] M. Nazari, A. Oroojlooy, L. Snyder, and M. Takác. Reinforcement learning for solving the vehicle routing problem. In Advances in Neural Information Processing Systems, pages 9839- 9849, 2018.
[27] G. Reinelt. Tsplib traveling salesman problem library. ORSA journal on computing, 3(4):376-384, 1991.
[28] M. M. Solomon. Algorithms for the vehicle routing and scheduling problems with time window constraints. Operations research, 35(2):254-265, 1987.
[29] Y. Sun, A. Ernst, Li, and J. Weiner. Generalization of machine learning for problem reduction: a case study on travelling salesman problems. OR Spectrum, 4373):607-633, 2021.
[30] E. Uchoa, D. Pecin, A. Pessoa, M. Poggi, T. Vidal, and A. Subramanian. New benchmark instances for the capacitated vehicle routing problem. European Journal of Operational Research, 257(3):845-858, 2017.
[31] T. Vidal, T. G. Crainic, M. Gendreau, N. Lahrichi, and W. Rei. A hybrid genetic algorithm for multidepot and periodic vehicle routing problems. Operations Research, 60(3):61 1 1-64,, 2012.
[32] O. Vinyals, M. Fortunato, and N. Jaitly. Pointer networks. In Advances in Neural Information Processing Systems, volume 28, 2015.
[33] Y. Wu, W. Song. Z. Cao. Zhang, and A. Lim. Learning improvement heuristics for solving routing problems. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[34] L. Xin, W. Song, Z. Cao, and J. Zhang. Step-wise deep learning models for solving routing problems. IEEE Transactions on Industrial Informatics, 17(7)4461-4471, 2020.
[35] L. Xin, W. Song, Z. Cao, and J. Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, pages 12042- 12049, 2021.
[36] J. Zheng, K. He, J. Zhou, Y. Jin, and C.-m. Li. Combining reinforcement learning with lin- kernighan-helsgaun algorithm for the traveling salesman problem. Proceedings of the AAAI Conference Artificial Intelligence, 2021.
12


Reviews:
[{"review_id": "x5gH8Oh6fHP", "reviewer": "Reviewer_dK7t", "summary": "The paper proposes a framework for solving the traveling salesman problem by combining deep learning with the heuristic Lin-Kernighan-Helsgaun (LKH) algorithm. The basic idea is to use a sparse graph network model to transform edge distance and create edge candidate sets for LKH searching trials. The experiment results show that the proposed framework improves the performance of the LKH algorithm for TSP and its variations.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The paper uses the sparse graph network model to learn the edge scores with supervised learning and node penalties with unsupervised learning respectively. The output is then used to guide the original LKH algorithm. The entire framework combines the strength of deep learning and the traditional heuristic approach.\n\nQuality: The paper is complete.\n\nClarity: The paper is clear to understand and easy to follow.\n\nSignificance: The authors claim that the proposed algorithm significantly outperforms the LKH algorithm. However, the experiment results only indicate a less than 0.1% improvement with 1000 instances under the same time limit for TSP when |V| = 100, 200, 500 respectively. Such marginal difference should not be considered statistically significant. This is a major weakness of the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Some comments:\n\n1) It is known that the LKH algorithm is powerful by itself. Given the marginal improvements (objective value) obtained by the proposed framework, I would expect some more intuitive illustration, such as a detailed analysis of an instance, to demonstrate how such framework adds value to boost or improve the original LKH algorithm. \n\n2) The entire framework is based on the LKH algorithm, which is known to be time-consuming. Usually one purpose of adding a deep learning wrap is to speed up the process. Thus, I would expect the authors to also show that the algorithm can achieve the same performance as LKH while significantly reducing the computational time.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "q-kASJ_M7B9", "reviewer": "Reviewer_2rWn", "summary": "The current paper proposes a new algorithm that combines deep learning with the heuristic solver for solving various vehicle routing problems, TSP, CVRP, PDP, CVRPTW. The method proposes a Sparse Graph Network (SGN) that produces edge scores and node penalties, which are then used to the edge candidate set and transformed edge distance. Once the edge candidate set and the transformed edge distances are computed, a conventional /lambda-opt algorithm is repeatedly used to produce the final best routing solution. The proposed sore functions are trained with small-sized problems sampled uniformly using supervised learning and unsupervised learning. Finally, the trained model is used to solve the test problems whose node distribution is different and whose node number is large. The proposed algorithm consistently outperforms its baseline, LKH, and VSR-LKH. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "<Originality>\nCombining a well-known and effective search heuristic with a learning-based score model sounds reasonable and effective in solving the target problem. In solving vehicle routing problems, various approaches devise improvement heuristics. While comparing with these improvement heuristics, please mention the unique novelty of the proposed method.\n\n<Quality>\nThe architecture of SGN is pretty simple. It would be good author justify the design choice of the architecture and the hyperparameters used. \n\nQ1. What is the rationale behind choosing the proper number of $\\gamma$? \n\nQ2. Why embedding layer uses skip connection? \n\nQ3. How to determine the number of layers? When the size of the graph changes, the same number of layers is used? The number of layers can be possibly jointly determined with the sparsity parameter $\\gamma$?\n\nThe SGN is trained with both supervised loss and unsupervised loss. I have several concerns about this approach:\n\nQ4. The biggest concern is that the method requires the optimal solution or a good solution to produce a good supervise label. What are the benefits of using the imitation learning approach comparing to reinforcement learning approach? It would be nice to compare the pros and cons and justify your selection. \n\nQ5. What is the rationale behind using the unsupervised loss? How to compute $d_i(\\pi)$ and is ti differentiable? And why this factor should close to 2? \n\nQ6. Can you train both edge score function and node penalty function simultaneously using reinforcement learning without accessing the optimum solution? What is more beneficial for generalizing over different node sizes and node distribution between imitation learning and reinforcement learning?\n\n<Clarity>\nThe paper is well written and well organized.\n\n<Significance >\nThe current paper replaces the existing empirical procedure for constructing edge candidate sets with a neural network-based edge candidate selection procedure. The proposed sparse graph network is a simple variant of a graph neural network with limited edge connectivity. I believe replacing a rule-based heuristic module with a learning-based module can naturally result in performance improvement. The paper does not propose a different way of problem-solving but proposes a straightforward extension to the original approach; the technical significance and novelty are limited. \n\nIn addition, the proposed method does not effectively generalize over the new test problems with different node distribution and different node numbers. The proposed method seems to work better than other baselines but often fails to solve the problems. In addition, it is hard to see that the method is completely transferable in that it requires fine-tuning.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The author comments that the proposed method generalizes well to large sizes and different distributions of nodes and does not generalize well to other distributions of demands and time windows without further training.\n\nTo achieve the generalization capability of the trained model over other aspects, the specially designed architecture or specially designed training scheme are generally required. The current study does not explicitly mentions such additional efforts but evaluates their performance. It would be good if the authors discuss the indented design choice to achieve what desirable performance and validate that argument through several ablation studies. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8 hours"}, "extra_text": {}}, {"review_id": "c33cUSDBYxJ", "reviewer": "Reviewer_U8cn", "summary": "This paper proposes a neural version of the classic LKH algorithm, where the subgradient optimization of LKH is replaced by a sparse GNN that predicts edge candidates and node penalties that can directly be used by the LKH search algorithm. The edge candidate predictions are learned supervised from optimal solutions whereas the penalties are learned 'unsupervised' to push the node degrees of the 1-tree induced by the penalties towards 2 (I consider this is more like RL where the deviation from node degree 2 determines the reward). After training, the NeuroLKH algorithm gives better results than the standard LKH algorithm given equal time, for the travelling salesman problem (TSP) and different vehicle routing problems (VRPs).\n\nAs the node penalties are only used for TSP, the major contribution seems to show the empirical benefit of the edge candidate predictions. While this is not a new idea in itself, the combination with LKH is and as the empirical results are very promising, I think this paper makes valuable progress and therefore should be accepted.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\nOn one hand, this paper presents a nice and simple idea of taking the subgradient objective for LKH but amortize its optimization over a set of instances to predict the node penalties for new instances without needing the subgradient optimization. On the other hand, the idea of learning to predict an edge candidate set is closely related to the cited works [19,21] and the differences are merely in architecture (most notably using a sparse network for increased scalability), so this in itself is not very novel. The combination with LKH, while relatively straightforward, has not been done before as far as I know.\n\n**Quality**\nThe paper is technically of high quality. The claims are supported by a large number of experiments and ablations on different datasets, instance distributions and problem sizes. Care has been taken to present fair comparison by forcing running times equal, and helpful ablations are included in the appendix. My major concern is that the learned node penalties are only used with TSP, for which the practical differences in performance seem small. For CVRP and TSPTW, the difference with LKH and other baselines is much more convincing, but this seems only result of the edge selection by the network. Therefore I consider the main contribution to show the strong empirical performance of using a carefully implemented neural network for edge selection in combination with an existing heuristic (LKH).\n\n**Clarity**\nOverall I found this paper very clear. Whereas a lot of space is used for the extensive empirical evaluation, I find most of the important details very clear, without going more in depth than necessary. The background on the LKH algorithm is very helpful and Figure 1 is very clear in pointing out the differences and similarities between LKH and NeuroLKH. I do have some questions about the unsupervised training and I think some of the mathematical notation can be improved (see below).\n\n**Significance**\nThis paper shows promising results on relatively large instances for different problems using a relatively simple methodology, namely the selection of edges by a graph neural network (and the prediction of node penalties used to change the edge cost for TSP), which can then be used by LKH, a relatively general solver for VRP problems. Whereas the ideas are quite simple, I consider it valuable to demonstrate the empirical performance of this methodology, which encourages further exploration on larger or other problems, as well as combination with other, more powerful solvers than LKH (which do exist and could/should be mentioned in the paper, see e.g. Hybrid Genetic Search for CVRP, Vidal 2020).\n\n**Minor comments**\n- I find the equations (1), (2) a bit confusing. I think attn_ij^l is a vector which may be helpful to mention explicitly (and as such exp(.) is also element wise). Also, it may be worth to mention the dimension over which BatchNorm operates / that it is not an element wise function as in the equations.\n- Also the notation (i,m) in E* is confusing since it looks as if i is a summation index while it is actually fixed. I suggest introducing N(i) as neighbours of i and sum over m in N(i).\n- I think some more explanation could be helpful for the loss in eq. (8). I understand now that it is correct but I was a bit surprised at first not to see absolute signs or a squared difference (as in a MSE loss).\n- L213, it would be helpful to explain/remind why the candidate sets are of much better quality than LKH (because they are trained on 'optimal' solutions, which I missed the first time reading).\n\n**Questions**\n- Why do you consider the attention 'cheap' (L160)? I think it is actually more expensive as it computes a softmax per individual feature.\n- How does the unsupervised learning of the penalties work exactly? Does the model predict penalties, then find a minimum 1-tree, then use this for the loss? How are min-1-trees found for a batch of instances? What is the batch size used? This sounds very costly. Also, I would consider this more like a reinforcement learning problem then where the action is deciding the penalties.\n- Do I understand correctly that starting from a sparse graph, some edges can never be in a solution? How does this affect solution quality? Especially for large scale VRPTW, I can imagine the optimal solution would not use the gamma = 20 shortest outgoing edges?\n- How does C (eq. 6) relate to the scale of the coordinates / size of the problem instance? This seems important as the penalty and thus C is actually in distance units.\n- Why can't node penalties be used with CVRP? When you fix the penalty for the depot to 0 (as number of visits to the depot may vary) I do not immediately see a problem?\n- The penalties affect the quality of the alpha measure, but do I understand correctly that this is not used in NeuroLKH as the edge set is predicted directly? So the penalties are only used to change the edge distances used in the search?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The major limitation is the dependence on the actual distribution of the instances and is discussed in the discussion. While not discussed, I do not see any direct negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "-lyyL2Yku7U", "reviewer": "Reviewer_rptX", "summary": "This paper proposes a machine learning approach to enhance the performance of a traditional heuristic LKH to solve routing problems. Specifically, a Sparse Graph Network is trained to select candidate edges for LKH to refine solutions. The experiments confirm that the proposed method can improve the performance of the original LKH and the VSR-LKH (based on RL) in general. ", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper combines the merits of both ML and traditional optimization techniques to solve large and hard routing problems. The approach sounds reasonable and appealing. The proposed method is well described, and the experiment is extensive.\n\n1. For the Sparse Graph Network to work, the original TSP instances need to be pre-processed, i.e., only keeping \\gamma shortest edges for each node and removing others. I guess the parameter \\gamma has a large impact on the performance of the model. Why \\gamma is set to 20 in the experiments, and what is the impact if changing \\gamma to a different value?\n\n2. Following the previous comment, sparsifying TSP graphs based on edge distance might lead to a sub-optimal solution in the first place, as mentioned by the authors that a certain amount of edges in the optimal tour were removed in pre-processing. Techniques [1,2] that sparsify TSP graphs based on machine learning might work better here. \n\n    [1] Sun, Yuan, et al. \"Generalization of machine learning for problem reduction: a case study on travelling salesman problems.\" OR Spectrum (2020): 1-27.\n\n    [2] Fitzpatrick, James, Deepak Ajwani, and Paula Carroll. \"Learning to Sparsify Travelling Salesman Problem Instances.\" International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research. Springer, Cham, 2021.\n\n3. In Equation (6), why the parameter C is set to 10? Justification is required. \n\n4. In Tables 1 and 2, the improvement in terms of solution gaps is quite small, partially because all the methods can find very near-optimal solutions. I think some results from Appendix A should be presented and discussed in the main paper to justify this, for example the converging curves and detailed statistical test results. \n\n5. The comparison between the LKH variants ignoring preprocessing time in Appendix A is interesting and important, which demonstrates that the improvement of the proposed method comes from a better ML prediction rather than simply running for more trials. I think this result should be discussed in the main paper in particular. \n\n6. In Figure S.1, why does NeuroLKH start from with an objective value much smaller than that of other methods? From my understanding, all the methods start with the same random generated solution, and so the initial objective value should be the same. \n\n7. In figure S.1, why do LKH and VSR-LKH ignoring pre-processing time terminate earlier? What happens if letting them running for the same amount of time? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations of the proposed method, especially the generalization of the method to other TSP instances with different characteristics and other routing problems.  ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all agreed that the combination of learning techniques to augment traditional problem solvers is an exciting direction, and this paper adds to that direction.  Reviewer dK7t was ultimately convinced by the author's response and material in the appendix (Figure S.1) that the contribution was worthwhile even if the objective improvement was small in many of the TSPs.  I think the paper could be improved by directly addressing this in the paper: both the discussion of relative improvement and highlighting the larger improvements in other domains.

Decision: Accept (Poster)