Title: Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Neural Combinatorial Optimization

Generalization

Large scale problem

Heavy decoder
Abstract: Neural combinatorial optimization (NCO) is a promising learning-based approach for solving challenging combinatorial optimization problems without specialized algorithm design by experts. However, most constructive NCO methods cannot solve problems with large-scale instance sizes, which significantly diminishes their usefulness for real-world applications. In this work, we propose a novel Light Encoder and Heavy Decoder (LEHD) model with a strong generalization ability to address this critical issue. The LEHD model can learn to dynamically capture the relationships between all available nodes of varying sizes, which is beneficial for model generalization to problems of various scales. Moreover, we develop a data-efficient training scheme and a flexible solution construction mechanism for the proposed LEHD model. By training on small-scale problem instances, the LEHD model can generate nearly optimal solutions for the Travelling Salesman Problem (TSP) and the Capacitated Vehicle Routing Problem (CVRP) with up to 1000 nodes, and also generalizes well to solve real-world TSPLib and CVRPLib problems. These results confirm our proposed LEHD model can significantly improve the state-of-the-art performance for constructive NCO.

Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalization

Abstract

Neural combinatorial optimization (NCO) is a promising learning-based approach for solving challenging combinatorial optimization problems without specialized algorithm design by experts. However, most constructive NCO methods cannot solve problems with large-scale instance sizes, which significantly diminishes their usefulness for real-world applications. In this work, we propose a novel Light Encoder and Heavy Decoder (LEHD) model with a strong generalization ability to address this critical issue. The LEHD model can learn to dynamically capture the relationships between all available nodes of varying sizes, which is beneficial for model generalization to problems of various scales. Moreover, we develop a data-efficient training scheme and a flexible solution construction mechanism for the proposed LEHD model. By training on small-scale problem instances, the LEHD model can generate nearly optimal solutions for the Travelling Salesman Problem (TSP) and the Capacitated Vehicle Routing Problem (CVRP) with up to 1000 nodes, and also generalizes well to solve real-world TSPLib and CVRPLib problems. These results confirm our proposed LEHD model can significantly improve the state-of-the-art performance for constructive NCO. The code is available at https://github.com/CIAM-Group/NCO_code/tree/main/single_objective/LEHD.

1 Introduction

Combinatorial optimization (CO) holds significant practical value across various domains, such as vehicle routing [48], production planning [11], and drug discovery [34]. Due to the NP-hardness and the presence of numerous complex variants, solving CO problems remains an extremely challenging task [31]. In the past few decades, many powerful algorithms have been proposed to tackle different CO problems, but they mainly suffer from two major limitations. Firstly, solving each new problem requires extensive domain knowledge to design a proper algorithm by experts, which could lead to a significant development cost. In addition, these algorithms usually require an excessively long execution time due to the NP-hardness of CO problems [49]. These limitations make the classical algorithms undesirable and impractical for many real-world applications.

Recently, many learning-based neural combinatorial optimization (NCO) methods have been proposed to solve CO problems without handcrafted algorithm design. These methods build neural network models to generate the solution for given CO problem instances, which can be trained by supervised learning (SL) [51, 22, 13, 23, 28, 18] or reinforcement learning (RL) [4, 27, 17, 19, 6, 9, 53, 37, 29, 36, 24, 39, 40, 54, 52, 1]. Although these methods can achieve promising performance on

small-scale problems, they perform poorly on solving large-scale problems. Due to the NP-hardness of large-scale problems, the SL-based methods struggle to obtain enough high-quality solutions as labeled training data. Meanwhile, the RL-based methods could suffer from the critical issues of sparse rewards [47, 15] and device memory limitations for solving large-scale problems. Therefore, it is impractical to directly train the NCO model on large-scale problem instances. One feasible approach is to train the NCO model on small-scale problems and then generalize it to solve large-scale problems. However, the current purely learning-based constructive NCO methods have a very poor generalization ability, which diminishes their usefulness in solving large-scale real-world problems.

The current constructive NCO models typically have a Heavy Encoder and Light Decoder (HELD) structure, which could be an underlying reason for their poor generalization ability. The HELD-based model aims to learn the embeddings of all nodes via a heavy encoder in one shot and then sequentially construct the solution with the static node embeddings via a light decoder. This one-shot embedding learning may incline the model to learn scale-related features to perform well on small-scale instances. When the model generalizes to large-scale problem instances, the learned scale-related features may hinder the model from capturing the necessary relations among a significantly larger number of nodes. This work proposes a constructive NCO model with a novel Light Encoder and Heavy Decoder (LEHD) structure to tackle this issue. Instead of one-shot embedding learning, our proposed LEHDbased model learns to dynamically capture the relationships among the current partial solution and all the available nodes at each construction step via the heavy decoder. As the size of the available nodes varies with construction steps, the model tends to learn scale-independent features. Moreover, the model can iteratively adjust and refine the learned node relationships during the training. Therefore, it could be less sensitive to the instance size and has much better generalization performance on large-scale problem instances.

It becomes impractical to train the proposed LEHD model with RL due to the huge memory and computational cost of the heavy decoder structure. In this paper, we proposed a data-efficient training scheme, called learn to construct partial solution , to train the LEHD model in a supervised manner. With this scheme, the model learns to reconstruct partial solutions during the optimization process, which can be treated as a kind of data augmentation for robust model training. Finally, similar to the training process, we propose a flexible solution construction mechanism called Random Re-Construct (RRC) for efficient active improvement at the inference stage, which can significantly improve the solution quality with iterative local reconstructions.

Our contributions can be summarized as follows:

We propose a novel Light Encoder and Heavy Decoder (LEHD) model for generalizable neural combinatorial optimization. By only training on small-scale problem instances, the proposed LEHD model can achieve robust and promising performance on problems with much larger sizes.

We develop a data-efficient training scheme and a flexible solution construction mechanism for the proposed LEHD model. The data-efficient training scheme enables LEHD to be trained efficiently via supervised learning, while the solution construction mechanism can continuously improve the solution quality through a customized inference budget.

Our purely learning-based constructive method can achieve state-of-the-art performance in solving TSP and CVRP at various scales up to size 1000, and also generalizes well to solve real-world TSPLib/CVRPLib problems.

2 Related Work

Constructive NCO with Balanced Encoder-Decoder In their seminal work, Vinyals et al. [51] propose the Pointer Network as a neural solver to directly construct solutions for combinatorial optimization problems in an autoregressive manner. This model has a balanced encoder-decoder structure where both the encoder and decoder are recurrent neural networks (RNNs) with the same number of layers. The original Pointer Network is trained by supervised learning, while Bello et al. [4] proposes an efficient reinforcement learning method for NCO model training. Some Pointer Network variants have been proposed to tackle different combinatorial optimization problems [37]. However, these models can only solve small-scale problem instances with sizes up to 100 and have poor generalization performance.

Constructive NCO with Heavy Encoder and Light Decoder Kool et al. [27] and Deudon et al. [10] leverage the Transformer architecture [46] to design more powerful NCO models. The Attention Model (AM) proposed by Kool et al. [27] has a Heavy Encoder and Light Decoder (HELD) structure with three attention layers in the encoder and one layer in the decoder. This model can obtain promising performance on various small-scale problems with no more than 100 nodes. Since then, AMhas become a typical model choice, and many AM variants have been proposed for constructive NCO [53, 29, 54, 30, 24]. Among them, POMO proposed by Kwon et al. [29] is an AM model with a more powerful learning and inference strategy based on multiple optimal policies. The POMO model has a heavy encoder with six attention layers and a light one-layer decoder. It can achieve remarkable performance on small-scale problems with up to 100 nodes.

However, these constructive models with heavy encoder and light decoder structures still have a very poor generalization performance on problem instances with larger sizes, even for those with up to 200 nodes [23]. Directly training these models on the large-scale problem is also very difficult if not infeasible [23]. Different methods have been proposed to enhance the generalization ability for constructive NCO models in the inference stage, such as Efficient Active Search (EAS) [19] and Simulation-guided Beam Search (SGBS) [8]. While these methods can improve the constructive NCO model's generalization performance on problems with up to size 200, they struggle to solve problems with larger scales.

Very recently, a few approaches have been proposed to use the AM model to tackle large-scale TSP instances. Pan et al. [38] propose a Lower-Level Model that learns to construct the solution of sub-problems obtained by an Upper-Level Model. Another method proposed by Cheng et al. [7] uses an AM model to learn to reconstruct segments of a given TSP solution and selects the most improving segment to optimize it. However, these methods heavily rely on specialized designs for TSP by human experts, and cannot be used to solve other problems such as CVRP. In a concurrent work, Drakulic et al. [12] propose a novel Bisimulation Quotienting (BQ) method for generalizable neural combinatorial optimization by reformulating the Markov Decision Process (MDP) of the solution construction. Our proposed LEHD model has a different motivation and model structure from BQ, and a comprehensive comparison is provided in the experiment section.

Non-Constructive NCO Methods In addition to constructive NCO, there are also other learningbased approaches that work closely with search and improvement methods [33, 6, 18, 21]. Joshi et al. [22] propose a graph convolutional network to predict the heat map of each edge's probability in the optimal solution for a TSP instance. Afterward, approximate solutions can be obtained via the heat map guided beam search [22], Monte Carlo tree search (MCTS) [13], dynamic programming [28], and guided local search [20]. The heatmap-based approach was originally proposed to solve smallscale problems, and some recent works generalize it for solving large-scale TSP instances [13, 42]. However, they heavily rely on the carefully expert-designed MCTS strategy for TSP [13] and cannot be used to solve other problems.

Learning-augmented methods can also be applied to improve and accelerate heuristic solvers via operation selection [6, 35, 9], guided improvement [17, 5, 55] and problem decomposition [44, 32]. Many improvement-based methods have also been proposed to iteratively improve a feasible initial solution [3, 52, 36]. However, these methods typically rely on human-designed heuristic operators or more advanced solvers [14, 16] for solving different problems. In this work, we mainly focus on the purely learning-based NCO solver without expert knowledge.

3 Model Architecture: Light Encoder and Heavy Decoder

In this section, we propose a novel NCO model with the Light Encoder and Heavy Decoder (LEHD) structure to tackle the critical issue regarding large-scale generalization.

3.1 Encoder

For a problem instance S with n node features ( s 1 , . . . , s n ) (e.g., the coordinates of n cities), a constructive model parameterized by θ generates the solution in an autoregressive manner, i.e., it constructs the solution by selecting nodes one by one. Our LEHD-based model consists of a light encoder and a heavy decoder, as shown in Figure 1. The light encoder has one attention layer, and the heavy decoder has L attention layers. Given the problem instance with node features

Decoder

Starting Node

Embedding

Available Node

Embeddings

Linear Projection

Linear Projection

Attention Layer

Linear Projection

Softmax

Selected Probability

Encoder

Node Features

Attention Layer

Linear Projection

FF

Add

Add

MHA

Attention Layer

Destination Node

Embedding

S = ( s 1 , . . . , s n ) , the light encoder transforms each node features s i to its initial embedding h (0) i through a linear projection. The initial embeddings { h (0) 1 , . . . , h (0) n } are then fed into one attention layer to get the node embedding matrix H (1) = ( h (1) 1 , . . . , h (1) n ) .

Attention layer The attention layer comprises two sub-layers: the multi-head attention (MHA) sub-layer and the feed-forward (FF) sub-layer [46]. Different from generic NCO models such as AM [27], the normalization is removed from our model to enhance generalization performance (see Appendix A). Let H ( l -1) = ( h ( l -1) 1 , . . . , h ( l -1) n ) be the input of the l -th attention layer for l = 1 , . . . , L , the output of the attention layer in terms of the i -th node is calculated as:

We denote this embedding process as H ( l ) = AttentionLayer( H ( l -1) ) .

3.2 Decoder

The decoder sequentially constructs the solution in n steps by selecting node by node. In the t -th step for t ∈ { 1 , . . . , n } , the current partial solution can be written as ( x 1 , . . . , x t -1 ) , the first selected node's embedding is denoted as h (1) x 1 , the embedding of the node selected in the previous step is denoted as h (1) x t -1 , and the embeddings of all available nodes is denoted by H a = { h (1) i | i ∈ { 1 , . . . , n } \ { x 1 , . . . , x t -1 }} . Since the decoder has L attention layers, the t -th construction step of LEHD can be described as:

̸

where W 1 , W 2 , W O are learnable matrices. The matrices W 1 and W 2 are used to re-calculate the embeddings of the starting node (i.e., the node selected in the previous step) and the destination node (i.e., the node selected in the first step), respectively. The matrix W O is used to transform the relation-denoting embedding matrix ˜ H ( L ) into a vector u for the purpose of calculating the selection probability p t . The most suitable node x t is selected based on p t at each decoding step t . Finally, a complete solution x = ( x 1 , . . . , x n ) ⊺ is constructed by calling the decoder n times.

For the HELD model such as AM [27], the heavy encoder with L layers outputs the node embedding matrix as H ( L ) = ( h ( L ) 1 , . . . , h ( L ) n ) . The average of all nodes' embeddings (i.e., 1 n ∑ n i =1 h ( L ) i ), the

1

2

3

4

5

7

6

8

9

2

3

4

5

6

TSP solution                               TSP partial solution                        CVRP solution                                  CVRP partial solution

1

2

3

4

5

7

6

8

9

2

3

4

5

6

sample

0

sample

starting node's embedding (i.e., h ( L ) x t -1 ), the destination node's embedding (i.e., h ( L ) x 1 ) are contacted to form the context node embedding (i.e., h (0) c ). Thereafter, the light decoder takes the context node embedding and the embeddings of all nodes (i.e., H ( L ) ) as input, and computes the selection probability through an MHA operation, the compatibility calculation, and the softmax function, i.e.,

̸

where W C and W E are learnable matrices, d is a parameter determined by the embedding dimension. Since the HELD model's decoder only has a static node embedding matrix H ( L ) , the relationships among the nodes it captures are not updated throughout the decoding process. In other words, the HELD model's performance relies heavily on the quality of the static node embedding matrix. When the problem size becomes larger, the ability of the static node embedding matrix to represent the relationships among all nodes becomes worse, resulting in the poor generalization performance of the model.

In our LEHD model, the heavy decoder dynamically re-embeds the embeddings of the starting node, destination node, and available nodes via L attention layers, thus updating the relationships among the nodes at each decoding step. Such a dynamic learning strategy enables the model to adjust and refine its captured relationships between the starting/destination and available nodes. In addition, as the size of the nodes varies during the construction steps, the model tends to learn the scale-independent features. These good properties enable the model to dynamically capture the proper relationships between the nodes, thereby making more informed decisions in the node selection on problem instances of various sizes.

4 Learn to Construct Partial Solution

The constructive NCO aims to construct the solution node-by-node using the decoder. The heavy decoder of our LEHD model can result in a higher computational cost than that of the HELD model at each construction step. Whereas the RL training method needs to generate the complete solution before computing the reward, which implies it requires significantly huge memory and computational costs. Furthermore, RL suffers from the challenging issue of sparse reward, especially for problems with large sizes.

According to Yao et al. [56], data augmentation (DA) can significantly reduce the number of required high-quality solutions (i.e., labels) for SL training, and DA-based SL can even outperform RL regarding solution accuracy and generalization performance. Operations based on the property of multiple optimality [29] and optimality invariance [25] are the most commonly used DA methods in NCO. In this work, we develop an efficient DA-based SL training scheme also based on the property of optimality invariance.

For each training data (i.e., a set of nodes), its label is a sequence of all the nodes (i.e., a valid tour). According to the property of optimality invariance, the partial solution of the optimal solution must

also be optimal. Therefore, we can randomly sample labeled partial solutions with various sizes and different directions from each labeled data, thus greatly enriching our training set. Based on these characteristics, we propose a training scheme called learning to construct partial solutions . The model learns to construct optimal partial solutions of various sizes and random directions, thereby resulting in a more efficient and robust training process.

4.1 Generate Partial Solutions during the Training Phase

For a routing problem instance S with n nodes ( s 1 , . . . , s n ) , its optimal solution can be represented by a specific node sequence x (also called the tour). The partial solution x sub is a consecutive subsequence sampled from x with a random size and direction. In this work, we let each partial solution have a length with w sampled from the discrete uniform distribution Unif ([4 , | V | ]) where | V | is the number of nodes for each TSP/CVRP instance, since a problem with size less than 4 is trivial to solve. Each partial solution x sub is one augmented data point in our method. Two examples of generating partial solutions for TSP and CVRP are provided in Figure 2.

4.2 Learn to Construct Partial Solutions via Supervised Learning

The model learns to construct the partial solution node by node. The first node of x sub is the starting node, the last node of x sub is the destination node, and the remaining nodes constitute the available nodes to be selected by the model. For the example x sub = [6 , 3 , 5 , 4 , 1] , node 6 is the starting node, node 1 is the destination node, and nodes 3 , 5 , 4 are available nodes.

In each training step, the model provides each available node s i with a probability p i of being selected. Meanwhile, the label x sub tells whether the node s i should be chosen in the current step (i.e., y i = 1 or 0 ). The cross-entropy loss loss = -∑ u i =1 y i log( p i ) is employed to measure the error made by the current model, where u is the number of available nodes. Then we can update the model parameters by a gradient-based algorithm (e.g., Adam) with respect to the loss. Throughout the training process, the number of available nodes gradually decreases, and the starting node dynamically shifts to the node selected in the previous step while the destination node remains constant. The training epoch ends when there are no more nodes available. The next epoch starts with new augmented data.

4.3 Generate the Complete Solution during the Inference Phase

During the inference phase, our proposed LEHD model employs a greedy step-by-step approach to construct the whole solution for the given instance S . In the beginning, a single node is randomly selected to serve as the destination node and also the initial starting node. The remaining nodes constitute the set of available nodes that the model can select step-by-step. Once a node is selected, it will be treated as the new starting node and removed from the set of available nodes, i.e., the starting node is dynamically shifting while the destination node remains constant. When no more available nodes exist, the whole solution is generated. In this way, LEHD can easily construct solutions for problems of various sizes as well as unseen large-scale problems.

5 Random Re-Construct for Further Improvement

The constructive model has an inductive bias, which means it could construct a better solution by following its preferred direction. Similarly, different starting and destination nodes will lead to different solution quality since the model may perform better when starting from some well-learned local patterns. The tour generated via greedy search could be not optimal and contain multiple suboptimal local segments (e.g., partial solutions). Therefore, rectifying these suboptimal segments during the inference phase can effectively enhance the overall solution quality.

Our model stems from learning to construct partial solutions, making it appropriate for iteratively reconstructing partial solutions from different directions and different starting and destination nodes. It naturally leads to our proposed flexible construction mechanism, Random Re-Construct (RRC). Specifically, being the same as generating the partial solution during training, RRC randomly samples a partial solution from the initial solution and restructures it to obtain a new partial solution. If the new partial solution is superior, it replaces the old one. This iterative process can enhance the solution quality within a stipulated time budget.

6 Experiment

In this section, we empirically compare our proposed LEHD model with other learning-based and classical solvers on TSP and CVRP instances with various sizes and distributions.

Problem Setting We follow the standard data generation procedure of the previous work [27] to generate TSP and CVRP datasets. The training sets consist of one million instances for TSP100 and CVRP100, respectively. The TSP test set includes 10,000 instances with 100 nodes and 128 instances for each of 200, 500, and 1000 nodes, respectively. Similarly, the CVRP test set is constructed with the same number of instances. We obtain optimal solutions for the TSP training set with the Concorde solver [2] and obtain optimal solutions for the CVRP training set with HGS [50]. The training and test datasets can be downloaded from Google Drive 3 or Baidu Cloud 4 .

Model Setting For our LEHD model, the embedding dimension is set to 128, and the number of attention layers in the decoder is set to 6. In each attention layer, the head number of MHA is set to 8, and the dimension of the feed-forward layer is set to 512. In all experiments, we train and test our LEHD models using a single NVIDIA GeForce RTX 3090 GPU with 24GB memory.

Training Both the TSP model and CVRP model are trained on one million instances of size 100, respectively. The optimizer is Adam [26] with an initial learning rate of 1e-4. The value of the learning rate decay is set to 0.97 per epoch for the TSP model and 0.9 per epoch for the CVRP model. With a batch size of 1024, we train the TSP model for 150 epochs and the CVRP model for 40 epochs since it converges much faster.

Baselines We compare our method with (1) Classical Solvers: Concorde [2], LKH3 [16], HGS [50], and OR-Tools [41]; (2) Constructive NCO: POMO [29], MDAM [54], EAS [19], SGBS [8], and BQ [12]; (3) Heatmap-based Method: Att-GCN+MCTS [13].

Metrics and Inference We report the optimality gap and inference time for all methods. The optimality gap measures the difference between solutions achieved by different learning and nonlearning-based methods and the optimal solutions obtained using Concorde for TSP and LKH3 for CVRP. Since the classical solvers are run on a single CPU, their reported inference times are not directly comparable to those of the learning-based methods executed on GPU.

For MDAM, POMO, SGBS, and EAS, we directly run their source code with default settings on our test set. For Att-GCN+MCTS, we report their original results for the corresponding setting. For BQ, following the settings described in the original paper, we reproduce and train the BQ model by ourselves and report the obtained results on our test set. For our proposed LEHD model, we report both the results for greedy inference as well as for Random Re-Construct (RRC) with various computational budgets.

6.1 Experimental Results

The main experimental results on uniformly distributed TSP and CVRP instances are reported in Table 1. For TSP, our proposed LEHD method can obtain good greedy inference performance with a very fast inference time on instances of all sizes. With only 100 RRC iterations, LEHD can significantly outperform all other learning-based methods with a reasonable inference time. If we have a larger budget of more than 500 RRC iterations, LEHD can achieve very promising performance for all instances, of which the gap is nearly optimal for TSP100 and TSP200, less than 0 . 2% for TSP500, and around 0 . 8% for TSP1000.

For CVRP, LEHD can also achieve promising greedy inference performance for all instances. With 300 RRC iterations, LEHD can outperform most learning-based methods on all CVRP instances, except for the EAS method on CVRP100, which requires a much longer inference time. The EAS's performance significantly decreases for solving larger instances such as CVRP500, and we fail to obtain a result for EAS on CVRP1000 with a reasonable budget. SGBS struggles with poor generalization and long inference time for solving large-scale instances. LEHD can also outperform

LKH3 on CVRP100-CVRP500 with a budget of 500 RRC iterations and can achieve an around 1% gap to LKH3 on CVRP1000 with 1000 RCC iterations. To the best of our knowledge, our method is the first purely learning-based NCO method that can outperform LKH3 on CVRP200 and CVRP500.

Table 2 shows the test results on real-world TSPLib [43] and CVRPLib [45] instances with different sizes and distributions. We report the results with greedy inference and 1000 RRC iterations for our proposed LEHD method. It is clear that LEHD has a good greedy inference performance and can significantly outperform POMO and BQ on all instances with 1000 RRC iterations. These results demonstrate the robust generalization ability of LEHD.

6.2 Ablation Study

Heavy Decoder vs. Heavy Encoder We compare the performance of models with the Heavy Encoder structure (POMO) and the Heavy Decoder structure (LEHD). We also train POMO with SL on TSP100 and report the result in Table 3. According to the results, POMO trained by SL can also reach a good training performance on TSP100 while it still performs poorly on large-scale problems. Therefore, the promising generalization performance of our proposed method does come from the heavy decoder structure rather than the SL training.

Supervised Learning vs. Reinforcement Learning In Table 4, we conduct a comparison with reinforcement learning and supervised learning for training our proposed LEHD model. Due to the high computational cost, the reinforcement learning method with the POMO strategy cannot converge in a reasonable time. Therefore, we use each method to train a LEHD model separately on TSP50 with the same computational budget. The results in Table 4 confirm that our proposed LEHD model is more suitable to be trained by the supervised learning method.

The Effect of RRC We also conduct comparisons with POMO with random sampling (POMOSample100) as well as POMO with RRC (POMO-RRC100) to clearly present the advantages of LEHD and RRC.

The experimental results are shown in Table 5, and we have the following two main observations:

POMO-RRC > POMO-Sample : The random sampling strategy is inefficient in improving POMO's solution quality, especially for large-scale problems. In contrast, our proposed RRC can efficiently improve POMO's performance at inference time.

LEHD-RRC > POMO-RRC : Our proposed LEHD model structure provides enough model capacity for learning to construct partial solutions for different sizes. It is crucial for the strong generalization ability to large-scale problems.

6.3 Comparison with Search-based/Improvement-based Methods

Table 6: Comparison with SO-mixed [7] . The results of the SO-mixed method are directly obtained from the original paper.

Recently, Cheng et al. [7] and Pan et al. [38] propose two learning-based methods which can tackle large-scale TSP instances. However, these two methods are search-based/improvement-based approaches that require powerful solvers like LKH3 or specifically designed heuristics, and can only tackle TSP but not other VRP variants. In this work, we propose a purely learning-based construction method to tackle the large-scale TSP/VRP instances, and hence mainly compare with other construction-based methods in the original paper.

We conduct a comparison with these methods for a more comprehensive experimental study. The results are reported in Table 6 and Table 7. Our proposed method clearly outperforms those methods in terms of both performance and running time. It highlights the effectiveness of our proposed LEHD+RRC as a purely learning-based method to tackle large-scale problems.

7 Conclusion, Limitation, and Future Work

Conclusion In this work, we propose a novel LEHD model for generalizable constructive neural combinatorial optimization. By leveraging the Light Encoder and Heavy Decoder (LEHD) model structure, the LEHD model achieves a strong and robust generalization ability as a purely learningbased method. We also develop a learn to construct partial solution strategy for efficient model training, and a flexible random solution reconstruction mechanism for online solution improvement with customized computational budgets. Extensive experimental comparisons with other representative methods on both synthetic and real-world instances fully demonstrate the promising generalization ability of our proposed LEHD model. Since the generalization ability is one crucial issue for the current constructive NCO solver, we believe LEHD can provide valuable insights and inspire follow-up works to explore the heavy decoder structure for powerful NCO model design.

Limitation and Future Work Although with a strong generalization performance, the current LEHD model can only be properly trained by supervised learning. It could be interesting to develop an efficient reinforcement learning-based method for LEHD model training. Another promising future work is to design a powerful learning-based method for more efficient partial solution reconstruction.

Acknowledgments and Disclosure of Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62106096, in part by the Shenzhen Technology Plan under Grant JCYJ20220530113013031, in part by the Characteristic Innovation Project of Colleges and Universities in Guangdong Province under Grant 2022KTSCX110, in part by the Research Grants Council of the Hong Kong Special Administrative Region under Grant CityU 11215622, and in part by the 2023 Graduate Innovation Practice Fund Project of Southern University of Science and Technology.

References

Sungsoo Ahn, Younggyo Seo, and Jinwoo Shin. Learning what to defer for maximum independent sets. In International Conference on Machine Learning , pages 134-144. PMLR, 2020.

David Applegate, Ribert Bixby, Vasek Chvatal, and William Cook. Concorde tsp solver, 2006.

Thomas Barrett, William Clements, Jakob Foerster, and Alex Lvovsky. Exploratory combinatorial optimization with reinforcement learning. In AAAI Conference on Artificial Intelligence (AAAI) , 2020.

Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 , 2016.

Quentin Cappart, Thierry Moisan, Louis-Martin Rousseau, Isabeau Prémont-Schwarz, and Andre A Cire. Combining reinforcement learning and constraint programming for combinatorial optimization. In AAAI Conference on Artificial Intelligence (AAAI) , 2021.

Xinyun Chen and Yuandong Tian. Learning to perform local rewriting for combinatorial optimization. Advances in Neural Information Processing Systems , 32, 2019.

Hanni Cheng, Haosi Zheng, Ya Cong, Weihao Jiang, and Shiliang Pu. Select and optimize: Learning to aolve large-scale tsp instances. In International Conference on Artificial Intelligence and Statistics , pages 1219-1231. PMLR, 2023.

Jinho Choo, Yeong-Dae Kwon, Jihoon Kim, Jeongwoo Jae, André Hottung, Kevin Tierney, and Youngjune Gwon. Simulation-guided beam search for neural combinatorial optimization. arXiv preprint arXiv:2207.06190 , 2022.

Paulo R d O Costa, Jason Rhuggenaath, Yingqian Zhang, and Alp Akcay. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In Asian Conference on Machine Learning , pages 465-480. PMLR, 2020.

Michel Deudon, Pierre Cournut, Alexandre Lacoste, Yossiri Adulyasak, and Louis-Martin Rousseau. Learning heuristics for the tsp by policy gradient. In International conference on the integration of constraint programming, artificial intelligence, and operations research , pages 170-181. Springer, 2018.

Alexandre Dolgui, Dmitry Ivanov, Suresh P Sethi, and Boris Sokolov. Scheduling in production, supply chain and industry 4.0 systems by optimal control: fundamentals, state-of-the-art and applications. International Journal of Production Research , 57(2):411-432, 2019.

Darko Drakulic, Sofia Michel, Florian Mai, Arnaud Sors, and Jean-Marc Andreoli. Bq-nco: Bisimulation quotienting for generalizable neural combinatorial optimization. arXiv preprint arXiv:2301.03313 , 2023.

Zhang-Hua Fu, Kai-Bin Qiu, and Hongyuan Zha. Generalize a small pre-trained model to arbitrarily large tsp instances. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 7474-7482, 2021.

Gurobi Optimization, Inc. Gurobi optimizer reference manual, 2012. URL http://www.gurobi.com .

Joshua Hare. Dealing with sparse rewards in reinforcement learning. arXiv preprint arXiv:1910.09281 , 2019.

Keld Helsgaun. An extension of the lin-kernighan-helsgaun tsp solver for constrained traveling salesman and vehicle routing problems. Roskilde: Roskilde University , 12, 2017.

André Hottung and Kevin Tierney. Neural large neighborhood search for the capacitated vehicle routing problem. In 24th European Conference on Artificial Intelligence (ECAI 2020) , 2020.

André Hottung, Bhanu Bhandari, and Kevin Tierney. Learning a latent search space for routing problems using variational autoencoders. In International Conference on Learning Representations , 2021.

André Hottung, Yeong-Dae Kwon, and Kevin Tierney. Efficient active search for combinatorial optimization problems. arXiv preprint arXiv:2106.05126 , 2021.

Benjamin Hudson, Qingbiao Li, Matthew Malencia, and Amanda Prorok. Graph neural network guided local search for the traveling salesperson problem. arXiv preprint arXiv:2110.05291 , 2021.

Chaitanya K. Joshi and Rishabh Anand. Recent advances in deep learning for routing problems. In ICLR Blog Track , 2022. URL https://iclr-blog-track.github.io/2022/03/25/ deep-learning-for-routing-problems/ . https://iclr-blog-track.github.io/2022/03/25/deep-learningfor-routing-problems/.

Chaitanya K Joshi, Thomas Laurent, and Xavier Bresson. An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv:1906.01227 , 2019.

Chaitanya K Joshi, Quentin Cappart, Louis-Martin Rousseau, and Thomas Laurent. Learning the travelling salesperson problem requires rethinking generalization. Constraints , 27(1-2):70-98, 2022.

Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems , 34:10418-10430, 2021.

Minsu Kim, Junyoung Park, and Jinkyoo Park. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. arXiv preprint arXiv:2205.13209 , 2022.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Wouter Kool, Herke van Hoof, and Max Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations , 2019. URL https://openreview.net/forum? id=ByxBFsRqYm .

Wouter Kool, Herke van Hoof, Joaquim Gromicho, and Max Welling. Deep policy dynamic programming for vehicle routing problems. In Integration of Constraint Programming, Artificial Intelligence, and Operations Research: 19th International Conference, CPAIOR 2022, Los Angeles, CA, USA, June 20-23, 2022, Proceedings , pages 190-213. Springer, 2022.

Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems , 33:21188-21198, 2020.

Yeong-Dae Kwon, Jinho Choo, Iljoo Yoon, Minah Park, Duwon Park, and Youngjune Gwon. Matrix encoding networks for neural combinatorial optimization. In Advances in Neural Information Processing Systems , 2021.

Bingjie Li, Guohua Wu, Yongming He, Mingfeng Fan, and Witold Pedrycz. An overview and experimental study of learning-based optimization algorithms for the vehicle routing problem. IEEE/CAA Journal of Automatica Sinica , 9(7):1115-1138, 2022.

Sirui Li, Zhongxia Yan, and Cathy Wu. Learning to delegate for large-scale vehicle routing. Advances in Neural Information Processing Systems , 34:26198-26211, 2021.

Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Combinatorial optimization with graph convolutional networks and guided tree search. In Advances in Neural Information Processing Systems , page 539, 2018.

Ruiwu Liu, Xiaocen Li, and Kit S Lam. Combinatorial chemistry in drug discovery. Current opinion in chemical biology , 38:117-126, 2017.

Hao Lu, Xingwen Zhang, and Shuang Yang. A learning-based iterative method for solving vehicle routing problems. In International Conference on Learning Representations , 2019.

Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Le Zhang, Zhenghua Chen, and Jing Tang. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. In Advances in Neural Information Processing Systems , 2021.

Mohammadreza Nazari, Afshin Oroojlooy, Lawrence Snyder, and Martin Takác. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems , 31, 2018.

Xuanhao Pan, Yan Jin, Yuandong Ding, Mingxiao Feng, Li Zhao, Lei Song, and Jiang Bian. H-tsp: Hierarchically solving the large-scale traveling salesman problem. In AAAI 2023 , February 2023. URL https://www.microsoft.com/en-us/research/publication/ h-tsp-hierarchically-solving-the-large-scale-traveling-salesman-problem/ .

Junyoung Park, Sanjar Bakhtiyar, and Jinkyoo Park. Schedulenet: Learn to solve multi-agent scheduling problems with reinforcement learning. arXiv preprint arXiv:2106.03051 , 2021.

Junyoung Park, Jaehyeong Chun, Sang Hun Kim, Youngkook Kim, and Jinkyoo Park. Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning. International Journal of Production Research , 59(11):3360-3377, 2021.

Laurent Perron and Vincent Furnon. Or-tools, 2023. URL https://developers.google.com/ optimization/ .

Ruizhong Qiu, Zhiqing Sun, and Yiming Yang. Dimes: A differentiable meta solver for combinatorial optimization problems. arXiv preprint arXiv:2210.04123 , 2022.

Gerhard Reinelt. TSPLIB-a traveling salesman problem library. ORSA Journal on Computing , 3(4): 376-384, 1991.

Jialin Song, Yisong Yue, Bistra Dilkina, et al. A general large neighborhood search framework for solving integer linear programs. Advances in Neural Information Processing Systems , 33:20012-20023, 2020.

Eduardo Uchoa, Diego Pecin, Artur Pessoa, Marcus Poggi, Thibaut Vidal, and Anand Subramanian. New benchmark instances for the capacitated vehicle routing problem. European Journal of Operational Research , 257(3):845-858, 2017.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang, Olivier Pietquin, Bilal Piot, Nicolas Heess, Thomas Rothörl, Thomas Lampe, and Martin Riedmiller. Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards. arXiv preprint arXiv:1707.08817 , 2017.

Matthew Veres and Medhat Moussa. Deep learning for intelligent transportation systems: A survey of emerging trends. IEEE Transactions on Intelligent Transportation Systems , 21(8):3152-3168, 2019.

Natalia Vesselinova, Rebecca Steinert, Daniel F Perez-Ramirez, and Magnus Boman. Learning combinatorial optimization on graphs: A survey with applications to networking. IEEE Access , 8:120388-120416, 2020.

Thibaut Vidal. Hybrid genetic search for the cvrp: Open-source implementation and swap* neighborhood. Computers & Operations Research , 140:105643, 2022.

Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. Advances in neural information processing systems , 28, 2015.

Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim. Learning improvement heuristics for solving routing problems. IEEE transactions on neural networks and learning systems , 33(9):5057-5069, 2021.

Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Step-wise deep learning models for solving routing problems. IEEE Transactions on Industrial Informatics , 17(7):4861-4871, 2020.

Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 12042-12049, 2021.

Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Neurolkh: Combining deep learning model with linkernighan-helsgaun heuristic for solving the traveling salesman problem. Advances in Neural Information Processing Systems , 34:7472-7483, 2021.

Shunyu Yao, Xi Lin, Zhenkun Wang, and Qingfu Zhang. Data-efficient supervised learning is powerful for neural combinatorial optimization. 2023. URL https://openreview.net/forum?id=a_yFkJ4-uEK .

A Ablation Study of Normalization

A.1 LEHD Model

In Table 8, we explore the effects of eliminating normalization from the attention layer in our LEHD model. We train three LEHD models with the same training scheme and training budget, differing solely in the attention layer: one with batch normalization (BN), one with instance normalization (IN), and one without normalization (w/o). Our experimental results demonstrate that the LEHD model without normalization in the attention layer can significantly outperform the other two models with normalization.

A.2 POMOModel

We also compare the performance of POMO [29] with different types of normalization: one with batch normalization (BN), one with instance normalization (IN), and one without normalization (w/o) in Table 9. We train all three POMO models with the same reinforcement learning method with POMO strategy and training budget ( 1 , 000 epochs). The results show that different types of normalization have little effect on POMO.

The results in Table 9 show that removing normalization from the attention layer has little impact on the model with a heavy encoder and a light decoder. However, the results in Table 8 show that removing normalization from the attention layer has a positive impact on the performance of the LEHD model, but it is not the critical factor for the LEHD model's strong generalization ability since the LEHD model with batch normalization still performs significantly better than POMO and SGBS in the case of TSP1000. Instead, we can conclude that the underlying reason for the model's strong generalization ability lies in the heavy decoder structure.

B Efficient Self-Improved Training

LEHD uses SL for training, which raises concerns that the performance of LEHD may be upper bounded by the provided (nearly) optimal solutions and the requirement of optimal solutions for LEHD training may make it hard to solve more complex VRPs. We tackle these two concerns as follows.

B.1 RRC can break the performance upper limit of the labeled solution

Firstly, the performance of our proposed LEHD model with RRC is not limited by the quality of labeled solutions. To demonstrate this point, we train a new LEHD model with labeled solutions generated by OR-Tools of which the quality is far from the optimal solutions (e.g., with a 6 . 762% optimal gap). The results are shown in Table 10.

In this experiment, we use OR-Tools to generate suboptimal solutions for 100 , 000 CVRP100 instances, and use them to train the LEHD model with only 10 epochs (about 1 . 25 hours). Then we compare their performance on the same test set with 10k instances. According to the results,

although LEHD with greedy inference is outperformed by OR-Tools, LEHD+RRC can significantly outperform OR-Tools by a large margin with a reasonable runtime. If more training budget is allowed, the performance of LEHD(+RRC) can be further improved.

This result also demonstrates the importance and effectiveness of RRC in our proposed method.

B.2 LEHD can be trained by RL without any labeled solution

Secondly, it is possible to directly train the LEHD model using reinforcement learning that does not require any labeled data. Experimental results can be found in Table 11 where we train the LEHD model by RL on the small-scale TSP20 instances, and then test its generalization performance on TSP100. The training budget is 40 epochs each with 100 , 000 instances on TSP20 (roughly 9 hours). According to the results, trained by RL, LEHD can achieve a nearly optimal solution for TSP20, and RRC makes LEHD have a reasonably good generalization performance on TSP100.

One concern for the purely RL training for LEHD is the high computational cost and slow convergence speed, as analyzed in Table 4 in the main paper. We design an efficient training method to tackle this concern as explained in the next point.

B.3 An efficient self-improved training method for LEHD without any labeled solution

Finally, combining the properties from the above two points, we can design an efficient self-improved training method without any labeled solutions for LEHD that can generalize well to large-scale problems. The three key steps are:

Train LEHD by reinforcement learning with a reasonable computational budget;

Use LEHD+RRC to generate good solutions for a set of problem instances;

Further train LEHD by supervised learning with the solutions generated in step 2.

The results of this new training method (LEHD-SI) can be found in Table 12. For the TSP, we first train LEHD by RL and use LEHD+RRC to generate solutions for 200 , 000 TSP100 instances as labels for supervised learning. The training budget is 40 RL epochs and 185 SL epochs, which costs 2.7 days in total. The entire training process on TSP does not require external solvers to generate labeled solutions.

According to the results, with such a training method, LEHD+RRC can still obtain good generalization performance on TSP instances with up to 1 , 000 nodes with a reasonable inference time. Notably, it can outperform the very strong BQ (with bs16) method trained by supervised learning, especially on large-scale problem instances. The performance of LEHD can be further improved if more computational budget is available.

In summary, LEHD can be efficiently trained without any already labeled (nearly) optimal solutions. Therefore, it is possible to extend LEHD to tackle other practical CO problems where the optimal solution is hard to obtain. We will investigate how to design an even more efficient training method for LEHD as well as apply LEHD to solve other CO problems in future work.

C Implementation Details for TSP

C.1 Problem Setup

The task of solving a TSP instance with n nodes involves finding the shortest loop that visits each node exactly once and eventually returns to the first visited node. We generate TSP instances following the approach in Kool et al. [27], where the coordinates of n nodes are sampled uniformly at random from the unit square.

C.2 Implementation Details

For a TSP instance S , the node features ( s 1 , . . . , s n ) are the 2-dimentional coordinates of the n nodes in the graph.

In the original AM decoder, irrelevant nodes are masked during each construction step. In our model, we remove the embeddings of irrelevant nodes from the decoder input. This removal serves the same purpose as masking them in every decoder attention layer but also saves computational resources since the decoder is not required to perform computations related to irrelevant nodes. Consequently, for each construction step, the input node embeddings for the decoder only consist of the starting node embedding, the destination node embedding, and the available node embeddings.

̸

Here is an extended explanation of Equation 2 in the case of TSP. After L attention layers, ˜ H (0) is transformed to ˜ H ( L ) . Each vector ˜ h ( L ) i ∈ R d e ( i = 1 or 2 ) is transformed into a scalar u i by applying the linear projection W O ∈ R d e × 1 (i.e., u i = W O ˜ h ( L ) i ), where d e is the embedding dimension. Then p t = softmax( u ) is calculated.

D Implementation Details for CVRP

D.1 Problem Setup

A CVRP instance involves n customer nodes and one depot node, with each customer node i having a specific demand δ i that must be fulfilled. We aim to determine a set of sub-tours starting and ending at the depot such that the sum of demand satisfied by each sub-tour is within the capacity constraint D of the vehicle. Given the capacity constraint D , the objective is to minimize the total distance of the set of sub-tours. Similarly, following Kool et al. [27], we generate CVRP instances where the coordinates of customer nodes and depot nodes are sampled uniformly from the unit square. The demand δ i is sampled uniformly form { 1 , . . . , 9 } . The vehicle capacities, denoted by D , are 50, 80, 100, and 250 for corresponding values of N , which are 100, 200, 500, and 1000, respectively.

Following Kool et al. [28] and Drakulic et al. [12], we define the formation of a feasible solution for CVRP. Rather than treating a visit to the depot as a separate step, we use binary variables to indicate whether a customer node is reached via the depot or another customer node. Specifically, in a feasible solution, a node is assigned a value of 1 if it is reached via the depot and a value of 0 if it is reached through another customer node.

For example, a feasible CVRP solution { 0 , 1 , 2 , 3 , 0 , 4 , 5 , 0 , 6 , 7 , 0 } where 0 represents the depot, can be denoted as follows:

In this notation, the first row represents the sequence of visited nodes in the solution, and the second row indicates whether each node is reached via the depot or another customer node.

The purpose of using this notation is to ensure solution alignment. In CVRP instances, solutions with the same number of customer nodes may have varying numbers of sub-tours, leading to potential misalignment. By employing this notation, we can avoid such issues.

D.2 Implementation details

For CVRP, the node feature s i is represented as a 3-dimensional vector, comprising the 2-dimensional coordinates and the demand of node i . The demand of the depot is assigned a value of 0. Without loss of generality, we normalize the vehicle capacity D to ˆ D = 1 , and the demand δ i to ˆ δ i = δ i D [27].

In the decoder, the dynamically changing remaining capacity is added to both the starting node and destination node embeddings, resembling the approach employed in Kwon et al. [29]. Similar to TSP, the irrelevant node embeddings are excluded from the decoder input.

̸

Here is an extended explanation of Equation 2 in the case of CVRP. After L attention layers, ˜ H (0) is transformed to ˜ H ( L ) . Each vector ˜ h ( L ) i ∈ R d e ( i = 1 or 2 ) is projected to a 2-dimensional vector u i using the linear projection W O ∈ R d e × 2 ( i.e., u i = W O ˜ h ( L ) i ), where d e is the embedding dimension. Each u i corresponds to two actions associated with the node i : either being reached via the depot or another customer node. This relation corresponds to the notation mentioned in equation 4. Subsequently, u is flattened, and the softmax is utilized to compute the probability associated with each possible action.

E Solution Visualizations

Table 2 shows the test results on TSPLib and CVRPLib instances with different sizes and distributions. For TSPLib, we report the results on 2D Euclidean TSP instances with sizes smaller than 5000 (up to 4461). For CVRPLib, we report the results on the 2D Euclidean instances without additional constraints such as time windows.

Figures 3 show the solutions of "pr2392" instance in TSPLib. Figures 4 show the solutions of "X-n1001-k43" instance in CVRPLib. For each figure, panel (a) shows the optimal solution, and panel (b), (c), and (d) shows the solution generated by POMO, BQ, and LEHD, respectively.

E.1 Solution visualizations of two TSPLib instances

(a) Optimal solution

(b) POMO aug

×

8: gap 69.6%

(c) BQ bs16: gap 29.4%

E.2 Solution visualizations of two CVRPLib instances

(a) Optimal solution

(b) POMO aug

×

8: gap 24.8%

(c) BQ bs16: gap 6.9%

F Licenses

The licenses for the codes and the datasets used in this work are listed in Table 13.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a neural network model in the form of a Light Encoder Heavy Decoder, as well as its training method for solving combinatorial optimization problems. In addition, this paper suggests an RPC method based on updating partial solutions as a way to improve solutions during the inference process. These methods show promising results in experiments with large scale Travelling Salesman Problems (TSP) and Capacitated Vehicle Routing Problems (CVRP).\n\nThe approach of learning the state of various node sizes through a heavy decoder is promising. However, using supervised learning as a training method and optaining an optimal solution for training with an existing algorithm has limitations in general use.\n\nSTRENGTHS:\n**S1.** The architecture of Heavy Decoder Model to yield optimal outputs across a range of node sizes\n\n**S2.** Showing promising experimental results on large-scale TSP and CVRP instances\n\n**S3.** Well-written and easy to read\n\nWEAKNESSES:\n**W1.** Weak novelty of training method and RPC\n\n**W2.** If there is no optimal solution, or if there is no existing high-performance algorithm such as Concorde/HGS, learning by the method of this paper is not possible, but many practical CO problems cannot secure an optimal solution. Therefore, for most practical CO problems where the optimal solution cannot be known, it is difficult to apply the method in this paper.\n\n**W3.** The optimization degree of the algorithm that outputs the optimal solution (or near optimal solution) of the training data seems to be the upper limit of the optimization degree of LEHD. \n\n**W4.** The number of instances used in the node experiments of 200, 500, and 1000 is too small at 128. It is expected that the experimental values may change as the 128 samples are altered. On the other hand, previous studies [20,8,12] mainly used 1k instances in experiments with 200 or more nodes.\nexperiments.\n\nQUESTIONS:\n**Q1.** From the experimental results, the inference of the LEHD greedy model was performed faster than the POMO using the HELD model. In the case of TSP 1000, the encoder performs 1 operation and the decoder 1000 operations, so I don't understand how LEHD can be faster than HELD(POMO). If the reason why HELD (POMO) is slower is because of 'augx8', I would like to know the difference in inference time between the HELD(POMO/AM) without augx8 and LEHD.\n\n**Q2.** How to select partial solution to perform update in RPC?\n\n**Q3.** In the CVRP results in Table 1, HGS performed better than LKH3 and was used as the optimal solution for LEHD learning, so it seems better to organize the table with HGS as GAP 0.\n\n**Q4.** Publication information of reference [56] is missed.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a learning-based approach called the Light Encoder and Heavy Decoder (LEHD) to solve combinatorial optimization (CO) problems. The proposed method learns less from the static graphs (via a light encoder), and pays more attention to dynamically capturing the relationships between all available nodes of varying sizes (via a heavy decoder), which allows for better generalization to various scales. The devised paradigm naturally divides the problem into sub-problems, and the pipeline is designed to first predict the general route greedily, followed by refining the route using its local construction policy. Experimental evaluations demonstrate the state-of-the-art performance of the proposed method, particularly in terms of its generalization capability.\n\nSTRENGTHS:\n1.The methodology design is insightful and logical. The division into the shortest Hamilton path problem is meaningful for addressing routing problems such as the Traveling Salesman Problem (TSP) and Vehicle Routing Problem (VRP), as it facilitates local improvement naturally.\n\n2.The empirical results are promising especially for the generalization ability.\n\n3.The paper is well-structured and easy to follow.\n\nWEAKNESSES:\n1.The RCC algorithm bears resemblance to methods that employ a divide-and-conquer strategy, such as [1] and [2]. It is recommended for the authors to explore and discuss this line of research.\n\n[1] Generalize Learned Heuristics to Solve Large-scale Vehicle Routing Problems in Real-time. ICLR 2023.\n\n[2] Learning to Delegate for Large-scale Vehicle Routing. NeurIPS 2021.\n\n2.The RCC algorithm does not learn to select regions for local improvement, instead randomly enforcing the local policy, which hampers its efficiency.\n\nMinor:\n\n1.Line 166: refind -> refine.\n2.Line 224: construction-base -> construction-based.\n\nQUESTIONS:\n1.Why is LKH not included in Table 1? LKH is expected able to achieve optimality for the problem scales in Table 1, but the runtime comparison is essential and should not be missed.\n\nPlease also see the discussion of weaknesses.\n\nLIMITATIONS:\nPlease see above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on a class of NP-hard combinatorial optimization problems called Vehicle Routing Problems (VRP) which is of wide practical interest. This paper proposes a novel Light Encoder and Heavy Decoder (LEHD) model for enabling neural construction methods well handling large-scale routing problems. It incorporates a data-efficient training scheme in the supervised manner and a flexible solution construction mechanism for further enhancing the performance. By training on small-scale problem instances, the LEHD model achieves robust and promising performance on problems with much larger sizes, especially for TSP and CVRP.\n\nSTRENGTHS:\n1. The motivation and training process is clear and easy to read.\n2. LEHD could dynamically capture the relationships between all available nodes of varying sizes, which enables the model to generalize well to problems of various scales.\n3. Sufficient experiments are conducted to prove the effectiveness and the generalization of LEHD, which has promising results on large-scale TSP and CVRP.\n\nWEAKNESSES:\n1. There is some confusion about the training time, since training time is also important.\n2. The proposed method might be limited in simple VRPs, since obtaining optimal labeled solutions for more complex VRP variants is hard.\n3. The LEHD does not show much superiority to HGS.\n4. The presentation of this paper needs to be further improved. Some details are not clearly described and there are some typos.\n\nQUESTIONS:\n1. What is \"w\" in the experiments? Does it also a random number?\n2. For generalizing LEHD to unseen large-scale problems, how does LEHD construct the whole solution? \n3. Why not apply \"sampling\" strategy on POMO with the same iterations of RRC? For example, POMO (sampling) with 100 trials should be compared to LEHD RRC 100.\n4. Are the solutions obtained by HGS optimal solutions? To my knowledge, HGS is a meta-heuristic method.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

comments: SUMMARY:
In summary, this work proposed a new solution that generalizes well on large-scale problem instances for routing problems. A data-efficient training scheme and solution construction mechanism are further proposed for efficient training. Experiments verify the effectiveness of the proposed method, and an impressive gap is observed between the proposed method and the existing works.

STRENGTHS:
1. The presentation of this manuscript was great, and I enjoy reading this paper.

2. This work proposed a simple yet effective strategy to solve routing problems, which is able to generate nearly optimal solutions for large-scale settings. It also generalized well on real-world datasets. In addition, the effectiveness of such a design is well studied in Section 6.2. I  believe this work would be very insightful for the community.

WEAKNESSES:
1. The training details need more illustration. How did you train on small-scale problem instances? Is the "small-scale problem instances" means the "argument data" you generated?

QUESTIONS:
1. How did you train on small-scale problem instances? Is the "small-scale problem instances" means the "argument data" you generated? 

2. There are some existing works that aim to tackle large-scale TSP instances, for example, [40] and [7] mentioned in your paper. Why no comparison with these works?

LIMITATIONS:
N.A.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07