SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dataset. To ensure the hardness and variety of training distributions and to guarantee that the training instances are not seen during testing, we generate 7.2 million training instances from mixed distributions. Data generation is done by first sampling a uniform instance and then non-repetitively applying three randomly mutation operators $^4$  of TSPGEN to the same instance. So a single training instance is of mixed distributions and does not belong to any single distributions in the testing. During the inference, unlike most existing neural heuristics that are only tested on a uniform distribution, we evaluated all methods on various distributions, including Explosion, Compression, Cluster, Expansion, and Rotation, to assess the generalization performance against distribution shift. These distributions are more diverse in terms of both visual and quantitative measures compared to the uniform distribution [59], making it fairly challenging for neural heuristics to generalize. We test baselines on 10,000 instances in total, with each of the five testing distributions containing 2,000 instances. Therefore, the training instances do not fall into any type of testing distribution and all the test instances are sampled from distribution not seen during training. HAC is trained on instances created by the built-in data generator as its design relies on this process. The training of DROP requires class labels of the input instance and we specify the class of the last applied mutation operation for it. AMDKD requires manually assigning a single distribution for training each teacher model, which is less applicable to our setting where each single training instance is of mixed distributions. Thus, we follow its original setting for the best performance, and directly utilize the pre-trained AMDKD to solve instances from TSPLib and CVRPLib (Table 5).
$^{2}$ https://www.math.uwaterloo.ca/tsp/concorde.html
<sup>3</sup>https://github.com/chkwon/PyHygese
<sup>4</sup>https://github.com/jakobbossek/tpsgen/tree/master/R
4.1 Cross-distribution Generalization on TSP
Table 1: Results of cross-distribution generalization on TSP50
Distribution	Metric	Concorde	AM	POMO	MDAM	HAC	DROP	EL-DRL
Explosion	Len.	4.74	4.88	4.84	4.87	4.85	4.88	4.83
Gap	0.00%	2.95%	2.11%	2.74%	2.32%	2.95%	1.90%
Compression	Len.	5.22	5.37	5.33	5.36	5.35	5.34	5.32
Gap	0.00%	2.87%	2.11%	2.68%	2.49%	2.30%	1.92%
Cluster	Len.	5.37	5.56	5.50	5.54	5.53	5.52	5.48
Gap	0.00%	3.54%	2.42%	3.17%	2.98%	2.79%	2.05%
Expansion	Len.	4.44	4.58	4.55	4.57	4.58	4.60	4.53
Gap	0.00%	3.15%	2.48%	2.93%	3.15%	3.60%	2.03%
Rotation	Len.	4.54	4.69	4.64	4.67	4.66	4.64	4.63
Gap	0.00%	3.30%	2.20%	2.86%	2.64%	2.20%	1.98%
Avg. Inf. Time (s)	0.08	0.07	0.01	0.04	0.08	0.01	0.02
Table 2: Results of cross-distribution generalization on TSP100
Distribution	Metric	Concorde	AM	POMO	MDAM	HAC	DROP	EL-DRL
Explosion	Len.	6.09	6.31	6.22	6.29	6.38	6.27	6.21
Gap	0.00%	3.61%	2.13%	3.28%	4.76%	2.96%	1.97%
Compression	Len.	6.89	7.16	7.06	7.12	7.18	7.12	7.03
Gap	0.00%	3.92%	2.47%	3.34%	4.21%	3.34%	2.03%
Cluster	Len.	7.26	7.58	7.45	7.53	7.63	7.46	7.41
Gap	0.00%	4.41%	2.62%	3.72%	5.10%	2.75%	2.07%
Expansion	Len.	5.57	5.80	5.72	5.78	5.88	5.74	5.69
Gap	0.00%	4.13%	2.69%	3.77%	5.57%	3.05%	2.15%
Rotation	Len.	6.02	6.28	6.17	6.24	6.34	6.23	6.14
Gap	0.00%	4.32%	2.49%	3.65%	5.32%	3.49%	1.99%
Avg. Inf. Time (s)	0.50	0.22	0.02	0.16	0.23	0.03	0.04
We evaluated the cross-distribution generalization of our EL-DRL on TSP50 and TSP100. Tables 1 and 2 gathered the averaged tour length and gaps of all baselines compared to the exact solver Concorde on unseen instances from five different distributions. Although all baselines were trained on mixed distributions, they did not generalize well to other specific distributions (especially for AM), resulting in drastically deteriorated performance (about  $2.5\%$  to  $4\%$  gaps) compared to the i.i.d testing results they reported in the original paper. For example, POMO reports only  $0.03\%$  and  $0.14\%$  gaps compared to LKH3 [58] on TSP50 and TSP100, respectively, when trained and tested on a uniform distribution in [9]. However, its gaps increase up to  $2.69\%$  in our cross-distribution experiments. This phenomenon implies that these neural baselines are relatively incapable of learning from the underlying common patterns on complex training distributions. In contrast, our EL-DRL significantly improved generalization performance on the five respective testing distributions, achieving the smallest gap among neural heuristic baselines. For example, our EL-DRL reduced the gaps by  $2.34\%$  ( $2.07\%$  vs.  $4.32\%$ ) compared to AM and achieved a  $0.55\%$  ( $2.07\%$  vs  $2.62\%$ ) gap reduction on the TSP100 with Cluster distribution compared to the best neural baseline POMO. Notice that while multiple decoders in MDAM slightly improved its performance compared to AM, it failed to generalize to various distributions with gaps exceeding  $3.2\%$  on TSP100. This indicates that simply composing multiple models (even with KL divergence) does not necessarily result in good generalization performance. Meanwhile, EL-DRL also consistently beats the recent generalization-specialized DROP and HAC in terms of both gaps and time efficiency. Especially, DROP exhibits unstable generalization performance and HAC does not generalize well on TSP100 ( $4.21 - 5.32\%$  gaps), possibly due to special data processing requirements for training, which hinder their application to our scenario. In short, our EL-DRL outperforms these two state-of-the-art approaches in generalizing to cross-distribution.
Table 3: Results of cross-distribution generalization on CVRP50
Distribution	Metric	HGS	AM	POMO	MDAM	DROP	EL-DRL
Explosion	Len.	9.79	10.02	9.92	9.98	9.91	9.88
Gap	0.00%	2.35%	1.33%	1.94%	1.23%	0.82%
Compression	Len.	10.14	10.39	10.28	10.35	10.32	10.24
Gap	0.00%	2.47%	1.38%	2.07%	1.78%	0.69%
Cluster	Len.	10.35	10.60	10.49	10.56	10.49	10.45
Gap	0.00%	2.42%	1.35%	2.03%	1.35%	0.87%
Expansion	Len.	9.40	9.64	9.53	9.61	9.61	9.50
Gap	0.00%	2.55%	1.38%	2.23%	2.23%	0.85%
Rotation	Len.	9.43	9.66	9.56	9.62	9.58	9.53
Gap	0.00%	2.44%	1.38%	2.01%	1.59%	0.74%
Avg. Inf. Time (s)	30	0.22	0.01	0.13	0.01	0.02
Table 4: Results of cross-distribution generalization on CVRP100
Distribution	Metric	HGS	AM	POMO	MDAM	DROP	EL-DRL
Explosion	Len.	14.30	14.79	14.65	14.76	14.70	14.62
Gap	0.00%	3.43%	2.45%	3.22%	2.80%	2.24%
Compression	Len.	14.82	15.36	15.21	15.28	15.32	15.18
Gap	0.00%	3.64%	2.63%	3.10%	3.37%	2.43%
Cluster	Len.	15.44	15.99	15.83	15.91	15.90	15.79
Gap	0.00%	3.56%	2.53%	3.04%	2.98%	2.27%
Expansion	Len.	13.70	14.18	14.02	14.17	14.19	13.99
Gap	0.00%	3.50%	2.34%	3.43%	3.58%	2.12%
Rotation	Len.	13.97	14.46	14.30	14.40	14.39	14.28
Gap	0.00%	3.51%	2.36%	3.08%	3.01%	2.22%
Avg. Inf. Time (s)	30	0.29	0.03	0.17	0.05	0.08
4.2 Cross-distribution Generalization on CVRP
Pertaining to CVRP, we present the averaged tour lengths and gaps to the best solutions (obtained from the HGS with a 30s runtime), along with the average time to inference on unseen instances of CVRP50 and CVRP100 from five distributions in Tables 3 and 4, respectively. Overall, the HGS solver performs the best regarding tour length, as this heuristic is highly specialized for CVRP. Notably, we observe greater advantages of our method against neural baselines on this harder problem than on TSPs. Our EL-DRL achieves superior results on unseen instances of various distributions, with significantly small gaps  $(0.69 - 0.87\%)$  on CVRP50, and especially outperforms POMO by around  $0.7\%$  on Compression distribution. Table 4 shows that for neural baselines, generalizing to various distributions on CVRP100 is also challenging, with considerable gaps ranging from  $2.36\%$  to  $3.64\%$ . In contrast, our method consistently delivers high-quality solutions with shorter lengths and is less sensitive to distribution shifts, which attains gaps of close to  $2\%$  for cross-distribution generalization on CVRP100. This superiority verifies the effectiveness of our idea of leveraging the strength of handling different distributions through cooperation between sub-policies. Regarding efficiency, all neural baselines can solve an instance in less than one second. Owing to our design of lightweight ensemble architecture and parallel inference, EL-DRL does not incur much inference time than the counterparts using a single model while yielding better performance and even exhibiting faster inference than AM and MDAM.
4.3 Real World benchmarks
We continue to evaluate our EL-DRL by taking public benchmark datasets TSPLib [53] and CVRPLib [54] as the testbed, to prove our approach is also efficient in solving more realistic and varied distributions. TSPLib is a library of TSP instances from various sources and of various types, making it desirable yet challenging for assessing generalization. For TSPLib, results are collected from 25 instances which problem sizes ranging from 50 to 300. For CVRPLib, the dataset is XML100, which
Table 5: Results on Real-World Benchmarks
Dataset	Metric	Opt.	AM	POMO	MDAM	HAC	DROP	AMDKD	EL-DRL
TSPLib	Len.	6.86	8.02	7.44	7.73	8.65	7.48	7.45	7.42
Gap	0.00%	10.53%	5.16%	7.82%	16.75%	5.79%	5.30%	4.85%
Avg. Time (s)	-	0.48	0.47	0.36	0.48	0.35	0.53	0.95
CVRPLib	Len.	16.97	17.82	17.70	17.76	-	17.84	17.67	17.65
Gap	0.00%	6.05%	4.51%	4.90%	-	5.25%	4.32%	4.12%
Avg. Time (s)	-	0.29	0.03	0.15	-	0.05	0.03	0.08
contains 10,000 CVRP100 instances from heterogeneous distributions. We normalize the coordinates in each instance to a range of [0, 1] for all neural methods, which also include the neural baseline AMDKD [15].
The upper half of Table 5 illustrates the averaged results on TSPLib instances. These results demonstrate that EL-DRL generalizes well to realistic distributions and varied sizes, in a small gap  $(4.85\%)$ , and it is much lower than the gaps of neural heuristics (i.e., around  $5\% -17\%$ ). In particular, the advantage of our EL-DRL over MDAM implies learning similar ensembles may hold back the generalization performance. Compared to the poor results of HAC, which learns only from Gaussian distribution, we can conclude that training the models on various heterogeneous distributions could be more beneficial. In comparison to the most recent generalization-specialized AMDKD, our EL-DRL brings a reduction in the gap by  $0.51\%$ . The lower half of Table 5 shows the averaged lengths and gaps on (harder) CVRPLib instances. Our EL-DRL again outperforms the top two baselines, AMDKD and POMO, by margins of  $0.20\%$  and  $0.40\%$ , respectively. The results on the realistic benchmarks underscore the remarkable capability of our EL-DRL in solving miscellaneous instances that were not present during the training phase, highlighting its robustness in coping with distribution shifts.
Table 6: Ablation Studies on CVRP100
Method	POMO	Multi-POMO	EL-DRL	w/o BRI	w/o LT	w/o LP
Avg. Length	14.801	14.784	14.747	14.768	14.752	14.756
Avg. Gap	2.423%	2.311%	2.055%	2.201%	2.090%	2.118%
We also carry out ablation studies to demonstrate the effectiveness of key designs in our EL-DRL, by generating 10,000 CVRP100 instances as an exemplary case and presenting the results in Table 6. To investigate the disadvantage of high similarity between ensemble policies, we deploy a single POMO and an ensemble of four POMOs (Multi-POMO), respectively, to compare with our EL-DRL. Additionally, we ablate Bootstrap with Random Initialization (BRI), the Theil index regularization  $(\mathcal{L}_T)$  and parameter dissimilarity regularization  $(\mathcal{L}_P)$ , then report the averaged results for the 10,000 instances of five distributions on CVRP100. First, the single POMO model relies on its own learning ability, increasing the susceptibility to over-fitting and leading to poor generalization performance. However, simply stacking POMO to multiple copies (Multi-POMO) cannot overcome this limitation and only brings slight improvement compared to the single POMO, which is primarily attributed to the larger amount of model parameters and in accordance with the finding in [60]. By contrast, our EL-DRL attains a  $0.37\%$  reduction of the overall gaps compared to the single POMO by collaboratively solving instances on various distributions. Excluding the Bootstrap with random initialization, sub-policies cannot receive distinct loss signals and converge to similar ones, and thus the gap of the variant without BRI increases significantly to  $2.201\%$  compared to that of EL-DRL  $(2.055\%)$ . Moreover, removing two regularization terms for maximizing diversity results in redundant parameters for sub-policies and hinders their expressive ability, as evidenced by the drop in generalization performance shown in the last two columns of Table 6. In summary, the designs of our EL-DRL are largely orthogonal and can be effectively integrated to maintain ensembles of diverse sub-models, which are capable of capturing different aspects of various distributions.
This paper presents an ensemble-based DRL approach for solving vehicle routing problems, which fosters the capability of existing neural heuristics in generalizing against distribution shifts. By enforcing distinct reinforcement loss signals on each sub-policy and integrating regularization to ensure diversity across sub-policies, our EL-DRL learns to leverage the strengths of individuals in solving different instances with various distributions. In our experiments, we demonstrate that our proposed method delivers high-quality solutions for both synthetic instances of various distributions and heterogeneous instances from real-world benchmarks, confirming the potential of the ensemble idea in contributing to more generalizable and robust neural heuristics.
While it is not the focus of this paper, our method might be less effective in handling large instances. A major reason is that the scalability of our EL-DRL highly depends on the sub-model, where POMO (the used sub-model) performs inferior on large ones. Moreover, since our EL-DRL follows the ensemble paradigm, the training needs more computational resources than the single one. In the future, we would like to, 1) foster the scalability of our method by exploring decomposition schemes such as the divide-and-conquer [61], so as to allow it to solve large instances more efficiently; 2) enhance the computational efficiency of our method by introducing sparsity in ensemble learning [62], so as to allow it to train more sub-models while reducing the training overhead; 3) enable the interpretability [63] and further strengthen the collaboration among the sub-models of our method; and 4) extend our method to non-routing combinatorial optimization problems.
This research was supported by the National Natural Science Foundation of China under Grant 62102228, the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1 grant, and the RIE2025 MTC IAF-PP funding (M23L4a0001).
[54] Eduardo Queiroga, Ruslan Sadykov, Eduardo Uchoa, and Thibaut Vidal. 10,000 optimal cvrp solutions for testing machine learning based heuristics. In AAAI-22 Workshop on Machine Learning for Operations Research (ML4OR), 2021.
[55] David Applegate, Robert Bixby, Vasek Chvatal, and William Cook. Concorde tsp solver. https://www.math.uwaterloo.ca/tsp/concorde.html, 2006.
[56] Thibaut Vidal. Hybrid genetic search for the cvrp: Open-source implementation and swap neighborhood. Computers & Operations Research, 140:105643, 2022.
[57] André Hottung and Kevin Tierney. Neural large neighborhood search for the capacitated vehicle routing problem. In Proceedings of the 24th European Conference on Artificial Intelligence (ECAI), 2020.
[58] Keld Helsgaun. An extension of the lin-kernighan-helsgaun tsp solver for constrained traveling salesman and vehicle routing problems. Roskilde: Roskilde University, pages 24-50, 2017.
[59] Jakob Bossek, Pascal Kerschke, Aneta Neumann, Markus Wagner, Frank Neumann, and Heike Trautmann. Evolving diverse tsp instances by means of novel and creative mutation operators. In Proceedings of the 15th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, pages 58-71, 2019.
[60] G Brown. Diversity in neural network ensembles, [ph. d. thesis]. University of Birmingham, 2004.
[61] Sirui Li, Zhongxia Yan, and Cathy Wu. Learning to delegate for large-scale vehicle routing. Advances in Neural Information Processing Systems, 34, 2021.
[62] Li Zhang and Wei-Da Zhou. Sparse ensembles using weighted combination methods based on linear programming. Pattern Recognition, 44(1):97-106, 2011.
[63] Chia-Hsiu Chen, Kenichi Tanaka, Masaaki Kotera, and Kimito Funatsu. Comparison and improvement of the predictability and interpretability with ensemble learning models in qspr applications. Journal of cheminformatics, 12:1-16, 2020.