SlowGuess/NeurIPS_2023_OCR huggingface dataset:
E.2 Generalization on real-world datasets
We further evaluate the generalization performance of our NeuOpt models, which were trained on TSP100 and CVRP100, by testing them on real-world TSPLIB and CVRPLIB instances containing up to 200 nodes (customers). These real-world instances differ significantly from our training ones
Table 10: Generalization results of our NeuOpt (10 runs) on TSPLIB real-world dataset.
Instances	AM-mix	POMO-mix	AMDKD (POMO)	DACT (sol. = 10k)	Ours (sol. = 10k)	DACT (sol. = 40k)	Ours (sol. = 25k)
Avg.	Best	Avg.	Best	Avg.	Best	Avg.	Best
KroA100	4.02%	0.02%	0.02%	1.13%	0.41%	0.00%	0.00%	0.21%	0.00%	0.00%	0.00%
KroB100	4.73%	0.25%	0.41%	0.67%	0.26%	0.05%	0.00%	0.38%	0.26%	0.00%	0.00%
KroC100	7.60%	0.01%	0.02%	2.56%	0.39%	0.00%	0.00%	0.77%	0.00%	0.00%	0.00%
KroD100	8.45%	0.27%	0.09%	2.23%	0.45%	0.05%	0.00%	0.64%	0.45%	0.02%	0.00%
KroE100	3.61%	0.50%	0.53%	1.14%	0.46%	0.02%	0.00%	0.60%	0.28%	0.02%	0.00%
eil101	5.45%	1.90%	2.55%	2.23%	1.59%	0.02%	0.00%	1.75%	1.27%	0.00%	0.00%
lin105	17.29%	0.36%	0.36%	6.37%	2.69%	0.00%	0.00%	0.89%	0.00%	0.00%	0.00%
pr107	71.89%	0.78%	1.62%	4.33%	2.87%	1.33%	0.81%	4.07%	2.87%	0.70%	0.54%
pr124	5.16%	0.00%	0.43%	0.75%	0.65%	0.12%	0.00%	0.30%	0.00%	0.05%	0.00%
bier127	133.13%	0.80%	0.65%	3.39%	2.01%	0.77%	0.51%	2.43%	1.71%	0.59%	0.32%
ch130	1.98%	0.60%	0.69%	2.72%	0.41%	0.40%	0.29%	1.09%	0.41%	0.35%	0.29%
pr136	3.54%	1.76%	1.49%	5.65%	1.45%	1.06%	0.26%	4.58%	0.58%	0.80%	0.02%
pr144	13.82%	0.85%	0.72%	2.63%	2.07%	0.62%	0.17%	1.16%	0.44%	0.19%	0.03%
ch150	2.33%	0.70%	1.25%	1.77%	0.58%	0.36%	0.00%	1.23%	0.87%	0.26%	0.00%
KroA150	11.22%	0.75%	1.07%	4.21%	1.74%	0.30%	0.00%	5.15%	0.66%	0.11%	0.00%
KroB150	9.39%	0.78%	0.76%	3.22%	1.64%	0.95%	0.57%	2.45%	1.64%	0.55%	0.15%
pr152	16.31%	1.35%	2.16%	3.10%	1.57%	0.79%	0.34%	2.23%	0.77%	0.26%	0.00%
rat195	39.35%	4.25%	4.68%	3.92%	2.02%	2.40%	1.42%	3.28%	2.02%	2.02%	1.25%
KroA200	16.77%	1.61%	1.83%	5.43%	3.05%	2.18%	1.67%	3.01%	2.33%	1.59%	1.39%
KroB200	15.75%	0.77%	2.36%	8.27%	5.46%	5.52%	3.88%	6.39%	5.23%	4.07%	2.93%
Avg. Gap	19.59%	0.92%	1.18%	3.29%	1.59%	0.85%	0.50%	2.13%	1.09%	0.58%	0.35%
with much larger sizes and different node (customer, depot) distributions. We compare our NeuOpt with the state-of-the-art L2S solver, DACT [9], reporting both the best and average gaps across 10 independent runs. We also compare with the results from a recent work [11] that focused on enhancing the generalization capabilities of existing L2C solvers, where we include AM-mix, POMO-mix, and AMDKD (POMO) as baselines. According to Bi et al. [11], AM-mix and POMO-mix are enhanced AM and POMO models, respectively, which were trained on a combination of uniform, cluster, and a mix of uniform and cluster instances to bolster their generalization performance. The AMDKD approach leverages adaptive multi-distribution knowledge distillation to glean various forms of knowledge from multiple teachers trained on exemplar distributions, so as to yield a lightweight yet highly adaptable student POMO model. Since our primary focus is on the generalization capability of the deep model itself, we do not consider post-hoc per-instance processing boosters (e.g., EAS [5]). The results for TSPLIB and CVRPLIB are presented in Table 10 and Table 11, respectively.
TSPLIB results. As depicted in Table 10, our NeuOpt model consistently yields lower average gaps than the state-of-the-art DACT solver when both are allowed to explore 10k solutions. Meanwhile, our NeuOpt exploring 25k solutions could even surpass the DACT exploring 40k solutions. When compared to L2C solvers, NeuOpt yields lower gaps than AM-mix and POMO-mix, and even the AMDKD (POMO) model that is explicitly designed to enhance cross-distribution generalization performance. These results indicate the desired generalization capabilities of our NeuOpt model.
CVRPLIB results. As depicted in Table 11, our NeuOpt could still deliver the lowest average gaps on CVRPLIB instances compared to other baselines. Despite the promising results, we acknowledge that there is still room for improvement in enhancing the generalization of L2S solvers (including our NeuOpt), which is akin to the efforts made by AMDKD in advancing L2C solvers. Meanwhile, as discussed in Section 7, the integration of post-hoc per-instance processing boosters (e.g., EAS [5]) could potentially further improve both the in-distribution and cross-distribution generalization performance of existing L2S solvers. We consider the above ideas as important directions for future research.
E.3 Results on TSP-200 and CVRP-200
We carry out experiments on larger scales, i.e., training our NeuOpt on TSP200 and CVRP200. For the size-specific hyper-parameters, we use  $\xi = 0.125$  for both problems; and use the CVRP problem settings: capacity  $\Delta = 70$  and 30 dummy depots. All other hyper-parameters remain unchanged. The performance of our NeuOpt models for TSP-200 and CVRP-200 are depicted in Table 12, where we compare our models with traditional heuristics (as per Table 1). The results indicate that our models consistently find close-to-optimal solutions, even outperforming the LKH3 solver on CVRP-200. We note that due to prohibited longer training times, existing L2S solvers (like DACT [9]) may not be efficient for training on size 200, which underscores the better scalability of our approach.
Table 11: Generalization results of our NeuOpt-GIRE (10 runs) on CVRPLIB real-world dataset.
Instances	AM-mix	POMO-mix	AMDKD (POMO)	DACT (sol. = 10k)	Ours (sol. = 10k)	DACT (sol. = 60k)	Ours (sol. = 60k)
Avg.	Best	Avg.	best	Avg.	Best	Avg.	best
X-n101-k25	9.92%	10.58%	6.19%	3.11%	1.68%	3.40%	1.77%	1.86%	1.32%	1.84%	0.51%
X-n106-k14	6.06%	2.71%	1.84%	2.77%	2.28%	1.66%	1.01%	2.09%	1.72%	1.41%	1.01%
X-n110-k13	4.66%	1.36%	2.30%	2.38%	0.37%	2.26%	0.98%	1.10%	0.00%	1.11%	0.27%
X-n115-k10	14.83%	6.76%	5.27%	1.43%	0.08%	1.18%	0.03%	0.68%	0.02%	0.61%	0.00%
X-n120-k6	20.71%	4.99%	2.04%	6.01%	3.49%	2.34%	0.89%	4.00%	2.71%	0.95%	0.52%
X-n125-k30	24.00%	12.32%	5.46%	6.01%	4.76%	4.71%	3.15%	5.51%	4.76%	3.55%	2.58%
X-n129-k18	6.54%	2.27%	1.76%	5.28%	3.98%	2.84%	1.64%	3.91%	3.48%	2.03%	1.26%
X-n134-k13	16.43%	3.74%	3.79%	6.40%	3.77%	3.48%	2.42%	3.28%	2.32%	2.50%	1.73%
X-n139-k10	10.03%	3.41%	2.69%	3.05%	1.65%	3.85%	2.39%	1.13%	0.53%	2.28%	0.99%
X-n143-k7	16.84%	5.01%	4.12%	5.15%	3.92%	2.05%	1.29%	3.37%	0.89%	1.27%	0.78%
X-n148-k46	42.24%	22.48%	8.16%	4.75%	3.61%	6.30%	4.70%	4.24%	2.60%	4.66%	2.96%
X-n153-k22	16.50%	11.35%	11.17%	7.62%	4.63%	9.16%	6.89%	4.26%	3.22%	6.71%	6.00%
X-n157-k13	17.86%	6.36%	3.40%	3.36%	2.70%	3.50%	2.71%	2.60%	2.09%	2.67%	2.33%
X-n162-k11	4.41%	5.75%	5.41%	2.68%	1.82%	3.02%	1.92%	1.57%	1.10%	2.43%	1.82%
X-n167-k10	5.49%	4.94%	4.11%	5.75%	4.36%	5.55%	3.64%	4.83%	4.05%	4.34%	3.04%
X-n172-k51	36.86%	9.29%	9.06%	5.42%	3.66%	8.85%	5.92%	3.78%	3.37%	6.48%	5.43%
X-n176-k26	11.40%	13.25%	11.25%	10.91%	7.74%	11.20%	8.00%	9.13%	7.74%	7.81%	5.86%
X-n181-k23	6.83%	13.47%	4.64%	3.65%	3.05%	2.82%	2.19%	2.51%	1.89%	2.33%	2.07%
X-n186-k15	7.04%	6.97%	9.06%	8.01%	6.80%	5.54%	3.45%	6.10%	5.01%	4.71%	4.01%
X-n190-k8	23.79%	11.38%	3.50%	7.86%	6.36%	8.06%	6.18%	6.42%	5.81%	6.65%	5.76%
X-n195-k51	30.76%	10.59%	15.96%	7.74%	6.60%	9.11%	7.53%	6.25%	5.14%	7.35%	6.12%
Avg. Gap	15.87%	8.05%	5.77%	5.21%	3.68%	4.80%	3.27%	3.74%	2.85%	3.51%	2.62%
Table 12: Results of our NeuOpt approach on solving TSP-200 and CVRP-200 instances.
Methods	Model Type	TSP-200	CVRP-200
Obj.	Gap	Time	Obj.	Gap	Time
Concorde [54]	Exact	10.687	-	23m	-
HGS [21]	Heuristics	-	21.756	-	19.8h
LKH [20, 51]	Heuristics	10.687	0.00%	2.3h	22.010	1.17%	21.6h
Ours (DA=5, T=1k)	L2S/RL	10.732	0.42%	28m	22.214	2.11%	58m
Ours (DA=5, T=5k)	L2S/RL	10.696	0.09%	2.4h	21.960	0.94%	4.8h
Ours (DA=5, T=10k)	L2S/RL	10.692	0.04%	4.7h	21.904	0.68%	9.6h
Ours (DA=5, T=20k)	L2S/RL	10.689	0.02%	9.4h	21.861	0.48%	19.2h
Ours (DA=5, T=30k)	L2S/RL	10.688	0.01%	14.1h	21.842	0.39%	1.2d
Ours (DA=5, T=40k)	L2S/RL	10.688	0.01%	18.8h	21.830	0.34%	1.6d
E.4 Inference with multiple GPUs
Our implementation allows for accelerated inference using multiple GPUs. Recall that in Table 1, the recorded time is for solving a total of 10,000 instances on a single GPU. For practical applications, users can achieve significantly shorter runtimes using multiple GPUs, as illustrated in Table 13.
Table 13: Inference time of our NeuOpt for solving 10k instances using different numbers of GPUs.
Methods	TSP-100	CVRP-100
1GPU (11GB)	2GPU (22GB)	4GPU (44GB)	1GPU (11GB)	2GPU (22GB)	4GPU (44GB)
NeuOpt (DA=1,T=1k)	17m	8m	5m	28m	14m	8m
NeuOpt (DA=1,T=5k)	1.4h	42m	23m	2.3h	1.2h	36m
NeuOpt (DA=1,T=10k)	2.8h	1.4h	45m	4.6h	2.3h	1.2h
E.5 Visualization of exploration behaviour
Figure 12 presents the objective values (search trajectories) of the visited feasible or infeasible solutions within the first 50 search steps, when using a pre-trained NeuOpt-GIRE model to solve five randomly generated CVRP-20 instances, respectively. The blue line depicts the objective values of each visited solution where we use green dots to mark feasible solutions and use red crosses to mark infeasible ones. The red and green dotted lines represent the best-so-far infeasible and feasible objective values, respectively. As observed from the figure, our learned model exhibits a propensity to explore infeasible regions at the beginning of the search, so as to rapidly decrease the objective values. It then learns to correct the constraint violations on the found best-so-far infeasible solutions and begins exploring the feasible regions, leading to new best-so-far feasible solutions more
efficiently. Moreover, there is a trend of alternating searches between feasible and infeasible regions. This behaviour aligns with our motivation in Section 5, suggesting that exploring infeasible regions could foster shortcut discovery, promising boundary searches, and identification of possibly isolated feasible regions. This showcases the significance and effectiveness of our GIRE method.
Figure 12: Search trajectories (50 steps) on 5 random CVRP-20 instances. The blue line shows the objective values of each visited solution (green dot - feasible one; red cross - infeasible one). The red and green dotted lines represent the best-so-far infeasible and feasible objective values, respectively.
In Figure 13, we further visualize the estimated distribution of the number of infeasible solutions within the last 5 visited solutions before finding a new best-so-far feasible solution. Results suggest that the probability of encountering at least one infeasible solution before finding a better best-so-far solution is around  $80\%$ , showcasing the importance of exploring both feasible and infeasible regions by GIRE (compared to purely visiting feasible solutions in the masking scheme).
Figure 13: Estimated distribution of the number of infeasible solutions explored in 5 visited solutions before finding a new best-so-far feasible solution (as step size T increases).
E.6 Used assets and licenses
We list the used assets in Table 14. All of them are open-source for academic research use. Our code and pre-trained models have been made available on GitHub (https://github.com/yining043/NeuOpt) using the MIT License.
Table 14: List of used assets (pre-trained models, codes, and datasets).
Type	License	Asset
Code	MIT license	HGS [21], GCN+BS [14], DPDP [42], AM+LCP* [33], Pointerformer [32], Sym-NCO [13], POMO [4], POMO+EAS [5], POMO+EAS+SGBS [34], DACT [9], DIFUSCO [15]
BSD-3-Clause license	Concorde [54]
GPL-3.0 license	NLNS [8]
Available for academic use	LKH-2 [51], LKH-3 [20]
No license	Costa et al. [16], Wu et al. [39]
Dataset	Available for academic use	TSPLIB (http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/), CVRPLIB (http://vrp.galgos.inf.puc-rio.br/index.php/en/)