SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We suggest an  $L$ -level and  $a$ -section hierarchy to gradually produce more specific submodels. Particularly, all  $N^{(l)}$  weight vectors in level  $l$  are defined as the centers of their corresponding subspaces that uniformly divide the whole weight vector space based on the Das and Dennis [44] method.  $N^{(l)}$  submodels are then fine-tuned to derive  $N^{(l+1)} = aN^{(l)}$  submodels with  $K^{(l+1)}$  fine-tuning steps in level  $l+1$ . Finally,  $N^{(L-1)}$  submodels are fine-tuned to derive  $N^{(L)} = N$  submodels, which are used to solve the  $N$  given subproblems. More details of this fine-tuning are presented in Appendix C.
For the  $L$ -level and  $a$ -section hierarchy with  $a^L = N$  and  $K^{(1)} = \dots = K^{(L)} = K$ , the total fine-tuning step is  $T_f^h = \sum_{l=1}^L Ka^l = Ka(a^L - 1) / (a - 1)$ . For the vanilla process, each submodel needs  $KL$  fine-tuning steps to achieve comparable performance, thus  $T_f^v = KLN = KL a^L$  steps in total. Hence, the hierarchical fine-tuning only needs  $T_f^h / T_f^v \approx 1 / L$  of the vanilla fine-tuning steps.
5.1 Experimental settings
We conduct computational experiments to evaluate the proposed method on the multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP). Following the convention in [26, 12], we consider the instances of different sizes  $n = 20 / 50 / 100$  for MOTSP/MOCVRP and  $n = 50 / 100 / 200$  for MOKP. All experiments are run on a PC with an Intel Xeon 4216 CPU and an RTX 3090 GPU.
Problems. Four classes of MOTSP [45] are considered, including bi-objective TSP type 1&2 (Bi-TSP-1, Bi-TSP-2) and tri-objective TSP type 1&2 (Tri-TSP-1, Tri-TSP-2). For the  $M$ -objective TSP type 1, node  $i$  has  $M$  2D coordinates  $\{\pmb{x}_i^1,\dots ,\pmb{x}_i^M\}$ , where the  $m$ -th cost between node  $i$  and  $j$  is defined as the Euclidean distance  $c_{ij}^{m} = \| \pmb{x}_{i}^{m} - \pmb{x}_{j}^{m}\|_{2}$ . For the  $M$ -objective TSP type 2, a node contains  $M - 1$  2D coordinates together with the altitude, so the  $M$ -th cost is defined as the altitude variance. For the bi-objective CVRP (Bi-CVRP) [46], two conflicting objectives with imbalanced domains, i.e., the total traveling distance and the traveling distance of the longest route (also called makespan), are considered. For the bi-objective KP (Bi-KP) [47], item  $i$  has a single weight and two distinct values. More details of these MOCOPs are presented in Appendix D.
Hyper-parameters. The meta-learning rate  $\epsilon$  is linearly annealed to 0 from  $\epsilon_0 = 1$  initially. A constant learning rate of the Adam optimizer is set to  $10^{-4}$ . We set  $B = 64$ ,  $T_{m} = 3000$ ,  $T_{u} = 100$ , and  $\tilde{N} = M$ . The  $N$  weight vectors in  $\mathcal{PF}$  construction are produced according to [44], where  $N = C_{H + M - 1}^{M - 1}$ .  $H$  is set to 100 ( $N = 101$ ) and 13 ( $N = 105$ ) for  $M = 2$  and  $M = 3$ , respectively. WS scalarization is adopted. In the hierarchy of fine-tuning, the whole weight space is uniformly divided by the Das and Dennis [44] method with  $H^{(l)} = 2^l$ , i.e., level  $l$  has  $2^l$  or  $4^l$  subspaces and  $L = 7$  or  $L = 4$  for  $M = 2$  or  $M = 3$ . The number of fine-tuning steps at each level is  $K^{(l)} = K = 20$  or  $K^{(l)} = K = 25$  for  $M = 2$  or  $M = 3$ .
Baselines. Two kinds of strong baselines are introduced, and all of them adopt WS (weighted sum) scalarization for fair comparisons. (1) The state-of-the-art neural heuristics, including MDRL [13], PMOCO [12], and DRL-based multiobjective optimization algorithm (DRL-MOA) [8]. Same as EMNH, all these neural heuristics adopt POMO as the base model for single-objective subproblems. (2) The state-of-the-art traditional heuristics, including PPLS/D-C [20], WS-LKH, and WS-DP. PPLS/D-C is a local-search-based MOEA proposed for MOCOPs, where a 2-opt heuristic is used for MOTSP and MOCVRP, and a greedy transformation heuristic [48] is used for MOKP, running with 200 iterations for all the three problems. WS-LKH and WS-DP are based on the state-of-the-art heuristics for decomposed single-objective subproblems, i.e., LKH [49, 50] for MOTSP and dynamic programming (DP) for MOKP, respectively. Our code is publicly available<sup>2</sup>.
Metrics. Our EMNH is evaluated in terms of solution quality and learning efficiency. Solution quality is mainly measured by hypervolume (HV) [51], where a higher HV means a better solution set (the definition is presented in Appendix E). Learning efficiency mainly refers to the training and fine-tuning efficiency, which are measured by training time for the same amount of training instances and solution quality with the same total fine-tuning steps (see Appendix F), respectively.
$^{2}$ https://github.com/bill-cjb/EMNH
Table 1: Results on 200 random instances for MOCOPs with balanced objective domains.
Method	Bi-TSP-1 (n=20)	Bi-TSP-1 (n=50)	Bi-TSP-1 (n=100)
HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
WS-LKH	0.6270	0.02%	10m	0.6415	-0.11%	1.8h	0.7090	-0.95%	6.0h
PPLS/D-C	0.6256	0.24%	26m	0.6282	1.97%	2.8h	0.6844	2.55%	11h
DRL-MOA	0.6257	0.22%	6s	0.6360	0.75%	9s	0.6970	0.75%	21s
PMOCO	0.6259	0.19%	6s	0.6351	0.89%	10s	0.6957	0.94%	19s
MDRL	0.6271	0.00%	5s	0.6364	0.69%	9s	0.6969	0.77%	17s
EMNH	0.6271	0.00%	5s	0.6364	0.69%	9s	0.6969	0.77%	16s
PMOCO-Aug	0.6270	0.02%	45s	0.6395	0.20%	2.3m	0.7016	0.10%	15m
MDRL-Aug	0.6271	0.00%	33s	0.6408	0.00%	1.7m	0.7022	0.01%	14m
EMNH-Aug	0.6271	0.00%	33s	0.6408	0.00%	1.7m	0.7023	0.00%	14m
Bi-KP (n=50)	Bi-KP (n=100)	Bi-KP (n=200)
Method	HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
WS-DP	0.3561	0.00%	22m	0.4532	0.07%	2.0h	0.3601	0.06%	5.8h
PPLS/D-C	0.3528	0.93%	18m	0.4480	1.21%	47m	0.3541	1.72%	1.5h
DRL-MOA	0.3559	0.06%	9s	0.4531	0.09%	18s	0.3601	0.06%	1.0m
PMOCO	0.3552	0.25%	6s	0.4523	0.26%	22s	0.3595	0.22%	1.3m
MDRL	0.3530	0.87%	6s	0.4532	0.07%	21s	0.3601	0.06%	1.2m
EMNH	0.3561	0.00%	6s	0.4535	0.00%	21s	0.3603	0.00%	1.2m
Tri-TSP-1 (n=20)	Tri-TSP-1 (n=50)	Tri-TSP-1 (n=100)
Method	HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
WS-LKH	0.4712	0.00%	12m	0.4440	-0.50%	1.9h	0.5076	-2.07%	6.6h
PPLS/D-C	0.4698	0.30%	1.4h	0.4174	5.52%	3.9h	0.4376	12.00%	14h
DRL-MOA	0.4699	0.28%	6s	0.4303	2.60%	9s	0.4806	3.36%	19s
PMOCO	0.4693	0.40%	5s	0.4315	2.33%	8s	0.4858	2.31%	18s
MDRL	0.4699	0.28%	5s	0.4317	2.29%	9s	0.4852	2.43%	16s
EMNH	0.4699	0.28%	5s	0.4324	2.13%	9s	0.4866	2.15%	16s
PMOCO-Aug	0.4712	0.00%	3.2m	0.4409	0.20%	28m	0.4956	0.34%	1.7h
MDRL-Aug	0.4712	0.00%	2.6m	0.4408	0.23%	25m	0.4958	0.30%	1.7h
EMNH-Aug	0.4712	0.00%	2.6m	0.4418	0.00%	25m	0.4973	0.00%	1.7h
5.2 Solution quality
The results for MOTSP, MOCVRP, and MOKP are recorded in Tables 1 and 2, including the average HV, gap, and total running time for solving 200 random test instances. To further show the significant differences between the results, a Wilcoxon rank-sum test at  $1\%$  significance level is adopted. The best result and the one without statistical significance to it are highlighted as bold. The second-best result and the one without statistical significance to it are highlighted as underline. The method with "-Aug" represents the inference results using instance augmentation [12] (see Appendix G).
Optimality gap. The gaps of HV with respect to EMNH-Aug are reported for all methods. According to the results in Tables 1 and 2, EMNH outperforms other neural heuristics without instance augmentation on all problems. When equipped with instance augmentation, EMNH-Aug further improves the solution, and performs superior to most of the baselines, while slightly inferior to WS-LKH on MOTSP with  $n = 50$  and  $n = 100$ . However, as iteration-based methods, WS-LKH and MOEAs take quite long running time on MOCOPs. For the neural heuristics, DRL-MOA is less agile since it trains multiple fixed models for a priori weight vectors. For new weight vectors concerned by decision makers, EMNH, MDRL, and PMOCO are able to efficiently produce high-quality trade-off solutions, where EMNH achieves the smallest gap.
Imbalanced objective domains. As demonstrated in Table 2, the gaps between EMNH and PMOCO or MDRL are further increased on the problems with imbalanced objective domains, i.e., Bi-CVRP, Bi-TSP-2, and Tri-TSP-2. EMNH exhibits more competitive performance on these problems, revealing
Table 2: Results on 200 random instances for MOCOPs with imbalanced objective domains.
Method	Bi-CVRP (n=20)	Bi-CVRP (n=50)	Bi-CVRP (n=100)
HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
PPLS/D-C	0.4287	0.35%	1.6h	0.4007	2.41%	9.7h	0.3946	3.26%	38h
DRL-MOA	0.4287	0.35%	10s	0.4076	0.73%	12s	0.4055	0.59%	33s
PMOCO	0.4267	0.81%	7s	0.4036	1.70%	12s	0.3913	4.07%	32s
MDRL	0.4291	0.26%	8s	0.4082	0.58%	13s	0.4056	0.56%	32s
EMNH	0.4299	0.07%	8s	0.4098	0.19%	12s	0.4072	0.17%	32s
PMOCO-Aug	0.4294	0.19%	13s	0.4080	0.63%	36s	0.3969	2.70%	2.7m
MDRL-Aug	0.4294	0.19%	11s	0.4092	0.34%	36s	0.4072	0.17%	2.8m
EMNH-Aug	0.4302	0.00%	11s	0.4106	0.00%	35s	0.4079	0.00%	2.8m
Bi-TSP-2 (n=20)	Bi-TSP-2 (n=50)	Bi-TSP-2 (n=100)
Method	HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
WS-LKH	0.6660	0.13%	11m	0.7390	-0.07%	1.8h	0.8055	-0.74%	6.1h
PPLS/D-C	0.6662	0.10%	27m	0.7300	1.15%	3.3h	0.7859	1.71%	10h
DRL-MOA	0.6657	0.18%	6s	0.7359	0.35%	8s	0.7965	0.39%	18s
PMOCO	0.6590	1.18%	5s	0.7347	0.51%	8s	0.7944	0.65%	17s
MDRL	0.6669	0.00%	5s	0.7361	0.32%	9s	0.7965	0.39%	15s
EMNH	0.6669	0.00%	5s	0.7361	0.32%	9s	0.7965	0.39%	15s
PMOCO-Aug	0.6653	0.24%	13s	0.7375	0.14%	33s	0.7988	0.10%	3.5m
MDRL-Aug	0.6669	0.00%	8s	0.7385	0.00%	22s	0.7996	0.00%	2.8m
EMNH-Aug	0.6669	0.00%	8s	0.7385	0.00%	22s	0.7996	0.00%	2.8m
Tri-TSP-2 (n=20)	Tri-TSP-2 (n=50)	Tri-TSP-2 (n=100)
Method	HV↑	Gap↓	Time	HV↑	Gap↓	Time	HV↑	Gap↓	Time
WS-LKH	0.5035	0.00%	13m	0.5305	-0.49%	2.0h	0.5996	-1.70%	6.6h
PPLS/D-C	0.5035	0.00%	1.4h	0.5045	4.42%	4.1h	0.5306	10.01%	15h
DRL-MOA	0.5019	0.32%	6s	0.5101	3.37%	8s	0.5488	6.92%	19s
PMOCO	0.5020	0.30%	5s	0.5176	1.95%	8s	0.5777	2.02%	18s
MDRL	0.5024	0.22%	5s	0.5183	1.82%	9s	0.5806	1.53%	16s
EMNH	0.5024	0.22%	5s	0.5205	1.40%	9s	0.5813	1.41%	16s
PMOCO-Aug	0.5035	0.00%	55s	0.5258	0.40%	6.1m	0.5862	0.58%	32m
MDRL-Aug	0.5035	0.00%	37s	0.5267	0.23%	4.2m	0.5886	0.17%	30m
EMNH-Aug	0.5035	0.00%	37s	0.5279	0.00%	4.2m	0.5896	0.00%	30m
the effectiveness of the proposed scaled symmetric sampling method in tackling the imbalance of objective domains. We further equip PMOCO with the same sampling method, which actually also improved the solutions for those problems, but still performed inferior to EMNH (see Appendix H).
Generalization ability. We test the generalization ability of the model on 200 larger-scale random instances  $(n = 150 / 200)$  and 3 commonly used MOTSP benchmark instances (KroAB100/150/200) in TSPLIB [52]. The zero-shot generalization performance of the model trained and fine-tuned both on the instances with  $n = 100$  is reported in Appendix I. The results suggest that EMNH has a superior generalization ability compared with MOEAs and other neural heuristics for larger problem sizes.
Hyper-parameter study. The results in Appendix J showed that the number of sampled weight vectors  $\tilde{N} = M$  is a more desirable setting and WS is a simple yet effective scalarization method.
5.3 Learning efficiency
As verified above, EMNH is able to produce superior solutions to the state-of-the-art neural heuristics, especially demonstrating a significant advantage over PMOCO in terms of solution quality. Then we further show that EMNH also has favorable learning efficiency against MDRL.
Training efficiency. EMNH, MDRL, and PMOCO only train one model to tackle all subproblems, where EMNH and MDRL use the same amount of training instances, and PMOCO requires a few
(b)
Figure 4: Learning efficiency. (a) Training efficiency. (b) Training stability. (c) Fine-tuning efficiency.
more training instances according to its original setting. DRL-MOA needs to train multiple submodels with more training instances. More details are presented in Appendix F. Figure 4(a) displays the speed-up ratio, i.e., the ratio of the training time of MDRL to EMNH. Due to the multi-task based training for (partial) architecture reuse, training time of EMNH is only about  $1 / \tilde{N}$  of that of MDRL.
Training stability. Figure 4(b) shows the stability of the training process, where EMNH-S and EMNH-R refer to EMNH with the symmetric sampling and the random sampling, respectively. EMNH achieves the stablest and best training performance, compared to EMNH-S, EMNH-R, and MDRL, as the proposed sampling method in EMNH considers symmetric weight vectors and (imbalanced) objective domains. More details are presented in Appendix F.
Fine-tuning efficiency. Figure 4(c) compares the hierarchical fine-tuning in EMNH with the vanilla fine-tuning method in MDRL, where the HV of EMNH with  $K$  fine-tuning steps at each level and the HV of MDRL with approximately equal total fine-tuning steps are presented. The results demonstrate that EMNH attains higher fine-tuning efficiency than MDRL, i.e., larger gaps with smaller fine-tuning steps. Furthermore, PMOCO is also equipped with the hierarchical fine-tuning (for both versions) for a fair comparison. It is worth noting that our fine-tuning process is performed individually for each weight vector on dedicated fine-tuning instances. As can be seen, PMOCO could hardly get improved by fine-tuning for zero-shot inference on test instances, since it has already converged for the corresponding subproblems. This means that EMNH has favorable potential to derive more desirable submodels to tackle specific tasks. Notably, EMNH with a few fine-tuning steps (e.g.,  $K = 5$ ) outperforms PMOCO in most cases, as demonstrated in Appendix F.
This paper proposes an efficient meta neural heuristic (EMNH) for MOCOPs. Specifically, EMNH reduces training time, stabilizes the training process, and curtails total fine-tuning steps via (partial) architecture-shared multi-task learning, scaled symmetric sampling (of weight vectors), and hierarchical fine-tuning, respectively. The experimental results on MOTSP, MOCVRP, and MOKP verified the superiority of EMNH in learning efficiency and solution quality. A limitation is that our EMNH, as the same as other neural heuristics, can not guarantee obtaining the exact Pareto front. In the future, we will (1) extend EMNH to other MOCOPs with complex constraints; (2) investigate other advanced meta-learning algorithms and neural solvers as the base model to further improve the performance.
This work is supported by the National Natural Science Foundation of China (62072483), and the Guangdong Basic and Applied Basic Research Foundation (2022A1515011690, 2021A1515012298).
[1] Matthias Ehrgott and Xavier Gandibleux. A survey and annotated bibliography of multiobjective combinatorial optimization. OR-spektrum, 22(4):425-460, 2000.
[2] Kostas Florios and George Mavrotas. Generation of the exact pareto set in multi-objective traveling salesman and set covering problems. Applied Mathematics and Computation, 237:1-19, 2014.
[44] I Das and JE Dennis. Normal-boundary intersection: A new method for generating pareto-optimal points in multieriteria optimization problems. SIAM Journal on Optimization, 8(3):631-657, 1998.
[45] Thibaut Lust and Jacques Teghem. The multiobjective traveling salesman problem: A survey and a new approach. In Advances in Multi-Objective Nature Inspired Computing, pages 119–141, 2010.
[46] Nicolas Jozefowiez, Frédéric Semet, and El-Ghazali Talbi. Multi-objective vehicle routing problems. European Journal of Operational Research, 189(2):293-309, 2008.
[47] Cristina Bazgan, Hadrien Hugot, and Daniel Vanderpooten. Solving efficiently the 0-1 multi-objective knapsack problem. Computers & Operations Research, 36(1):260-279, 2009.
[48] Hisao Ishibuchi, Naoya Akedo, and Yusuke Nojima. Behavior of multiobjective evolutionary algorithms on many-objective knapsack problems. IEEE Transactions on Evolutionary Computation, 19(2):264-283, 2015.
[49] Keld Helsgaun. An effective implementation of the lin-kernighan traveling salesman heuristic. European Journal of Operational Research, 126(1):106-130, 2000.
[50] Renato Tinós, Keld Helsgaun, and Darrell Whitley. Efficient recombination in the lin-kernighan-helsgaun traveling salesman heuristic. In International Conference on Parallel Problem Solving from Nature, pages 95–107, 2018.
[51] Charles Audet, Jean Bigeon, Dominique Cartier, Sébastien Le Digabel, and Ludovic Salomon. Performance indicators in multiobjective optimization. European Journal of Operational Research, 292(2):397-422, 2021.
[52] Gerhard Reinelt. TSPLIB—a traveling salesman problem library. ORSA Journal on Computing, 3(4): 376-384, 1991.