SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When solving subproblem  $i$ , more than one solution can be attained by sampling, e.g., sampling with multiple start nodes as did in POMO [36]. In this case,  $\mathcal{F}^i = \mathrm{MPO}(\mathcal{F}^{i - 1}\cup \mathcal{G}^i)$ , where  $\mathcal{G}^i$  contains all the candidate points (find by sampling) to be introduced in the new Pareto front and  $\mathrm{MPO}(\cdot)$  is an operator that updates the Pareto front. However, as the complexity of  $\mathrm{MPO}(\mathcal{F}^{i - 1}\cup \mathcal{G}^i)$  is  $O((|\mathcal{F}^{i - 1}| + |\mathcal{G}^i|)|\mathcal{G}^i|)$ , it may take a relatively long time, especially when there are thousands of points in  $\mathcal{F}^{i - 1}$  and  $\mathcal{G}^i$ . Thus, we suggest an efficient update mechanism executed on the surrogate Pareto fronts,  $\mathcal{F}^i = \mathrm{MPO}(\hat{\mathcal{F}}^{i - 1}\cup \tilde{\mathcal{G}}^i)$ , where  $\tilde{\mathcal{G}}^i\subset \mathcal{G}^i$  includes at most  $J$  (usually setting  $J > K$ ) best points selected from  $\pmb {f}(\pmb {\pi})\in \mathcal{G}^i$  according to  $g(\pi |s,\lambda^i)$ . The complexity of  $\mathrm{MPO}(\tilde{\mathcal{F}}^{i - 1}\cup \tilde{\mathcal{G}}^i)$  is then reduced to  $O((K + J)J)$ , which is able to curtail the overall solving time in practice.
4.4 Training and inference
Our NHDE can be applied to different decomposition-based DRL methods, e.g., PMOCO [14] and MDRL [15], and the training algorithm is easy to adapt with slight adjustments, where the one for NHDE-P is presented in Algorithm 1. The key points include three aspects. (1) Multiple weights are sampled to train the same instance (Line 6), since the solving processes for those subproblems are dependent. (2) The HV indicator is adopted in the reward (Line 10). (3) Multiple Pareto-optimal solutions are preserved via MPO (Line 15). Note that when training with a batch, the point sets with different sizes of the instances are padded with repetitive reference points, which are masked in the attention, based on the maximum size. The training algorithm of NHDE-M is given in Appendix E.
In the inference phase, for  $N$  given weights and diversity factors, the well-trained model is used to sequentially solve  $N$  corresponding subproblems, as shown in Figure 1. Moreover, instance augmentation [14] can be also brought into our MPO for each subproblem, i.e., the points of all sampled solutions from an instance and its augmented instances are included in  $\mathcal{G}^i$ . Our NHDE can achieve desirable performance by using only parts of instance augmentation (see Appendix D), since it can already deliver more diverse solutions.
Problems. We evaluate the proposed NHDE on three typical MOCO problems that are commonly studied in the neural MOCO literature [13-15], namely the multi-objective traveling salesman problem (MOTSP) [51], multi-objective capacitated vehicle routing problem (MOCVRP) [3], and multi-objective knapsack problem (MOKP) [52]. More specifically, we solve bi-objective TSP (Bi-TSP), tri-objective TSP (Tri-TSP), the bi-objective CVRP (Bi-CVRP) and the bi-objective KP (Bi-KP). For the  $M$ -objective TSP with  $n$  nodes, each node has  $M$  sets of 2-dimensional coordinates, where the  $m$ -th objective value of the solution is calculated with respect to the  $m$ -th coordinates. Bi-CVRP consists of  $n$  customer nodes and a depot node, with each node featured by a 2-dimensional coordinate and each customer node associated with a demand. Following the literature, we consider two conflicting objectives in Bi-CVRP, i.e., the total tour length and the makespan (that is the length of the longest route). Bi-KP is defined by  $n$  items, with each taking a weight and two separate values. The  $m$ -th objective is to maximize the sum of the  $m$ -th values but not exceed the capacity. Three sizes of these problems are considered, i.e.,  $n = 20/50/100$  for MOTSP and MOCVRP, and  $n = 50/100/200$  for MOKP. The coordinates, demands, and values are uniformly sampled from  $[0,1]^2$ ,  $\{1,\ldots,9\}$ , and  $[0,1]$ , respectively. The vehicle capacity is set to 30/40/50 for MOCVRP20/50/100. The knapsack capacity is set to 12.5/25/25 for MOKP50/100/200.
Hyperparameters. We train NHDE-P with 200 epochs, each containing 5,000 randomly generated instances. We use batch size  $B = 64$  and the Adam [53] optimizer with learning rate  $10^{-4}$  ( $10^{-5}$  for MOKP) and weight decay  $10^{-6}$ . During training,  $N' = 20$  weights are sampled for each instance. During inference, we generate  $N = 40$  and  $N = 210$  uniformly distributed weights for  $M = 2$  and  $M = 3$ , respectively, which are then shuffled so as to counteract biases. The diversity factors linearly shift through the  $N$  subproblems from (1,0) to (0,1), which implies a gradual focus from achieving convergence (scalar objective) with a few solutions to ensuring comprehensive performance with a multitude of solutions. We set  $K = 20$  and  $J = 200$ . See Appendix F for the settings of NHDE-M.
Baselines. We compare NHDE with two classes of state-of-the-art methods. (1) The neural methods, including PMOCO [14], MDRL [15], and DRL-based multiobjective optimization algorithm (DRL-MOA) [13], all with POMO as the backbone for single-objective CO subproblems. Our NHDE-P and NHDE-M each train a unified model with the same gradient steps as PMOCO and MDRL, respectively, while DRL-MOA trains 101 (105) models for  $M = 2$  ( $M = 3$ ) with more gradient steps, i.e., the first model with 200 epochs and the remaining models with 5-epoch per model via parameter transfer. (2) The non-learnable methods, including the state-of-the-art MOEA and strong heuristics. Particularly, PPLS/D-C [23] is a specialized MOEA for MOCO with local search techniques, including a 2-opt heuristic for MOTSP and MOCVRP, and a greedy transformation heuristic [52] for MOKP, implemented in Python. In addition, LKH [54, 55] and dynamic programming (DP), are employed to solve the weighted-sum (WS) based subproblems for MOTSP and MOKP, denoted as WS-LKH and WS-DP, respectively. All the methods use WS scalarization for fair comparisons. All the methods are tested with an RTX 3090 GPU and an Intel Xeon 4216 CPU. Our code is publicly available<sup>3</sup>.
Metrics. We use hypervolume (HV) and the number of non-dominated solutions (|NDS|). A higher HV means better overall performance in terms of convergence and diversity, while |NDS| reflects the diversity when HVs are close. The average HV, gaps with respect to NHDE, and total running time for 200 random test instances are reported. The best (second-best) and its statistically insignificant results at  $1\%$  significance level of a Wilcoxon rank-sum test are highlighted in **bold** (underline).
All results of NHDE-P and the baselines are displayed in Table 1. Given the same number of weights (wt.), NHDE-P significantly surpasses PMOCO for all problems and sizes in terms of HV and |NDS|, which indicates that NHDE-P has the potential to discover diverse and high-quality solutions. When instance augmentation (aug.) is equipped, NHDE-P achieves the smallest gap among the methods in most cases, except Bi-TSP100 and Bi-CVRP100 where WS-LKH and DRL-MOA perform better. However, WS-LKH consumes much longer runtime than NHDE-P due to iterative search (2.7 hours vs 5.6 minutes), and DRL-MOA costs much more training overhead to prepare multiple models for respective weights. Besides, another reason why NHDE-P is inferior to DRL-MOA on Bi-CVRP100 might be that the hypernetwork (inherited from PMOCO) could be hard to cope with the objectives
<sup>3</sup>https://github.com/bill-cjb/NHDE
Table 1: Results of NHDE-P on 200 random instances for MOCO problems.
Method	Bi-TSP20	Bi-TSP50	Bi-TSP100
HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-LKH (40 wt.)	0.6266	14	0.46%	4.1m	0.6402	29	0.42%	42m	0.7072	37	-0.31%	2.7h
PPLS/D-C (200 iter.)	0.6256	71	0.62%	26m	0.6282	213	2.29%	2.8h	0.6844	373	2.92%	11h
DRL-MOA (101 models)	0.6257	23	0.60%	6s	0.6360	57	1.07%	9s	0.6970	70	1.13%	21s
PMOCO (40 wt.)	0.6258	17	0.59%	4s	0.6331	31	1.52%	5s	0.6938	36	1.59%	8s
PMOCO (600 wt.)	0.6267	23	0.44%	27s	0.6361	68	1.06%	53s	0.6978	131	1.02%	2.1m
NHDE-P (40 wt.)	0.6286	56	0.14%	19s	0.6388	127	0.64%	53s	0.7005	193	0.64%	1.9m
PMOCO (40 wt. aug.)	0.6266	17	0.46%	23s	0.6377	32	0.81%	1.6m	0.6993	37	0.81%	3.0m
PMOCO (100 wt. aug.)	0.6270	20	0.40%	1.4m	0.6395	53	0.53%	3.8m	0.7016	76	0.48%	15m
NHDE-P (40 wt. aug.)	0.6295	81	0.00%	1.5m	0.6429	269	0.00%	2.5m	0.7050	343	0.00%	5.6m
Bi-CVRP20	Bi-CVRP50	Bi-CVRP100
Method	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
PPLS/D-C (200 iter.)	0.4283	14	0.46%	1.3h	0.4007	17	2.15%	9.7h	0.3946	20	1.13%	38h
DRL-MOA (101 models)	0.4287	7	0.37%	10s	0.4076	10	0.46%	12s	0.4055	12	-1.60%	33s
PMOCO (40 wt.)	0.4266	6	0.86%	4s	0.4035	7	1.47%	7s	0.3912	6	1.98%	12s
PMOCO (300 wt.)	0.4268	7	0.81%	20s	0.4039	9	1.37%	35s	0.3914	8	1.93%	1.2m
NHDE-P (40 wt.)	0.4284	12	0.44%	18s	0.4062	14	0.81%	36s	0.3933	10	1.45%	1.1m
PMOCO (40 wt. aug.)	0.4292	6	0.26%	8s	0.4078	7	0.42%	15s	0.3968	7	0.58%	1.1m
PMOCO (300 wt. aug.)	0.4294	9	0.21%	1.0m	0.4081	10	0.34%	1.8m	0.3969	9	0.55%	7.0m
NHDE-P (40 wt. aug.)	0.4303	21	0.00%	1.2m	0.4095	22	0.00%	1.5m	0.3991	16	0.00%	2.4m
Bi-KP50	Bi-KP100	Bi-KP200
Method	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-DP (40 wt.)	0.3560	10	0.11%	9.6m	0.4529	16	0.26%	1.3h	0.3598	23	0.39%	3.8h
PPLS/D-C (200 iter.)	0.3528	13	1.01%	18m	0.4480	19	1.34%	47m	0.3541	20	1.97%	1.5h
DRL-MOA (101 models)	0.3559	21	0.14%	9s	0.4531	38	0.22%	18s	0.3601	48	0.30%	1.0m
PMOCO (40 wt.)	0.3550	14	0.39%	6s	0.4518	22	0.51%	9s	0.3590	28	0.61%	25s
PMOCO (300 wt.)	0.3552	17	0.34%	29s	0.4524	31	0.37%	1.1m	0.3597	46	0.42%	3.0m
NHDE-P (40 wt.)	0.3564	30	0.00%	29s	0.4541	83	0.00%	1.0m	0.3612	243	0.00%	2.7m
Tri-TSP20	Tri-TSP50	Tri-TSP100
Method	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-LKH (210 wt.)	0.4727	78	0.82%	23m	0.4501	189	1.90%	3.5h	0.5165	209	0.84%	12h
PPLS/D-C (200 iter.)	0.4698	876	1.43%	1.4h	0.4174	3727	9.02%	3.9h	0.4376	8105	15.99%	14h
DRL-MOA (105 models)	0.4675	72	1.91%	5s	0.4285	98	6.61%	9s	0.4850	101	6.89%	19s
PMOCO (210 wt.)	0.4714	113	1.09%	11s	0.4381	198	4.51%	18s	0.4946	207	5.05%	39s
PMOCO (3003 wt.)	0.4741	264	0.52%	2.3m	0.4484	1339	2.27%	4.6m	0.5087	2330	2.34%	10m
NHDE-P (210 wt.)	0.4758	675	0.17%	1.2m	0.4506	2547	1.79%	4.4m	0.5111	4984	1.88%	10m
PMOCO (210 wt. aug.)	0.4727	104	0.82%	21m	0.4471	201	2.55%	1.1h	0.5044	209	3.17%	4.2h
PMOCO (153 wt. aug.)	0.4722	89	0.92%	15m	0.4447	150	3.07%	47m	0.5009	153	3.84%	3.1h
NHDE-P (210 wt. aug.)	0.4766	527	0.00%	14m	0.4588	9047	0.00%	30m	0.5209	16999	0.00%	1.5h
with imbalanced scales. Considering this drawback of PMOCO, we also apply our method to MDRL, i.e., NHDE-M, and demonstrate that NHDE-M outperforms DRL-MOA on Bi-CVRP100 in Table 2. Also, NHDE-M outperforms MDRL in all cases. More results of NHDE-M are given in Appendix G.
Regarding the inference efficiency, NHDE-P generally takes (tolerably) more runtime than the other learning based methods. To be fair, we further enhance the state-of-the-art PMOCO by adjusting the number of the weights, so that it takes similar or even more runtime compared with NHDE-P. The results in the last two lines for each problem show that NHDE-P still attains the smaller gaps. We also observe that |NDS| hardly grows along with the increase of number of weights for PMOCO, since numerous solutions to different subproblems could be repetitive. In contrast, NHDE-P is able to produce more diverse solutions with much fewer weights.
5.2 Generalization study
To assess the generalization capability of NHDE-P, we compare the trained models (from neural methods) for Bi-TSP100 and the other baselines on 200 random Bi-TSP instances with larger sizes, i.e., Bi-TSP150/200. Three commonly used benchmark instances developed from TSPLIB [56],
Table 2: Results of NHDE-M on 200 random instances for MOCO problems.
Method	Bi-TSP20	Bi-TSP50	Bi-TSP100
HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-LKH (40 wt.)	0.6266	14	0.46%	4.1m	0.6402	29	0.42%	42m	0.7072	37	-0.33%	2.7h
PPLS/D-C (200 iter.)	0.6256	71	0.62%	26m	0.6282	213	2.29%	2.8h	0.6844	373	2.91%	11h
DRL-MOA (101 models)	0.6257	23	0.60%	6s	0.6360	57	1.07%	9s	0.6970	70	1.12%	21s
MDRL (40 wt.)	0.6264	20	0.49%	2s	0.6342	33	1.35%	3s	0.6940	36	1.55%	8s
NHDE-M (40 wt.)	0.6287	58	0.13%	20s	0.6393	132	0.56%	57s	0.7008	195	0.58%	2.0m
MDRL (40 wt. aug.)	0.6267	18	0.44%	21s	0.6384	34	0.70%	1.5m	0.6995	38	0.77%	3.3m
NHDE-M (40 wt. aug.)	0.6295	81	0.00%	1.5m	0.6429	273	0.00%	2.6m	0.7049	339	0.00%	5.5m
Method	Bi-CVRP20	Bi-CVRP50	Bi-CVRP100
HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
PPLS/D-C (200 iter.)	0.4287	15	0.42%	1.6h	0.4007	17	2.34%	9.7h	0.3946	20	3.14%	38h
DRL-MOA (101 models)	0.4287	7	0.42%	10s	0.4076	10	0.66%	12s	0.4055	12	0.47%	33s
MDRL (40 wt.)	0.4284	9	0.49%	3s	0.4057	5	1.12%	5s	0.4015	0	1.45%	10s
NHDE-M (40 wt.)	0.4296	16	0.21%	23s	0.4086	20	0.41%	47s	0.4053	18	0.52%	1.4m
MDRL (40 wt. aug.)	0.4293	9	0.28%	5s	0.4073	11	0.73%	16s	0.4040	11	0.83%	1.0m
NHDE-M (40 wt. aug.)	0.4305	24	0.00%	1.2m	0.4103	29	0.00%	1.6m	0.4074	26	0.00%	2.7m
Method	Bi-KP50	Bi-KP100	Bi-KP200
HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-DP (40 wt.)	0.3560	10	0.17%	9.6m	0.4529	16	0.29%	1.3h	0.3598	23	0.30%	3.8h
PPLS/D-C (200 iter.)	0.3528	13	1.07%	18m	0.4480	19	1.37%	47m	0.3541	20	1.88%	1.5h
DRL-MOA (101 models)	0.3559	21	0.20%	9s	0.4531	38	0.24%	18s	0.3601	48	0.22%	1.0m
MDRL (40 wt.)	0.3559	17	0.20%	4s	0.4528	25	0.31%	8s	0.3594	31	0.42%	24s
NHDE-M (40 wt.)	0.3566	41	0.00%	31s	0.4542	93	0.00%	1.0m	0.3609	160	0.00%	2.8m
Method	Tri-TSP20	Tri-TSP50	Tri-TSP100
HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time	HV↑	NDS↑	Gap↓	Time
WS-LKH (210 wt.)	0.4727	78	0.82%	23m	0.4501	189	2.00%	3.5h	0.5165	209	-0.92%	12h
PPLS/D-C (200 iter.)	0.4698	876	1.43%	1.4h	0.4174	3727	9.12%	3.9h	0.4376	8105	14.50%	14h
DRL-MOA (105 models)	0.4675	72	1.91%	5s	0.4285	98	6.71%	9s	0.4850	101	5.24%	19s
MDRL (210 wt.)	0.4723	126	0.90%	14s	0.4388	199	4.46%	20s	0.4956	207	3.17%	40s
NHDE-M (210 wt.)	0.4763	783	0.06%	1.4m	0.4512	2636	1.76%	4.7m	0.4997	4056	2.36%	11m
MDRL (210 wt. aug.)	0.4727	107	0.82%	14m	0.4473	202	2.61%	53m	0.5056	209	1.21%	4.3h
NHDE-M (210 wt. aug.)	0.4766	748	0.00%	13m	0.4593	10850	0.00%	30m	0.5118	13216	0.00%	1.5h
Table 3: Results on 200 random instances for larger-scale problems.
Method	Bi-TSP150	Bi-TSP200
HV↑	|NDS|↑	Gap↓	Time	HV↑	|NDS|↑	Gap↓	Time
WS-LKH (40 wt.)	0.7075	39	-0.90%	5.3h	0.7435	40	-1.52%	8.5h
PPLS/D-C (200 iter.)	0.6784	473	3.25%	21h	0.7106	512	2.98%	32h
DRL-MOA (101 models)	0.6901	73	1.58%	45s	0.7219	75	1.43%	87s
PMOCO (40 wt.)	0.6891	37	1.73%	22s	0.7215	38	1.49%	41s
PMOCO (400 wt.)	0.6938	160	1.06%	3.7m	0.7259	186	0.89%	6.8m
NHDE-P (40 wt.)	0.6964	231	0.68%	3.0m	0.7280	259	0.60%	4.3m
PMOCO (40 wt. aug.)	0.6944	38	0.97%	20m	0.7264	39	0.82%	40m
NHDE-P (40 wt. aug.)	0.7012	372	0.00%	14m	0.7324	384	0.00%	26m
i.e., KroAB100, KroAB150, and KroAB200, are also tested. The comparison results and Pareto fronts are demonstrated in Table 3 and Figure 3, respectively. As shown, NHDE-P outperforms the state-of-the-art MOEA (i.e., PPLS/D-C) and other neural methods significantly, in terms of HV and |NDS|, which means a superior generalization capability. The figure again verifies that NHDE-P generates a more extended Pareto front than PMOCO, showing a better diversity. Although PPLS/D-C finds a large number of solutions, they are inferior in terms of the optimality, with a biased distribution (e.g., crowded in certain regions). In contrast, NHDE-P generates a more well-distributed Pareto front with stronger convergence. More results on generalization are given in Appendix H.
Figure 3: Pareto fronts of benchmark instances. (a) KroAB100. (b) KroAB150. (c) KroAB200.
(a)
(b)
Figure 4: Ablation study. (a) Effects of indicator-enhanced DRL, HGA, and MPO. (b) Effects of the efficient update of MPO and partial instance augmentation.
To analyze the effect of the indicator-enhanced DRL, we compare NHDE-P with decomposition-based DRL without indicator (NHDE w/o I) and indicator-based DRL without decomposition (NHDE w/o D). To verify the valid design of HGA, three attention-based variants, which are formed by removing point-to-node attention (NHDE w/o P2N), removing node-to-point attention (NHDE w/o N2P), and adding point-to-point attention (NHDE w P2P), are involved for comparison. To assess the impact of MPO, NHDE w/o MPO is also evaluated. More details about these variants are presented in Appendix I. As seen from Figure 4(a). The performance of NHDE-P is significantly impaired when any of the components is ablated. Instead, NHDE w P2P degrades a bit, which reveals that the extra point-to-point attention may bring noises into the model. Moreover, we evaluate the effectiveness of the efficient update of MPO and partial instance augmentation in Figure 4(b), which shows that either of them saliently diminishes solving time with only a little sacrifice of the performance.
This paper proposes a novel NHDE for MOCO problems. NHDE impedes repetitive solutions from different subproblems via indicator-enhanced DRL with a HGA model, and digs more solutions in the neighborhood of each subproblem with an MPO strategy. Our generic NHDE can be deployed to different neural MOCO methods. The experimental results on three classic MOCO problems showed the superiority of NHDE, especially with regard to the diversity of Pareto set. A limitation is that the HV calculation would expend additional computational time, which might hinder the scalability of NHDE for solving much larger problems with many objectives. In the future, we will explore alternative schemes like the HV approximations [57, 58] to further promote the training efficiency of NHDE, and we also intend to apply it to tackle real-world MOCO problems.