Title: Efficient Meta Neural Heuristic for Multi-Objective Combinatorial Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: neural heuristic

meta learning

deep reinforcement learning

multi-objective combinatorial optimization
Abstract: Recently, neural heuristics based on deep reinforcement learning have exhibited promise in solving multi-objective combinatorial optimization problems (MOCOPs). However, they are still struggling to achieve high learning efficiency and solution quality. To tackle this issue, we propose an efficient meta neural heuristic (EMNH), in which a meta-model is first trained and then fine-tuned with a few steps to solve corresponding single-objective subproblems. Specifically, for the training process, a (partial) architecture-shared multi-task model is leveraged to achieve parallel learning for the meta-model, so as to speed up the training; meanwhile, a scaled symmetric sampling method with respect to the weight vectors is designed to stabilize the training. For the fine-tuning process, an efficient hierarchical method is proposed to systematically tackle all the subproblems. Experimental results on the multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP) show that, EMNH is able to outperform the state-of-the-art neural heuristics in terms of solution quality and learning efficiency, and yield competitive solutions to the strong traditional heuristics while consuming much shorter time.

Efficient Meta Neural Heuristic for Multi-Objective Combinatorial Optimization

Abstract

Recently, neural heuristics based on deep reinforcement learning have exhibited promise in solving multi-objective combinatorial optimization problems (MOCOPs). However, they are still struggling to achieve high learning efficiency and solution quality. To tackle this issue, we propose an efficient meta neural heuristic (EMNH), in which a meta-model is first trained and then fine-tuned with a few steps to solve corresponding single-objective subproblems. Specifically, for the training process, a (partial) architecture-shared multi-task model is leveraged to achieve parallel learning for the meta-model, so as to speed up the training; meanwhile, a scaled symmetric sampling method with respect to the weight vectors is designed to stabilize the training. For the fine-tuning process, an efficient hierarchical method is proposed to systematically tackle all the subproblems. Experimental results on the multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP) show that, EMNH is able to outperform the state-of-the-art neural heuristics in terms of solution quality and learning efficiency, and yield competitive solutions to the strong traditional heuristics while consuming much shorter time.

1 Introduction

Multi-objective combinatorial optimization problems (MOCOPs) [1] are widely studied and applied in many real-world sectors, such as telecommunication, logistics, manufacturing, and inventory. Typically, an MOCOP requires the simultaneous optimization of multiple conflicting objectives, where the amelioration of an objective may lead to the deterioration of others. Therefore, a set of trade-off solutions, known as Pareto-optimal solutions, are usually sought for MOCOPs.

Generally, it is difficult to exactly find all the Pareto-optimal solutions of an MOCOP [2], especially given that the decomposed single-objective subproblem might already be NP-hard. Hence, heuristic methods [3] are usually preferred to solve MOCOPs in reality, as they can attain approximate Pareto-optimal solutions in (relatively) reasonable time. Nevertheless, the traditional heuristics are still lacking due to their reliance on handcrafted rules and massive iterative steps, and superfluous computation even for instances of the same (or similar) class.

Recently, inspired by the success of deep reinforcement learning (DRL) in learning neural heuristics for solving the single-objective combinatorial optimization problems (COPs) [4-7], a number of

weights

Meta-learning process weights

Fine-tuning process

𝜃𝑁

6

1

2

3

4

5

Inference process

𝜃 1

(1)

N

test

weights

Fine-tunning

instances

𝜃init

𝜃meta

෩

𝑁

symmetric

sampled weights

Training

instances

T u

steps

T m

iterations

෨

𝜃body

෨

𝜃head1

෨

𝜃head෩ 𝑁

෨

…

…

𝜃body𝜃head

𝜃 2

(1)

𝜃 1

(2)

𝜃 2

(2)

𝜃 3

(2)

𝜃 4

(2)

𝜃1

𝜃𝑖

𝜃𝑁

…

…

…

…

…

6

1

2

3

4

5

1

2

3

4

5

6

1

2

3

5 4

6

Accelerated

meta-learning process

Hierachical

fine-tuning process Inference process

Test instances

DRL-based neural heuristics [8-13] have also been investigated for MOCOPs. While bypassing the handcrafted rules, these neural heuristics adopt an end-to-end paradigm to construct solutions without iterative search. Benefiting from a large amount of data (i.e., problem instances), a well-trained deep model allows the neural heuristic to automatically extract informative and expressive features for decision making and generalize to unseen instances as well.

Although demonstrating promise for solving MOCOPs, those neural heuristics still struggle to achieve high learning efficiency and solution quality. Particularly, in this line of works, the early attempts [8-11] always train multiple deep models, i.e., one for each preference (weight) combination, making them less practical. While the well-known preference-conditioned multi-objective combinatorial optimization (PMOCO) [12] realized a unified deep model by introducing a huge hypernetwork, it leaves considerable gaps in terms of solution quality. The recent Meta-DRL (MDRL) [13] has demonstrated the capability to enhance solution quality over existing state-of-the-art algorithms. However, it still faces challenges related to inefficient and unstable training procedures, as well as undesirable fine-tuning processes. This paper thereby proposes an efficient meta neural heuristic (EMNH) to further strengthen the learning efficiency and solution quality, the framework of which is illustrated in Figure 1. Following the meta-learning paradigm [13, 14], EMNH first trains a meta-model and then quickly fine-tunes it according to the weight (preference) vector to tackle the corresponding single-objective subproblems.

Our contributions are summarized as follows. (1) We propose an efficient meta-learning scheme to accelerate the training. Inspired by the feature reuse of meta-learning [15], we introduce a multi-task model composed of a parameter-shared body (all other layers of neural network except the last one) and respective task-related heads (the last layer of the network), rather than respective submodels, to train the meta-model, which is able to handle different tasks (subproblems) in parallel. (2) We design a scaled symmetric sampling method regarding the weight vectors to stabilize the training. In each meta-iteration, once a weight vector is sampled, its scaled symmetric weight vectors will also be engendered, which help avoid fluctuations of the parameter update for the meta-model, especially on problems with objectives of imbalanced domains. (3) We present a hierarchical fine-tuning method to efficiently cope with all the subproblems. Since it is redundant to derive a specific submodel for each subproblem in the early tuning process, our meta-model can be gradually fine-tuned to act as specialized submodels from low level to high level, which takes much fewer fine-tuning steps in total. Moreover, experimental results on three classic MOCOPs confirmed the effectiveness of our designs.

2 Related works

Exact and heuristic methods for MOCOPs. Over the past decades, MOCOPs have been attracting increasing attention from the computational intelligence community, and tremendous works have been proposed. Generally, exact methods [2] are able to gain the accurate Pareto-optimal solutions, but their computational time may exponentially grow. Heuristic methods such as multi-objective

𝜃෩ 𝑁

evolutionary algorithms (MOEAs) are popular in practice, where the representative paradigms for the general multi-objective optimization include NSGA-II [16] and MOEA/D [17]. Among them, a number of MOEAs outfitted with local search [18-20] are specifically designed for MOCOPs, which iteratively perform a search in the solution space to find approximate Pareto-optimal solutions.

Neural heuristics for COPs. As the early attempts, neural construction heuristics based on deep models [21-23] were proposed to learn to directly construct solutions for (single-objective) COPs. Among them, the classic Attention Model (AM) [24] was proposed inspired by Transformer architecture [25], and known as a milestone for solving vehicle routing problems. A number of subsequent works [26-30] were developed based on AM, including the popular policy optimization with multiple optima (POMO) [26], which leveraged the solution symmetries and significantly enhanced the performance over AM. Besides, graph neural networks were also employed to learn graphic embeddings for solving COPs [31-34]. Different from the above neural construction heuristics, neural improvement heuristics [35-39] were proposed to assist iteration-based methods to refine an initial solution.

Neural heuristics for MOCOPs. MOCOPs seek for a set of trade-off solutions, which could be obtained by solving a series of single-objective subproblems. According to the number of trained models, neural heuristics could be roughly classified into multi-model and single-model ones. The former adopts multiple networks to tackle respective subproblems, where the set of networks could also be collaboratively trained with a neighborhood-based parameter-transfer strategy [8, 9], or evolved with MOEA [10, 11]. Differently, the latter usually exploits a unified network to tackle all the subproblems, which is more flexible and practical. For example, taking the weight (preference) vector of a subproblem as the input, PMOCO introduces a hypernetwork to learn the decoder parameters [12]. However, the hypernetwork may cause extra complexity to the original deep reinforcement learning model, rendering it less effective to learn more accurate mapping from the preference to the optimal solution of the corresponding subproblem. By contrast, MDRL [13] leverages the meta-learning to train a deep reinforcement learning model that could be fine-tuned for various subproblems. However, its learning efficiency is far from optimal, especially given the slow and unstable training due to sequential learning of randomly sampled tasks, and inefficient fine-tuning due to redundant updates.

3 Preliminary

3.1 MOCOP

In general, an MOCOP could be defined as

where f ( x ) is an M -objective vector, X is a discrete decision space, and the objectives might be conflicted. The Pareto-optimal solutions concerned by decision makers are defined as follows.

Definition 1 (Pareto dominance). Let u, v ∈ X , u is said to dominate v ( u ≺ v ) if and only if f i ( u ) ≤ f i ( v ) , ∀ i ∈ { 1 , . . . , M } and ∃ j ∈ { 1 , . . . , M } , f j ( u ) < f j ( v ) .

Definition 2 (Pareto optimality). A solution x ∗ ∈ X is Pareto-optimal if x ∗ is not dominated by any other solution in X , i.e., ∄ x ′ ∈ X such that x ′ ≺ x ∗ . The set of all Pareto optimal solutions P = { x ∗ ∈ X | ∄ x ′ ∈ X : x ′ ≺ x ∗ } is called the Pareto set. The image of Pareto set in the objective space, i.e., PF = { f ( x ) | x ∈ P} is called the Pareto front.

3.2 Decomposition

The decomposition strategy is always applied for solving MOCOPs [17], since it is straightforward yet effective to guide the search towards prescribed directions. Specifically, MOCOPs can be decomposed into N scalarized single-objective subproblem g ( x | λ ) , where the weight vector λ ∈ R M satisfies λ m ≥ 0 and ∑ M m =1 λ m = 1 . Then, PF is approximated by solving the subproblems systematically.

Regarding the decomposition strategy, the weighted sum (WS) and Tchebycheff [40] are commonly used. As a simple representative, WS considers the linear combination of M objectives as follows,

Given the weight vector λ , the corresponding single-objective subproblem could be cast as a sequential decision problem and solved through DRL. In particular, a solution is represented as a sequence π = { π 1 , . . . , π T } with length T , and a stochastic policy for yielding solution π from instance s is calculated as P ( π | s ) = ∏ T t =1 P θ ( π t | π 1: t -1 , s ) , where P θ ( π t | π 1: t -1 , s ) is the node selection probability parameterized by θ (the subscript of θ i , π i , P i for subproblem i is omitted for readability).

3.3 Meta-learning

Meta-learning [14, 41] aims to train a model that can learn to tackle various tasks (i.e., subproblems) via fine-tuning. Naturally, a meta-model can be trained to quickly adapt to N corresponding subproblems given N weight vectors. Generally, meta-learning comprises three processes for MOCOPs. In the meta-learning process, a model θ is trained by repeatedly sampling tasks from the whole task space. In the fine-tuning process, N submodels θ 1 , . . . , θ N based on the given weight vectors are fine-tuned from θ using fine-tuning instances (also called query set). In the inference process, N subproblems of an instance are solved using the submodels to approximate the Pareto set.

4 Methodology

Our efficient meta neural heuristic (EMNH) includes an accelerated and stabilized meta-learning process, a hierarchical fine-tuning process, and an inference process, as depicted in Figure 1. EMNH is generic, where we adopt the first-order gradient-based Reptile [14] as the backbone of metalearning, and employ the popular neural solver POMO [26] as the base model. In our meta-learning process, the model θ is trained with T m meta-iterations, where a multi-task model is exploited to speed up the training and a scaled symmetric sampling method is designed to stabilize the training. In our fine-tuning process, the meta-model θ is hierarchically and quickly fine-tuned with a few gradient updates to solve the corresponding subproblems. The details for each design are presented below.

4.1 Accelerated training

In the accelerated training process, as shown in Algorithm 1, the meta-model θ is trained with T m meta-iterations. In each iteration, the model needs to be optimized by learning ˜ N sampled tasks (subproblems) with T u -step gradient updates. Inspired by the feature reuse of meta-learning [15], it is unnecessary and expensive to serially update the meta-model through the ˜ N respective submodels of the same architecture. On the other hand, it is reasonable to assume that only the head θ head is specified for a task, while the body θ body can be reused for all tasks. To realize such a lightweight meta-learning scheme, we further introduce a multi-task model ˜ θ which is composed of a shared body and ˜ N respective heads to learn the ˜ N tasks in parallel.

Specifically, since our EMNH adopts the encoder-decoder structured POMO as the base model, as depicted in Figure 2(a), 1) θ head could be defined as the decoder head, i.e., W K ∈ R d × d in the last single-head attention layer, where d is empirically set to 128 [26]; 2) θ body could be composed of the whole encoder θ en and the decoder body θ de -body . For a problem instance with n nodes, the node embeddings h 1 , . . . , h n ∈ R d are computed by θ en at the encoding step. At each decoding step, the query q c ∈ R d is first computed by θ de -body using the node embeddings and problem-specific context embedding h c . Then, the last single-head attention layer computes the probability of node selection P θ ( π | s ) using q c and the key k 1 , . . . , k n ∈ R d , where k i ′ for node i ′ is computed by θ head , i.e., k i ′ = W K h i ′ . More details are presented in Appendix A.

As demonstrated in Figure 2(b), for natural accommodation, the multi-task model ˜ θ consists of ˜ θ body and ˜ θ head 1 , . . . , ˜ θ head ˜ N , where ˜ θ body and ˜ θ head i have the same architecture as θ body and θ head , respectively. Note that ˜ θ head i is individually updated for subproblem i , while ˜ θ body is shared for ˜ N tasks. Concretely, since the subproblems are captured by different weight vectors but the same (or similar) node features, the shared node embeddings are computed by ˜ θ en . At each decoding step, for subproblem i , the query q c,i is first computed by ˜ θ de -body using the node embeddings and context embedding h c,i . Then, the key is computed as k i ′ ,i = W i K h i ′ for node i ′ . Finally, the single-head attention layer computes the probability of node selection P ˜ θ i ( π | s ) for subproblem i using q c,i and k 1 ,i , . . . , k n,i . In practice, W K 1 , . . . , W K ˜ N are concatenated to tackle the ˜ N tasks in parallel.

Algorithm 1 Accelerated training process

Input: initialized meta-model θ , number of symmetric sampled weight vectors ˜ N , initialized multitask model ˜ θ , initial meta-learning rate ϵ 0 , number of meta-iterations T m , number of update steps of the multi-task model T u , batch size B , problem size n

1: ϵ ← ϵ 0

2: for t m = 1 to T m do

3: λ i is obtained by the scaled symmetric sampling method, ∀ i ∈ { 1 , . . . , ˜ N }

4: ˜ θ body ← θ body

5: ˜ θ head i ← θ head , ∀ i

6: for t u = 1 to T u do

7: s j ∼ SampleInstance ( S ) , ∀ j ∈ { 1 , . . . , B }

8: { π k | s j , λ i } ∼ SampleRollout ( P ˜ θ i ( ·| s j )) , ∀ k ∈ { 1 , . . . , n } , ∀ i , ∀ j

9: b ij ← 1 n ∑ n k =1 g ( π k | s j , λ i )

10: ∇L ( ˜ θ ) ← 1 ˜ NBn ˜ N ∑ i =1 B ∑ j =1 n ∑ k =1 [( g k ij -b ij ) ∇ log P ˜ θ i ( π k | s j )]

11: ˜ θ ← Adam ( ˜ θ , ∇L ( ˜ θ ))

12: end for

13: θ body ← ˜ θ body

14: θ head ← θ head + ϵ ( 1 ˜ N ∑ ˜ N i =1 ˜ θ head i -θ head )

15: ϵ ← ϵ -ϵ 0 /T m

16: end for

Output: The trained meta-model θ

LP

𝑥1

...

𝑥𝑛

×

𝒩

Layers

×

T

Steps

Solution

MHA

ADD & BN

FF

Node Feature

𝑙

...

ℎ𝑛

𝑙

෠

𝑙

...

෠

ℎ𝑛

𝑙

ADD & BN

...

ℎ𝑛

𝒩

MHA

(

𝜃de-body

)

𝑊 𝐾

(

𝜃head

)

𝑞𝑐

𝑘1

...

𝑘𝑛

Compatibility

Masked Softmax

𝑝1

𝑡

...

𝑝𝑛

𝑡

𝜋𝑡

ℎ𝑐

𝜃en

Decoder

Encoder

(a)

ℎ𝑐

ℎ𝑐

𝜋𝑡

𝜋𝑡

𝑝1

𝑡

...

𝑝𝑛

𝑡

𝑝1

𝑡

...

𝑝𝑛

𝑡

𝑞𝑐

𝑞𝑐

𝑘1

...

𝑘𝑛

𝑘1

...

𝑘𝑛

𝑊 𝐾

(

𝜃head

)

𝑊 𝐾

(

𝜃head

)

LP

𝑥1

...

𝑥𝑛

×

𝒩

Layers

×

T

Steps

Decoder

Encoder

෩

𝑁

Solutions

MHA

ADD & BN

FF

Node Feature

𝑙

...

ℎ𝑛

𝑙

෠

𝑙

...

෠

ℎ𝑛

𝑙

ADD & BN

...

ℎ𝑛

𝒩

MHA

(

෨

𝜃de-body

)

𝑊1

𝐾

(

෨

𝜃head1

)

𝑞𝑐

𝑘1

...

𝑘𝑛

Compatibility

Masked Softmax

𝑝1

𝑡

...

𝑝𝑛

𝑡

𝜋𝑡

ℎ𝑐

෨

𝜃en

×

෩

𝑁

×

෩

𝑁

×

෩

𝑁

(b)

We extend REINFORCE [42] to train the multi-task model ˜ θ , including ˜ θ body for all tasks and ˜ θ head i for task i . Specifically, ˜ θ is optimized by gradient descent with the multi-task loss ∇L = 1 ˜ N ∑ ˜ N i =1 ∇L i , where L i ( ˜ θ | s ) is the i -th task loss and its gradient is estimated as follows,

where b i ( s ) = 1 n ∑ n k =1 g ( π k | s, λ i ) is a baseline to reduce the gradient variance; π 1 , . . . , π n are solutions produced by POMO with n different starting nodes. In practice, ∇L is approximated using Monte Carlo sampling, and computed by a batch with B instances as follows,

where g k ij = g ( π k | s j , λ i ) and b ij = b i ( s j ) . ˜ θ is then optimized with T u steps using Adam [43].

(0,1)

(1,0)

…

(0,1)

(1,0)

…

(0,1)

(1,0)

…

𝝀1

𝝀2

෨

𝝀1

෨

𝝀2

෨

𝝀1

෨

𝝀2

Balanced space

𝝀1

𝑠

𝝀2

𝑠

Meta-model

𝝀

Sampled weight

෨ 𝝀

Symmetric weight

(0.2,0.8)

(0.4,0.6)

(0.2,0.8)

f

1

=

f

2

, Balance

f

1

=

f

2

, Balance

f

1

/

f

2

= 7/3, Imbalance

(0.2,0.8)

(0.37,0.63)

(0.63,0.37)

(0.43,0.57)

Rotational symmetry

Scale

Meta-model update

(a)

(b)

(c)

(0,1)

(1,0)

4.2 Stabilized training

In each meta-iteration, the deviation of ˜ N (usually a small number) randomly sampled weight vectors may cause fluctuations to the parameter update of the meta-model, thereby leading to unstable training performance. For example, as shown in Figure 3(a), the biased samples of weight vectors, i.e., λ 1 and λ 2 , make the meta-model tend to favor the second objective more in this iteration.

To address this issue, we first design a symmetric sampling method for the case of balanced objective domain, as illustrated in Figure 3(b). Once a weight vector ˜ λ 1 is sampled, its M -1 rotational symmetric weight vectors ˜ λ 2 , . . . , ˜ λ M are calculated and used to reduce the bias as follows,

For the case of imbalanced objective domains, we further design a scaled symmetric sampling method. In particular, the m -th weight λ m is scaled as λ m /f ∗ m , where f ∗ m is the ideal objective value but difficult to be calculated beforehand. Thus, f ∗ m is replaced with f ′ m , which is dynamically estimated using the meta-model on a validation dataset with λ = (1 /M,..., 1 /M ) during training.

Concretely, for a sampled ˜ λ 1 , it is first multiplied by f ′ to scale objectives into the balanced domains. Then, its M -1 rotational symmetric weight vectors are obtained by Eq. (5). Finally, they are divided by f ′ to scale back to the original domain to attain the genuine rotational symmetric weight vectors. An example for ˜ λ 1 = (0 . 2 , 0 . 8) and its scaled symmetric ˜ λ 2 = (0 . 43 , 0 . 57) are demonstrated in Figure 3(c). In summary, the scaled symmetric ˜ λ 2 , . . . , ˜ λ M for a given ˜ λ 1 are calculated as follows,

Then, the vectors are normalized as ˜ λ i,m = λ s i,m ∑ M m =1 λ s i,m . The pseudo code of the scaled symmetric sampling method for the weight vectors is presented in Appendix B.

4.3 Hierarchical fine-tuning

Once the meta-model is trained, it can be fine-tuned with a number of gradient updates to derive customized submodels for given weight vectors. However, it still might be expensive for MOCOPs since numerous submodels need to be individually fine-tuned from the meta-model. From a systematic perspective, the early coarse-tuning processes in which the parameters of neighboring submodels might be close, could be merged. In this sense, we propose a hierarchical method to efficiently fine-tune the numerous subproblems from the lowest to the highest level, as illustrated in Figure 1.

We suggest an L -level and a -section hierarchy to gradually produce more specific submodels. Particularly, all N ( l ) weight vectors in level l are defined as the centers of their corresponding subspaces that uniformly divide the whole weight vector space based on the Das and Dennis [44] method. N ( l ) submodels are then fine-tuned to derive N ( l +1) = aN ( l ) submodels with K ( l +1) fine-tuning steps in level l +1. Finally, N ( L -1) submodels are fine-tuned to derive N ( L ) = N submodels, which are used to solve the N given subproblems. More details of this fine-tuning are presented in Appendix C.

For the L -level and a -section hierarchy with a L = N and K (1) = · · · = K ( L ) = K , the total fine-tuning step is T h f = ∑ L l =1 Ka l = Ka ( a L -1) / ( a -1) . For the vanilla process, each submodel needs KL fine-tuning steps to achieve comparable performance, thus T v f = KLN = KLa L steps in total. Hence, the hierarchical fine-tuning only needs T h f /T v f ≈ 1 /L of the vanilla fine-tuning steps.

5 Experimental results

5.1 Experimental settings

We conduct computational experiments to evaluate the proposed method on the multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP). Following the convention in [26, 12], we consider the instances of different sizes n =20/50/100 for MOTSP/MOCVRP and n =50/100/200 for MOKP. All experiments are run on a PC with an Intel Xeon 4216 CPU and an RTX 3090 GPU.

Problems. Four classes of MOTSP [45] are considered, including bi-objective TSP type 1&2 (BiTSP-1, Bi-TSP-2) and tri-objective TSP type 1&2 (Tri-TSP-1, Tri-TSP-2). For the M -objective TSP type 1, node i has M 2D coordinates { x 1 i , . . . , x i M } , where the m -th cost between node i and j is defined as the Euclidean distance c m ij = ∥ x i m -x j m ∥ 2 . For the M -objective TSP type 2, a node contains M -1 2D coordinates together with the altitude, so the M -th cost is defined as the altitude variance. For the bi-objective CVRP (Bi-CVRP) [46], two conflicting objectives with imbalanced domains, i.e., the total traveling distance and the traveling distance of the longest route (also called makespan), are considered. For the bi-objective KP (Bi-KP) [47], item i has a single weight and two distinct values. More details of these MOCOPs are presented in Appendix D.

Hyper-parameters. The meta-learning rate ϵ is linearly annealed to 0 from ϵ 0 = 1 initially. A constant learning rate of the Adam optimizer is set to 10 -4 . We set B = 64 , T m = 3000 , T u = 100 , and ˜ N = M . The N weight vectors in PF construction are produced according to [44], where N = C M -1 H + M -1 . H is set to 100 ( N = 101 ) and 13 ( N = 105 ) for M = 2 and M = 3 , respectively. WS scalarization is adopted. In the hierarchy of fine-tuning, the whole weight space is uniformly divided by the Das and Dennis [44] method with H ( l ) = 2 l , i.e., level l has 2 l or 4 l subspaces and L = 7 or L = 4 for M = 2 or M = 3 . The number of fine-tuning steps at each level is K ( l ) = K = 20 or K ( l ) = K = 25 for M = 2 or M = 3 .

Baselines. Two kinds of strong baselines are introduced, and all of them adopt WS (weighted sum) scalarization for fair comparisons. (1) The state-of-the-art neural heuristics, including MDRL [13], PMOCO [12], and DRL-based multiobjective optimization algorithm ( DRL-MOA ) [8]. Same as EMNH, all these neural heuristics adopt POMO as the base model for single-objective subproblems. (2) The state-of-the-art traditional heuristics, including PPLS/D-C [20], WS-LKH , and WS-DP . PPLS/D-C is a local-search-based MOEA proposed for MOCOPs, where a 2-opt heuristic is used for MOTSP and MOCVRP, and a greedy transformation heuristic [48] is used for MOKP, running with 200 iterations for all the three problems. WS-LKH and WS-DP are based on the state-of-the-art heuristics for decomposed single-objective subproblems, i.e., LKH [49, 50] for MOTSP and dynamic programming (DP) for MOKP, respectively. Our code is publicly available 2 .

Metrics. Our EMNH is evaluated in terms of solution quality and learning efficiency. Solution quality is mainly measured by hypervolume (HV) [51], where a higher HV means a better solution set (the definition is presented in Appendix E). Learning efficiency mainly refers to the training and fine-tuning efficiency, which are measured by training time for the same amount of training instances and solution quality with the same total fine-tuning steps (see Appendix F), respectively.

5.2 Solution quality

The results for MOTSP, MOCVRP, and MOKP are recorded in Tables 1 and 2, including the average HV, gap, and total running time for solving 200 random test instances. To further show the significant differences between the results, a Wilcoxon rank-sum test at 1% significance level is adopted. The best result and the one without statistical significance to it are highlighted as bold . The second-best result and the one without statistical significance to it are highlighted as underline. The method with '-Aug" represents the inference results using instance augmentation [12] (see Appendix G).

Optimality gap. The gaps of HV with respect to EMNH-Aug are reported for all methods. According to the results in Tables 1 and 2, EMNH outperforms other neural heuristics without instance augmentation on all problems. When equipped with instance augmentation, EMNH-Aug further improves the solution, and performs superior to most of the baselines, while slightly inferior to WS-LKH on MOTSP with n =50 and n =100. However, as iteration-based methods, WS-LKH and MOEAs take quite long running time on MOCOPs. For the neural heuristics, DRL-MOA is less agile since it trains multiple fixed models for a priori weight vectors. For new weight vectors concerned by decision makers, EMNH, MDRL, and PMOCO are able to efficiently produce high-quality trade-off solutions, where EMNH achieves the smallest gap.

Imbalanced objective domains. As demonstrated in Table 2, the gaps between EMNH and PMOCO or MDRL are further increased on the problems with imbalanced objective domains, i.e., Bi-CVRP, BiTSP-2, and Tri-TSP-2. EMNH exhibits more competitive performance on these problems, revealing

the effectiveness of the proposed scaled symmetric sampling method in tackling the imbalance of objective domains. We further equip PMOCO with the same sampling method, which actually also improved the solutions for those problems, but still performed inferior to EMNH (see Appendix H).

Generalization ability. We test the generalization ability of the model on 200 larger-scale random instances ( n =150/200) and 3 commonly used MOTSP benchmark instances (KroAB100/150/200) in TSPLIB [52]. The zero-shot generalization performance of the model trained and fine-tuned both on the instances with n =100 is reported in Appendix I. The results suggest that EMNH has a superior generalization ability compared with MOEAs and other neural heuristics for larger problem sizes.

Hyper-parameter study. The results in Appendix J showed that the number of sampled weight vectors ˜ N = M is a more desirable setting and WS is a simple yet effective scalarization method.

5.3 Learning efficiency

As verified above, EMNH is able to produce superior solutions to the state-of-the-art neural heuristics, especially demonstrating a significant advantage over PMOCO in terms of solution quality. Then we further show that EMNH also has favorable learning efficiency against MDRL.

Training efficiency. EMNH, MDRL, and PMOCO only train one model to tackle all subproblems, where EMNH and MDRL use the same amount of training instances, and PMOCO requires a few

6.0

1

2

3

4

5

6

1

2

3

4

5

6

1.0

2.0

2.9

3.8

4.7

5.6

1.0

2.0

3.0

4.0

5.0

Speed-up Ratio

N

Bi-TSP-1

(a)

0

600

1200

1800

2400

3000

0.38

0.39

0.40

0.41

0.42

0.43

(

n

=20

)

HV

T m

EMNH

EMNH-S

EMNH-R

MDRL

(b)

1

5

10

15

20

0.640

0.645

0.650

0.655

0.660

0.665

0.670

Bi-TSP-2

(

n

=20

)

HV

K

EMNH-Aug

MDRL-Aug

PMOCO-Aug

EMNH

MDRL

PMOCO

(c)

more training instances according to its original setting. DRL-MOA needs to train multiple submodels with more training instances. More details are presented in Appendix F. Figure 4(a) displays the speed-up ratio, i.e., the ratio of the training time of MDRL to EMNH. Due to the multi-task based training for (partial) architecture reuse, training time of EMNH is only about 1 / ˜ N of that of MDRL.

Training stability. Figure 4(b) shows the stability of the training process, where EMNH-S and EMNH-R refer to EMNH with the symmetric sampling and the random sampling, respectively. EMNH achieves the stablest and best training performance, compared to EMNH-S, EMNH-R, and MDRL, as the proposed sampling method in EMNH considers symmetric weight vectors and (imbalanced) objective domains. More details are presented in Appendix F.

Fine-tuning efficiency. Figure 4(c) compares the hierarchical fine-tuning in EMNH with the vanilla fine-tuning method in MDRL, where the HV of EMNH with K fine-tuning steps at each level and the HVof MDRL with approximately equal total fine-tuning steps are presented. The results demonstrate that EMNH attains higher fine-tuning efficiency than MDRL, i.e., larger gaps with smaller fine-tuning steps. Furthermore, PMOCO is also equipped with the hierarchical fine-tuning (for both versions) for a fair comparison. It is worth noting that our fine-tuning process is performed individually for each weight vector on dedicated fine-tuning instances. As can be seen, PMOCO could hardly get improved by fine-tuning for zero-shot inference on test instances, since it has already converged for the corresponding subproblems. This means that EMNH has favorable potential to derive more desirable submodels to tackle specific tasks. Notably, EMNH with a few fine-tuning steps (e.g., K = 5 ) outperforms PMOCO in most cases, as demonstrated in Appendix F.

6 Conclusion

This paper proposes an efficient meta neural heuristic (EMNH) for MOCOPs. Specifically, EMNH reduces training time, stabilizes the training process, and curtails total fine-tuning steps via (partial) architecture-shared multi-task learning, scaled symmetric sampling (of weight vectors), and hierarchical fine-tuning, respectively. The experimental results on MOTSP, MOCVRP, and MOKP verified the superiority of EMNH in learning efficiency and solution quality. A limitation is that our EMNH, as the same as other neural heuristics, can not guarantee obtaining the exact Pareto front. In the future, we will (1) extend EMNH to other MOCOPs with complex constraints; (2) investigate other advanced meta-learning algorithms and neural solvers as the base model to further improve the performance.

Acknowledgments and disclosure of funding

This work is supported by the National Natural Science Foundation of China (62072483), and the Guangdong Basic and Applied Basic Research Foundation (2022A1515011690, 2021A1515012298).

References

Matthias Ehrgott and Xavier Gandibleux. A survey and annotated bibliography of multiobjective combinatorial optimization. OR-spektrum , 22(4):425-460, 2000.

Kostas Florios and George Mavrotas. Generation of the exact pareto set in multi-objective traveling salesman and set covering problems. Applied Mathematics and Computation , 237:1-19, 2014.

Arne Herzel, Stefan Ruzika, and Clemens Thielen. Approximation methods for multiobjective optimization problems: A survey. INFORMS Journal on Computing , 33(4):1284-1299, 2021.

Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, Jiwoo Pak, Andy Tong, Kavya Srinivasa, William Hang, Emre Tuncer, Quoc V. Le, James Laudon, Richard Ho, Roger Carpenter, and Jeff Dean. A graph placement methodology for fast chip design. Nature , 594:207-212, 2021.

Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research , 290(2):405-421, 2021.

Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, and Evgeny Burnaev. Reinforcement learning for combinatorial optimization: A survey. Computers & Operations Research , 134:105400, 2021.

Qi Wang and Chunlei Tang. Deep reinforcement learning for transportation network combinatorial optimization: A survey. Knowledge-Based Systems , 233:107526, 2021.

Kaiwen Li, Tao Zhang, and Rui Wang. Deep reinforcement learning for multiobjective optimization. IEEE Transactions on Cybernetics , 51(6):3103-3114, 2021.

Hong Wu, Jiahai Wang, and Zizhen Zhang. MODRL/D-AM: Multiobjective deep reinforcement learning algorithm using decomposition and attention model for multiobjective optimization. In ISICA 2019 , volume CCIS 1205, pages 575-589, 2020.

Yinan Shao, Jerry Chun-Wei Lin, Gautam Srivastava, Dongdong Guo, Hongchun Zhang, Hu Yi, and Alireza Jolfaei. Multi-objective neural evolutionary algorithm for combinatorial optimization problems. IEEE Transactions on Neural Networks and Learning Systems , 2021.

Yongxin Zhang, Jiahai Wang, Zizhen Zhang, and Yalan Zhou. MODRL/D-EL: Multiobjective deep reinforcement learning with evolutionary learning for multiobjective optimization. In International Joint Conference on Neural Networks , 2021.

Xi Lin, Zhiyuan Yang, and Qingfu Zhang. Pareto set learning for neural multi-objective combinatorial optimization. In International Conference on Learning Representations , 2022.

Zizhen Zhang, Zhiyuan Wu, Hang Zhang, and Jiahai Wang. Meta-learning-based deep reinforcement learning for multiobjective optimization problems. IEEE Transactions on Neural Networks and Learning Systems , 34(10):7978-7991, 2023.

Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms, 2018.

Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. In International Conference on Learning Representations , 2020.

Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation , 6(2):182-197, 2002.

Qingfu Zhang and Hui Li. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation , 11(6):712-731, 2007.

Andrzej Jaszkiewicz. Genetic local search for multi-objective combinatorial optimization. European Journal of Operational Research , 137(1):50-71, 2002.

Jialong Shi, Qingfu Zhang, and Jianyong Sun. PPLS/D: Parallel pareto local search based on decomposition. IEEE transactions on Cybernetics , 50(3):1060-1071, 2020.

Jialong Shi, Jianyong Sun, Qingfu Zhang, Haotian Zhang, and Ye Fan. Improving pareto local search using cooperative parallelism strategies for multiobjective combinatorial optimization. IEEE Transactions on Cybernetics , 2022.

Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In Advances in Neural Information Processing Systems , 2015.

Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. In International Conference on Learning Representations , 2017.

Mohammadreza Nazari, Afshin Oroojlooy, Martin Takáˇ c, and Lawrence V. Snyder. Reinforcement learning for solving the vehicle routing problem. In Advances in Neural Information Processing Systems , 2018.

Wouter Kool, Herke Van Hoof, and Max Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations , 2019.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems , 2017.

Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. POMO: Policy optimization with multiple optima for reinforcement learning. In Advances in Neural Information Processing Systems , 2020.

Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve np-hard routing problems. In Advances in Neural Information Processing Systems , 2021.

Yeong-Dae Kwon, Jinho Choo, Iljoo Yoon, Minah Park, Duwon Park, and Youngjune Gwon. Matrix encoding networks for neural combinatorial optimization. In Advances in Neural Information Processing Systems , 2021.

Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In AAAI Conference on Artificial Intelligence , pages 12042-12049, 2021.

Bo Peng, Jiahai Wang, and Zizhen Zhang. A deep reinforcement learning algorithm using dynamic attention model for vehicle routing problems. In ISICA 2019 , volume CCIS 1205, pages 636-650, 2020.

Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song. Learning combinatorial optimization algorithms over graphs. In Advances in Neural Information Processing Systems , 2017.

Sahil Manchanda, Akash Mittal, Anuj Dhawan, Sourav Medya, Sayan Ranu, and Ambuj Singh. GCOMB: Learning budget-constrained combinatorial algorithms over billion-sized graphs. In Advances in Neural Information Processing Systems , 2020.

Lu Duan, Yang Zhan, Haoyuan Hu, Yu Gong, Jiangwen Wei, Xiaodong Zhang, and Yinghui Xu. Efficiently solving the practical vehicle routing problem: A novel joint learning approach. In ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 3054-3063, 2020.

Jinbiao Chen, Huanhuan Huang, Zizhen Zhang, and Jiahai Wang. Deep reinforcement learning with twostage training strategy for practical electric vehicle routing problem with time windows. In International Conference on Parallel Problem Solving from Nature , 2022.

Xinyun Chen and Yuandong Tian. Learning to perform local rewriting for combinatorial optimization. In Advances in Neural Information Processing Systems , 2019.

Hao Lu, Xingwen Zhang, and Shuang Yang. A learning-based iterative method for solving vehicle routing problems. In International Conference on Learning Representations , 2020.

André Hottung and Kevin Tierney. Neural large neighborhood search for the capacitated vehicle routing problem. In European Conference on Artificial Intelligence , 2020.

Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim. Learning improvement heuristics for solving routing problems. IEEE Transactions on Neural Networks and Learning Systems , 33(9):5057-5069, 2022.

Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Le Zhang, Zhenghua Chen, and Jing Tang. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. In Advances in Neural Information Processing Systems , 2021.

Xiaoliang Ma, Qingfu Zhang, Guangdong Tian, Junshan Yang, and Zexuan Zhu. On tchebycheff decomposition approaches for multiobjective evolutionary optimization. IEEE Transactions on Evolutionary Computation , 22(2):226-244, 2017.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning , pages 1126-1135, 2017.

Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning , 8(3):229-256, 1992.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations , 2015.

I Das and JE Dennis. Normal-boundary intersection: A new method for generating pareto-optimal points in multieriteria optimization problems. SIAM Journal on Optimization , 8(3):631-657, 1998.

Thibaut Lust and Jacques Teghem. The multiobjective traveling salesman problem: A survey and a new approach. In Advances in Multi-Objective Nature Inspired Computing , pages 119-141, 2010.

Nicolas Jozefowiez, Frédéric Semet, and El-Ghazali Talbi. Multi-objective vehicle routing problems. European Journal of Operational Research , 189(2):293-309, 2008.

Cristina Bazgan, Hadrien Hugot, and Daniel Vanderpooten. Solving efficiently the 0-1 multi-objective knapsack problem. Computers & Operations Research , 36(1):260-279, 2009.

Hisao Ishibuchi, Naoya Akedo, and Yusuke Nojima. Behavior of multiobjective evolutionary algorithms on many-objective knapsack problems. IEEE Transactions on Evolutionary Computation , 19(2):264-283, 2015.

Keld Helsgaun. An effective implementation of the lin-kernighan traveling salesman heuristic. European Journal of Operational Research , 126(1):106-130, 2000.

Renato Tinós, Keld Helsgaun, and Darrell Whitley. Efficient recombination in the lin-kernighan-helsgaun traveling salesman heuristic. In International Conference on Parallel Problem Solving from Nature , pages 95-107, 2018.

Charles Audet, Jean Bigeon, Dominique Cartier, Sébastien Le Digabel, and Ludovic Salomon. Performance indicators in multiobjective optimization. European Journal of Operational Research , 292(2):397-422, 2021.

Gerhard Reinelt. TSPLIB-a traveling salesman problem library. ORSA Journal on Computing , 3(4): 376-384, 1991.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article presents an efficient neural heuristic method based on meta-learning, referred to as EMNH (Efficient Meta-learning Neural Heuristic), for solving Multi-Objective Combinatorial Optimization Problems (MOCOPs). The authors employ a shared multi-task model to expedite the meta-learning process and introduce a weight vector-based scaling symmetric sampling approach to stabilize this process. Furthermore, they propose a layered fine-tuning method to effectively address all sub-problems. Experimental results demonstrate that EMNH outperforms existing neural heuristic methods on three classical MOCOPs and can also compete with traditional heuristic methods.\n\nSTRENGTHS:\n1. The article addresses the practical and challenging issue of MOCOPs, presenting a novel neural heuristic approach that combines ideas from meta-learning, multi-task learning, and layered fine-tuning.\n2. The proposed method's advantages are theoretically analyzed, including accelerated and stabilized meta-learning processes, along with reduced fine-tuning steps.\n3. The article conducts comprehensive experiments on three different types of MOCOPs, comparing against various benchmark methods. It showcases the proposed method's advantages in terms of solution quality, learning efficiency, and generalization ability.\n\nWEAKNESSES:\n1. The article inadequately discusses the limitations and shortcomings of the proposed method, such as its applicability to more complex or higher-dimensional MOCOPs, sensitivity to weight vector selection, and risks of overfitting or underfitting.\n2. The article lacks sufficient details to explain the model architecture, hyperparameter settings, experimental setup, etc. This might affect the replicability and credibility of the results.\n\nQUESTIONS:\n1. Did the authors consider using other types of neural network structures or meta-learning algorithms to implement the proposed method? If so, how do they compare to POMO and Reptile?\n2. Did the authors attempt to test the proposed method on other types or scales of MOCOPs? If yes, what were the results? If not, what were the reasons?\n\nLIMITATIONS:\n1. The proposed method might require substantial training data and computational resources to achieve ideal results, potentially limiting its feasibility and scalability in practical applications.\n2. The proposed method might struggle with MOCOPs featuring nonlinear or non-convex objective functions, dynamic or stochastic constraints, multi-modal or multi-peak distributions, and other complex features.\n3. The proposed method might not guarantee the identification of true Pareto-optimal solution sets, only approximated solution sets, potentially affecting evaluators' judgments on solution quality and diversity.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper proposes efficient meta neural heuristic (EMNH) for solving multi-objective combinatorial optimization problems (MOCOP). The paper provides novel scaled sampling method for stability and a hierarchical fine-tune method for sub-task specific performance improvement over MDRL. The idea is sound and the experiments yield better performance than the state-of-the-art methods for MOCOP.

STRENGTHS:
The idea of using multi-task body with different heads in meta-learning is sound.
The experiments are detailed and convincing.

WEAKNESSES:
I’m not an expert in MOCOP problem, as far as I’m concerned, I detect no weakness of this paper.

QUESTIONS:
1. According to Table 2, although the traditional heuristics may yield the best performance, it would take them much more time to solve the problem. Given the same amount of fine-tuning time, what is the performance of the proposed MOCOP method EMNH? (i.e. set K to a large number).

LIMITATIONS:
The proposed EMNH provides no guarantee for the pareto front.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes EMNH, an efficient meta neural heuristic, for solving multi-objective combinatorial optimization problems. It builds a single meta model to tackle different trade-offs among multiple objectives during training, which can be efficiently fine-tuned into specialized submodels to solve different trade-offed subproblems. The main contributions are: 1) a multi-task meta model with parameter-shared body and task-related heads that can handle different trade-offs at the same time; 2) a scaled symmetric sampling method to stabilize the multi-objective training with imbalanced objectives; and 3) a hierarchical fine-tuning method to gradually fine-tune a set of submodel with much fewer steps. Experimental results show that the proposed EMHN method can outperform other neural combinatorial optimization methods on different combinatorial optimization problems such as MOTSP, MOCVRP, and MOKP.\n\nSTRENGTHS:\n+ This paper is well-written and easy to follow.\n\n+ Multi-objective combinatorial optimization is important for real-world applications. Neural heuristic is a promising approach to tackle this problem, but only a few methods have been proposed recently. This work is a timely contribution to a promising research direction.\n\n+ The proposed method obtains promising performances on various multi-objective combinatorial optimization problems.\n\nWEAKNESSES:\nI have the following concerns for the proposed method: \n\n**1. Runetime and Efficiency of Fine-tune**\n\nThe runtime of the fine-tuning approach is not clearly discussed in the paper. In my understanding, for solving a new problem instance, the meta-learning based methods need to first fine-tune the meta model into different submodels, and then run each submodel to generate corresponding approximate solutions. In other words, the prediction is not zero-shot inference. However, in all experiments, the reported runtimes for MDRL and EMNH are similar to DRL-MOA and PMOCO which support zero-shot inference. Why the cost of fine-tuning is not included in the runtime? \n\n**2. Structure of Multi-Task Meta Model**\n\nThe proposed multi-task meta model has a single model body that is shared by all tasks and specialized heads for different tasks. Recent work has also investigated fine-tuning a small part of the neural combinatorial optimization model to improve the performance of each instance for single-objective problems [1]. What is the advantage of the proposed model structure for multi-objective optimization? Is there any guideline for model design? \n\n[1] Efficient Active Search for Combinatorial Optimization Problems, ICLR 2022.\n\n**3. Solution Distribution**\n\nEMNH mainly uses the weighted-sum strategy to decompose a multi-objective optimization problem into multiple subproblems. According to PMOCO [1], different decomposition methods will lead to very different solution distributions on a given problem instance, especially those with more than two objectives. In addition, as reported in a current work [2], the decomposition-based PMOCO will generate redundant solutions for different preferences. Will EMNH also have these two issues? Can these issues be addressed by the scaled symmetric weight sampling method? \n\n[2] Pareto Set Learning for Neural Multi-Objective Combinatorial Optimization, ICLR 2022.\n\n[3] Graph Learning Assisted Multi-Objective Integer Programming, NeurIPS 2022.\n\n**4. Fine-Tuning PMOCO**\n\nAs reported in this work, PMOCO usually has a promising zero-shot prediction performance, but it cannot be further improved with the fine-tuned approach. In my understanding, PMOCO is also a variant of the AM structure in POMO. According to recent works [1], only fine-tuning a small part of the model parameters in POMO can significantly improve the performance of single-objective optimization. It is interesting to know why fine-tuning does not work for PMOCO.\n\nQUESTIONS:\n- Please address the concerns raised in Weaknesses.\n\nLIMITATIONS:\nThe limitation of the proposed method has been discussed in the conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a meta neural heuristic in which a meta model is first trained and then fine-tuned with a few steps\nto solve corresponding single-objective subproblems. For the training process, a partial architecture-shared multi-task model is leveraged to achieve parallel learning for the meta model, so as to speed up the training. Meanwhile, a scaled symmetric sampling method with respect to the weight vectors is designed to stabilize the training. For the fine-tuning process, an efficient hierarchical method is proposed to systematically tackle all the subproblems.\n\nThe article contains a review of related works and preliminaries, then it presents the introduced methodology.\nThis is followed by the description of experiments, their settings, results, and analysis. The experiments were carried out for\nthe multi-objective traveling salesman problem, multi-objective capacitated vehicle routing problem, and multi-objective knapsack problem. They showed that the introduced method is able to outperform the state-of-the-art neural heuristics in terms of solution quality and learning efficiency and yield competitive solutions to the strong traditional heuristics while consuming a much shorter time.\n\nThe main text is followed by supplementary materials.\n\nSTRENGTHS:\nThe introduced methodology is quite advanced and seems to be innovative and successful. \nI also like that it was tested on several problems from different domains and besides reporting efficiency and the quality of the found solutions, the required time is reported too. The paper is well written and it is good that it is followed by the supplementary materials. I like that there is a pseudocode in the supplementary materials and the authors declared that the codes for all the methods will be made available.\n\nWEAKNESSES:\nThe main weakness I found is that the code and the datasets used in experiments are not available, so it is difficult to verify and reproduce the results. However, the authors provided some pseudocode and declared that the code will be made publicly available. Besides, didn't find serious weaknesses of this paper, but it is possible that I didn't understand some parts.\n\nQUESTIONS:\nI don't have any specific questions.\n\nLIMITATIONS:\nThe main limitation, according to the authors, is that the method can not guarantee to obtain the exact Pareto front (similar to other neural heuristics). Also, the code and the datasets used in experiments are not available, so it is difficult to verify and reproduce the results. However, the authors provided some pseudocode and declared that the code will be made publicly available. I didn't find other limitations, but it is possible that I didn't understand some parts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn order to achieve higher learning efficiency and better solution quality, this paper proposed an efficient meta neural heuristic (EMNH), in which a meta model is first trained and then fine-tuned with a few steps to solve corresponding single-objective subproblems. For the training process, an architecture-shared multi-task model is leveraged to achieve parallel learning for the meta model, so as to accelerate the training; meanwhile, a scaled symmetric sampling method with respect to the weight vectors is designed to stabilize the training. For the fine-tuning process, an efficient hierarchical method is proposed to systematically tackle all the subproblems. The general idea of this paper is clear and logical.\n\nSTRENGTHS:\n1. The methodology part of the paper is clearly described and has a certain degree of logic.\n\nWEAKNESSES:\n1. In the Methodology part, this paper proposed three methods for accelerate training, stabilize training and hierarchical Fine-tuning. However, in the Introduction part, the connection between motivation of this paper and the proposed stabilized training method should be clearer described.\n2. In the Experimental Results part, ablation experiment should be added in order to verify the effect of the proposed method. To be more specific, as for the experiments towards learning efficiency, the comparison between original method and method without adding stabilized training part should be done.\n\nQUESTIONS:\n1. In the Experimental Results part, the experiment results in tables should be further analyzed. \n2. In the Training Efficiency part, more current comparison method should be added in order to support the experimental conclusion with more sufficient and comprehensive experimental results.\n3. Are there any control parameters in the proposed method? how sensitive are them?  \n4. The picture in Figure 4 is too small for clearly understanding of readers.\n5. The writing of the paper could be improved for better description and clarification.\n\nLIMITATIONS:\n1. The experimental problems of this method are limited to real-world problems, and further experimental verification should be carried out on synthetic problems.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}