SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jinbiao Chen $^{1}$ , Jiahai Wang $^{1,2,3,*}$ , Zizhen Zhang $^{1,*}$ , Zhiguang Cao $^{4}$ , Te Ye $^{1}$ , Siyuan Chen $^{1}$
$^{1}$ School of Computer Science and Engineering, Sun Yat-sen University, P.R. China
$^{2}$ Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, Sun Yat-sen University, P.R. China
<sup>3</sup>Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, P.R. China
<sup>4</sup>School of Computing and Information Systems, Singapore Management University, Singapore chenjb69@mail2.sysu.edu.cn, {wangjiah, zhangzzh7}@mail.sysu.edu.cn zgcao@smu.edu.sg, {yete, chensy47}@mail2.sysu.edu.cn
Recently, neural heuristics based on deep reinforcement learning have exhibited promise in solving multi-objective combinatorial optimization problems (MOCOPs). However, they are still struggling to achieve high learning efficiency and solution quality. To tackle this issue, we propose an efficient meta neural heuristic (EMNH), in which a meta-model is first trained and then fine-tuned with a few steps to solve corresponding single-objective subproblems. Specifically, for the training process, a (partial) architecture-shared multi-task model is leveraged to achieve parallel learning for the meta-model, so as to speed up the training; meanwhile, a scaled symmetric sampling method with respect to the weight vectors is designed to stabilize the training. For the fine-tuning process, an efficient hierarchical method is proposed to systematically tackle all the subproblems. Experimental results on the multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP) show that, EMNH is able to outperform the state-of-the-art neural heuristics in terms of solution quality and learning efficiency, and yield competitive solutions to the strong traditional heuristics while consuming much shorter time.
Multi-objective combinatorial optimization problems (MOCOPs) [1] are widely studied and applied in many real-world sectors, such as telecommunication, logistics, manufacturing, and inventory. Typically, an MOCOP requires the simultaneous optimization of multiple conflicting objectives, where the amelioration of an objective may lead to the deterioration of others. Therefore, a set of trade-off solutions, known as Pareto-optimal solutions, are usually sought for MOCOPs.
Generally, it is difficult to exactly find all the Pareto-optimal solutions of an MOCOP [2], especially given that the decomposed single-objective subproblem might already be NP-hard. Hence, heuristic methods [3] are usually preferred to solve MOCOPs in reality, as they can attain approximate Pareto-optimal solutions in (relatively) reasonable time. Nevertheless, the traditional heuristics are still lacking due to their reliance on handcrafted rules and massive iterative steps, and superfluous computation even for instances of the same (or similar) class.
Recently, inspired by the success of deep reinforcement learning (DRL) in learning neural heuristics for solving the single-objective combinatorial optimization problems (COPs) [4-7], a number of
* Jiahai Wang and Zizhen Zhang are the corresponding authors.
Figure 1: The overall framework of EMNH.
DRL-based neural heuristics [8-13] have also been investigated for MOCOPs. While bypassing the handcrafted rules, these neural heuristics adopt an end-to-end paradigm to construct solutions without iterative search. Benefiting from a large amount of data (i.e., problem instances), a well-trained deep model allows the neural heuristic to automatically extract informative and expressive features for decision making and generalize to unseen instances as well.
Although demonstrating promise for solving MOCOPs, those neural heuristics still struggle to achieve high learning efficiency and solution quality. Particularly, in this line of works, the early attempts [8-11] always train multiple deep models, i.e., one for each preference (weight) combination, making them less practical. While the well-known preference-conditioned multi-objective combinatorial optimization (PMOCO) [12] realized a unified deep model by introducing a huge hypernetwork, it leaves considerable gaps in terms of solution quality. The recent Meta-DRL (MDRL) [13] has demonstrated the capability to enhance solution quality over existing state-of-the-art algorithms. However, it still faces challenges related to inefficient and unstable training procedures, as well as undesirable fine-tuning processes. This paper thereby proposes an efficient meta neural heuristic (EMNH) to further strengthen the learning efficiency and solution quality, the framework of which is illustrated in Figure 1. Following the meta-learning paradigm [13, 14], EMNH first trains a meta-model and then quickly fine-tunes it according to the weight (preference) vector to tackle the corresponding single-objective subproblems.
Our contributions are summarized as follows. (1) We propose an efficient meta-learning scheme to accelerate the training. Inspired by the feature reuse of meta-learning [15], we introduce a multi-task model composed of a parameter-shared body (all other layers of neural network except the last one) and respective task-related heads (the last layer of the network), rather than respective submodels, to train the meta-model, which is able to handle different tasks (subproblems) in parallel. (2) We design a scaled symmetric sampling method regarding the weight vectors to stabilize the training. In each meta-iteration, once a weight vector is sampled, its scaled symmetric weight vectors will also be engendered, which help avoid fluctuations of the parameter update for the meta-model, especially on problems with objectives of imbalanced domains. (3) We present a hierarchical fine-tuning method to efficiently cope with all the subproblems. Since it is redundant to derive a specific submodel for each subproblem in the early tuning process, our meta-model can be gradually fine-tuned to act as specialized submodels from low level to high level, which takes much fewer fine-tuning steps in total. Moreover, experimental results on three classic MOCOPs confirmed the effectiveness of our designs.
Exact and heuristic methods for MOCOPs. Over the past decades, MOCOPs have been attracting increasing attention from the computational intelligence community, and tremendous works have been proposed. Generally, exact methods [2] are able to gain the accurate Pareto-optimal solutions, but their computational time may exponentially grow. Heuristic methods such as multi-objective
evolutionary algorithms (MOEAs) are popular in practice, where the representative paradigms for the general multi-objective optimization include NSGA-II [16] and MOEA/D [17]. Among them, a number of MOEAs outfitted with local search [18-20] are specifically designed for MOCOPs, which iteratively perform a search in the solution space to find approximate Pareto-optimal solutions.
Neural heuristics for COPs. As the early attempts, neural construction heuristics based on deep models [21-23] were proposed to learn to directly construct solutions for (single-objective) COPs. Among them, the classic Attention Model (AM) [24] was proposed inspired by Transformer architecture [25], and known as a milestone for solving vehicle routing problems. A number of subsequent works [26-30] were developed based on AM, including the popular policy optimization with multiple optima (POMO) [26], which leveraged the solution symmetries and significantly enhanced the performance over AM. Besides, graph neural networks were also employed to learn graphic embeddings for solving COPs [31-34]. Different from the above neural construction heuristics, neural improvement heuristics [35-39] were proposed to assist iteration-based methods to refine an initial solution.
Neural heuristics for MOCOPs. MOCOPs seek for a set of trade-off solutions, which could be obtained by solving a series of single-objective subproblems. According to the number of trained models, neural heuristics could be roughly classified into multi-model and single-model ones. The former adopts multiple networks to tackle respective subproblems, where the set of networks could also be collaboratively trained with a neighborhood-based parameter-transfer strategy [8, 9], or evolved with MOEA [10, 11]. Differently, the latter usually exploits a unified network to tackle all the subproblems, which is more flexible and practical. For example, taking the weight (preference) vector of a subproblem as the input, PMOCO introduces a hypernetwork to learn the decoder parameters [12]. However, the hypernetwork may cause extra complexity to the original deep reinforcement learning model, rendering it less effective to learn more accurate mapping from the preference to the optimal solution of the corresponding subproblem. By contrast, MDRL [13] leverages the meta-learning to train a deep reinforcement learning model that could be fine-tuned for various subproblems. However, its learning efficiency is far from optimal, especially given the slow and unstable training due to sequential learning of randomly sampled tasks, and inefficient fine-tuning due to redundant updates.
3.1 MOCOP
In general, an MOCOP could be defined as
$$
\min  _ {x \in \mathcal {X}} f (x) = \left(f _ {1} (x), f _ {2} (x), \dots , f _ {M} (x)\right), \tag {1}
$$
where  $\pmb{f}(\pmb{x})$  is an  $M$ -objective vector,  $\mathcal{X}$  is a discrete decision space, and the objectives might be conflicted. The Pareto-optimal solutions concerned by decision makers are defined as follows.
Definition 1 (Pareto dominance). Let  $u, v \in \mathcal{X}$ ,  $u$  is said to dominate  $v$  ( $u \prec v$ ) if and only if  $f_i(u) \leq f_i(v), \forall i \in \{1, \dots, M\}$  and  $\exists j \in \{1, \dots, M\}$ ,  $f_j(u) < f_j(v)$ .
Definition 2 (Pareto optimality). A solution  $x^{*} \in \mathcal{X}$  is Pareto-optimal if  $x^{*}$  is not dominated by any other solution in  $\mathcal{X}$ , i.e.,  $\nexists x' \in \mathcal{X}$  such that  $x' \prec x^{*}$ . The set of all Pareto optimal solutions  $\mathcal{P} = \{x^{*} \in \mathcal{X} \mid \nexists x' \in \mathcal{X} : x' \prec x^{*}\}$  is called the Pareto set. The image of Pareto set in the objective space, i.e.,  $\mathcal{PF} = \{f(x) \mid x \in \mathcal{P}\}$  is called the Pareto front.
3.2 Decomposition
The decomposition strategy is always applied for solving MOCOPs [17], since it is straightforward yet effective to guide the search towards prescribed directions. Specifically, MOCOPs can be decomposed into  $N$  scalarized single-objective subproblem  $g(x|\lambda)$ , where the weight vector  $\lambda \in \mathcal{R}^M$  satisfies  $\lambda_{m}\geq 0$  and  $\sum_{m = 1}^{M}\lambda_{m} = 1$ . Then,  $\mathcal{PF}$  is approximated by solving the subproblems systematically. Regarding the decomposition strategy, the weighted sum (WS) and Tchebycheff [40] are commonly used. As a simple representative, WS considers the linear combination of  $M$  objectives as follows,
$$
\min  _ {x \in \mathcal {X}} g _ {\mathrm {w s}} (x | \boldsymbol {\lambda}) = \sum_ {m = 1} ^ {M} \lambda_ {m} f _ {m} (x). \tag {2}
$$
Given the weight vector  $\lambda$ , the corresponding single-objective subproblem could be cast as a sequential decision problem and solved through DRL. In particular, a solution is represented as a sequence  $\pi = \{\pi_1,\dots ,\pi_T\}$  with length  $T$ , and a stochastic policy for yielding solution  $\pi$  from instance  $s$  is calculated as  $P(\pi |s) = \prod_{t = 1}^{T}P_{\theta}(\pi_t|\pi_{1:t - 1},s)$ , where  $P_{\theta}(\pi_t|\pi_{1:t - 1},s)$  is the node selection probability parameterized by  $\pmb{\theta}$  (the subscript of  $\pmb{\theta}_i$ ,  $\pmb{\pi}_i$ ,  $P_i$  for subproblem  $i$  is omitted for readability).
3.3 Meta-learning
Meta-learning [14, 41] aims to train a model that can learn to tackle various tasks (i.e., subproblems) via fine-tuning. Naturally, a meta-model can be trained to quickly adapt to  $N$  corresponding subproblems given  $N$  weight vectors. Generally, meta-learning comprises three processes for MOCOPs. In the meta-learning process, a model  $\theta$  is trained by repeatedly sampling tasks from the whole task space. In the fine-tuning process,  $N$  submodels  $\theta_{1},\ldots ,\theta_{N}$  based on the given weight vectors are fine-tuned from  $\theta$  using fine-tuning instances (also called query set). In the inference process,  $N$  subproblems of an instance are solved using the submodels to approximate the Pareto set.
Our efficient meta neural heuristic (EMNH) includes an accelerated and stabilized meta-learning process, a hierarchical fine-tuning process, and an inference process, as depicted in Figure 1. EMNH is generic, where we adopt the first-order gradient-based Reptile [14] as the backbone of meta-learning, and employ the popular neural solver POMO [26] as the base model. In our meta-learning process, the model  $\theta$  is trained with  $T_{m}$  meta-iterations, where a multi-task model is exploited to speed up the training and a scaled symmetric sampling method is designed to stabilize the training. In our fine-tuning process, the meta-model  $\theta$  is hierarchically and quickly fine-tuned with a few gradient updates to solve the corresponding subproblems. The details for each design are presented below.
4.1 Accelerated training
In the accelerated training process, as shown in Algorithm 1, the meta-model  $\theta$  is trained with  $T_{m}$  meta-iterations. In each iteration, the model needs to be optimized by learning  $\tilde{N}$  sampled tasks (subproblems) with  $T_{u}$ -step gradient updates. Inspired by the feature reuse of meta-learning [15], it is unnecessary and expensive to serially update the meta-model through the  $\tilde{N}$  respective submodels of the same architecture. On the other hand, it is reasonable to assume that only the head  $\theta_{\mathrm{head}}$  is specified for a task, while the body  $\theta_{\mathrm{body}}$  can be reused for all tasks. To realize such a lightweight meta-learning scheme, we further introduce a multi-task model  $\tilde{\theta}$  which is composed of a shared body and  $\tilde{N}$  respective heads to learn the  $\tilde{N}$  tasks in parallel.
Specifically, since our EMNH adopts the encoder-decoder structured POMO as the base model, as depicted in Figure 2(a), 1)  $\theta_{\mathrm{head}}$  could be defined as the decoder head, i.e.,  $W^{K} \in \mathcal{R}^{d \times d}$  in the last single-head attention layer, where  $d$  is empirically set to 128 [26]; 2)  $\theta_{\mathrm{body}}$  could be composed of the whole encoder  $\theta_{\mathrm{en}}$  and the decoder body  $\theta_{\mathrm{de - body}}$ . For a problem instance with  $n$  nodes, the node embeddings  $h_{1}, \ldots, h_{n} \in \mathcal{R}^{d}$  are computed by  $\theta_{\mathrm{en}}$  at the encoding step. At each decoding step, the query  $q_{c} \in \mathcal{R}^{d}$  is first computed by  $\theta_{\mathrm{de - body}}$  using the node embeddings and problem-specific context embedding  $h_{c}$ . Then, the last single-head attention layer computes the probability of node selection  $P_{\theta}(\pi | s)$  using  $q_{c}$  and the key  $k_{1}, \ldots, k_{n} \in \mathcal{R}^{d}$ , where  $k_{i'}$  for node  $i'$  is computed by  $\theta_{\mathrm{head}}$ , i.e.,  $k_{i'} = W^{K} h_{i'}$ . More details are presented in Appendix A.
As demonstrated in Figure 2(b), for natural accommodation, the multi-task model  $\tilde{\theta}$  consists of  $\tilde{\theta}_{\mathrm{body}}$  and  $\tilde{\theta}_{\mathrm{head}_1},\ldots ,\tilde{\theta}_{\mathrm{head}_N}$ , where  $\tilde{\theta}_{\mathrm{body}}$  and  $\tilde{\theta}_{\mathrm{head}_i}$  have the same architecture as  $\theta_{\mathrm{body}}$  and  $\theta_{\mathrm{head}}$ , respectively. Note that  $\tilde{\theta}_{\mathrm{head}_i}$  is individually updated for subproblem  $i$ , while  $\tilde{\theta}_{\mathrm{body}}$  is shared for  $\tilde{N}$  tasks. Concretely, since the subproblems are captured by different weight vectors but the same (or similar) node features, the shared node embeddings are computed by  $\tilde{\theta}_{\mathrm{en}}$ . At each decoding step, for subproblem  $i$ , the query  $q_{c,i}$  is first computed by  $\tilde{\theta}_{\mathrm{de - body}}$  using the node embeddings and context embedding  $h_{c,i}$ . Then, the key is computed as  $k_{i^{\prime},i} = W_i^K h_{i^{\prime}}$  for node  $i^{\prime}$ . Finally, the single-head attention layer computes the probability of node selection  $P_{\tilde{\theta}_i}(\pi |s)$  for subproblem  $i$  using  $q_{c,i}$  and  $k_{1,i},\ldots ,k_{n,i}$ . In practice,  $W_1^K,\ldots ,W_N^K$  are concatenated to tackle the  $\tilde{N}$  tasks in parallel.
Algorithm 1 Accelerated training process
Input: initialized meta-model  $\theta$ , number of symmetric sampled weight vectors  $\tilde{N}$ , initialized multi-task model  $\tilde{\theta}$ , initial meta-learning rate  $\epsilon_0$ , number of meta-iterations  $T_{m}$ , number of update steps of the multi-task model  $T_{u}$ , batch size  $B$ , problem size  $n$
1:  $\epsilon \gets \epsilon_0$
2: for  $t_m = 1$  to  $T_m$  do
3:  $\lambda_{i}$  is obtained by the scaled symmetric sampling method,  $\forall i\in \{1,\dots ,\tilde{N}\}$
4:  $\pmb{\theta}_{\mathrm{body}}\gets \pmb{\theta}_{\mathrm{body}}$
5:  $\tilde{\pmb{\theta}}_{\mathrm{head}_i}\gets \pmb{\theta}_{\mathrm{head}},\quad \forall i$
6: for  $t_u = 1$  to  $T_{u}$  do
7:  $s_j \sim \mathbf{SampleInstance}(\mathcal{S}), \quad \forall j \in \{1, \dots, B\}$
8:  $\{\pmb{\pi}^k | s_j, \pmb{\lambda}_i\} \sim \mathbf{SampleRollout}(P_{\tilde{\theta}_i}(\cdot | s_j)), \quad \forall k \in \{1, \dots, n\}, \forall i, \forall j$
9:  $b_{ij}\gets \frac{1}{n}\sum_{k = 1}^{n}g(\pmb{\pi}^k |s_j,\pmb{\lambda}_i)$
0:  $\nabla \mathcal{L}(\tilde{\boldsymbol{\theta}})\gets \frac{1}{\tilde{N}Bn}\sum_{i = 1}^{\tilde{N}}\sum_{j = 1}^{B}\sum_{k = 1}^{n}[(g_{ij}^{k} - b_{ij})\nabla \log P_{\tilde{\boldsymbol{\theta}}_i}(\boldsymbol{\pi}^k |s_j)]$
11:  $\tilde{\theta} \gets \mathbf{Adam}(\tilde{\theta}, \nabla \mathcal{L}(\tilde{\theta}))$
12: end for
13:  $\pmb{\theta}_{\mathrm{body}}\gets \tilde{\pmb{\theta}}_{\mathrm{body}}$
14:  $\pmb{\theta}_{\mathrm{head}}\gets \pmb{\theta}_{\mathrm{head}} + \epsilon (\frac{1}{\tilde{N}}\sum_{i = 1}^{\tilde{N}}\tilde{\pmb{\theta}}_{\mathrm{head}_i} - \pmb{\theta}_{\mathrm{head}})$
15:  $\epsilon \gets \epsilon -\epsilon_0 / T_m$
16: end for
Output: The trained meta-model  $\theta$
(a)
Figure 2: Model architectures. (a) Base model (POMO). (b) Multi-task model.
We extend REINFORCE [42] to train the multi-task model  $\tilde{\pmb{\theta}}$ , including  $\tilde{\pmb{\theta}}_{\mathrm{body}}$  for all tasks and  $\tilde{\pmb{\theta}}_{\mathrm{head}_i}$  for task  $i$ . Specifically,  $\tilde{\pmb{\theta}}$  is optimized by gradient descent with the multi-task loss  $\nabla \mathcal{L} = \frac{1}{\tilde{N}} \sum_{i=1}^{\tilde{N}} \nabla \mathcal{L}_i$ , where  $\mathcal{L}_i(\tilde{\pmb{\theta}}|s)$  is the  $i$ -th task loss and its gradient is estimated as follows,
$$
\nabla \mathcal {L} _ {i} (\tilde {\boldsymbol {\theta}} | s) = \mathbf {E} _ {P _ {\tilde {\boldsymbol {\theta}} i} (\boldsymbol {\pi} | s)} \left[ \left(g \left(\boldsymbol {\pi} | s, \boldsymbol {\lambda} _ {i}\right) - b _ {i} (s)\right) \nabla \log P _ {\tilde {\boldsymbol {\theta}} i} (\boldsymbol {\pi} | s) \right], \tag {3}
$$
where  $b_{i}(s) = \frac{1}{n}\sum_{k = 1}^{n}g(\pmb{\pi}^{k}|s,\pmb{\lambda}_{i})$  is a baseline to reduce the gradient variance;  $\pmb{\pi}^1,\dots ,\pmb{\pi}^n$  are solutions produced by POMO with  $n$  different starting nodes. In practice,  $\nabla \mathcal{L}$  is approximated using Monte Carlo sampling, and computed by a batch with  $B$  instances as follows,
$$
\nabla \mathcal {L} (\tilde {\boldsymbol {\theta}}) \approx \frac {1}{\tilde {N} B n} \sum_ {i = 1} ^ {\tilde {N}} \sum_ {j = 1} ^ {B} \sum_ {k = 1} ^ {n} \left[ \left(g _ {i j} ^ {k} - b _ {i j}\right) \nabla \log P _ {\tilde {\boldsymbol {\theta}} i} \left(\boldsymbol {\pi} ^ {k} \mid s _ {j}\right) \right], \tag {4}
$$
where  $g_{ij}^{k} = g(\pmb{\pi}^{k}|s_{j},\pmb{\lambda}_{i})$  and  $b_{ij} = b_{i}(s_{j})$ .  $\tilde{\pmb{\theta}}$  is then optimized with  $T_{u}$  steps using Adam [43].
Figure 3: (a) Random sampling. (b) Symmetric sampling. (c) Scaled symmetric sampling.
4.2 Stabilized training
In each meta-iteration, the deviation of  $\tilde{N}$  (usually a small number) randomly sampled weight vectors may cause fluctuations to the parameter update of the meta-model, thereby leading to unstable training performance. For example, as shown in Figure 3(a), the biased samples of weight vectors, i.e.,  $\lambda_{1}$  and  $\lambda_{2}$ , make the meta-model tend to favor the second objective more in this iteration.
To address this issue, we first design a symmetric sampling method for the case of balanced objective domain, as illustrated in Figure 3(b). Once a weight vector  $\tilde{\lambda}_1$  is sampled, its  $M-1$  rotational symmetric weight vectors  $\tilde{\lambda}_2, \ldots, \tilde{\lambda}_M$  are calculated and used to reduce the bias as follows,
$$
\tilde {\lambda} _ {i, m} = \left\{ \begin{array}{l l} \tilde {\lambda} _ {i - 1, M}, & m = 1 \\ \tilde {\lambda} _ {i - 1, m - 1}, & \text {o t h e r w i s e} \end{array} . \right. \tag {5}
$$
For the case of imbalanced objective domains, we further design a scaled symmetric sampling method. In particular, the  $m$ -th weight  $\lambda_{m}$  is scaled as  $\lambda_{m} / f_{m}^{*}$ , where  $f_{m}^{*}$  is the ideal objective value but difficult to be calculated beforehand. Thus,  $f_{m}^{*}$  is replaced with  $f_{m}^{\prime}$ , which is dynamically estimated using the meta-model on a validation dataset with  $\lambda = (1 / M,\dots ,1 / M)$  during training.
Concretely, for a sampled  $\tilde{\lambda}_1$ , it is first multiplied by  $f'$  to scale objectives into the balanced domains. Then, its  $M-1$  rotational symmetric weight vectors are obtained by Eq. (5). Finally, they are divided by  $f'$  to scale back to the original domain to attain the genuine rotational symmetric weight vectors. An example for  $\tilde{\lambda}_1 = (0.2, 0.8)$  and its scaled symmetric  $\tilde{\lambda}_2 = (0.43, 0.57)$  are demonstrated in Figure 3(c). In summary, the scaled symmetric  $\tilde{\lambda}_2, \ldots, \tilde{\lambda}_M$  for a given  $\tilde{\lambda}_1$  are calculated as follows,
$$
\lambda_ {i, m} ^ {s} = \left\{ \begin{array}{l l} \tilde {\lambda} _ {i - 1, M} \times f _ {M} ^ {\prime} / f _ {m} ^ {\prime}, & m = 1 \\ \tilde {\lambda} _ {i - 1, m - 1} \times f _ {m - 1} ^ {\prime} / f _ {m} ^ {\prime}, & \text {o t h e r w i s e} \end{array} . \right. \tag {6}
$$
Then, the vectors are normalized as  $\tilde{\lambda}_{i,m} = \frac{\lambda_{i,m}^s}{\sum_{m=1}^{M}\lambda_{i,m}^s}$ . The pseudo code of the scaled symmetric sampling method for the weight vectors is presented in Appendix B.
4.3 Hierarchical fine-tuning
Once the meta-model is trained, it can be fine-tuned with a number of gradient updates to derive customized submodels for given weight vectors. However, it still might be expensive for MOCOPs since numerous submodels need to be individually fine-tuned from the meta-model. From a systematic perspective, the early coarse-tuning processes in which the parameters of neighboring submodels might be close, could be merged. In this sense, we propose a hierarchical method to efficiently fine-tune the numerous subproblems from the lowest to the highest level, as illustrated in Figure 1.