Pretrained Optimization Model for Zero-Shot Black Box Optimization
Authors: Xiaobin Li
Kai Wu
Yujian Betterest Li
Xiaoyu Zhang
Handing Wang
Jing Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Pretrained Optimization Models
Black-box Optimization
Zero-shot Optimization
Primary area: optimization
TL;DR: This work considers zero-shot black box optimization with a population-based pretrained optimization model (POM).

Abstract: Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot optimization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose a Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions to zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation on the BBOB benchmark and two robot control tasks demonstrates that POM outperforms state-of-the-art black-box optimization methods, especially for high-dimensional tasks. Fine-tuning POM with a small number of samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimensions, population sizes, and optimization horizons. For code implementation, see https://github.com/ninja-wm/POM/.

Pretrained Optimization Model for Zero-Shot Black Box Optimization
Xiaobin Li Xidian University
Kai Wu' Xidian University kwu@xidian edu. en
221712147840suu xxid//....stti.iiiia...a edu.
Yujian Betterrest
Xiaoyu Zhang Xidian University
Handing Wang
Xidian University
Xidian University
bebettereett@outlok..coo com
xiaoyuzhang@ricien Sxdiinn edu. cn
hdwang@xidian. edu. cn
Jing Liu Xidian University
neouma@mail wxiiiiiii...m edu. cn
Abstract
Zero-shot optimization involves optimizing target task that was not seen during training, aiming to provide the optimal solution without with minimal adjust- ments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot opti- mization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation the BBOB benchmark and two robot control tasks demon- strates that POM outperforms state-of-the-art black-box optimization methods, especially for high- digh-dimensional tasks. Fine-tuning POM with small number samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimen- sions, population sizes, and optimization horizons. For code implementation, see https://gthhb..cmmnnjjjwww/////
Introduction
Black box optimization, including tasks like hyperparameter optimization (HPO) I. neuroevolution 4 neural architecture search (NAS) [5], and algorithm selection 6 is very important. In these scenarios, the algorithm information about f, such as the Hessian and gradients, is unavailable.
evaluate fxx for any solution x; however, access to additional
Addressing diverse BBO problems necessitates the tailored design of specific algorithms to achieve satisfactory performance. Crafting these algorithms typically demands substantial expertise. There- fore, it is crucial to ensure reliable and robust performance of the optimizer in various applications, called zero-shot optimization. Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer.
Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The studies 77/9
employed Transformer or diffusion models to pretrain model-based optimizers using offline datasets. While effective, these methods primarily fit optimization trajectories of other BBO algorithms to a specific task, potentially requiring retraining for new tasks, limiting their ability to zero-shot optimization. Subsequently, [ [1] introduced two learned optimization frameworks for meta-learning evolution strategy (ES) and genetic algorithm (GA). However, the performance of these two methods on zero-shot optimization is weaker than that of CMA-ES [12] (see Section 4.2)).
To address zero-shot optimization, especially for continuous optimization, we introduce a population- based Pretrained Optimization Model, called POM. Leveraging multiple individuals, population- based optimizers gain a better understanding of the fitness landscape. The core of the optimizer how to design optimization strategies that sample better solutions. Inspired by the solution- producing mechanism of evolutionary computation, we design powerful POM blocks to form a general optimization strategy representation framework. Drawing inspiration from [3], we introduce an end-to-end gradient-based training method for POM, termed MetaGBT (Meta Gradient-Base Training), ensuring stable and rapid training for POM. Pretraining POM on a set of training functions with MetaGBT ensures good optimization strategy. Our contributions can be summarized follows:
Excellent ability to solve zero-shot BBO. We develop a efficient POM for zero-shot BBO, demonstrating a substantial performance advantage over state-of-the-art black-box optimizers.
Excellent ability to solve few-shot BBO. Few-shot optimization is the existence of a small budget of function evaluations for the target task to tune the optimizer for better performance. More than 30% performance improvement can be obtained with 25 random function evaluations.
2 Related Work
Heuristic Population-based BBO Algorithms. Numerous metaheuristic population- algo- rithms, such as genetic algorithms 14) evolution strategies [15[7]] particle swarm optimization [18 191. and differential evolution 20 21, have been devised to address optimization problems. Notably, CMA-ES 12] and L-SHADE [22 stand out as sfate-of-the-art methods for BBO. However, these approaches rely on manually designed components, exhibiting inefficiency and fragility when confronted with new tasks. In contrast, the proposed POM can autonomously acquire optimization strategies from problem instances, mitigating the aforementioned limitations.
Pretrained Population-based BBO Algorithms. Pre-training BBO algorithms be categorized into two types within the meta-learning framework. The first type frames meta-learning BBO algorithms as bi-level optimization problem [23]. For instance, (24] leverages meta-learning to infer population-based black-box optimizers that automatically adapt to specific task classes. LES [25] designs a self-attention-bautt based search strategy for discovering effective update rules for evolution strategies through meta-learning. Subsequent works like LGA [10] utilize this framework to discover the update rules of Gaussian genetic algorithms via Open-ES (26). The second type models the meta-lcarning of a BBO algorithm as reinforcement learning problem. [27] meta-learn a policy that adjusts the mutation step-size parameters of CMA-ES [12. Category one faces the curse of dimensionality, where an escalating number of model parameters leads to skyrocketing training difficulty, impeding the development of intricate strategies. In contrast, category two, which models meta-learning optimizers as reinforcement learning tasks, grapples with training instability. POM, employing a gradient- based end-to-end training approach, successfully bypasses the curse of dimensionality, ensuring stable training.
LLM for Optimization. In line with POMs, various optimization approaches leveraging Large Language Models (LLMs) have emerged to address diverse problem domains, including NP-hard algorithm evolution [30,33 reward design [34 and Neural Architecture Search Notably, LLMs play role in sampling new solutions. However, their optimization strategies depend on externally introduced natural selection mechanisms and are less effective in numerical optimization scenarios [37. LLaMoCo [8] and EoH (39 use LLM to generate code to solve optimization problems, but the performance LLaMoCo depends on carefully designed instructions and prompts, and EoH has expensive evaluation costs. TNPs (40], ExPT 41 and LICO
problems 28
29
(NAS)
35 36
42 use transformer structures to solve the BBO problem and have achieved good results. However,
TNPs requires contextual information of the target problem, and neither ExPT nor LICO can be

---

directly used to solve tasks with different dimensions from the training task. These methods lack the universal applicability as pretrained BBO models due to a deficiency in generating capabilities across tasks.
All the ahove methods cannot be the zero-shot optimizer. The first two categories need to adjust the hyper hyperrarameters when optimizing the new tasks, while the latter must fine-tune the instructions to achieve satisfactory results.
Pretrained Optimization Model
3.1 Problem Definition
black-box optimization problem can be transformed as minimization problem, and constraints may exist for corresponding solutions: min f(x), s.t. 2; [4, uz], where (21, Fd)
represents the solution of optimization problem f. the lower and upper bounds 1 - (4,, 22,22.... 1a)
and u = (u1, U2.... ..... ua), and is the dimension of x. For more background information on evolutionary algorithms, see Appendix
Definition Zero-shot Optimization. Zero-shot optimization refers to an optimizer that is applied directly to solve continuous black-box optimization problem I without any tuning. This means that the optimizer does not require any contextual information about f and can be directly used to handle problems of any dimensionality.
Definition Few-shot Optimization. Alternatively, it is permissible to fine-tune the optimizer using a small portion of the function evaluation budget for the objective task, and then use the fine-tuned optimizer solve f.
3.2 Classic Population Optimization Algorithm
In this section, we Differential Evolution (DE) example review classic evolutionary algorithms. DE (20) 4 is a prominent family within evolutionary algorithms (EAs), known for its
advantageous properties such as rapid convergence and robust performance [44 45). The optimization strategy of DE primarily involves mutation and crossover operations.
The classic DE/rand/1 crossover operator is illustrated in Eq. ①I (additional examples listed in Appendix (A.2). Each mutation strategy can be viewed as a specific instance of Eq. (2); Further represent the mutation strategy form, as shown in Eq. 3 The matrix S evolves with the generation index 1, indicating that the mutation strategy adapts across different generations. Consequently, we propose module to enhance the performance of the mutation operation, which leverages the information from the population of the ith generation generate St, This serves the motivation for our design of the LMM.
details are provided in Appendix A.2, Additionally,
matrix
x F (x
(I)
In the crossover phase at step f, DE uses a fixed crossover probability cr¹ [0. 1] for each individual xf in the population, as shown in Eq. (9). The crossover strategy for the entire population can then be expressed as vector cr' = (cri, CT ... xr Our goal is to design module that adaptively generates cr' using the information from the population. This approach allows for the automatic design of the crossover strategy by controlling the parameter er. This serves as the motivation for our design of LCM.
3.3 Design of POM
population consists of n individuals, denoted X [x1. (x..X2..... x..... ,xx]. treated as X X2, "x,", to support matrix operations. We feed POM an initial random population X at step 0, specify the evolution generation for it, and hope that it can generate population X close to the global optimum at step T, as shown in X = POM(X" T(0). where E Ω is the parameters of POM, where Q stands for the strategy space. The goal of training POM to find an optimal 0 in Ω. As shown in Fig, POM consists of LMM, LCM and SM.
In this paper, X is also
3

---

shown as LMM LMM generates candidate solutions v for individual xj through Eq. 2 which enables the
population information be fully utilized in the process of generating candidate solutions v
x (Vwi. E IR, Wi,s # 0)
Further, we organize Eq.
2 into matrix form, as shown in Eq. 33)
S'= x X
X RNxd is the population in generation and S E RNNN evolves with each change I, signifying a mutation strategy that adapts across generations. Consequently, it is imperative to devise a module that leverages information from the population at generation t to generate St. Any mutation operator of differential evolution, such as the classic DE/rand/1 mutation operator, can be converted into Equation (3) in the specific case of S (sce Appendix [A.2 for details). At the same time, the
erossover operation of GAs can also be generalized into the form of Equation (3) 46
The function of LMM designed based on Multi-head self-attention (MSA) follows:
S L.MMLMMHHH
where 81 W ml. W m2, W m3. 1 4m bm3) denotes the trainable parameters within LMM, while H' = bi. hg.... h serves as LMM's input, encapsulating population information. Each h incorporates details about x,, encompassing: 1) f/: the normalized fitness f(x{) of x 2) ff: the centralized ranking of xt. The method for calculating ft is:
f(x) f(x)) 4²
(5)
where 2² and of denote the mean and standard deviation, respectively, of individual fitness values within the population at time We build A follows:
(xnn X²) 0.5) x 2
N
where rank yields the ranking of x within the population X with values ranging from 1 to N. Thus, LMM utilizes information on the relative fitness of individuals to dynamically generate the strategy Sr. F serves as position encoding, explicitly offering the ranking information of individuals. Equation 7 details the computation of St.
H' Tanh(H' W ml bm)),
Q' Tanh Wmz + bm2)
Kt_ = Tanh A x Wma ma + bma), S' = Tanh( Q' x (Kt)T)
((dm))
where Tanh is activation function. W ml E pxd and W m2, W m3 IR bm bm and bms are vector with dimension d,m. H' E Nxdm, Q',
K' R Nxd. and S E NXN.
The topological structure of the population significantly influences their information exchange [48] When all individuals engage in information exchange, the algorithm's 's convergence may suffer, diversity could diminish, and susceptibility to local optima increases. To address this, we introduce mask operation during both training and testing phases, where the probability of setting each element in St to 0 is T moakk. This operation enhances POM's ability to learn efficient and robust strategies, as validated in our experiments. Consequently, St is derived using Eq. (8)
S mask(S' ( (mmas))
(8)
Finally, we get V via Eq.
LCM For each individual x² at step i, crossover probability or E [0, 1] is established. Conse- quently, the population's 's crossover strategy is encapsulated the vector er" (orf orl. ,or,) ).
The crossover operation, depicted in Eq. (9), can be elucidated as follows:
if rand(0, 1) S ort Vie [1
u k
(9)
otherwise

---

The module design should facilitate the adaptive generation of cr' by leveraging population informa- tion. Executing the crossover operation with cr' yields U [u], u2. u
LCM is designed hased on FFN 47], as shown in Eq.
cr' LCM (Z' (02)
(10)
where 02 (We1, b. W c2,, b2,,T is the parameter of LCM and Z' E RRNx is the population information used by LCM. Here, Z' |z zz. . z represents the relevant information of individual x² and X'. For example, it can include the ranking information of x the fitness information of x the Euclidean distance between x and V;, and the distribution information of individuals within the population (such as the fitness distribution, the distance between pairs of individuals), etc. In this paper, z includes the following information
case study: 1) fü: the normalized fitness (xx) of x 2) ff: the centralized ranking of x,; 3) simf: the cosine similarity between o and of.
h' Tanh (Z' Wet be1), h' = lay)rnorm(h(()) cr = -Sigmoddhh (h's x Wez bb2) (11)
where the activation function Sigmoid maps inputs the range (0, 1). Wel RR.d. Wa Rd.x1 r is the learnable parameters of layernorm 49 bei and bę2 are vectors with dimensions de and respectively.
Although we derive cr' from Eq. 11 as in Eq. (9), the discrete nature of the crossover operator renders it non-differentiable, impeding gradient-based training of the LCM module. To address this limitation, we introduce the hembel_sfftmax method (50), providing an efficient gradient estimator that replaces dinfeffeerettible samples from categorical distribution with differentiable samples from a novel Gumbel- Softmax distribution.
Eq. (2) shows how to perform crossover operations between x and vł in LCM (Vi E [1,
tile(er), d))), u cv'.0 x + cv'.1 vi, U' [', u u'y]
ri rand(d), cv; commbe_sofftmazdde
(12)
First, the rand function samples uniformly from the range [0, 1] to obtain vector r!. Then get or from cr according to the index. The tile function expands or into a d-dimensional vector: [or], orf,,- ccr]]. The cat function concatenates them into matrix shown below:
T crf
:::
r cr
71
(13)
Here, gumbel_sofinng is executed column- For any column, the larger element becomes I after numbel_softmac and 0 otherwise. Therefore, cvę E 2xdd may be matrix like this:
0
0
0
cv
(14)
0
target task f
samples
porometers
training set
U
X
POM
Loos Fundionn
9 Gemmndee 1)
LINE
LCM
CM
gene
x
populnt
loop if tsT
convergence
(a) POM
(b) training
(c) testing
Figure 1: In the figure, X is the initial random population. (a) The overall architecture of the POM. (b) POM training process. Here T is the size the inner loop iteration step during training, and the training function should be differentiable. (c) POM testing process. Here, T is the number of iterations the testing process and f is the target task. f does not have to be differentiable. Here we directly apply the trained POM solve ff without requiring gradient information.
Overall Framework We design LMM and LCM to achieve the generation of sample strategy (that is, generate St) and crossover strategy (that is, generate cr'), respectively. The overall architecture of POM is shown in Fig. [] The parameters that need to be trained in POM are 0 = (0,,02), At time step the population X Initially, we amalgamate the information from X to construct descriptive representations of the population, H' and Z'. LMM adaptively generates St based on H
5

---

The multiplication of X' and S' yields V (see Eq. 33. Next, LCM adaptively generates cr" based on its imput Z', and performs a crossover operation based on cr' to obtain Us. Finally, SM [ST) 1-to-/ selection strategy is executed between U' and X to produce the next-generation population X
X SM U*) = (ilelle>o(M -M)) X + lile(1 12 Mp)) о U (15)
where (x))(x) = > and (2)0()) - if < O. and the rile copy function extends the indication matrix tensor with size (N,d), M, (M denotes the fitness matrix of ('")),, and C indicates the pairwise multiplication between inputs.
Algorithm 1 MetaGBT
Algorithm 2 Driving POM to Solve Problem
Input: T. n. training set TS.
Input:
Generations T, population size BBO
Output: The optimal
problem f.
1: Randomly sample the parameter 0 of POM.
Output: The optimal X found. 1: POM loads the trained parameter 0.
2: while not done do
Sample [TS] tain [ X
populations of size n to ob-
Randomly sample an initial population X
X rs
of size n.
4: for 1, 2,.. [TS] do
do 4: Construct H based on X' and f.
for = 0,,,,.,,
..... 5: Randomly sample w' for the f; in TS.
6: end for
S' (H)
7:
for t.
1,2...., .......,aul do for i 1,2,,.... TTS do
6:
V < S' X'.
Build Z' based X', V and f.
8:
8: cr +- LCM((Z
9:
X < POM 1(0). loss < 14 (X) X Si, A).
Construct U using Equation 12
10:
10: X <- SM(X' U²).
11:
end for
11: end for
0 - Update / based on 17'T Σ loss
13:
end for end while
14:
3.4 Tasks, Loss Function & MetaGBT
POM is meticulously crafted model amenable end-to-end training based on gradients. While POM necessitates gradient information from the training task during the training phase, it exhibits the ability to tackle BBO problems the testing phase without relying on any gradient information. To ensure the acquisition of an efficient, highly robust, and broadly generalizable optimization strategy, POM undergoes training on a diverse set of tasks. Training on these tasks sequentially poses the risk of domain overfitting, local optima entrapment, and diminished generalization performance. Consequently, we introduce a training methodology named MelaGBT.
Tasks. We form a training task set TS (XWw)) J..kkw'))wherr where i E [1,5] and E N]. comprising 4N tasks derived from Table 3 in appendix, where w, denotes the task parameter influencing the function's 's landscape offset. Our selection of these functions for the training task is motivated by their diverse landscape features. The specific landscape features encompassed in TS are detailed in Appendix B
Loss Function. To avoid bias of different output scales in TS, for any function fa in TS, we design the normalized loss function 1,(X' X wt A). In Equation (16). 23 calculates the average fitness difference between the input and output of the POM, further normalized within [0, 1]. This
encourages convergence of the algorithm. 12 uses standard deviation to simulate the distribution of
the output population, encouraging diversity in the output population. std(X', :) is the standard deviation of the jth dimension of the population. A is hyperparameter, and we find that setting it to 0.005 can make model training more stable.
X (X 1|0)
R Σ [x(x(w") XEXI
1 Σ [x(((w)) XEXI-1
stdl x),))
(16)
12
1 1-=0 =1 N22
Σ f((x(wo)
X XEX!

---

MetaGBT. The pseudocode for MetaGBT is presented in Algorithm| I Initially, we sample the POM parameter 0 from a standard normal distribution. The objective of MetaGBT is to iteratively update to bring it closer to the global optimum 0*. In line 2, we sample population for each task in 7S. Lines 3, and involve the resampling of task parameters for tasks 7S, thereby altering the task landscape, augmenting training complexity, and enhancing the learning of robust optimization strategies by POM. The final loss function (line 10) is determined by computing the average of the loss functions for all tasks. Subsequently, in line 12, we update 0 using gradient-based optimizer, such as Adam [52]. The trained POM is then ready for application solving an unknown BBO
problem, as depieted in Algorithm
POM LM00 LSHADE
1289 DE ES LES
PUM
CMA-ES LGA
Figure The critical difference diagram illus-
CHALSS
trates the performance ranking of seven algorithms across 24 BBOB problems with dimensions d 30, 100, employing Wilcoxon-Holne analysis [53] at a significance level of p = 0.05. Algorithm po- sitions are indicative of their mean scores across multiple datasets, with higher scores signifying method consistently outperforming competitors. Thick horizontal lines denote scenarios where there statistically significant difference in algorithm performance.
LOA
(a) Bipedal Walker
(b) Enduro
Figure 3: Experimental results are presented for the Bipedal Walker (a) and Enduro (b), with the vertical axis denoted as R. representing the strategy score. The score corresponds to the total reward acquired by the agent during interactions with the environment.
4
Experiments
4.1 Experimental Setup
We test the performance of POM the widely used BBO benchmark and two complex real-world problems (see Appendix C.. Selected methods include DE (DE/rand/1/bin) [54] and ES ((,,A)-ES) as nopulation-based baselines, L-SHADE [22) and CMA-ES 112 as state-of-the-arr population-based
BBO methods, and LES [25] and LGA 110 as state-of-the-ar POMs. POM is trained on TS with 100, n
100, and 10. Detailed parameters for all compared methods are provided in
Appendix B
Please refer to Appendix D for the reasons for choosing these algorithms.
4.2 Results
BBOB [55]. We evaluate the generalization ability of POM across 24 BBOB functions with dimen- sions d = 30 and = 100, where optimal solutions are located at 0. Figure presents the critical difference diagram comparing all algorithms (refer Appendix Tables 4 and 0 and Figures II 12 and 13 for detailed results). POM significantly outperforms all methods, showcasing its eficacy across varying dimensions. Despite being trained solely on TF1-TF4 with d = 10, POM excels in higher dimensions (d = {30, 100, 500)), with its performance advantage becoming more pronounced with increasing dimensionality. Particularly on complex problems F21-F24, where global structure is weak, POM lags behind LSHADE but surpasses other methods, attributed to its adaptability through fine-tuning. TurBO
(56) is the Bayesian optimization algorithm with the best performance BBOB 57] Under little budget conditions, the performance of POM outperforms that of TurBO in most cases (see Appendix G for details).
Bipedal Walker (58). The Bipedal Walker task involves optimizing a fully connected neural network with d 874 parameters over & 800 time steps to enhance robot locomotion control. In Fig. ()) LSHADE shows ineffectiveness,, while CMA-ES, LSHADE, and LGA suffer from premature convergence. Conversely, POM achieves stable and swift convergence, ultimately attaining the highest score.
Enduro 58 Enduro task entails controlling a strategy with d = 4149 parameters across k = 500 steps, posing greater difficulty than Bipedal Walker. As depicted in Fig. 3bb), LGA and LES exhibit premature convergence and limited exploration. While CMA-ES initially converges slightly
7

---

faster than POM, the latter maintains a superior balance between exploration and exploitation, outperforming LSHADE.
RCM UNTRAKED
NO HASK
XL
Loar
VS
(a)
(b)
Figure 4: (a) Results of ablation study. The metric used to evaluate performance is the optimal value of the function found, with smaller values being better. Here, 30. (b) Results of POMs with different sizes on BBOB tests (d = 100).
6
3
5
4
(b)
Figure 5: The impact of training dataset size on the performance POM. d 100. means that the training set only contains 7F1, and 2 means that the training set only contains TF1 and FF2, and so on.
0.0
10
15
Sample Stze
(a) Time Cost
(b) Testing Cost
Figure 6: Experimental results of fine- tne-ttning tests.
Figure 7: (a) Time cost of POM. (b) Testing cost
RFT per oormance improvement perfornance of base POM
baselines and POM.
4.3 Analysis
Ablation Study The ablation study results for the designed modules are presented in Fig. (refer to Appendix Table(7for additional details). Configurations include UNTRAINED, representing an untrained POM with randomly initialized parameters; NO LMM, where the LMM is excluded, and simple DE/rand/1/bit mutation operator is employed; NO LCM, indicating the absence of the learnable crossover operation, using only binomial crossover; and NO MASK, signifying the omission of the mask operation described in Eq.
While UNTRAINED yields optimal results for F9 and F16, untrained POM is inherently optimization strategy, the adaptability of trained POM surpasses the baselines in most scenarios. In simpler tasks with d 30, UNTRAINED underperforms, demonstrating the advantage of trained POM on more complex tasks. Notably, NO L.M and NO I.CM excel on F5, F1 1, and F19, respectively. This could be attributed potential overfitting of POM the relatively simple training set. The exclusion of mask operation (NO MASK) significantly diminishes POM's performance, highlighting the importance the mask for global information sharing and population interaction, erucial for maintaining diversity. All modules contribute to POM's overall performance, with the negative impact on POM's performance ranked as follows: NO MASK > NO LMM UNTRAINED NO LCM.
Fine-tuning Test Appendix Table 3 We replace LCM with standard transformer encoder to obtain more stable experimental results. For x; and V,, we normalize their features and then concatenate them by dimension to obtain xv; dx2 XV; and the normalized [fitness, ranking] information of x; in the parent population are concatenated to obtain xvf, xd+11)22 Based on this input, the transformer encoder will generate cv, in Eq. [22). Different numbers are generated as fine-tuning samples for T FF-TFS and Algorithm used to fine-tune POM for each function. The base POM is
We evaluate the fine-tuned POM's performance on TFG-TFS as detailed in

---

A15
E.00
A12
(a) F8 step
(b) F8 step 50
(c) F8 step 100
Figure 8: Displayed are visualized outcomes of LMM S in BBOB with d 100 using 10 for clarity. Blank squares in the matrix denote masked portions from Eq. (8) Steps 1, 50, and 100 correspond the 1st, 50th, and 100th generations in population evolution. The horizontal and vertical axes denote individual rankings, with 1 as the best and 10 as the worst in the population. Each row illustrates the weight assigned to other individuals when executing mutation operations for the respective individual.
Io- 0906e-1
C00E Pulc
06
Qusseer
4.3
0.2
F31
F24
3.5
20
80
1.00
-0
50
80
100
40
300
Cietenation
Cinmenation
(a) rank 5
(b) rank 51
(c) rank 100
Figure 9: Visual analysis results of LCM on BBOB FI, FII, and F24 with d 100, employing n = 100, are presented. "Rank" signifies an individual's position, with rank 5 representing the fifth-ranked individual in the population. Subgraphs depict the evolution of the probability that an individual will undergo crossover across three tasks as the population progresses. For example, (a) illustrates the crossover probability change for the top-ranked individual on F1, F11, and F24 with the number of generations.
initially trained on TF1-TF5. We calculate the relative performance improvement (RFI) achieved by the fine-tuned POM compared to the base POM, with results displayed Figure Experimental results indicate that fine-tuning POM leads to significant performance improvements even with small sample size. The method for obtaining fine-tuning samples is not restricted; for black-box tasks, surrogate model can be constructed to facilitate fine-tuning.
Size of Training Dataset Any complex problem can be simulated by polynomial composed of simple basic function terms. To ensure that the optimization strategy learned by POM has robust generalization ability and performance, we should train POM on a set of basic functions.
First, we tested the impact of increasing the number of basic functions in the training set on model performance. Next, we examined the effect of introducing complex functions into training set. Functions TF1 - 7F5 are basic simple terms. For example, F1 is an absolute value term, and TF5 is square summation term. Functions FF TF8 are composite terms composed of several basic functions. For instance, FFF includes both cumulative multiplier term and cosine term. The test results are shown in Figure (a) and (b) (see Appendix Table g for details), respectively.
Experimental results indicate that increasing the number of basic functions leads to an overall im- provement in POM performance, whereas the introduction of composite terms results in significant performance decline. This aligns with our hypothesis.
Scale of POM We explore the performance of POM difffernt scales, which is shown in Fig. (b) (refer to Appendix Table 9 for additional details). We increase POM's parameter count by perturbing the hidden layers of each module (dm- d.). Six models are constructed in ascending order of parameter count, labeled as VS (very small), S (small), M (medium), L (large), VL (very large), and XL (extra large) (details the Appendix Table XL. achieves the best performance, while VS and M also perform well. S exhibits the worst performance, and VL performs worse than L. Two core factors contribute to this phenomenon: the number of parameters training. We observe complex relationship between the number of parameters and training difficulty. VS, with the fewest

---

parameters, is the easiest to train and performs well on BBOB. Conversely, XL, with a large number of parameters, exhibits the strongest capability to represent strategies, resulting the best performance. The performance of XI. aligns with our expectations. We obtain the following principles: 1) Larger models can have stronger capabilities but are more challenging to train; 2) Training difficulty and model scale do not exhibit simple linear relationship, warranting further research; 3) Larger models require more functions for effective training.
Time Budget We assess the training and test time efficiency of POM across various architectures on BBOB (d = 10) and BBOB (d = 100) respectively, as illustrated in Figure 7 POM demon- strates remarkable efficiency in tackling BBO problems, with negligible training costs relative to its exceptional generalization ability and high performance.
4.4 Visualization Analysis
LMM Learning Analysis F S for an in-depth analysis of the LMM strategy (refer to Appendix Figure 20) for additional details). Key observations and conclusions include: I) Generally, superior individuals receive higher weights during LMM, showcasing POM's ability to balance exploration and exploitation as the population converges. 2) Across diverse function problems, POM dynamically generates optimization strategies, highlighting its adaptability and contributing to robust generalization. 3) Disadvantaged individuals exhibit more uniform weight distribution, potentially aiding in their escape from local optima and enhancing algorithm convergence.
LCM Learning Analysis We visually examine the L.C strategy, presenting the results in Fig.
9 (refer to Appendix Figure 21 26 26 for additional details). LCM displays the capacity to adaptively generate diverse strategies for individuals across different ranks in the population, revealing distinct patterns among tasks and rankings. Notably, top-ranking individuals within the top 20, such as those ranked 1st, 5th, and 18th, exhibit a flexible crossover strategy. The dynamic adjustment of crossover probability with population evolution aids in preserving dominant genes and facilitating escape from local optima. Conversely, lower-ranking individuals show increasing overall probability of crossover, promoting exploration of disadvantaged individuals and enhancing the algorithm's 's explo- ration capability. LCM proficiently generates adaptive crossover strategies across tasks, individuals, and convergence stages, significantly boosting both convergence and exploration capabilities.
5 Conclusions
We present POM, a novel Pretrained Optimization Model designed to address the inefficiencies of existing methods in zero-shot optimization. Evaluation on BBOB and robot control tasks demonstrates POM's superiority over other black-box optimizers, particularly in high-dimensional scenarios. Additionally, POM excels in solving few-shot optimization problems. Future research avenues include designing enhanced loss functions to optimize POM for both population convergence and diversity, thereby improving overall algorithm performance. In addition, the limitations of model scale and time performance deserve further study (see Appendix for details).
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62206205 and 62471371, in part by the Young Talent Fund of Association for Science and Technology in Shaanxi, China under Grant 20230129, in part by the Guangdong High-level Innovation Research Institution Project under Grant 202 202180909050008, and in part by the Guangzhou Key Research and Development Program under Grant 202206030003.
References
[1] Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. Automated machine learning: methods, systems, challenges. Springer Nature, 2019.
10

---

[2] Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O Stanley, and Jeff Clune. Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprini arXiv: arXiv: 171.00667, 2017.
[3] Jiacheng Chen, Zeyuan Ma. Hongshu Guo, Yining Ma, Jie Zhang, and Yue-Jiao Gong. SYMBOL: Generating flexible black-box optimizers through symbolic equation learning. In The Twelfth International Conference on Learning Representations, 2024. URL https
7/ //pperreeeee..eefffor/otddeeed
[4] Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Zhenrui Li, Guojun Peng, Yue-Jiao Gong, Yining Ma, and Zhiguang Cao. Metabox: A benchmark platform for meta-black-box optimization with reinforcement learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 10775-10795. Curran Associates, Inc., 2023.
[5] Qing Ye, Yanan Sun, Jixin Zhang, and Jiancheng Lv. A distributed framework for ea-based nas. IEEE Transactions on Parallel and Distributed Systems, 32(7):1753-1764 2021. doi: TA1109/TPDS.2000000
[6] Hongshu Guo, Yining Ma. Zeyuan Ma. Jiacheng Chen, Xinglin Zhang, Zhiguang Cao. Jun Zhang, and Yue-Jiao Gong. Deep reinforcement learning for dynamic algorithm selection: A mroof-of-principhe study on differential evolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems, pages 1-13, 2024. doi: 10.1 10.109/TSMC.22243334888
[7] Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Richard Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc aurelio Ranzato, et al. Towards learning universal hyperparameter optimizers with transformers. Advances in Neural Information Processing Systems, 35:32053-32068, 2022.
[8] Siddarth Krishnamoorthyy Satvik Mehul Mashkaria, and Aditya Grover. Generative pretraining for black-box optimization. arXiv preprint arXiv:2206. 10786, 2022.
[9] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. arXiv preprint vrXiv:2306.07/880 2023.
[10] Robert Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, and Sebastian Flennerhag. Discovering attention-based genetic algorithms via meta-black-box optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 929 937, 2023.
[11] Robert Tjarko Lange Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flennerhag. Discovering evolution strategies via meta-blat sata-bbackk-bor optimization. The Eleventh International Conference on Learning Representations, 2023. URL https:: //operevie./ www.teetouuueeeee formmmrmmmi mDDUUUFPPE
[12] Nikolaus Hansen. The ema evolution strategy: A tutorial. arXiv preprint arXiv: 1604.0077 2016.
[13] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International conference on machine learning, pages 1126-1135. PMLR, 2017.
[14] John Holland. Genetic algorithms. Scientific american, 267(():,6-73, 1992.
[15] Nikolaus Hansen and Andreas Ostermeier. Completely derandomized aelf-adaptation in evolu- tion strategies. Evolutionary Computation, 9(2):159-195, 2001.
[16) Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (ema-es). Evolutionary computation, 1 111):1-18, 2003.
[17] Raymond Ros and Nikolaus Hansen. A simple modification in cma-es achieving linear time and space complexity. International Conference on Parallel Problem Solving from Nature, pages 296-305. Springer, 2008

---

[18] James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of ICNN'95- International Conference on Neural Networks, volume 4, pages 1942-1948. IEEE, 1995.
[19] Yue-Jiao Gong, Jing-Jing Li, Yicong Zhou, Yun Li, Henry Shu-Hung Chung, Yu-Hui Shi, and Jun Zhang. Genetic learning particle swarm optimization. IEEE Transactions on Cybernetics, 46(10):2777-2290, 2015.
[20] Rainer Storn and Kenneth Price. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11:341-355 1997.
[21] Vladimir Stanovov, Shakhnaz Akhmedova, and Eugene Semenkin. NI-shade-Ihe algorithm with linear parameter adaptation bias change for cec 2022 numerical optimization. In 2022 IEEE Congress Evolutionary Computation (CEC), pages 01-08 IEEE, 2022.
[22] Ryoji Tanabe and Alex S. Fukunaga. Improving the search performance of shade using linear population size reduction. In 2014 IEEE Congress on Evolutionary Computation (CEC), pages 1658-1665, 2014. doi: 10.1109//E 69900000...
[23] Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 44(12):00445-1006 2022. doi: 10. 10.1009TPAMI.2011. 111.22224733377 1326744.
[24] Hugo Siqueira Gomes, Benjamin Léger, and Christian Gagné. Meta learning black-box population-based optimizers. arXiv preprint urXiv:2103.0333 2021.
[25] Robert Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flennerhag. Discovering evolution strategies via meta-black-bor optimiza- tion. In Proceedings of the Companion Conference on Cenetic and Evolutionary Computation, pages 29-30,2200 2023.
[26] Xingwen Zhang, Jeff Clune, and Kenneth O Stanley. On the relationship between the openai evolution strategy and stochastic gradient descent. arXiv preprint (exXiv:/112.06500 2017.
[27] Gresa Shala, André Biedenkapp. Noor Awad, Steven Adriaensen, Marius Lindauer, and Frank Hutter. Learning step-size adaptation in ema-es. In International Conference on Parallel Problem Solving from Nature, pages 691-706. Springer, 2020.
[28] Bernardino Romera-Paredess Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, pages 1-3, 2023.
[29] Elliot Meyerson, Mark Nelson, Herbie Bradley, Arash Moradi, Amy Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. arXiv preprint MxXi::2302.12170, 2023.
[30] Fei Liu, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang. Algorithm evolution using large language model. arXiv preprint arXiv:2311.15224 2023.
[31] Chengrun Yang, Xuezhi Wang. Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. arXiv preprint arXiv:2309.0344 2023.
[32] Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley. Evolution through large models. In Handbook of Evolutionary Machine Learning. pages 331-366. Springer, 2023.
[33] Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language mode, 2024.
[34] Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Fureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931 2023.
12

---

[35] Angelica Chen, David M Dohan, and David R So. Evoprompting: Language models for code-level neural architecture search. arXiv preprint arXiv:2302.14836 2023.
[36] Muhammad U Nasir, Sam Earle, Julian Togelius, Steven James, and Christopher Cleghom. Llmatie: Neural architecture search via large language models and quality-diversity optimization. arXiv preprini arXiv:2306.01 102, 2023.
[37] Beichen Huang. Xingyu Wu, Yu Zhou, Jibin Wu, Liang Feng, Ran Cheng, and Kay Chen Tan. Exploring the true potential: Evaluating the black-bos optimization capability of large language models. arXiv preprint ar wrXiv2240.006290 2024.
[38] Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Guojun Peng. Zhiguang Cao, Yining Ma, and Yue-Jiao Gong. Llamoco: Instruction tuning of large language models for optimization code generation, 2024. URL htttt:://aaxiiii org/ abs 2403 220030111111..
[39] Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model, 2024. URL http::t/ps//a..... http///axivvv// org/ abs/ 2/////241.000000
[40] Tung Nguyen and Aditya Grover. Transformer neural processes: Uncertainty-wware meta learning via sequence modeling, 2023.
041799
[41] Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. Expt: experimental design. Advances in Neural Information Processing Systems, 36, 2024.
synthetic pretraining for few-shot
[42] Tung Nguyen and Aditya Grover. Lico: Large language models for in-context molecular optimization. arXiv preprint arXiv:2406. I 18851, 2024.
[43] Radha Thangaraj, Millie Pant, and Ajith Abraham. A simple adaptive differential evolution algorithm. In 2009 world congress on nature & biologically inspired computing (nabic), pages 457-462. IEEE, 2009.
[44] Swagatam Das, Sankha Subhra Mullick, and Ponnuthurai N Suganthan. Recent advances in differential evolution-an updated survey. Swarm and evolutionary computation, 27:1-30, 2016.
[45] Ferrante Neri and Ville Tirronen. Recent advances in differential evolution: a survey and experimental analysis. Artificial intelligence review, 33:61-106, 2010.
[46] Jiangning Zhang, Chao Xu, Jian Li, Wenzhou Chen, Yabiao Wang. Ying Tai, Shuo Chen, Chengjie Wang, Feiyue Huang, and Yong Liu. Analogous evolutionary algorithm: Designing a unified sequence model. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 26674-26000 26688. Curran Associates, Inc., 2021.
[47] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[48] Yang Yu, Zhenyu Lei, Yirui Wang, Tengfei Zhang, Chen Peng. and Shangce Gao. Improving dendritic neuron model with dynamic scale-free network-based differential evolution. JEEE/CAA Journal of automatica sinica, 911:999-110, 2021.
[49] Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in Neural Information Processing Systems, 32, 2019.
[50] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softman arXiv preprint arXiv: 1611.01 144, 2016.
[51] Kai Wu, Penghui Liu, and Jing Liu. Decn: Automated evolutionary algorithms via evolution inspired deep convolution network, 2023.
[52] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69800 2014.
13

---

[53] Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pienee- Alain Muller. Deep learning for time series classification: review. Data Mining and Knowledge Discovery, 33(4):117-963, 2019.
[54] Swagatam Das and Ponnuthurai Nagaratnam Suganthan. Differential evolution: A survey of the state-of-the-arr, IEEE transactions on evolutionary computation, 15(1):4-31) 31, 2010.
[55] Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff. Coco: A platform for comparing continuous optimizers in black-box setting. Optimization Methods and Software, 36(1): 2021.
[56] David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. In H. Wallach, H. Larochelle, A. Beygelzimer, d' d'Alché-Ban Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[57] Maria Laura Santoni, Elena Raponi, Renato De Leone, and Carola Doerr. Comparison of high-dimensional bayesian optimization algorithms on bbob. arXiv preprint arXiv:2303.00890 2023.
[58] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym, 2016.
[59] Steffen Finck, Nikolaus Hansen, Raymond Ros, and Anne Auger. heal-parameter black-box optimization benchmarking 2009: Presentation of the noiseless functions. Technical report, Citeseer, 2010.
[60] Jazzbin. Geatpy: The genetic and evolutionary algorithm toolbox with high performance in python, 2020.
14

---

A Preliminaries
A.1 Genetic Algorithms
The crossover, mutation, and selection operators form the basic framework of GAs. GA starts with a randomly generated initial population. Then, genetic operations such crossover and mutation will be carried out. After the fitness evaluation of all individuals in the population, a selection operation is performed identify fitter individuals undergo reproduction generate offspring. Such an evolutionary process will be repeated until specific predefined stopping criteria are satisfied.
Crossover
The crossover operator generates new individual xf E IR by Eq. (7 and er is the probability of the crossover operator.
rand(0, < cr otherwise
(17)
where k E [1,... d]. and E 21, (i + j). d represents the dimension of the problem. zig and orf represent the k-th element of x" and x respectively. This operator is commonly conducted on n individuals; n represents the population size. After an expression expansion, we reformulate Eq. 11 as Ei= x,W? [6]. X, IR represents the ith individual in X, where X {x1, X2, -.. Xn a population. Wi Rdxd is the diagonal matrix. If W is full of zeros, the ith individual has no contribution.
The mutation operator brings about random changes in the population. Specifically, an individual x; in the population goes through the mutation operator to form the new individual x.", formulated as follows:
Mutation
rand(le, ux) zx
rand(0, 1) mr otherwise
(18)
where mr is the probability of mutation operator and & E [1,... d). ot and of represent the k-th element of x and x" respectively. Similarly, Equation (18) can be reformulated as x(W,, where W!Th Rxd is the diagonal matrix.
Selection We introduce the binary tournament mating selection operator in Eq. (19 The selection operator survives individuals of higher quality for the next generation until the number of individuals chosen. As shown in Eq. (9
f(x:) < f(xx)
Ps 0
f(x:) f(xx) (x,, xx) E XUX"
(19)
where ps reflects the probability that x; is selected for the next generation, and XT {xT x x
A.2 Mutation Strategy in DE
The core components of the optimization model include modules that generate solutions and modules that select solutions. GA and DE basically include crossover modules, mutation modules and selection modules. The evolutionary strategy represented by CMA-ES needs sample population from a certain distribution (such as Gaussian distribution), and further select individuals to update this distribution. In this paper, we design parameterized trainable LMM and LCM as modules for generating solutions. The function of LMM is to generate a candidate population, and LCM further performs crossover between the candidate population and the original population to obtain the offspring population.
We list some classic DE mutation strategies.
DE/rand/1
v = x + F. (xf2 -x/3)
(20)
DDE/rand/2
v = x + F (x/² - x + x x(ss)
(21)
15

---

DE/best/1
v = xteet + F. (xt1 -x/2)
(22)
DE//urreentto-randd//
vs F)x + F. (x/1 x x333
(23)
DE/curcent-to-best//
v = (1= F. Xbext + F. (x/1 x422
(24)
DE/current-to-pben
=1- +
x xbestt + F- -xxx x
(25)
The integer index rl (and similarly, r2 and r3) is randomly selected from the range [0. N]. pbeat
is randomly selected from the indices of the best p individuals. whest is the individual with the best fitness in the population at generation
The generalized form of the mutation strategy is
W (Vwa.) E R)
(26)
For example, when Wi.g = 1, Wi.k = Wi, F 0, and wi,1 = 0 (Al {g. # j). becomes DE/rand If individuals of the population has been sorted from good bad by fitness, when win = 1, W Wij & O, and wi = 0 (vl & {0.j. kJ, k + j), it becomes DE/best/1.
B Landscape Features of TF1-TF8
The landscape features included in TS are shown as follows:
TF1:
Unimodal
TF2:
Separable
TF3:
Unimodal, Separable
TF4:
Unimodal, Separable
TF5: Multimodal, Non-separable, Having a very narrow valley from local optimum global optimum, Ill-conditioned
TF6: Multi-modal, -on-sepaa,,l,, Rotated
TF7: Multimodal, Separable, Asymmetrical, Local optima's 's number is huge
TF8: Multi-modal, Non-separ,ble, Asymmetrical
C
Test Set
C.1
BBOB
BBOB [59 (55 is a widely researched and recognized collection of benchmark test problems to evaluate the performance of optimization algorithms. The dataset consists of series of high- dimensional continuous optimization functions, including single-peak, multi-peak, rotated, and distorted functions, as well as some functions with specific properties such as Lipschitz continuity and second-order differentiability.
C.2 Robot Control Tasks
We test the performance of POM on two complex robot control tasks.
16

---

C.2.1 Bipedal Walker
The continuous control task Bipedal Walker [58. implemented within the Box2D physics engine, has been designed to test the ability of walking agents to navigate varying terrain by controlling their joints and maintaining balance. The challenge requires the agent to learn efficient walking strategies that enable to traverse the intended path without falling or deviating from its trajectory. The robot's s state comprises range of variables, including the hull angle speed, angular velocity, horizontal speed, vertical speed, joint positions and angular speeds, legs contact with ground, and lidar rangefinder measurements. The robot's s actions involve determining motor speed values in the range of [-1, 1] for each the four joints at the hips and knees. The performance of the agent is evaluated through reward system, whereby it receives points for moving forward, with maximum of 300+ points awarded upon successfully reaching the end the designated course. However, the penalty of 100 points is imposed if the robot loses balance and falls. Furthermore, applying motor torque incurs a small cost terms of points. The score accrued by the agent serves as a measure of its optimal performance. The Bipedal Walker task represents challenging and dynamic environment that effectively evaluates the walking and balance control abilities of agents. As such, it provides a valuable benchmark for testing and comparing different reinforcement learning algorithms for robotic locomotion.
C.2.2 Enduro
Enduro [88] is one of the classic reinforcement learning environments provided by OpenAl Gym. is a driving racing game based on the Atari 2600 game, In this environment, your goal is to drive as far as possible by controlling the car. The Enduro game to avoid other vehicles and overtake as many other vehicles as possible within a limited time. You can avoid collisions with other vehicles by moving your car left and right, and be careful to control your speed to avoid accidents. The game rewards you based on how far you drive, so your goal is to learn good driving strategy maximize the distance traveled.
on an endless highway where you need
In these two test tasks, the agent interacts with the environment for time steps, and the reward at the i-th step is ri. We evaluate strategy performance follows:
(27)
In these two tasks conduct 10 sets of experiments, each set of experiments consists of 5 independent runs. We finally take the best results of each set of experiments to calculate the mean and standard deviation.
17

---

D Baselines
Our core the population-based pre-training BBO algorithm, so we do not compare with non- population methods such Bayesian optimization methods. Moreover, Bayesian optimization methods are difficult deal with continuous optimization problems of more than 100 dimensions. We do not use LLM--ased approaches 32 36 as baselines because they can only be used for a specific type of task.
Heuristic Population-based BBO Algorithm. DE(DE/rand///bi 54 ES-S),))-ES) L-SHADE
22.. and CMA-ES [22. where DE [54] and ES are implemented based Geatpy [60], CMA-ES and IPOP-CMA-ES are implemented by cmaes and L-SHADE is implemented by pyad The reasons for choosing these baselines are the following:
DE((EE/aan///bin)) A classic numerical optimization algorithm.
ES((s,))ESS): classic variant of the evolution strategy.
CMA-ES: CMA-ES is often considered the state-of-the-art method for continuous domain optimization under challenging settings (c.g., ill-conditioned, non-convex, non-continuous, multimodal).
L-SHADE: The state-of-the-an variant of DE.
Pretrained BBO Algorithm. We chose three state-of-the meta-learn BBO algorithms for com- parison with POM.
LES [25]: A recently proposed learnable ES. It uses data-driven approach to discover new ES with strong generalization performance and search efficiency.
LGA 10 recently proposed learnable GA that discovers new GA in data-driven manner. The learned algorithm can be applied to unseen optimization problems, search dimensions, and evaluation budgets.
We train POM TS. During training, the maximum number of evolution generations is 100, n 100 and the problem dimension is set to 10.
Phups: http://ggghhub.oooooyyrregggallt bbunss/githubb.co///kiz//zzp
18

---

E Parameters and Training Dataset
The primary control parameters of CMA-ES and L-SHADE are automatically adjusted. For LGA and LES, we utilized the optimal parameters provided by the authors without modifications. Other hyperparameters were tuned using grid search identify the optimal combinations, and multiple experiments were conducted accordingly. Detailed parameter settings are presented in Table Each experiment reports mean and standard deviation the results from various sets of experiments, with consistent population size of 100 across all trials. All experiments are performed on device with GeForce RTX 3090 24G GPU, Intel Xeon Gold 6126 CPU and 64G RAM.
Table 1: Detailed parameter settings for all baselines.
Algorithm
item
Setting
1000
POM
Standard Settings for POM (M).
de
2/5 is hyperparameter, and we determine this hy- perparameter between [0.1, using grid search, with step of 0.I.
Initial
upper bounds-lower.
CMA-ES
(upper_boung) lower bounds + (randn(d)
lower_ ) where rondn(d) stands for sampling d-dimensional vector from standard normal distribution.
Initial μ
We use grid search to determine this parameter. the search interval is [1, 10), and the search step is 1. grid search to determine search interval [dup, ecs, etour, otos, rcs, rps, rws, sus, tour, urs]
LSHADE
memory size 6
We
this parameter,
selFuc urs
ES
[60]
determine this hyperparameter between (0.1, 0.8)
Nsel = 0.5
using grid search, with step of 0.1.
we determine this hyperparameter between 0011, 0.9
F 0.5
using grid search, with step of 0.1. [60]
DE
determine this hyperparameter between [0.1, 0.9
XOV H 0.5
using grid search, with step of 0.1.
We use the pre-trained optimal parameters provided by the authors.
I.GA
All parameters
We the pre- pre-trained optimal parameters provided by the authors.
LES
All parameters
Table 2: POM parameters of different architectures and architecture settings.
STRUCTURE number of parameters
dm
dg
VS
40929
200
S
101529
500
M
202529
1000
L
404641
2000
VL
110851
5000
50
XI.
2021201
10000
100
19

---

Table 3: Additional Training Functions. 2 32 Wi-
ID
Functions
Range
TF1
Σ: 22 wsl
x E [-10, 10), w E -10, 10
E1 w)) + (a++1 witll + +E -wil
x 100 10) w [-10, 10
TF3
2.
100, 100). w [-50, 50
TF4
max[|=]1 1 i
# E 1-00, 100). w E -500 50
TFS(Rosenbncc)
(100(2² =;11)² + +(= (==-119) 1)9)
m [- 100, 100).
50, 50)
TF6(Grewaak))
1 4000
IT cas +1++1
E [- 600, 600), w [- 300, 300]
Σ
(of (2 10)
E [-5, 5]. w [-2.5, 2.5]
TP8(Ackley)
-20exp(-022 Disi
2?) exp( Ei= cos( exp(1)
1-32, w [-16,
F
Additional Experimental results on BBOB
F1
BBOB Test
Table BBOB RESULT. POM is trained on TF1-TF5 with =10.. The best results are indicated in bold, and the suboptimal results are underlined.
DE
CMA-ES
LSHADE
LES
F1
11378-1)3.228-11)
9.4650001L117..00
1.28E-08/7.366 36E-044
4.558.00400BE-00
LIBE4015 81E-00)
F2
1.49-12/4.099-13)
2. 2118--00..22E.011
1.10B-014.355:400
1ASE-02/52118-0
1.75L.014.800-00
F3
6577E-06655E501)
26E40))
1,02=0015
2,7T-00123984400
112E+010,,114400
8.10E-00/1.0ME (.ME--2))
2.826-02/1
(6E--1)
6.958-0116.95E411)
3.25E46366.70::11
2.218 (02):000)
6.11k=00211
(226-00))
3.766:4022331 144+-01)
F5
3.118-01(6611-00)
(0EE+01(1-45500
1.32E-002.700
20022270--000 01)
0.00E-000000E-0000
0.00E-0000000
1,99--02/1.66----1 (.46-011)
0.00E-0000000E000
F6
..99-0011609E099)
1.296x021177444400
8.99E-0216.01140033
1.54E-0118.944-0005
1.115=0107
F7
37788-193.78B-13)
161E+035,119440 19
7,62+00/9,02E-00
3.41E-00176788-1
L.25-0010..1E:400
1220--0113922-011
6.97E-01(.0002-01)
F8
6238-0666.288-66)
4 (55
3.266400544662
(0)
3,15E-0003.00EAA
L.GEE405(2.944-00 (.94)-0)
1605-4011.000-00
4.17E=0111.20:400
1.85==015.421:00
1571=-000.933600
L.NEE=0X44400-022
F10
4.56 148E4461148
2,33840776 306
100E-00(1.188440)
L.I6::44455722..
7,58E-004 9.33..04))
2.67E:49555 13E-044
7.38E-000.3888-000 139944333.311122
5.73E
(2))
LSSENEELL
2.50:-0212 (91
3.95
(400-002)
F12
5.13E-0465133-04) 4.18
62E+08)
1,37E+ JOUS 878-8))
20JE-11(2.99--10
1.10E-05(4,53440
9,59E-07
87E-07)
F13
67668-056.766E-55
9.66E-00X
(0)
8.618=01(3
01)
2.40E:02233.51
F14
2.29E-64/2.29E-04
00ME+011L.0EE
$41E-0//9936-001
1.928-001.144---
4,38E
6.01E-0X1.51BB
4.02E-0XX. .8E
F15
018.880011
6622 2332
4.31E=006 20)
4.27E-04)
3.068404)
8.75
(2)--0))
2.84
90E--01
F16
2.55B-01/2.588-00 6621+0144 (0))
5.44EH015744EH0
2.ISB-0133.668-000
L64E-0115.990 555
7.17E-099.99-000
3.24E-0108.22 73E-01)
F17
2779E-0((2.998.05)
2.47E46177336 36E400)
2,43E00 938.00)
3.788-01 36E-02)
4.67E-0199 78E-02)
9,74
30E-00))
2.21E:002 2.95--11))))
F18
(.00E-01(13308-01)
2.26E-00S.
9.54E
3.43b=01(1 02)
1.21::012263---0
4.2E-0004.82EE0000
5.4D=+014.16
40E=00(1.177-00
594E=000
5.40E+00K
1.672-113
1.1E-0112.
(110000))
7,06E-0X2.092
F20
1.326 -))
1.25500513,
(4)
(188:00(2.55B4000
4.27
(3H6-01)
3.13E:009.
2.71E:01(1.032.00
9.09E--010 -01
F21
3.368-01:33888-01 8.80E-0106.166 111.6/..6--1111
R.56E+0L(7 1.000L77.460000
(HIE-011122E+00)
1.99E-018.0EE:
9.98E+0022.8 36E-00
F22
1.77E-001157E--00
122461118824400
257E40166.998-00
1:9550005
(2E-00))
1.14E-007..178.00
1.68E-0013.62B-000
F23
3.68E-00336888-0000 1.38E-018..99
1.1E=O0L1 (00++00)
3,80E-00(362E-01
3.17E-00X8.6EE-11
-.01E-003.266-00
4.38E-00
2.818-0222.81:000
490444))))))
4.225:044.5555000
(228-002)547E.00
1.728:40255553k.m00
7.08F=00266 (7E-00))
3.69E 3.535-01)
5922-12/5.92F-12)
4.345-0114
29E4000)
1645-0108.785-01
2:20E-02:4
(7E--0)
1.135-02(1
60E-01)
F2
1.706-12//.708-13)
(6
4.17E-01
(9.2000))))))
7.58F-02/4
38F--12)
4,56F-000)
(.06.-00)
3.28F--004
70E-01)
..078-091007E-099
106
4.77ESO
3.24
4.7E:0029.000:-00
2.725=08(115
L.ZE-HIK 78E-001
1.30E-0701.300007 6.38F44((731F
136F405(2445..00
3.77E-00(3
T.20F=031.600
5.15F-03(1.0
(2F.--3)
2.49F-0331
(333-00)
E5
3.04B:0223.4444000
422444)
2.4450002.1334000
1.305=00(8
74
4.05 (6))-000))
9.32F-100.32V-110
0.155400(22255400)
2.655-00(1
(5F440))
4.00F+01
4.37F-0024
(7F--0))
2.09F-402/900
-SE-
F7
13)28-13024288-133)
L:1E:04(2.21E40
6.11E:0444
2.798-000/3 3 3.55E402)
T.96E:40277711
728--0)
F8
3.01F-080..01F-00))
2. 0094407(2 75E+05)
1.60=08(2.
1.06E-04(2447E-000
1,35E-04(7
05F-01)
2,405-0511
(SE-04)
9:43E:04455
31E-04)
F9
64115-02:6.4144133
(PFE45().59E446)
1.12F-05(8, 38E:04)
406E4039
3.718-0511
(1E-04)
1.87
F10
2.34E-01(2344401
5.73E+07(1.
(EH07) (EE40)
1995-0811115--00
177E--7((49911-00
4.100-0503.00
2.825-06
OTE-06)
3. 3.833-0665
1.71F-0110.11:-000
467F403(55 42F+02)
2411F00(20055-00)
8,25F:00415
7.82 F-F0))
1,27F:00X(
-7F--03
F12
1.33804/1,458-04
4.8654
8984 10)
1.918-12/5.1111111
9001
2.12E:00 (.77:-00)
F13
7.238-05(:223E-155
4 188 22F401)
6.65F50744
6TF
6.35F-0021 1. (55+40))
17F401)
1.535-07
(.03-00))
9.26F-402166
39F-01)
F14
9.07E-550.87F..5)
4.51E46266. (2E401)
3.85EJ0R,
41SE-0026,8SE440
7.45 (00)
3,57E-0115
(38.000)
F15
4.88F-0224.8844113
9 SEEE-630002E400
6.86F-048,50
-0))
3,37E-04L
93E+04)
10SE-00309
66E+01)
3.61E-0022
1655-0331
10F-01)
F16
4.738-01:4.23--00
1.9050021
408-00)
5.58F:01(1
36E-000)
F17
5.50E-07:5.500-00
26
1.77E=OH8. 83E+02)
5.718-0001
(24++00
2.61E00005
126-025
2.10-0012. 18E--00)
1.205+01(1
(9E-000
F18
(34F-0665.34F-66)
I 733 (F402)
2.66F-04
(333)
2.65F-00
T.666-+01(1 25F-000
5.60F-0199 S0F.-0)
4116F40(4-5TF--0
F19
674B--006.74E4000
5.326x0R(200SE443)
L.0E--011.11E4000
(8E-01)
2.75E=01(2.745-000
(22-000)
F20
5.00--005.000F--0000
1. (6F
3.95F-04((1
4.66F-0041
(555.04)
2.56F-0466.50F
4.058-.01:4.155..00
1.225408(.2288400
6.56E-01(1
(255.00)
1.77
7.628-0106.40
7.35E-01(7.5TE
(7577--11)
F22
5.056-0145055--00
2.33E402(1 93F+01)
A46
777
3.12F-01(18822
(1882+00)
7.62F-0114
80E-00)
8.01F:01166
36F-00)
F23
L.3E--00
1. 4.39 62E400)
5.0ZE400
3811-01)
5.21840011
558-01)
7.15E-0011
1. 37E--01)
1.44F-001.2444113
132E+07(2.44B4440
1.30E=08(1.61
(.61+707)
16116-006.44440)
124440322 28044))
3.525-013
10E-00)
3.37E-0X2.41E-00
46/0/2
35/2/11
43/0/5
44/4
20

---

F.2 BBOB Test With Optimal Solution Disturbed
We further test the performance of the algorithm on the BBOB. Here, the optimal solution of each function is randomly disturbed, that is, x' Xopt + 2, where x' represents the optimal solution after disturbing, Xopt represents the original optimal solution, Xopt is vector obtained by random sampling and z --111 The results are displayed in Table 5 and Figure I0] We found that the performance of POM can still dominate other algorithms when the function optimal solution is disturbed.
POM LAM
100 CMA-ES
LSHADE
Figure 10: Critical difference diagram of 3 algorithms on 24 BBOB problems with d 100. The locations of the optimal solutions are in the range of [-1, 1].
Table 5:
Additional Experimental results on BBOB (d = 100). The best results are indicated in bold, and the suboptimal results are underlined.
POM
ES
DE
CMA-ES
LSHADE
LES
LGA
FI
1.85--1(.3S--11
2,49F-022(1 20F401)
6.27F=42(7.800
5.76F400) (69F-000) 2.94F=0008 22F-01)
2:00F-02(001F-000
8.95400(1 19F-01)
3.11E-1((911--22)
5.95E400L
4.31B:0011
(00--0)))
101E00332234400
TASB- 95E-41E)
3.30
3.39E.006.1199000
9.64E-0119644F-01
2,14F403(6
76F401)
4.5544033
(0E4402)
1.26F40% 322--0)
268E400(11 (5F402)
1.71E403(1
F4
6.100-00(6.10E-I0))
(99E4))
1408404
(3)2
L68E405
5.24E4G3( (8844)))
2.76E403(3.
12E402)
3.50E=00/3.50E0
2.875400
2,22E402(3
4620-00047500-000
1.38E03()
744-0007774E-000
F6
2.05E-18K2.80 200
4.55E.402(L
(8E401)
(10FFA01(266FF-000)
1000402(1
(B401)
2.7084022
24F:0T)
1.22E-134,222-133
1991-0301.3
57E40))
R8IE+00(8
(6E+02)
220E402(2225E-02
1,38E-010
1.33E+0355
42E+02)
8.93E+02(L 21E+02)
1899-08: (899-118)
5.5440052222
29F404)
4.85E40KSS
34F405)
3.02F.4366665F.000
8.35F-4726.277
37F401)
(066405() 21F440)
2.378405(1 39F404)
F9
6.39E.02(6 (21.339))2))))
1.11
777444))
4.738400(6
(55)
7.94E402211
9. D.E54
63)
000
(3E405)
4-71540((2444-40
6.955-0422
14E400666
033-406/6
FII
2144.00112.4FF000
7.92E
49)
5.21E:41522
43B:402)
7.69E001(1111
1.000011111111011 13E500))
4
4.3 93E
F12
L.6E-04: (.66F-04)
(2E+0))
2.08 36E408)
1.51E+09(3.
66E-08)
6.044-070L
935-07)
70 08E+08)
2.25E+09(3 85E+08)
5.511-05(5.511.055)
161E
8OE40I)
2644403 44F402)
2.52E44221
(55.01))
1.40.4030 F.05))
43 60F401)
423-
22F402)
F14
5.05E-4555003T--55
5.67--01EL
52E+00)
4. 11IE-00/2.83--00
5.52E401(L
L85E-004
4.575-01
3.855-0133 60E+00)
4.11-01 73E+00)
5111:0225.11144
16E-40322444 43E40))
6,58E40366.
1.277.40%
6.41F-00066
03 4F402)
7070403 76F-02)
F16
111400111 67E400)
7.31B401 (777E400))
5.25E:011
(00:
3.89E-0114442E.00
1
4.31 25E 25E +00)
F17
5.90E--0155900
25E5+004 87E-01)
188440216
365401)
2,43F+00338898
14F--001
100-00000000001 70F-011
24
44F-01)
L. OZ22+01 OSE-01)
7.01E-066.011-066
3.4440201
(4E4402)
9,50E400L1 (511-00))
3.74
(000
4.7 3.39 (2005401)
(4)
F19
7.07E
2. I.E5 (3.02-01
1.445-0003
(0000-00)))))))
8.39E-0002.80E.
80E-01)
7.526-000
(RE--00)
46E4001 48E+00)
1.336+01(1. 28E+00)
(418540((347E40))
187F442
T.66-00
1.7F-41)
166444((1
62F.05)
F21
6.40E=0116 40E+01)
2.24+001( 20E+00)
1,0040026.7
73E+00)
26650012.677--00
7. 46E+00)
7.13E+01 37E+00
F22
1911FO0 (591F40))
04F-0122. 71540)
(3.2401 (00
T,05 81F-00)
78IF-01(S 26F-01)
7.79F401
(9F+80)
5A05-00 5,055000)
67TE-000.51E-01
161EB00(S.
3.5115002.21E-00
1.ME-00LL066-011
F24
T.31440N TEFFOONT.31R40 3314400)
2.57F-032
(4F-00(1.77--0))
1.17440/771 (2F--01)
9,63E-02/ (.1R-+02)
3,577403(1
69F402)
3.536+03(1
90F+02)
win/tie/loss
23/M1
23/01
20/041
17/3/1
21/2/1
21/2/1
21

---

F33 Higher-Dimensional BBOB Test
7
3
2
POM 6.417
DE
LSHADE A.120 ES 63750
2
LGA CMA-ES
Figure 11: Critical difference diagram of algorithms on 24 BBOB problems with d 500.
Table Additional Experimental results BBOB (d 500). The best results are indicated in bold, and the suboptimal results are underlined.
ES
CMA-ES
LRS
1.98E-120..98-11
23115400(6440440)
2,4NE
4.10E-0055.2
F2
1.
2.288=00/21166-001
2,11E=02/1 (223-00)
L998-4002.506000
7.4SE+013-ASEHH0
9,80E=01(1
1.38--112.358-11
167E40M(2448440)
3.02E-0448 3.02E-0M..6605400 (646440)
3.53443544 12:40)
1911
2:93E:044
2.47E-0K2.4EE-09
30GE+04143004400
8.90E=0M (145==0))
6.908:03111
1.315405(1
(288+00))
1.606=000(
402:
F5
A0TE40300 94E401)
1.269:44501 (055:42))))
48F:402)
8.44F.03(S (.88.4)2
2. 2.96F-0R8.51EE
8.96-1100.9E--11
391EM001
9.14Exd59.511:d0
231:011)
3555603
(2582402)
5.63
+2)
L606-11/1.60F-11
(0E4444(10114000
5:54F:0H2.244F-003
2,20F-0K(1,
(.44400)
52F4400(1
20F:47))
2.96F-04( 03B403)
FR
3.75E-08/37758.088
2.27
1.STE.0D(3.0SE4000
2.80E=00K
(.50E4077
1299400(1
(1B4407)
4.648-06
05H447)
A.24F-4003.24444333
2.07E408(
146F-4002.766--00
2.46F408(1
F1B
2.718-85(2.718-055
115E.08(6.3774000
4:57E.00,3.66B 3.63E-00)
2.36E-00R,344 (77
4066
9.21E407(1
(888407)
1.17E:-2211117::
1.79Ex04/22772-000
2.40E+0M
4,13E=00
(31::0))
C364
(22)
2.12F-05/2.1288
4:5344100600044000
13234-1/2.33--111
2.28F-11(4.22
48F4
87F408
4000
(0F40))4))))
1,00-11CC 93B409)
F13
1.29E-0/1.299E00
4,30 225401)
(TEE603(36626-00
12
11311.13115 1.31
5,285:03(1
(.445402)
O2))
1.50E-0661.00 1.1..66))
29FF4428.221F-000
6.18F-0211
5.33F-0015
F15
L.5E--02/111524003
(:38440(44-75440)
2.50E-04
(26x08)
29)
2,12E.04,2 15E
2. -4E4-D4(1 (7E)00))
F16
6,528401 (6.5---000
6.56F401 (2)66)40)
8.54F50144.44 (4F--01)
7.60F=0012.20F
2922:01(11 (00)))0))))
7:04--0r(
20F400)
2.6EE-07(2.6E--77
8.27E4016..17EE 178440))
4.11E-00/8.777 (1)
(.66-01) (66
(66E401(13004400
1.738-0022.500 40550))
F18
3758-47(2.75E-47
125E:+222244BB
3.77E=00244299-001
H.58E
172.01)
7:52EH0L3307TE-0
2.54 (5
8.19F-0088. 10))
7.85F.4N6.2FF-00
(8F:4)))
1.58
1.11EA03(2
2-92F-0344 S0E+01)
F20
2.59.011-2.668-00
1.445+0615.22
2.585:001.7775 (7775400)
(3E:801)
1.25 (9):
3.17 00:000
F21
(8.0400
9,09F401 90F-01)
174F/4027.622F-00
1.11F=00288 8TF300)
7.60F
101--0)
(3.71--2))
9. 9.80F-01(1
34F400)
L0NE=001/..0EE000
4.0154224.858:001
1.14E:00/8.099.00
7.89E:0112.799.000
8.22E4017.348-00
9.705-01C
(4)+00)
F23
1.8F-0001 1.60-000)
(2.5540)
2.09F-023.11--001
3.827--01(1 (2F.00))
A88840002 10F-02)
1.25F-01r1 24F400)
7,46F.00377 40)
6499440().114405)
3.258.4774
3,41E:0K(6666E4000
7.2N8::03(72258.000)
2.21E404(.365402)
1.259.06c (.777440)
wintio/lax
22/1/1
23/11
23/0/1
22/1/1
230V1
G Compare with TurBO
We compared POM and Bayesian optimization algorithms, finding that Bayesian optimization converges very slowly figh-dimensional problems. TurBO [56], noted for its fast convergence and strong performance [57]. was used as a benchmark. Although TurBO requires substantial time for 10,000 evaluations, POM completes the same task in under one second. Therefore, plotted the convergence curves of TurBO and POM with only 3,100 evaluations. As shown in Figure 14 POM demonstrates significant performance advantages TurBO in most cases.
22

---

PON
POM
26
00
LBS
LGA
LGA
50
POM
CHAES
DE
100
LONGDE
-10
-12
LGA
(a) F1
(b) F2
(c) F3
(d) F4
5.0
2.
2.0
0.
-25
CHARS
CHAES
-60
POM
CHAES
POM
CHALS
DE
ES
LGA
LGA
LOA
LSHADE
-10
(c) F5
(f) F6
(g) F7
(h) F8
PON
15
CHASS
CHARS
DE
DC
10
UGA
LSHADE
CHALES
LSHADE
POM
DE
3
LMADEE
LSHADE
40
00
80
100
20
40
50
80
100
20
40
60
100
20
60
00
80
100
(i)
(j) F10
(k) F11
())
POM
3.0
LES
2.5
LGA
20
CHAGS
LGA
LGA
LSHADE
(m) F13
(n) F14
(o) F15
(p) F16
ROM
CHAES
CHAES
DE
LES
106
7012
LSHADS
LSHADS
185
20
40
80
80
100
20
100
20
40
60
100
0
20
60
100
(q) F17
(r) F18
(S) F19
(t) F20
1.0
POM
PON
0.5
40
2
LES
LBS
PPM ChIRS
LSHADE
LSHADE
CHARS
DE
0.6
300
LOA
LGA
0.5
25
LSHADE
ESHADE
100
20
(u) F21
(v) F22
(w) F23
(x) F24
Figure 12: The log convergence curves of POM and other baselines. shows the convergence curve of these algorithms functions in BBOB with d = 30.
23

---

5.-
0.
25
25
POR CHACS
POM
50
CHAR1
CHARA
CHARS
-75
-50
LES
-5.0
5GA
-7.
LSHADE
LSHADE
00
100
20
40
00
100
20
42
80
100
20
1
300
(8) F1
(d) F4
7.0
25
5.0
2.5
POH
0.0
PON
15
LSHADS
LSHADS
LSHADE
LSHADE
60
50
100
100
20
40
90
80
100
20
200
(e) F5
(f) F6
F7
(h) F8
PON
PON
15
PGM
POA
CMARS
CHAES
6
LES
LGA
LOA
-6
100
(i) F9
(j) F10
(k) F1 1
(I) F12
PON CHARS
3.0
CMAES
235
ESHADE
200
LSHADE
LSH4DE
1.0
20
40
50
190
300
20
110
100
20
611
11
100
20
40
50
80
100
(m) F13
F14
(0)
(p) F16
PON
PGM
CM
CMA
LE5
LSS
LCA
LGA
(q) F17
(r) F18
(S) F19
(L) F20
POM
CHARS
4.0
POM CHAES
PON
CHAES
CHAPS
1.2
215
3.0
LGA
1.2
1.0
LGS
LGA
LGA
LSWADE
20
0.6
15
07
50
100
20
100
20
60
60
80
100
0
20
60
200
(u) F21
(v) F22
(w) F23
(x) F24
Figure 13: The log convergence curves of POM and other baselines. It shows the convergence curve of these algorithms on the functions in BBOB with 100.
24

---

TuRoo
4.0
50
2.3
(a) F1
(b) F2
(c) F3
(d) F4
3.2
PPM
Tu/80
PPM
1ur60
POM TurBo
Ture0
2
24
500
1003
1500
5000
2900 3000
500 1000 1500 2000 2503
5000
500
1000
1520
2000
2500 5003
500
1000
1500
2000
2500
3000
(c) F5
(f) F6
(g)
(h) F8
POM
PON
FOH
Turio
Tarto
Tar00
6-
2000
3000
2500 5000
(i) F9
(i) F10
(k) F1 1
()) F12
PDM
3.5
1.5
3.7
3.4
20
0.2
3.5
3.4
3.0
1.2
(m) F13
(n) F14
(0) F15
(p) F16
PON
2.0
POM
2.0
Turoo
Tureo
1.0
0.0
-0.5
-05
1.2
15
-1.0
1.0
2.0
500
1000
1600
2003
2500
9000
500
1000
1500
2000
2500
3000
0.8
800
1000
1500
2000
2500
3000
500
1000
1800
2000
2500 8000
(q) F17
(r) F18
(S) F19
(L) F20
2.0
1.8
POM
PDN
Turoo
1.6
1.5
0.85
2.5
1.4
3.4
12
0.80
3.5
12
100
0.75
0.70
08
POM
2.1
500
1000
1500
2000
2500
3000
900
2000
1500
2000
2500
3000
550
1000
1500
2000
2500
3000
8.0
500
1000
1500
2000
2500
3000
(u) F21
(v) F22
(w) F23
(x) F24
Figure 14: The log convergence curves of POM and TurBO. It shows the convergence curve of these algorithms on functions in BBOB with d = 100.
25

---

H
Results of Analysis Study
Table 7: Results of ablation experiments. The best results are indicated in bold, and the suboptimal
results are underlined. Here d = 30.
NO LMM
NO LCM
NO MASK
UNTRAINED
POM
8.83E+000.88EE000
5.86E-07( 86E-07)
3.09E-01(3 (9E+01)
4,74E-0M4,74B-44
5.58E-15(6.5SE-IB
6411E-06641EE-0)
3.26E-08(3.266) 26E-08)
2.97E-012.976-01)
200EE-062222E-0000
1.67E-17(1.6EE17))
F3
2.77))40))..772-22)
(55=+015)35E+01)
4.34B-024.344402
1.56B40201.566 1,56+002)
1.34E+0KK.34E+000
R4 3.55E+02(..5EE-10
(175E00((7955401
(.77-0027.77E-02)
1.00E+03(I (6E--03)
2.92E=012..22-000
2.399
4.75E=0144.5B400
3.0SE-013.0SB+01)
3.92E+013.922-400
1.835+01 (333++01
5.ME::015.44EE00)
6.85E-0616.8 85L-06)
6.70E-0166.70E:4
5.69E-02/5.69E
5.15E-135.15E-ID
F7
(WEE40033,30E-02
2.51E-022.511E44
1008-0221.700-000
1.718+0(1.711E-00
6.28E-033.28F.
F8
3.71E+03(3.112) +13))))
4. 4,26E-04(4 (46E-004 26E-04)
1.34E-04(1
(34E--041144E-04)
3344+04)
(22B+0003.2EE-000
1.24E-11(1.24E
5.37E:40355331 03)
1.51E+02(1. 515+02)
000E--03880944400
1.366=02/1
1.32E+00/1.32E440
F10
-.15E-029.155-0000 45$++5))45E++55
6.47E-0566.774405))
375E+05(2.33E-00
1.225+03(1.22E--4
F11
2. (9E-02)
9.26E+0009.26
177E-0221779E+00
8.1774018.17E-00
4.92E=01144.2EB
F12 181E:08(18111--
3.355-02(3.335---4
3.31E=08(333
(331--0033311-008) 31E+08)
2.07L-04(.07E--0M
2.87E-07(2.872
F13 2.95E+02(2.95E=022)
2.76B-03(2.76AA))
1.76E+01(1.
(6E-01)
(.85E-05(1.05E-0
1.33=+0((.53E+001
3.95E-04(3
5.66E-0015.6 66E+00)
4.46E-02(4.466--0
1.16E-04(1.16E-0
F15 3.71
1.14
1445402)
3.84E-02(3
-34EE-02338444400)
1.94
1.94))02)
F1G 2.92E=01(2.92EE-2
2.76E-01(2..6EAA
2.91E0012.91E-00
2.60E+01 (2.E=00(2.6EE-0))
1.24E-0011.244E-000
F17 6.19E400X6.11E22 19E+00)
1338-02222233-02
4.75E-00(4.554000
2.97E-01 (2177--1)))
5.06E-07(5.06E-07
2.211:+01(22211+000
2.08E-0122 08E-01)
1.98E-01(1 98E+01)
2225:4002.2555-000
2.71E-032.71E--33
F19
8. 8.122800 12E+000)
-00E-00000000-0000
(44B-009944E-000
5.63E+00(5.63B
F20 4.05E:02(4.05E:4020
2.755--0(-22.58----
1.56E-03(1
L.56E-03(1.66E-03)
1111=000-3.11E-000
-1 -L84B-01-188E4000
121 3.11E401(3.11E4M
21200-011(220E-00
(16E-0116655E+0)
6.455-01
3.655+01(3655E401
6.31E=0006.31E+0000
1.08E=01(L (8E+01)
5.10E-01(5.10E44 100++0))
6.32E+01
(2332-01))
166E+01(3.8EE-00
F23
3,33E+00(3663E20000
3.49E4003.19E400)
3.315-0003311-400)
3.23E+6003.233--000
3.775-003,77E-000
F24
3.55E+02(3.5524422
2.00E-00((2.0EE000) 198E--2(3.98E400)
2,885-02/2.88--022
3,415-02(34411-022)
Table 8: Results of Training Dataset Experiments. The best results are indicated in bold, and the suboptimal results are underlined.
F1
1 500:-O1C1.00 SO00 001
5.33E 04L5 33E 04)
8.2068-078..2
200 (IT)
5.56E QBLS 56E 08)
4.97E-08/4.97--0
7.57E 177E--667-77-00
7.54E Q577 (4L-00)
2,088--04/2 (0S--0))-044
F2
3,208-00(3, (29--02)
4 2.29-00544
29E-055
5.56840755
5QF-47)
6788-47766 78E-077
1.388-19 1338--09)
14418-0766411-000
6,548-47766 (4F-477)
13
0.528-00088. (2))
40E
02)
5.01400215.0
616+02)
2.84
8.4GE 460+00
5,27HH (7))
9,57E=0109
F4
80E+0216 80E+02)
4,06
(,06E+02)
1.88++2(7.588E4)
7.53E+02(7,53E-222
S.0EH0225
1 14EE-00/1.9444400
FS
9.145-02/9.1 02)
1.14
(2))
7.ME
7.666:00277
-Z)
P6
9.30E 01(9 300 OL) 2.89I 8912 03)
1.11B-00(1.118:0
05)
1.04--051
100--00110000 DOE 05)
1099-06(1.09-
7.17E (5)-172)-0)) 6.30E
0415330 01)
8.35E 1.35E-04/8.0 (5)-45)))4)
F?
1.06E-0001
(0) 2. 112
1.608-1201
2.H8- (2(2-98--12)
1408-13/L44E
1.99- 12L (2E-12)
7/33--2(TT438--
1,18B-1((1 ISE-11)
18
4.945-0004 (22) 1.
7-255-0907
258-33000 (3)
-97E-00/2297E--0
4.22:0114822-00
DOE (DE+00)
5,928-02255 (22--222)
P9
1.56E-03(1 -6E-03) 6223 43E402)
6411542(664414400
6.40E 40E-0E)
477 42E-422 5,41E-02
6.306-00226.
30E-002
210
(4)
1.171-00(1.178
3,2HE 01 (1)
L4SE-0411605E-44
332 (2)+01) 3.5ZE:
00)
1.85=+001
(3E-00)
FI1
2313--00/2. (38-02)) 2.04400 (4E4B1)
1,58P401(1
(38401)
2266401 (6F401)
1.97401(1
97E-01) 2.01400
1.234401
(3:33001) 9.63F-000
+.63+000)
F12 3 (7 O0) 1.5L 1. (115444
4.00E
(0))
200E: (0E401)
8.97E-018.97--00
TOOE (3)
3.1
OZ)
2.05B-00000 O0L 167E+0001 67E+00)
5.918-021590
91E: (2)
7.06E
3,420
0213 42E
63E
(LE
-SIE-OX1_SIEE44
F14 1 1.248-01(1 24E OL 4 1IEE-09
(1E-03)
9.208505(9922 298 (88-05)
1.37E-05(1 (1005511..7388 337--05)
1/24E-09(1
24E-03)
1.37E-09
STEEEEE
1.75B-008) 75E--13)
P13 N.SSE-0218 (Z) 6.
6,55E:0206
(2)+
7322+027
6.4EE-H2266 (8E+02)
7.21 21E+H2)
7.66L=0017
(Z))
11000----O1C -0 47
77E4B1)
4,70P-0144
465E441(4.65E--00
48444014
348401)
4.74F-0114 74E-01)
FL7 6 6.556-0166. S5E-01)
I. L.02--01
02E-01)
3.085-0003
LOESEE-000 00)
47E-02) 1-47E-02(1
4444E-0415.44E-441
3.90E-0363
906-03)
15E--222
15E-02)
6.75E-0226. 75E-02)
F18 25E-000 28E+00) 4.04E OL14 ONE
7.59E
9HE
1.47E
3.25E-0313.2EE-33)
TIME O2 OZ)
1.28E OL() 28E 0L) 2.79
01.2.7991118101 01)
F197.
8SE--0077 SSE-O00 2.28 28E+00) 7,04E-0077
-00-000
(07E+0007707++000
6.44E-0006.444-000 6.7EHH06
7,26E+0077 (.26))00) 7.0-0)00)225-00000
7,28E-00X7 (225--00)
F20 2
2,00E 01.20O OL) 133 1. 03)++00)
1.54E:000-11540 -.544
(1D)
1955+000
(0)) 2.00E-0X2.0000000
F21 A555--115 45E-0L) 66 S7E+01)
6,7E+0116667H00)
5.9HE-0115 9IC+01) 6.72E501
6.0E--O
(6E000) 6.36E-01(6.
P22
7.55 (SE--017755EE001
OLIT 65E+
01)+
7:3340017/735401)
7.38E-017755 1.1SEH017.EEE00 O8
7.02 00) 7,46)=001(7 A0L-00)
F13
200-
5.20
200+00)
$21EE000S
5.195-00ES. 19E+00)
4.92EH0064 92E+00)
5.08E-00LS
(.8+00)) 1,836-00 (.83E00))
F34
1.33--00 C..0NN333EEE 33E-00))
ALEHO 4LE+03) 1 24E-03)
1.26E40(1.20 266403)
1.3 31E+03)
1194+00 199-003)
1,31E-0X (13-0
26

---

0.23
0.200
03
0.50
62
03
0.255
5.75
2000
0.500
10
(a) step
(b) Fl step 50
(c) F1 stop 100
0.25
04
0.000
0375
0150
2122
5136
2075
2000
0.550
12
20
(d) F2 step
(c) F2 step 50
(f) F2 step 100
05
05
2175
5115
0.22
2120
8.0
6:8
2.50
13
(g) step
(h) F3 step 50
(i) step
2225
0.25
2230
0.00
02
0375
61
0.50
0325
-01
02
0,550
(i)
step step
(k) F4 step 50
(I) F4 step 100
Figure 15: Visualized results of mutation strategy St on BBOB (F1-F4) with d = 100. Here, 10 for the sake of clarity. The blank squares in the matrix indicate the masked parts in Eq. (8) Steps 1, 50 and 100 correspond to the 1st, 50th and 100th generations in the population evolution process. The horizontal and vertical axes show the ranking of individuals, with being the best and 10 being the worst in the population. Each row represents the weight assigned other individuals when performing mutation operations for the corresponding individual.
27

---

04
05
02
52
(a) FS step 1
(b) FS step 50
(c) F5 step 100
0.29
2230
0.25
0.700
0.100
2.75
0.755
50500
0.500
(d) F6 step
(c) F6 step 50
(f) F6 step 100
05
3225
2150
0.25
5125
0.00
0.75
0,550
10
13
10
(g) step
(h) F7 step 50
ii) step 100
2225
0.18
5175
02
0.2
5136
-01
10
13
(i) step
(k) F8 step 50
(1) step
Figure 16: Visualized results of mutation strategy St on BBOB (F5-F8) with 100.
28

---

05
24
-33
-02
03
30
.43
(a) P9 step 1
(b) P9 step 50
(c) F9 step 100
0.25
34
0.75
0.25
002
0.200
0,500
(d) FIO step
(e) F10 step 50
(I) F10 step 100
26
02
-2
63
0.2
30
-01
-02
-53
-0.3
(g) FI1 step
(h) F11 step
step
03
0.25
0.00
0.75
0.50
05
0.50
20
20
F12 step Figure 17: Visualized results of mutation strategy St BBOB (F9-F122 with d = 100.
ii) step
(k) F12 step
29

---

3230
0,25
04
220
5236
0.75
5175
0.25
0.20
0.239
2030
(a) F13 step
(b) F13 step 50
(c) F13 step 100
0.255
0.200
03
64
0135
2120
2073
0.755
2.00
0.059
20
12
10
(d) F14 step
(c) F14 step
F14 step 100
2228
0.25
04
2375
0375
0.25
5136
0.000
(8) F15 step
(h) F15 step 50
F15 step 100
0.8
06
02
00
(i) F16 step
(k) F16 step 50
F16 step 100
Figure 18: Visualized results of mutation strategy St on BBOB (F13-F16) with d = 100.
30

---

2200
0,20
0.39
5.15
2,30
0.00
5075
2030
(a) F17 step
(b) F17 step 50
(c) F17 step 100
05
64
2230
0.12
5125
-01
02
5.055
(d) F18 step
(c) F18 step 50
(f) F18 step 100
20
94
23
02
0.4
30
-62
(g) F19 step
(h) F19 step 50
(i) F19 step 100
DC
2225
64
5175
0125
05
5136
0200
0.75
2.00
0.50
F20 step Figure 19: Visualized results mutation strategy St BBOB (F17-F20) with d 100.
ii) F20 step
(k) F20 step 50
31

---

10
36
0.4
2A
04
52
-0.2
(a) F21 step
(b) F21 step 50
(c) F21 step 100
62
30
0.1
-02
-02
10
(d) F22 step
(c) F22 step
F22 step 100
06
36
0.2
30
-02
(g) F23 step
(h) F23 step 50
F23 step 100
26
2.6
04
02
( F24 step
(k) F24 step 50
F24 step 100
Figure 20: Visualized results of mutation strategy St on BBOB (F21-F24) with d = 100.
32

---

Table 9: Results of POMs of different sizes on BBOB tests (d 100). The best results are indicated in bold, and the suboptimal results are underlined.
XS
8
M
1.
VI.
XL.
F1
3. 1.89E16(3.88 16(3.89--11 89E- 16)
5.38B-07(5.5888 SKE-(T7)
6.00B-1966.06
7.888-24(7 S8E-24)
1998.19( 197-11999
4.998-27(4.99827))
7.66k 182260E-11 18)
9971-0099.977-00
LOSE/2H1 (1)
2.588
F3
3.11
5.500-07/440000 (0)
2.808-1662 89E-16)
4.174402(4 17E440)
3334-2663.399220
1-
8.898-0508.996.00
5.26 +2)
2.408-0012 42)
8.50g-0188.00 56E-011
3.10 (1)
6.158-2%6.15B-20
F5
4.00E+02240000-022
3,22E-02/3 221--02)
3.89E-02/3.899---
0.000-000.000 0.00E+00)
4.055422/00550022
366E-0222,56E-02
P6
2.94g 14/2141/24448 14)
8.11B-0x(8. 11E:00)
1,582- 1003-1611..8--111 1888-11)
1.678-191160
19)
681E-171631E-17)
3.928-2443.92---
14(9-48B-14)
661E EIEE-4466611-111
9,54B (498-1499544-144
F8
0.00E:00000000000000
6.708-04/6. 76E-04)
0:008-000000000000
0.308-0000000 (0)
000E:0000000E::000
0.808-4000.00000000
19
6,32E40210.322 322
6.428-0226. (2)
6.27E-02/6.272
6.88
6.38k (38)--2))
F10
6.80-0116 36F-01)
8.96E-03 02)
1.37E-019.577..00
1.89F-04(1 (9F-04)
1.265400(1 (6E400
2.45E-02/2445E.02)
F11
293:401122
1,34 02)
8.306-018.500--00
6,848=0116..8 01)-01
4.1G:+014.2524011
1.00E-01(110E. (01)
F12
2.84F-
(1
2.90E-002
6.53F-116 (165531-11
1.025-0541 (22F-03)
2.11F-09/2 (150111
3.80F-20
2.09-21
F13
176-07)
5477-02(3447B-41
1.299-05(1 (295-05)
L67E-06 667--06
F14
1512.85L 2.85E
1.99 (11900441-77
1.716 0517111-0
8.94E 050805088800 (.8.948--0
LOOE 100-0011..00 L.00
6.58E-06K
F15
5.18E-0225 (8):
8.228-40233.
5.77E-025.57E-02
4.955402(4
F16
3.455:001 (5E-011)
4.51E-0144.1100000
265E-01,,.6EE
3340-0113.44-000
3.13E+012. 13E+01)
(330E-00(3330E0000) (.E000))
1665--08(1 (.35-00)
1.15E-03(1.
5.808-10L5
6.798-106 79E-11)
L61E-14(1611
F18
3.126-08/3.124 122 08)
1106-044.166-000
1399:-004.885-00
1611:08811 0661-00)
5.50EE-14:5.502
F19
622F44006.222 22F+00)
7.26F-000 2.20F-007726FF000
6.50F=2006.50F.. 50F--0))
7.06F-000 (06F--007006FFO00)
7.34F400(7.333 340444))
6.21F=-0%6..
F20
-7.196-00- (11
9.00-0109 90E-011
4 4.400:000444000- (,000000)
-1,944:00X-- +4))
F21
5.24E-01(5.2
6.86E-01(6..8) 96E+01)
1168--01(3.6E=-000
2.22E-01:2222E401
6.43E+0166.8 (6.433+01)
4.88E+01( (.)E-0))))
7.24E:01(224E:001
7.20E-01(7.20 0177000-1))
7,80E:00117,
6738-0016.735401)
(C4E60177655400
7.34F
1/3
5,06E:0005 (66:+000
5.17E:005..0 17E:-00)
5.15E-0005.15
1177=-005.177--0
5.42840015 42E+000
5.04E-00 5.000000)
F24
L.1144))))331144333
131E (33E--03)
1.32F=003). (2E--03)
1.35E403( 35E40))
1.34F.05( (3E4-00)
61
Generation
Ceneration
Ceneration
(a) F1 & F2 & F3
(b) F4 & F5 & F6
(c) F7 & F8 & P9
(d) F10 & F11 & F12
64
Cemendbon
Comerttio
Cenmrntion
(c) F13 & F14 & F15
(f) & F17 & F18
(g) F19 & F20 & F21
(h) F22 & F23 & F24
Figure 21: Results of visual analysis of LCM on BBOB with 100. Here, n 100. This is the crossover strategy of the individual ranked No. 1. Rank denotes the ranking of an individual. subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
33

---

15
OPASANN
2
20
100
20
1.00
20
130
1.9
(a) F1 F2 & F3
(b) F4 & F5 & F6
(c) F7 & F9
(d) F10 & F11 & F12
1.0
20
60 1.2
20
60
20
1.0
20
80
120
20
40
20
1.0
(c) F13 & F14 & F15
F16 & F17 & F18
(g) F19 & F20 & F21
(h) F22 & F23 & F24
Figure 22: Results of a visual analysis LCM on BBOB with 100. Here, n = 100. This is the crossover strategy the individual ranked No. 5. Rank denotes the ranking of an individual. A subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
1.8
2022
20
20
1.30
(a) F1 & F2 & F3
(b) F4 & F5 & F6
(c) F7 & F8 & F9
(d) F10 & F11 & F12
3.2
F24
02
F38
20
60
180
20
60
30
130
20
20
20
1.20
20
40
1.00
(e) F13 & F14 F15
F16 & F17 & F18
F19 F20
F21
(h) F22 & F23 & F24
Figure 23: Results of a visual analysis LCM on BBOB with 100. Here, n = 100. This is the crossover strategy of the individual ranked No. 18. Rank denotes the ranking of individual. A subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
34

---

CLPAS
0.00000
P33
20
60
100
65
60
160
322
360
F1 F2 & F3
& F6
(c) P7 &
(d) FIO FII & F12
F24
25
60
60
100
24
20
65 90
20
40
30
130
10
300
(e) F13 & F14 & F15
F & & F18
F19 F20 F21
(h) F22 F23 & F24
Figure 24: Results of visual analysis of LCM on BBOB with 100. Here, 100. This is the crossover strategy of the individual ranked No. 51. Rank denotes the ranking of individual. A subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
F32
103
20
120
(a) F1 & F2 & F3
(b) F & F6
(c) F7 & F8 F9
(d) F10 & F11 & F12
656
8.5
735
20
60
180
10
20
100
20
40
65
30
120
20
40
30
1.0
(e) F13 F14 F15
(1) & F17 & F18
F19 F20 F21
(h) F22 F23 & F24
Figure 25: Results of a visual analysis of LCM on BBOB with d = 100. Here, n 100. This is the crossover strategy of the individual ranked No. 75, Rank denotes the ranking of individual. A subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
35

---

Oseigeer Age
23
5
2.10
711
Concration
(a) F1 & F2 & F3
(b) & F5 & F6
(c) F8 F9
(d) FIO & FI1 & F12
P15
F21
724
20
100
20
20
20
130
(e) & & F15
& F18
(g) F19 & F20 & F21
(h) F22 & F23 & F24
Figure 26: Results of visual analysis LCM on BBOB with 100. Here, n = 100. This is the crossover strategy of the individual ranked No. 100. Rank denotes the ranking of individual. subgraph illustrates the change in the probability of an individual crossing three tasks as the population evolves.
I Limitations
Model size: the experiment, we found that the relationship between the model size and the performance of POM is not strict linear relationship. Although the larger the model, the more difficult to train, there is still no very quantitative design criterion between model size, training data volume and training difficulty.
Time performance: We introduced an operation similar to the attention mechanism, whose time complexity is Onn)). which makes POM require lot time cost when processing large-scale populations. How to reduce and improve the time efficiency of POM is also worthy of further study.
Potential Impact
This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted
here.
36

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, we accurately reflect the contribution and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of work in the experimental analysis part.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
37

---

Justification:
Yes, we have a complete experimental proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We give all the details required to reproduce the main experimental results, see section 4.1 for details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
38

---

Answer: [Yes]
Justification:
We provide source code, and the data sets used are public data sets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
www.ooooolcyyy iy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwtii/gguie///eeeeceoocc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the details (see section 3.3 for details).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, we provide statistical experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
39

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we provide device resources information and time analysis (see section
4).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: sesttp:://neurppttt http:///w...pppc..cccccctetteeee
Answer: [Yes]
Justification:
Our research is in line with Neurips Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See the appendix 0 for specific content.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
40

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We respect the relevant original author and the open source agreement.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
41

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide anonymous code link.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
42


Reviews:
[{"review_id": "ECzOpOB9VL", "reviewer": "Reviewer_G2ar", "summary": "The paper introduces POM, a neural-network-based evolutionary algorithm for black-box optimization. POM is trained on diverse optimization tasks to enable adaptation to new tasks. POM outperforms the baselines on BBOB benchmark and two robot control tasks.", "questions": "- What do the authors mean by \"zero-shot\" optimization? Does it mean optimizing a new objective function without seeing any (x, y) pair from that function (is this even possible)? Or do the authors mean optimizing a new objective function without any gradient training/fine-tuning? If the latter is the case, then there already exists methods that can perform optimization without training/fine-tuning on the new objective (see my comments above).  \n- How did the author decide on the input to LMM (H) and LCM (Z)?", "limitations": "See above.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed method performs better than the baselines on the BBOB benchmark and two robotic tasks.\n\n**Increased score from 3 to 6 after discussion**", "weaknesses": "- The paper is missing some key references [1, 2, 3]. These methods learn from diverse tasks and are able to adapt to new tasks without any finetuning. This invalidates some of the claims made in the paper (lines 28-29, line 78)\n- Section 3 is difficult to read without a background section on population-based optimization. It should describe a general evolutionary algorithm first and how POM parameterizes different components of the algorithm with a neural network.\n- Section 3.2 is overly mathy with unnecessary equations. This distracts away from the main idea of the proposed model, which is using neural networks to parameterize evolution. For example, equations (6) and (10) are simply describing multi-head attention and feed-forward layers, which are very standard in deep learning and it's unnecessary to present the equations here. It is also redundant to specify each element of the network parameters $\\theta_1$ and $\\theta_2$, etc.\n- Section 3.3 sounds difficult to believe. How can we expect the same set of parameters ($\\theta_1$ and $\\theta_2$) to learn from many tasks without feeding metadata of each task? For example, if two tasks have conflicting gradient signals, they get canceled out, and the model basically learns nothing. Specifically, for a certain $x_t$, for task $i$, the model should produce $x_{t+1}$, but for task $j$, the model should produce $x_{t+1}'$, and $x_{t+1}$ and $x_{t+1}'$ are conflicting, then how does the model learn in this case?\n- Section 4 is confusing without any experiment setup information. Where are the zero-shot and few-shot settings mentioned in the introduction? How is POM adapted to a new objective after training? \n- POM has advantages over the baselines because it was pretrained while the baselines were not, so the better performance of POM is not surprising.\n\n[1] Nguyen, Tung, and Aditya Grover. \"Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling.\" International Conference on Machine Learning. PMLR, 2022.\n\n[2] Nguyen, Tung, Sudhanshu Agrawal, and Aditya Grover. \"ExPT: synthetic pretraining for few-shot experimental design.\" Advances in Neural Information Processing Systems 36 (2024).\n\n[3] Nguyen, Tung, and Aditya Grover. \"LICO: Large Language Models for In-Context Molecular Optimization.\" arXiv preprint arXiv:2406.18851 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We've carefully reviewed your feedback, addressed all queries, added the refs, and made revisions as per your suggestions. Your further review and acceptance of the updated manuscript would be highly valued.\n\n**Weaknesses 1**\n\nWe have included references to [1-3] in the related work section of the POM due to their significant contributions.\n\n*\"TNPs [40], ExPT [41] and LICO78 [ 42] use transformer to solve BBO problems and have achieved good results. TNPs requires contextual information of the target problem, and neither ExPT nor LICO can be directly used to solve tasks with different dimensions from the training task.\"*\n\nHowerver, we respectfully disagree with your viewpoint for the following reasons:\n\nIn [1], it is mentioned: \"*the number of evaluation points $N$ and the number of context points $m$ are generated from the same uniform distribution as in training.*\" And there are three experimental tasks in [1]. When solving these tasks, [1] is retrained separately. These might imply that [1] may not be suitable for addressing Zero-shot Optimization problems.\n\nIn Section 2.1 of [2], it is mentioned that “during the adaptation phase, one can use the pretrained model to optimize any objective function $f$ in the same domain $X \\subseteq \\mathbb{R}^d$ .” In section 4.3 of [3], it is mentioned that “after training, a single LICO model can be used for optimizing various objective functions within the domain $X \\subseteq \\mathbb{R}^d$ .”  This could suggest that references [2-3] may not be directly applicable to our particular context.\n\nPOM overcomes these limitations.\n\n\n\nAt the same time, we found these related papers, which also gave us a lot of inspiration.\n\n- Probing the Decision Boundaries of In-context Learning in Large Language Models.\n- ClimaX: A foundation model for weather and climate.\n- Temporal predictive coding for model-based planning in latent space.\n\n\n\n**Weaknesses 2** \n\nAppendix A provides an overview of the background knowledge related to population-based optimization. Owing to constraints on space, we have chosen to include this information in the appendix rather than the main text. Section 3 meticulously outlines the specifics of POM, thereby ensuring that the reproduction of POM is feasible without a priori knowledge of population-based optimization techniques.\n\n**Weaknesses 3** \n\nSec. 3.2 details POM's structure and mechanism for reproducibility. Eq. (6) and (10) present specialized model designs, not standard multi-head attention or feed-forward layers, including weights, activations, and normalization.\n\n\n\n**Weaknesses 4** \n\nPOM does not have the conflict you claim. The usage of POM is similar to CMA-ES. It can directly optimize the target task without any metadata. POM learns general optimization strategies instead of fitting a specific optimization task. It only needs to input the fitness of the population and some features within the population to adaptively generate optimization strategies. \n\n**Weaknesses 5**\n\nWe introduced the concepts of Zero-shot Optimization and Few-shot Optimization in section 3.1 (L87-L90).\n\nThe experiment in **Section 4.2** verified the performance of POM in the Zero-shot Optimization scenario (see Fig. 2, 3 and the corresponding appendix for the results). In this section, we directly used the trained POM to solve the target problem.  \n\nIn **Section 4.3 Fine-tuning Test**, we further verified the performance of POM in the **Few-shot Optimization** scenario. Here, we fine-tune POM using a small number of simulated functions of the target task. The experimental results are shown in Fig. 6. \n\nThe trained POM can be directly used to handle new target tasks, which is described in detail in Algorithm 2 (line 167).\n\n\n**Weaknesses 6** \n\nThis is the advantage of POM. The baselines include two algorithms (LES and LGA) pre-trained by Meta-BBO. Although they have also been pre-trained, their performance are far inferior to POM.  POM can obtain superior performance through pre-training, while baselines cannot.\n\n**Question 1**\n\nThe concept of Zero-shot Optimization introduced in section 3.1 is as follows:\n\n“**Definition 1 Zero-shot Optimization**. An optimizer is applied directly to solve $f$ without any tuning.”\n\nThe references [1-3] mentioned in Q1 cannot achieve zero-shot optimization. For details, see the reply to Q1.\n\n**Question 2**\n\nThe details of these designs are described in detail in section 3.2. \n\n**LMM**: The role of LMM is to generate a set of candidate solutions $\\mathbf{V}^t$ based on the population $\\mathbf{X}^t$. Therefore, the input of LMM is based on fitness information, that is, $\\mathbf{H}^t$ (see lines 107-118 for details). The input information of LMM does not include the solution $\\mathbf{x}$, because $\\mathbf{x}$ will have different dimensions and follow different distributions under different tasks. If the input information includes $\\mathbf{x}$, POM will not be able to generalize to different task dimensions.\n\n**LCM**:  The input of LCM needs to take into account the fitness information $[\\hat{f}_i^t,\\hat{r}^t_i]$ of  $\\mathbf{x}^t_i$. Intuitively,  if $\\mathbf{x}^t_i$ has a low fitness and a poor ranking in the population, then its genes are likely to be eliminated and should not enter the offspring. At the same time, LCM should take into account the cosine similarity between $\\mathbf{x}^t_i$ and its candidate solutions $\\mathbf{v}^t_i$. Intuitively, if the similarity between $\\mathbf{x}^t_i$ and its candidate solution $\\mathbf{v}^t_i$ is very low, selecting more genes of the candidate solution $\\mathbf{v}^t_i$ into the offspring individuals helps encourage the exploration of the model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lSGeCnBtfL", "reviewer": "Reviewer_eUpS", "summary": "This paper studies zero-shot optimizers for blackbox optimization problem. The core idea is to pretrain a hypernetwork that generate suitable optimization strategies on a subset of tasks; at test time, the hypernetwork can thus be deployed to propose the optimizer for a given unseen task. The key technical contribution of this work includes the architectural design for this hypernetwork as well as a meta learning algorithm for training it. Empirical studies are conducted on BBOB benchmark as well as Bipedal Walker and Enduro dataset, where the proposed method achieves superior final results compared with several prior arts.", "questions": "On Bipedal Walker task, the proposed method seems to exhibit substantially higher variance compared with other baselines. Are there any intuitions for this behavior?", "limitations": "I did not seem to find any discussion on the limitations.", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The idea is straightforward yet grounded in the rich existing literature of hypernetwork in AutoML.\n2. The hypernetwork, trained in a meta-learning fashion, exhibits fairly consistent generalization ability on the task considered.\n3. Components of the proposed method are extensively studied.", "weaknesses": "1. Presentation: Some parts of this paper are a bit vanilla and takes some efforts to follow. For example, section 3 jumps right into detailing the architectural design. Providing high level overview as well as prioritizing the most important designs could benefit the readability of this work.\n2. Analysis. The empirical results suggest that the proposed method performs strongly on high dimensional scenarios, but it seems the paper did not provide the intuition behind why it is the case.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**weakness 1**\n\nThank you very much for your valuable suggestions. We think your suggestion can really help us enhance the readability of the paper. We have made the following changes to Section 3: 1) First, we briefly introduced the overall model architecture of POM and gave the overall model structure diagram; 2) Further, we introduced the core components of POM in detail. 3) Finally, we introduced the details of POM training and testing.\n\nI hope our changes will be recognized by you!\n\n\n\n**weakness 2**\n\nIn the paper, we have made a visual analysis of why POM achieves such excellent performance (see section 4.4). We found that both LMM and LCM can adaptively adjust their own optimization strategies to achieve a good balance between exploration and utilization. Specifically, in the LMM module, we found the following phenomenon (lines 284-289):\n\n*“ 1) Generally, superior individuals receive higher weights during LMM, showcasing POM's ability to balance exploration and exploitation as the population converges. 2) Across diverse function problems, POM dynamically generates optimization strategies, highlighting its adaptability and contributing to robust generalization. 3) Disadvantaged individuals exhibit a more uniform weight distribution, potentially aiding in their escape from local optima and enhancing algorithm convergence.”*\n\nIn the LCM module, we found the following phenomenon (see lines 291-299):\n\n*“LCM displays the capacity to adaptively generate diverse strategies for individuals across different ranks in the population, revealing distinct patterns among tasks and rankings. Notably, top-ranking individuals within the top 20, such as those ranked 1st, 5th, and 18th, exhibit a flexible crossover strategy. The dynamic adjustment of crossover probability with population evolution aids in preserving dominant genes and facilitating escape from local optima. Conversely, lower-ranking individuals show an increasing overall probability of crossover, promoting exploration of disadvantaged individuals and enhancing the algorithm's exploration capability. LCM proficiently generates adaptive crossover strategies across tasks, individuals, and convergence stages, significantly boosting both convergence and exploration capabilities.”*\n\nPOM has obtained strong optimization and generalization capabilities through pre-training, and has a very strong optimization efficiency. The performance of POM on all problems far exceeds the baselines of other manually designed optimization strategies (DE, ES, LSHADE, CMAES) and the baselines that obtain optimization capabilities through pre-training (LGA, LES), which proves the effectiveness and rationality of POM.\n\n\n\n**Question 1**\n\nThe Bipedal Walker task is a very difficult robot control task with very sparse reward signals. This task is very challenging. We can find that LES almost directly failed on this task, while DE, ES, LGA and LSHADE all showed premature convergence. These poor-performing algorithms all exhibit a common characteristic: small variance. Both POM and CMAES, which perform best on this task, exhibit significantly higher variance than other algorithms, which implies that stronger exploration capabilities are required to solve this task. For example, when CMA-ES encounters an evolutionary failure, it will increase the standard deviation of the search distribution and explore better points from a larger space; POM will adaptively adjust the optimization strategy. Among them, the mask module enhances the randomness of POM and avoids the LMM module's over-reliance on outstanding individuals in the population, thus improving the exploration ability of POM.\n\nWe did observe in the experiment that there are a large number of failed evolutions in the population evolution process. It may be that after many consecutive generations of attempts, a new and better individual will suddenly be found. This results in a strong randomness in the convergence process, so the convergence curves of CMA-ES and POM appear to have relatively large variances.\n\n**limitations**\n\nThank you very much for your comments! We have conducted a very detailed analysis of POM in the paper. We also pointed out the advantages and limitations of POM that we observed from the experiments. We have added a summary of limitations as follows:\n\n*\"limitations*\n\n1) *Model size: In the experiment, we found that the relationship between the model size and the performance of POM is not a strict linear relationship. Although the larger the model, the more difficult it is to train, there is still no very quantitative design criterion between model size, training data volume and training difficulty.*\n2) *Time performance: We introduced an operation similar to the attention mechanism, whose time complexity is $O(n^2)$, which makes POM require a lot of time cost when processing large-scale populations. How to reduce and improve the time efficiency of POM is also worthy of further study.*\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OeWdZu3FUG", "reviewer": "Reviewer_Se7V", "summary": "This paper focus on zero-shot black box optimization. They propose a pretrained optimization model (POM) to pretrain on a training dataset, achieving good results on BBOB benchmark and two robot control tasks. The authors design several parts including LMM and LCM for the POM to achieve the generation of sample strategy. The experiment results are comprehensive with the main experiment and some in-depth experiments.", "questions": "1. What will the results be when functions in BBOB dataset being rotated and shifted simultaneously? In the main experiment, the optimal points are all located in 0. Although in appendix there are results for shifted version, but rotation is also another usual operation to examine the robustness of some algorithms.\n2. How do you implement those baseline methods? It will be nice to mention this.\n3. For LES and LGA, was them being trained in the same dataset under the setting in this paper?", "limitations": "Please see weaknees and question part.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The design of the POM make sense in general. The authors draw inspiration from optimization process and hand-crafted a lot of design for the LMM and LCM.\n2. The training process make sense too. The construction of the training dataset consider the coverage of the landscape feature. The design of loss function is really instructive, considering both convergence and diversity.\n3. The algorithm present good experiment results in both synthetic BBOB dataset and robot control task, and achieve good results on both low and high dimensional scenario.", "weaknesses": "1. POM contains so many hand-crafted design, but are still under the scheme of mutate and then crossover, what's the motivation to decide this is the best design?\n2. From the 'Ablation Study', we learn that the `mask` is the most important design of the POM which is just a random process without any theoretical guarantee. This would raise the concern of the whole design of the framework because those hand-crafted parts aren't even more important than the random mask. Also the NOTRAIN one perform better when one of the core design been discarded, which is weird.\n3. More results can be presented. (e.g. there are only results for 100D when BBOB test with optimal solution disturbed)\n4. The training dataset is relatively simple, it's easy for a model to overfit to it.\n5. Related work can be more comprehensive. For example, for the part of 'LLM for Optimization', there are several works working in resolving black-box optimization problems using LLMs, such as https://arxiv.org/abs/2403.01131, https://arxiv.org/abs/2401.02051...). It would be nice to have a more comprehensive introduction to these related works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**weakness 1**\n\nWe do not claim that the current version of POM is the best design, but it does show excellent performance in experiments. We also do not emphasize that POM must be under the scheme of mutate and then crossover. We just design LMM and LCM modules for solution generation according to the needs of POM, and ensure that POM can be pre-trained end-to-end through gradient.\n\n**weakness 2**\n\nThe problems you mentioned just prove the rationality of the POM module design. \n\n1) mask\n\nThe reason for introducing `mask` is explained in detail in lines 121-126. Its function is to limit the information exchange within the population, prevent the population from quickly converging to the optimal individual, resulting in loss of diversity. This design is similar to the dropout operation of deep neural networks. Without the dropout operation, the neural network will easily fall into the local optimum and overfit on the training set.\n\nIt is worth noting that the role of `mask` is to make LCM and LMM work better. `mask` is like a catalyst, while LCM and LMM can be likened to reactants. We cannot say that the catalyst itself is more important than the reactants.\n\n2) `NOTTRAIN` and `NO LMM`\n\n`NOTRAIN` performs better than `NO LMM`, which means LMM is a very core component. If LMM is lost, the trained model performance is not as good as `NOTRAIN`.\n\n**weakness 3**\n\nThe experimental results of BBOB (d=100 ,optimal solutions disturbed), Bipedal Walker and Enduro proved the excellent performance of POM.  These experimental results have been able to verify the optimization and generalization capabilities of POM. We further show the experimental results of BBOB(d=500, disturbed).\n\n|      **F**       |         **GLHF**         |       **CMA-ES**       |       **LSHADE**       |\n| :--------------: | :----------------------: | :--------------------: | :--------------------: |\n|      **F1**      |  **1.29E+02(1.29E+02**)  |   6.12E+02(3.15E+01)   |   2.06E+02(2.18E+01)   |\n|      **F2**      |    5.22E+00(5.22E+00)    |   6.28E+01(3.25E+00)   | **3.13E+00(5.25E-01)** |\n|      **F3**      |    4.55E+03(4.55E+03)    |   8.93E+03(2.57E+02)   |   4.00E+03(1.15E+02)   |\n|      **F4**      |    9.98E+03(9.98E+03)    |   2.62E+04(3.08E+03)   | **8.82E+03(5.22E+02)** |\n|      **F5**      |    5.41E+03(5.41E+03)    | **1.43E+03(7.16E+01)** |   1.49E+03(6.53E+02)   |\n|      **F6**      |    3.00E+05(3.00E+05)    |   8.20E+05(1.09E+05)   | **2.66E+05(3.11E+04)** |\n|      **F7**      |  **1.19E+03(1.19E+03)**  |   8.06E+03(3.59E+02)   |   1.34E+03(2.63E+01)   |\n|      **F8**      |  **8.37E+05(8.37E+05)**  |   2.03E+07(3.52E+05)   |   1.79E+06(2.36E+05)   |\n|      **F9**      |  **3.24E+03(3.24E+03)**  |   1.76E+07(1.87E+06)   |   5.02E+05(3.04E+04)   |\n|     **F10**      |    4.03E+06(4.03E+06)    |   5.75E+07(5.25E+06)   | **3.51E+06(4.43E+05)** |\n|     **F11**      |  **4.28E+02(4.28E+02)**  |   6.48E+03(2.62E+02)   |   5.80E+02(1.90E+02)   |\n|     **F12**      |  **1.24E+09(1.24E+09)**  |   1.74E+10(1.02E+09)   |   2.28E+09(1.34E+08)   |\n|     **F13**      |  **1.12E+03(1.12E+03)**  |   2.48E+03(1.37E+02)   |   1.42E+03(4.37E+01)   |\n|     **F14**      |  **1.02E+01(1.02E+01)**  |   1.14E+02(1.16E+01)   |   1.64E+01(9.05E-01)   |\n|     **F15**      |  **4.32E+03(4.32E+03)**  |   9.31E+03(1.84E+02)   |   4.75E+03(3.34E+02)   |\n|     **F16**      |  **4.91E+01(4.91E+01)**  |   6.70E+01(1.73E+00)   |   5.79E+01(2.12E+00)   |\n|     **F17**      |  **2.19E+00(2.19E+00)**  |   1.06E+01(8.56E-01)   |   3.14E+00(1.84E-01)   |\n|     **F18**      |  **8.29E+00(8.29E+00)**  |   3.54E+01(2.41E+00)   |   1.29E+01(7.45E-01)   |\n|     **F19**      |  **7.72E+00(7.72E+00)**  |   1.03E+02(6.85E+00)   |   1.56E+01(8.92E-01)   |\n|     **F20**      | **-4.67E+00(-4.67E+00)** |   3.41E+05(2.53E+04)   |   5.10E+03(7.19E+02)   |\n|     **F21**      |  **7.79E+01(7.79E+01)**  |   8.61E+01(1.45E-01)   | **7.02E+01(3.09E+00)** |\n|     **F22**      |    8.10E+01(8.10E+01)    |   8.46E+01(1.08E+00)   | **7.08E+01(7.65E-01)** |\n|     **F23**      |    1.67E+00(1.67E+00)    |   1.94E+00(1.02E-01)   | **1.64E+00(1.29E-02)** |\n|     **F24**      |    7.34E+03(7.34E+03)    |   1.56E+04(3.21E+02)   | **7.31E+03(4.71E+02)** |\n| **win/tie/loss** |          -/-/-           |         22/1/1         |         14/2/8         |\n\nWe hope our reply will be recognized by you.\n\n**weakness 4**\n\nPOM does not overfit the training set, and experimental results show that POM exhibits generalization capabilities far beyond its training distribution.\n\n\n\n**weakness 5**\n\nThank you for your valuable comments. We have added these references in the Related Work section, which makes our related work more complete （see Section 2, LLM for Optimization）.\n\n“\n\n*LLaMoCo [38] and EoH [39] use LLM to generate code to solve optimization problems, but the performance of LLaMoCo depends on carefully designed instructions and prompts, and EoH has expensive evaluation costs.*\n\n”\n\n\n\n**Question 1**\n\nWe think your suggestion is very correct and reasonable. In fact, in all BBOB-related experiments, we performed corresponding projections or rotations on the objective function. We only removed the deviation from the optimal solution in the main experimental part.\n\n**Question 2**\n\nThe implementation of these baseline methods is described in detail in Appendix section D, and their parameter settings are described in detail in Appendix section E.\n\n**Question 3**\n\nThey were not trained on the same training set. We did not need to train LES and LGA, as they claim to be trained models that can be used out of the box. Second, the training frameworks of LES and LGA are not open source, so we could not train them. Third, the training task sets of LES and LGA contain many BBOB functions, which is beneficial to them. Even so, their performance is not as good as POM.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work considers zero-shot black box optimization with a population-based pretrained optimization model (POM). The reviewers appreciate the design and training process of POM under a meta-learning framework,  and the performance is promising.  It receives positive scores after the rebuttal discussions. AC agrees this is a nice progress on zero-shot black box optimization and recommends acceptance. This work can be further strengthened with an improved presentation, more analysis and justification on its performance especially under high-dimensional scenarios, and generalization studies on more realistic complex cases.
Decision: Accept (poster)