Pretrained Optimization Model for Zero-Shot Black Box Optimization

Authors: xiaobin  li
kai  wu
yujian betterrest li
xiaoyu  zhang
handing  wang
jing  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 10:40:28

Abstract: Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot optimization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose a Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions to zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation on the BBOB benchmark and two robot control tasks demonstrates that POM outperforms state-of-the-art black-box optimization methods, especially for high-dimensional tasks. Fine-tuning POM with a small number of samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimensions, population sizes, and optimization horizons. For code implementation, see https://github.com/ninja-wm/POM/.

Main paper: 
 introduction: 
 Black box optimization, including tasks like hyperparameter optimization (HPO) [1] , neuroevolution [2] [3] [4] , neural architecture search (NAS) [5] , and algorithm selection [6] , is very important. In these scenarios, the algorithm can evaluate f (x) for any solution x; however, access to additional information about f , such as the Hessian and gradients, is unavailable.Addressing diverse BBO problems necessitates the tailored design of specific algorithms to achieve satisfactory performance. Crafting these algorithms typically demands substantial expertise. Therefore, it is crucial to ensure reliable and robust performance of the optimizer in various applications, called zero-shot optimization. Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer.The studies [7] [8] [9] employed Transformer or diffusion models to pretrain model-based optimizers using offline datasets. While effective, these methods primarily fit optimization trajectories of other BBO algorithms to a specific task, potentially requiring retraining for new tasks, limiting their ability to zero-shot optimization. Subsequently, [10, 11] introduced two learned optimization frameworks for meta-learning evolution strategy (ES) and genetic algorithm (GA). However, the performance of these two methods on zero-shot optimization is weaker than that of CMA-ES [12] (see Section 4.2).To address zero-shot optimization, especially for continuous optimization, we introduce a populationbased Pretrained Optimization Model, called POM. Leveraging multiple individuals, populationbased optimizers gain a better understanding of the fitness landscape. The core of the optimizer is how to design optimization strategies that sample better solutions. Inspired by the solutionproducing mechanism of evolutionary computation, we design powerful POM blocks to form a general optimization strategy representation framework. Drawing inspiration from [13] , we introduce an end-to-end gradient-based training method for POM, termed MetaGBT (Meta Gradient-Based Training), ensuring stable and rapid training for POM. Pretraining POM on a set of training functions with MetaGBT ensures good optimization strategy. Our contributions can be summarized as follows:• Excellent ability to solve zero-shot BBO. We develop a efficient POM for zero-shot BBO, demonstrating a substantial performance advantage over state-of-the-art black-box optimizers. • Excellent ability to solve few-shot BBO. Few-shot optimization is the existence of a small budget of function evaluations for the target task to tune the optimizer for better performance. More than 30% performance improvement can be obtained with 25 random function evaluations. 
related work: 
 Heuristic Population-based BBO Algorithms. Numerous metaheuristic population-based algorithms, such as genetic algorithms [14] , evolution strategies [15] [16] [17] , particle swarm optimization [18, 19] , and differential evolution [20, 21] , have been devised to address optimization problems. Notably, CMA-ES [12] and L-SHADE [22] stand out as state-of-the-art methods for BBO. However, these approaches rely on manually designed components, exhibiting inefficiency and fragility when confronted with new tasks. In contrast, the proposed POM can autonomously acquire optimization strategies from problem instances, mitigating the aforementioned limitations.Pretrained Population-based BBO Algorithms. Pre-training BBO algorithms can be categorized into two types within the meta-learning framework. The first type frames meta-learning BBO algorithms as a bi-level optimization problem [23] . For instance, [24] leverages meta-learning to infer population-based black-box optimizers that automatically adapt to specific task classes. LES [25] designs a self-attention-based search strategy for discovering effective update rules for evolution strategies through meta-learning. Subsequent works like LGA [10] utilize this framework to discover the update rules of Gaussian genetic algorithms via Open-ES [26] . The second type models the meta-learning of a BBO algorithm as a reinforcement learning problem. [27] meta-learn a policy that adjusts the mutation step-size parameters of CMA-ES [12] . Category one faces the curse of dimensionality, where an escalating number of model parameters leads to skyrocketing training difficulty, impeding the development of intricate strategies. In contrast, category two, which models meta-learning optimizers as reinforcement learning tasks, grapples with training instability. POM, employing a gradient-based end-to-end training approach, successfully bypasses the curse of dimensionality, ensuring stable training.LLM for Optimization. In line with POMs, various optimization approaches leveraging Large Language Models (LLMs) have emerged to address diverse problem domains, including NP-hard problems [28, 29] , algorithm evolution [30] [31] [32] [33] , reward design [34] , and Neural Architecture Search (NAS) [35, 36] . Notably, LLMs play a role in sampling new solutions. However, their optimization strategies depend on externally introduced natural selection mechanisms and are less effective in numerical optimization scenarios [37] . LLaMoCo [38] and EoH [39] use LLM to generate code to solve optimization problems, but the performance of LLaMoCo depends on carefully designed instructions and prompts, and EoH has expensive evaluation costs. TNPs [40] , ExPT [41] and LICO [42] use transformer structures to solve the BBO problem and have achieved good results. However, TNPs requires contextual information of the target problem, and neither ExPT nor LICO can be directly used to solve tasks with different dimensions from the training task. These methods lack the universal applicability as pretrained BBO models due to a deficiency in generating capabilities across tasks.All the above methods cannot be the zero-shot optimizer. The first two categories need to adjust the hyperParameters when optimizing the new tasks, while the latter must fine-tune the instructions to achieve satisfactory results.3 Pretrained Optimization Model 
problem definition: 
 A black-box optimization problem can be transformed as a minimization problem, and constraints may exist for corresponding solutions:min x f (x), s.t. x i ∈ [l i , u i ], where x = (x 1 , x 2 , • • • , x d )represents the solution of optimization problem f , the lower and upper bounds l = (l 1 , l 2 , • • • , l d ) and u = (u 1 , u 2 , • • • , u d ), and d is the dimension of x. For more background information on evolutionary algorithms, see Appendix A.Definition 1 Zero-shot Optimization. Zero-shot optimization refers to an optimizer that is applied directly to solve a continuous black-box optimization problem f without any tuning. This means that the optimizer does not require any contextual information about f and can be directly used to handle problems of any dimensionality.Definition 2 Few-shot Optimization. Alternatively, it is permissible to fine-tune the optimizer using a small portion of the function evaluation budget for the objective task, and then use the fine-tuned optimizer to solve f .In this section, we use Differential Evolution (DE) as an example to review classic evolutionary algorithms. DE [20, 43] is a prominent family within evolutionary algorithms (EAs), known for its advantageous properties such as rapid convergence and robust performance [44, 45] . The optimization strategy of DE primarily involves mutation and crossover operations.The classic DE/rand/1 crossover operator is illustrated in Eq. (1) (additional examples are listed in Appendix A.2). Each mutation strategy can be viewed as a specific instance of Eq. ( 2); Further details are provided in Appendix A.2. Additionally, we represent the mutation strategy in a matrix form, as shown in Eq. ( 3). The matrix S evolves with the generation index t, indicating that the mutation strategy adapts across different generations. Consequently, we propose a module to enhance the performance of the mutation operation, which leverages the information from the population of the tth generation to generate S t . This serves as the motivation for our design of the LMM.EQUATIONIn the crossover phase at step t, DE uses a fixed crossover probability cr t i ∈ [0, 1] for each individual x t i in the population, as shown in Eq. ( 9). The crossover strategy for the entire population can then be expressed as a vectorcr t = (cr t 1 , cr t 2 , • • • , cr t N ).Our goal is to design a module that adaptively generates cr t using the information from the population. This approach allows for the automatic design of the crossover strategy by controlling the parameter cr. This serves as the motivation for our design of LCM.A population consists of n individuals, denoted asX = {x 1 , x 2 , • • • , x n }. In this paper, X is also treated as X = [x 1 , x 2 , • • • ,x n ] T to support matrix operations. We feed POM an initial random population X 0 at step 0, specify the evolution generation T for it, and hope that it can generate a population X T close to the global optimum at step T , as shown in X T = P OM (X 0 , T |θ), where θ ∈ Ω is the parameters of POM, where Ω stands for the strategy space. The goal of training POM is to find an optimal θ in Ω. As shown in Fig. 1 , POM consists of LMM, LCM and SM.LMM LMM generates candidate solutions v t i for individual x t i through Eq. ( 2), which enables the population information to be fully utilized in the process of generating candidate solutions v t i .EQUATIONFurther, we organize Eq. ( 2) into a matrix form, as shown in Eq. (3).EQUATIONX t ∈ R N ×d is the population in generation t and S t ∈ R N ×N . S evolves with each change in t, signifying a mutation strategy that adapts across generations. Consequently, it is imperative to devise a module that leverages information from the population at generation t to generate S t . Any mutation operator of differential evolution, such as the classic DE/rand/1 mutation operator, can be converted into Equation (3) in the specific case of S (see Appendix A.2 for details). At the same time, the crossover operation of GAs can also be generalized into the form of Equation ( 3) [46] .The function of LMM is designed based on Multi-head self-attention (MSA) [47] , as shown as follows:S t = LM M (H t |θ 1 ) (4) where θ 1 = {W m1 , W m2 , W m3 , b m1 , b m2 , b m3 } denotes the trainable parameters within LMM, while H t = [h t 1 , h t 2 , • • • , h t N ]serves as LMM's input, encapsulating population information. Each h t i incorporates details about x t i , encompassing: 1) f t i : the normalized fitness f (x t i ) of x t i ; 2) rt i : the centralized ranking of x t i . The method for calculating f t i is:EQUATIONwhere µ t and σ t denote the mean and standard deviation, respectively, of individual fitness values within the population at time t. We build rt i as follows:EQUATIONwhere rank yields the ranking of x t i within the population X t , with values ranging from 1 to N . Thus, LMM utilizes information on the relative fitness of individuals to dynamically generate the strategy Ŝt . rt i serves as position encoding, explicitly offering the ranking information of individuals. Equation (7) details the computation of Ŝt .EQUATIONwhere Tanh is an activation function.W m1 ∈ R 2×dm and W m2 , W m3 ∈ R dm×dm . b m1 , b m2 , and b m3 are vector with dimension d m . Ĥt ∈ R N ×dm , Q t , K t ∈ R N ×dm , and Ŝt ∈ R N ×N .The topological structure of the population significantly influences their information exchange [48] .When all individuals engage in information exchange, the algorithm's convergence may suffer, diversity could diminish, and susceptibility to local optima increases. To address this, we introduce a mask operation during both training and testing phases, where the probability of setting each element in Ŝt to 0 is r mask . This operation enhances POM's ability to learn efficient and robust strategies, as validated in our experiments. Consequently, S t is derived using Eq. (8) .EQUATIONFinally, we get V t via Eq. (3).LCM For each individual x t i at step t, a crossover probability cr t i ∈ [0, 1] is established. Consequently, the population's crossover strategy is encapsulated in the vector cr t = (cr t 1 , cr t 2 , • • • , cr t N ). The crossover operation, as depicted in Eq. ( 9), can be elucidated as follows:EQUATIONThe module design should facilitate the adaptive generation of cr t by leveraging population information. Executing the crossover operation with cr t yields Ut = [u t 1 , u t 2 , • • • , u t N ]. LCM is designed based on FFN [47] , as shown in Eq. (10) ,EQUATIONwhereθ 2 = {W c1 , b c1 , W c2 , b c2 , τ } is the parameter of LCM and Z t ∈ R N ×3is the population information used by LCM. Here,Z t = [z t 1 , z t 2 , • • • , z t N ]. z t i represents the relevant information of individual x t i and X t . For example, it can include the ranking information of x t i , the fitness information of x t i , the Euclidean distance between x t i and V t i , and the distribution information of individuals within the population (such as the fitness distribution, the distance between pairs of individuals), etc. In this paper, z t i includes the following information as a case study: 1) f t i : the normalized fitness f (x t i ) of x t i ; 2) rt i : the centralized ranking of x t i ; 3) sim t i : the cosine similarity between x t i and v t i .h t = T anh(Z t ×W c1 +b c1 ), ĥt = layernorm(h t |τ ), cr t = Sigmoid( ĥt ×W c2 +b c2 ) (11)where the activation function Sigmoid maps inputs to the range (0, 1). W c1 ∈ R 3×dc , W c2 ∈ R dc×1 , τ is the learnable parameters of layernorm [49] . b c1 and b c2 are vectors with dimensions d c and 1, respectively.Although we derive cr t from Eq. ( 11) as in Eq. ( 9), the discrete nature of the crossover operator renders it non-differentiable, impeding gradient-based training of the LCM module. To address this limitation, we introduce the gumbel_softmax method [50] , providing an efficient gradient estimator that replaces non-differentiable samples from a categorical distribution with differentiable samples from a novel Gumbel-Softmax distribution.Eq. ( 12) shows how to perform crossover operations between x t i andEQUATIONFirst, the rand function samples uniformly from the range [0, 1] to obtain a vector r t i . Then get cr t i from cr t according to the index. The tile function expands cr t i into a d-dimensional vector:[cr t i , cr t i , • • • , cr t i ].The cat function concatenates them into a matrix as shown below:r t i,1 r t i,2 • • • r t i,d cr t i cr t i • • • cr t i (13)Here, gumbel_softmax is executed column-wise. For any column, the larger element becomes 1 after gumbel_softmax and 0 otherwise. Therefore, cv t i ∈ R 2×d may be a matrix like this: Here, T is the number of iterations of the testing process and f is the target task. f does not have to be differentiable. Here we directly apply the trained POM to solve f without requiring gradient information.cv t i = 1 0 0 0 1 1 • • • 1 1 0 1 1 1 0 0 • • • 0 0 (14) LMM LCM SM POM X t H t S t V t U t X t+1 Generate H f Generate Z Z t X (a) POMOverall Framework We design LMM and LCM to achieve the generation of sample strategy (that is, generate S t ) and crossover strategy (that is, generate cr t ), respectively. The overall architecture of POM is shown in Fig. 1 . The parameters that need to be trained in POM are θ = {θ 1 , θ 2 }. At time step t, the population is X t . Initially, we amalgamate the information from X t to construct descriptive representations of the population, H t and Z t . LMM adaptively generates S t based on H t .The multiplication of X t and S t yields V t (see Eq. ( 3)). Next, LCM adaptively generates cr t based on its input Z t , and performs a crossover operation based on cr t to obtain U t . Finally, SM [51] , a 1-to-1 selection strategy is executed between U t and X t to produce the next-generation populationX t+1 . X t+1 = SM (X t , U t ) = tile(l x>0 (M F ′ -M F )) ⊙ X t + tile(1 -l x>0 (M F ′ -M F )) ⊙ U t (15)where l x>0 (x) = 1 if x > 0 and l x>0 (x) = 0 if x < 0, and the tile copy function extends the indication matrix to a tensor with size (N, d), M F (M F ′ ) denotes the fitness matrix of X t (U t ), and ⊙ indicates the pairwise multiplication between inputs.Algorithm 2 Driving POM to Solve Problem Input: T , n, training set T S. Output: The optimal θ. 1: Randomly sample the parameter θ of POM. 2: while not done do3: Sample |T S| populations of size n to ob- tain [X 0 1 , X 0 2 , • • • , X 0 |T S| ].4:for i = 1, 2, . . . , |T S| do 5:Randomly sample ω i for the f i in T S.end for 7:for t = 1, 2, . . . , T do 8:for i = 1, 2, . . . , |T S| do 9:EQUATION).10: Construct H t based on X t and f . 5:loss t i ← l i (X t i , X t-1 i , f i , ω i , λ).S t ← LM M (H t |θ 1 ). 6: V t ← S t × X t . 7:Build Z t based on X t , V t and f . 8:cr t ← LCM (Z t |θ 2 ). 9:Construct U t using Equation (12) . 10:X t+1 ← SM (X t , U t ). 11: end forPOM is meticulously crafted as a model amenable to end-to-end training based on gradients. While POM necessitates gradient information from the training task during the training phase, it exhibits the ability to tackle BBO problems in the testing phase without relying on any gradient information. To ensure the acquisition of an efficient, highly robust, and broadly generalizable optimization strategy, POM undergoes training on a diverse set of tasks. Training on these tasks sequentially poses the risk of domain overfitting, local optima entrapment, and diminished generalization performance. Consequently, we introduce a training methodology named MetaGBT.Tasks. We form a training task set T S = {f i (X|ω j )},where i ∈ [1, 5] and j ∈ [1, N ], comprising 4N tasks derived from Table 3 in appendix, where ω i denotes the task parameter influencing the function's landscape offset. Our selection of these functions for the training task is motivated by their diverse landscape features. The specific landscape features encompassed in T S are detailed in Appendix B.Loss Function. To avoid bias of different output scales in TS, for any function f i in T S, we design the normalized loss function l i (X t , X t-1 , f i , ω i , λ). In Equation ( 16), l 1 i calculates the average fitness difference between the input and output of the POM, further normalized within [0, 1]. This encourages convergence of the algorithm. l 2 i uses standard deviation to simulate the distribution of the output population, encouraging diversity in the output population. std(X t , j) is the standard deviation of the jth dimension of the population. λ is a hyperparameter, and we find that setting it to 0.005 can make model training more stable.X t = P OM (X t-1 , 1|θ)EQUATION)MetaGBT. The pseudocode for MetaGBT is presented in Algorithm 1. Initially, we sample the POM parameter θ from a standard normal distribution. The objective of MetaGBT is to iteratively update θ to bring it closer to the global optimum θ * . In line 2, we sample a population for each task in TS. Lines 3, 4 and 5 involve the resampling of task parameters for all tasks in TS, thereby altering the task landscape, augmenting training complexity, and enhancing the learning of robust optimization strategies by POM. The final loss function (line 10) is determined by computing the average of the loss functions for all tasks. Subsequently, in line 12, we update θ using a gradient-based optimizer, such as Adam [52] . The trained POM is then ready for application in solving an unknown BBO problem, as depicted in Algorithm 1. 4 Experiments 
experimental setup: 
 We test the performance of POM on the widely used BBO benchmark and two complex real-world problems (see Appendix C). Selected methods include DE (DE/rand/1/bin) [54] and ES ((µ,λ)-ES) as population-based baselines, L-SHADE [22] and CMA-ES [12] as state-of-the-art population-based BBO methods, and LES [25] and LGA [10] as state-of-the-art POMs. POM is trained on T S with T = 100, n = 100, and d = 10. Detailed parameters for all compared methods are provided in Appendix E. Please refer to Appendix D for the reasons for choosing these algorithms.BBOB [55] . We evaluate the generalization ability of POM across 24 BBOB functions with dimensions d = 30 and d = 100, where optimal solutions are located at 0. Figure 2 presents the critical difference diagram comparing all algorithms (refer to Appendix Tables 4 and 6 , and Figures 11, 12 and 13 for detailed results). POM significantly outperforms all methods, showcasing its efficacy across varying dimensions. Despite being trained solely on TF1-TF4 with d = 10, POM excels in higher dimensions (d = {30, 100, 500}), with its performance advantage becoming more pronounced with increasing dimensionality. Particularly on complex problems F21-F24, where global structure is weak, POM lags behind LSHADE but surpasses other methods, attributed to its adaptability through fine-tuning. TurBO [56] is the Bayesian optimization algorithm with the best performance on BBOB [57] . Under little budget conditions, the performance of POM outperforms that of TurBO in most cases (see Appendix G for details).Bipedal Walker [58] . The Bipedal Walker task involves optimizing a fully connected neural network with d = 874 parameters over k = 800 time steps to enhance robot locomotion control. In Fig. 3 (a), LSHADE shows ineffectiveness, while CMA-ES, LSHADE, and LGA suffer from premature convergence. Conversely, POM achieves stable and swift convergence, ultimately attaining the highest score.Enduro [58] . Enduro task entails controlling a strategy with d = 4149 parameters across k = 500 steps, posing greater difficulty than Bipedal Walker. As depicted in Fig. 3(b),  LGA and LES exhibit premature convergence and limited exploration. While CMA-ES initially converges slightly faster than POM, the latter maintains a superior balance between exploration and exploitation, outperforming LSHADE. Ablation Study The ablation study results for the designed modules are presented in Fig. 4 (a) (refer to Appendix Table 7 for additional details). Configurations include UNTRAINED, representing an untrained POM with randomly initialized parameters; NO LMM, where the LMM is excluded, and a simple DE/rand/1/bin mutation operator is employed; NO LCM, indicating the absence of the learnable crossover operation, using only binomial crossover; and NO MASK, signifying the omission of the mask operation described in Eq. ( 8).While UNTRAINED yields optimal results for F9 and F16, as an untrained POM is inherently an optimization strategy, the adaptability of trained POM surpasses the baselines in most scenarios. In simpler tasks with d = 30, UNTRAINED underperforms, demonstrating the advantage of trained POM on more complex tasks. Notably, NO LMM and NO LCM excel on F5, F11, and F19, respectively. This could be attributed to potential overfitting of POM to the relatively simple training set. The exclusion of mask operation (NO MASK) significantly diminishes POM's performance, highlighting the importance of the mask for global information sharing and population interaction, crucial for maintaining diversity. All modules contribute to POM's overall performance, with the negative impact on POM's performance ranked as follows: NO MASK > NO LMM > UNTRAINED > NO LCM.We evaluate the fine-tuned POM's performance on T F 6-T F 8 as detailed in Appendix Table 3 . We replace LCM with a standard transformer encoder to obtain more stable experimental results. For x i and v i , we normalize their features and then concatenate them by dimension to obtain xv i ∈ R d×2 . xv i and the normalized [fitness, ranking] information of x i in the parent population are concatenated to obtain xvf i ∈ R (d+1)×2 . Based on this input, the transformer encoder will generate cv i in Eq. ( 12). Different numbers of ω are generated as fine-tuning samples for T F 6-T F 8, and Algorithm 1 is used to fine-tune POM for each function. The base POM is initially trained on T F 1-T F 5. We calculate the relative performance improvement (RFI) achieved by the fine-tuned POM compared to the base POM, with results displayed in Figure 6 . Experimental results indicate that fine-tuning POM leads to significant performance improvements even with a small sample size. The method for obtaining fine-tuning samples is not restricted; for black-box tasks, a surrogate model can be constructed to facilitate fine-tuning.Any complex problem can be simulated by a polynomial composed of simple basic function terms. To ensure that the optimization strategy learned by POM has robust generalization ability and performance, we should train POM on a set of basic functions. First, we tested the impact of increasing the number of basic functions in the training set on model performance. Next, we examined the effect of introducing complex functions into the training set. Functions T F 1 -T F 5 are basic simple terms. For example, T F 1 is an absolute value term, and T F 5 is a square summation term. Functions T F 6 -T F 8 are composite terms composed of several basic functions. For instance, T F 6 includes both a cumulative multiplier term and a cosine term. The test results are shown in Figure 5 (a) and (b) (see Appendix Table 8 for details), respectively.Experimental results indicate that increasing the number of basic functions leads to an overall improvement in POM performance, whereas the introduction of composite terms results in a significant performance decline. This aligns with our hypothesis.We explore the performance of POM at different scales, which is shown in Fig. 4 (b) (refer to Appendix Table 9 for additional details). We increase POM's parameter count by perturbing the hidden layers of each module (d m , d c ). Six models are constructed in ascending order of parameter count, labeled as VS (very small), S (small), M (medium), L (large), VL (very large), and XL (extra large) (details in the Appendix Table 2 ). XL achieves the best performance, while VS and M also perform well. S exhibits the worst performance, and VL performs worse than L. Two core factors contribute to this phenomenon: the number of parameters and training. We observe a complex relationship between the number of parameters and training difficulty. VS, with the fewest parameters, is the easiest to train and performs well on BBOB. Conversely, XL, with a large number of parameters, exhibits the strongest capability to represent strategies, resulting in the best performance. The performance of XL aligns with our expectations. We obtain the following principles: 1) Larger models can have stronger capabilities but are more challenging to train; 2) Training difficulty and model scale do not exhibit a simple linear relationship, warranting further research; 3) Larger models require more functions for effective training.We assess the training and test time efficiency of POM across various architectures on BBOB (d = 10) and BBOB (d = 100) respectively, as illustrated in Figure 7 . POM demonstrates remarkable efficiency in tackling BBO problems, with negligible training costs relative to its exceptional generalization ability and high performance.LMM Learning Analysis Figure 8 displays S t for an in-depth analysis of the LMM strategy (refer to Appendix Figure 15 -20 for additional details). Key observations and conclusions include: 1) Generally, superior individuals receive higher weights during LMM, showcasing POM's ability to balance exploration and exploitation as the population converges. 2) Across diverse function problems, POM dynamically generates optimization strategies, highlighting its adaptability and contributing to robust generalization. 3) Disadvantaged individuals exhibit a more uniform weight distribution, potentially aiding in their escape from local optima and enhancing algorithm convergence.We visually examine the LCM strategy, presenting the results in Fig. 9 (refer to Appendix Figure 21 -26 for additional details). LCM displays the capacity to adaptively generate diverse strategies for individuals across different ranks in the population, revealing distinct patterns among tasks and rankings. Notably, top-ranking individuals within the top 20, such as those ranked 1st, 5th, and 18th, exhibit a flexible crossover strategy. The dynamic adjustment of crossover probability with population evolution aids in preserving dominant genes and facilitating escape from local optima. Conversely, lower-ranking individuals show an increasing overall probability of crossover, promoting exploration of disadvantaged individuals and enhancing the algorithm's exploration capability. LCM proficiently generates adaptive crossover strategies across tasks, individuals, and convergence stages, significantly boosting both convergence and exploration capabilities. 
conclusions: 
 We present POM, a novel Pretrained Optimization Model designed to address the inefficiencies of existing methods in zero-shot optimization. Evaluation on BBOB and robot control tasks demonstrates POM's superiority over other black-box optimizers, particularly in high-dimensional scenarios. Additionally, POM excels in solving few-shot optimization problems. Future research avenues include designing enhanced loss functions to optimize POM for both population convergence and diversity, thereby improving overall algorithm performance. In addition, the limitations of model scale and time performance deserve further study (see Appendix I for details). 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: Yes, we provide device resources information and time analysis (see section 4).Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: See the appendix J for specific content.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We respect the relevant original author and the open source agreement.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: The crossover, mutation, and selection operators form the basic framework of GAs. GA starts with a randomly generated initial population. Then, genetic operations such as crossover and mutation will be carried out. After the fitness evaluation of all individuals in the population, a selection operation is performed to identify fitter individuals to undergo reproduction to generate offspring. Such an evolutionary process will be repeated until specific predefined stopping criteria are satisfied.Crossover The crossover operator generates a new individual x c i ∈ R d by Eq. ( 17), and cr is the probability of the crossover operator.EQUATION)wherek ∈ [1, • • • , d], i and j ∈ [1, 2, . . . , n] (i ̸ = j).d represents the dimension of the problem. x i k and x j k represent the k-th element of x i and x j respectively. This operator is commonly conducted on n individuals; n represents the population size. After an expression expansion, we reformulate Eq. (17) asn i=1 x i W c i [46]. x i ∈ R d represents the ith individual in X, where X = {x 1 , x 2 , • • • , x n } is a population. W c i ∈ R d×d is the diagonal matrix. If W c i is full of zeros, the ith individual has no contribution.Mutation The mutation operator brings about random changes in the population. Specifically, an individual x i in the population goes through the mutation operator to form the new individual x m i , formulated as follows:EQUATION)where mr is the probability of mutation operator and k∈ [1, • • • , d].x m k and x c k represent the k-th element of x m and x c respectively. Similarly, Equation ( 18) can be reformulated asx c i W m i , where W m i ∈ R d×d is the diagonal matrix.Selection We introduce the binary tournament mating selection operator in Eq. (19) . The selection operator survives individuals of higher quality for the next generation until the number of individuals is chosen. As shown in Eq. (19) ,EQUATIONwhere p i reflects the probability that x i is selected for the next generation, andX m = {x m 1 , x m 2 , • • • , x m n }.The core components of the optimization model include modules that generate solutions and modules that select solutions. GA and DE basically include crossover modules, mutation modules and selection modules. The evolutionary strategy represented by CMA-ES needs to sample a population from a certain distribution (such as Gaussian distribution), and further select individuals to update this distribution. In this paper, we design parameterized trainable LMM and LCM as modules for generating solutions. The function of LMM is to generate a candidate population, and LCM further performs crossover between the candidate population and the original population to obtain the offspring population.We list some classic DE mutation strategies.• DE/rand/1EQUATION• DE/best/1EQUATION• DE/current-to-rand/1EQUATION• DE/current-to-best/1EQUATION• DE/current-to-pbest/1EQUATIONThe integer index r1 (and similarly, r2 and r3) is randomly selected from the range [0, N ]. pbest is randomly selected from the indices of the best p individuals. x t best is the individual with the best fitness in the population at generation t.The generalized form of the mutation strategy isEQUATIONFor example, when w i,q = 1, w i,k = -w i,j ̸ = 0, and w i,l = 0 (∀l / ∈ {q, j, k}, q ̸ = k, k ̸ = j, q ̸ = j), it becomes DE/rand1/1. If individuals of the population has been sorted from good to bad by fitness, when w i,0 = 1, w i,k = -w i,j ̸ = 0, and w i,l = 0 (∀l / ∈ {0, j, k}, k ̸ = j), it becomes DE/best/1.The landscape features included in T S are shown as follows:• TF1: Unimodal BBOB [59, 55] is a widely researched and recognized collection of benchmark test problems to evaluate the performance of optimization algorithms. The dataset consists of a series of highdimensional continuous optimization functions, including single-peak, multi-peak, rotated, and distorted functions, as well as some functions with specific properties such as Lipschitz continuity and second-order differentiability.We test the performance of POM on two complex robot control tasks.The continuous control task Bipedal Walker [58] , implemented within the Box2D physics engine, has been designed to test the ability of walking agents to navigate varying terrain by controlling their joints and maintaining balance. The challenge requires the agent to learn efficient walking strategies that enable it to traverse the intended path without falling or deviating from its trajectory. The robot's state comprises a range of variables, including the hull angle speed, angular velocity, horizontal speed, vertical speed, joint positions and angular speeds, legs contact with the ground, and lidar rangefinder measurements. The robot's actions involve determining motor speed values in the range of [-1, 1] for each of the four joints at the hips and knees. The performance of the agent is evaluated through a reward system, whereby it receives points for moving forward, with a maximum of 300+ points awarded upon successfully reaching the end of the designated course. However, the penalty of -100 points is imposed if the robot loses balance and falls. Furthermore, applying motor torque incurs a small cost in terms of points. The score accrued by the agent serves as a measure of its optimal performance. The Bipedal Walker task represents a challenging and dynamic environment that effectively evaluates the walking and balance control abilities of agents. As such, it provides a valuable benchmark for testing and comparing different reinforcement learning algorithms for robotic locomotion.Enduro [58] is one of the classic reinforcement learning environments provided by OpenAI Gym. It is a driving racing game based on the Atari 2600 game. In this environment, your goal is to drive as far as possible by controlling the car. The Enduro game is set on an endless highway where you need to avoid other vehicles and overtake as many other vehicles as possible within a limited time. You can avoid collisions with other vehicles by moving your car left and right, and be careful to control your speed to avoid accidents. The game rewards you based on how far you drive, so your goal is to learn a good driving strategy to maximize the distance traveled.In these two test tasks, the agent interacts with the environment for k time steps, and the reward at the i-th step is r i . We evaluate strategy performance as follows:EQUATIONIn these two tasks we conduct 10 sets of experiments, each set of experiments consists of 5 independent runs. We finally take the best results of each set of experiments to calculate the mean and standard deviation.Our core is the population-based pre-training BBO algorithm, so we do not compare with nonpopulation methods such as Bayesian optimization methods. Moreover, Bayesian optimization methods are difficult to deal with continuous optimization problems of more than 100 dimensions. We do not use LLM-based approaches [28] [29] [30] [31] [32] [34] [35] [36] as baselines because they can only be used for a specific type of task.Heuristic Population-based BBO Algorithm. DE(DE/rand/1/bin) [54] , ES((µ,λ)-ES), L-SHADE [22] , and CMA-ES [12] , where DE [54] and ES are implemented based on Geatpy [60] , CMA-ES and IPOP-CMA-ES are implemented by cmaes2 , and L-SHADE is implemented by pyade 3 . The reasons for choosing these baselines are the following:• DE(DE/rand/1/bin): A classic numerical optimization algorithm.• ES((µ,λ)-ES): A classic variant of the evolution strategy.• CMA-ES: CMA-ES is often considered the state-of-the-art method for continuous domain optimization under challenging settings (e.g., ill-conditioned, non-convex, non-continuous, multimodal). • L-SHADE: The state-of-the-art variant of DE.Pretrained BBO Algorithm. We chose three state-of-the-art meta-learn BBO algorithms for comparison with POM.• LES [25] : A recently proposed learnable ES. It uses a data-driven approach to discover new ES with strong generalization performance and search efficiency. • LGA [10] : A recently proposed learnable GA that discovers new GA in a data-driven manner.The learned algorithm can be applied to unseen optimization problems, search dimensions, and evaluation budgets. • We train POM on T S. During training, the maximum number of evolution generations is 100, n = 100 and the problem dimension is set to 10.The primary control parameters of CMA-ES and L-SHADE are automatically adjusted. For LGA and LES, we utilized the optimal parameters provided by the authors without modifications. Other hyperparameters were tuned using grid search to identify the optimal combinations, and multiple experiments were conducted accordingly. Detailed parameter settings are presented in Table 1 . Each experiment reports the mean and standard deviation of the results from various sets of experiments, with a consistent population size of 100 across all trials. All experiments are performed on a device with GeForce RTX 3090 24G GPU, Intel Xeon Gold 6126 CPU and 64G RAM. We use a grid search to determine this parameter, the search interval is [1, 10] , and the search step is 1.We use a grid search to determine this parameter, the search interval is [dup, ecs, etour, otos, rcs, rps, rws, sus, tour, urs] [60] .N sel = 0.5we determine this hyperparameter between [0.1, 0.8] using a grid search, with a step of 0.1. DE F = 0.5 we determine this hyperparameter between [0.1, 0.9] using a grid search, with a step of 0.1. [60] .XOV R = 0.5 we determine this hyperparameter between [0.1, 0.9] using a grid search, with a step of 0.1.LGAWe use the pre-trained optimal parameters provided by the authors.We use the pre-trained optimal parameters provided by the authors. We further test the performance of the algorithm on the BBOB. Here, the optimal solution of each function is randomly disturbed, that is, x * = x opt + z, where x * represents the optimal solution after disturbing, x opt represents the original optimal solution, x opt is a vector obtained by random sampling and z ∈ [-1, 1] d . The results are displayed in Table 5 and Figure 10 . We found that the performance of POM can still dominate other algorithms when the function optimal solution is disturbed. We compared POM and Bayesian optimization algorithms, finding that Bayesian optimization converges very slowly on high-dimensional problems. TurBO [56] , noted for its fast convergence and strong performance [57] , was used as a benchmark. Although TurBO requires substantial time for 10,000 evaluations, POM completes the same task in under one second. Therefore, we plotted the convergence curves of TurBO and POM with only 3,100 evaluations. As shown in Figure 14 , POM demonstrates significant performance advantages over TurBO in most cases. .12E+00(8.12E+00) 5.06E+00(5.06E+00) 9.64E+00(9.64E+00) 5.63E+00(5.63E+00) 5.62E+00(5.62E+00) F20 4.05E+02(4.05E+02) -2.75E+01(-2.75E+01) 1.56E+03(1.56E+03) -3.11E+00(-3.11E+00) -1.84E+01(-1.84E+01) F21 3.11E+01(3.11E+01) 2.20E+01(2.20E+01) 6.55E+01(6.55E+01) 6.45E+01(6.45E+01) 3.65E+01(3.65E+01) F22 6.31E+00(6.31E+00) 1.08E+01(1.08E+01) 5.10E+01(5.10E+01) 6.32E+01(6.32E+01) 3.86E+01(3.86E+01) F23 3.63E+00(3.63E+00) 3.49E+00(3.49E+00) 3.31E+00(3.31E+00) 3.23E+00(3.23E+00) 3.77E+00(3.77E+00) F24 3.55E+02(3.55E+02) 2.60E+02(2.60E+02) 3.98E+02(3.98E+02) 2.88E+02(2.88E+02) 3.41E+02(3.41E+02) .88E+00(7.88E+00) 7.28E+00(7.28E+00) 7.04E+00(7.04E+00) 7.07E+00(7.07E+00) 6.44E+00(6.44E+00) 6.57E+00(6.57E+00) 7.26E+00(7.26E+00) 7.28E+00(7.28E+00) F20 2.60E-01(2.60E-01) 1.33E+00(1.33E+00) 2.07E+00(2.07E+00) -1.54E+00(-1.54E+00) 0(0) 6.32E-01(6.32E-01) 1.95E+00(1.95E+00) 2.00E+00(2.00E+00) F21 6.45E+01(6.45E+01) 6.87E+01(6.87E+01) 5.72E+01(5.72E+01) 6.67E+01(6.67E+01) 5.94E+01(5.94E+01) 6.72E+01(6.72E+01) 6.09E+01(6.09E+01) 6.36E+01(6.36E+01) F22 7.55E+01(7.55E+01) 7.65E+01(7.65E+01) 7.62E+01(7.62E+01) 7.73E+01(7.73E+01) 7.38E+01(7.38E+01) 7.69E+01(7.69E+01) 7.62E+01(7.62E+01) 7.46E+01(7.46E+01) F23 5.21E+00(5.21E+00) 4.74E+00(4.74E+00) 5.20E+00(5.20E+00) 5.21E+00(5.21E+00) 5.19E+00(5.19E+00) 4.92E+00(4.92E+00) 5.08E+00(5.08E+00) 4.83E+00(4.83E+00) 9.54E-14(9.54E-14) F8 0.00E+00(0.00E+00) 6.76E-04(6.76E-04) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) F9 6.32E+02(6.32E+02) 6.42E+02(6.42E+02) 6.37E+02(6.37E+02) 6.27E+02(6.27E+02) 6.38E+02(6.38E+02) 6.38E+02(6.38E+02) F10 6.86E-01( 5.50E-14(5.50E-14) F19 6.22E+00(6.22E+00) 7.26E+00(7.26E+00) 6.59E+00(6.59E+00) 7.06E+00(7.06E+00) 7.34E+00(7.34E+00) 6.21E+00(6.21E+00) F20 -5.39E+00(-5.39E+00) -7.19E+00(-7.19E+00) -7.71E+00(-7.71E+00) 9.90E-01(9.90E-01) -4.40E+00(-4.40E+00) -1.94E+00(-1.94E+00) F21 5.24E+01(5.24E+01) 6.86E+01(6.86E+01) 5.46E+01(5.46E+01) 2.22E+01(2.22E+01) 6.43E+01(6.43E+01) 4.88E+01(4.88E+01) F22 7.24E+01(7.24E+01) 7.70E+01(7.70E+01) 7.30E+01(7.30E+01) 6.73E+01(6.73E+01) 7.65E+01(7.65E+01) 7.34E+01(7.34E+01) F23 5.06E+00(5.06E+00) 5.17E+00(5.17E+00) 5.15E+00(5.15E+00) 5.17E+00(5.17E+00) 5.42E+00(5.42E+00) 5.04E+00(5.04E+00) • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.F24 1.33E+03(1.33E+03) 1.41E+03(1.41E+03) 1.24E+03(1.24E+03) 1.26E+03(1.26E+03) 1.22E+03(1.22E+03) 1.31E+03(1.31E+03) 1.19E+03(1.19E+03) 1.31E+03(1.31E+03)Justification: Our research is in line with Neurips Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).https: //github.com/CyberAgentAILabhttps://github.com/xKuZz/pyade 


Decision: This work considers zero-shot black box optimization with a population-based pretrained optimization model (POM). The reviewers appreciate the design and training process of POM under a meta-learning framework,  and the performance is promising.  It receives positive scores after the rebuttal discussions. AC agrees this is a nice progress on zero-shot black box optimization and recommends acceptance. This work can be further strengthened with an improved presentation, more analysis and justification on its performance especially under high-dimensional scenarios, and generalization studies on more realistic complex cases.

Reviews: review: paper_summary: paper_summary: The paper introduces POM, a neural-network-based evolutionary algorithm for black-box optimization. POM is trained on diverse optimization tasks to enable adaptation to new tasks. POM outperforms the baselines on BBOB benchmark and two robot control tasks.
strength_weakness: strength_weakness: - The paper is missing some key references [1, 2, 3]. These methods learn from diverse tasks and are able to adapt to new tasks without any finetuning. This invalidates some of the claims made in the paper (lines 28-29, line 78)
- Section 3 is difficult to read without a background section on population-based optimization. It should describe a general evolutionary algorithm first and how POM parameterizes different components of the algorithm with a neural network.
- Section 3.2 is overly mathy with unnecessary equations. This distracts away from the main idea of the proposed model, which is using neural networks to parameterize evolution. For example, equations (6) and (10) are simply describing multi-head attention and feed-forward layers, which are very standard in deep learning and it's unnecessary to present the equations here. It is also redundant to specify each element of the network parameters $\theta_1$ and $\theta_2$, etc.
- Section 3.3 sounds difficult to believe. How can we expect the same set of parameters ($\theta_1$ and $\theta_2$) to learn from many tasks without feeding metadata of each task? For example, if two tasks have conflicting gradient signals, they get canceled out, and the model basically learns nothing. Specifically, for a certain $x_t$, for task $i$, the model should produce $x_{t+1}$, but for task $j$, the model should produce $x_{t+1}'$, and $x_{t+1}$ and $x_{t+1}'$ are conflicting, then how does the model learn in this case?
- Section 4 is confusing without any experiment setup information. Where are the zero-shot and few-shot settings mentioned in the introduction? How is POM adapted to a new objective after training? 
- POM has advantages over the baselines because it was pretrained while the baselines were not, so the better performance of POM is not surprising.

[1] Nguyen, Tung, and Aditya Grover. "Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling." International Conference on Machine Learning. PMLR, 2022.

[2] Nguyen, Tung, Sudhanshu Agrawal, and Aditya Grover. "ExPT: synthetic pretraining for few-shot experimental design." Advances in Neural Information Processing Systems 36 (2024).

[3] Nguyen, Tung, and Aditya Grover. "LICO: Large Language Models for In-Context Molecular Optimization." arXiv preprint arXiv:2406.18851 (2024).
questions: questions: - What do the authors mean by "zero-shot" optimization? Does it mean optimizing a new objective function without seeing any (x, y) pair from that function (is this even possible)? Or do the authors mean optimizing a new objective function without any gradient training/fine-tuning? If the latter is the case, then there already exists methods that can perform optimization without training/fine-tuning on the new objective (see my comments above).  
- How did the author decide on the input to LMM (H) and LCM (Z)?
limitations: limitations: See above.
paper_summary: paper_summary: This paper studies zero-shot optimizers for blackbox optimization problem. The core idea is to pretrain a hypernetwork that generate suitable optimization strategies on a subset of tasks; at test time, the hypernetwork can thus be deployed to propose the optimizer for a given unseen task. The key technical contribution of this work includes the architectural design for this hypernetwork as well as a meta learning algorithm for training it. Empirical studies are conducted on BBOB benchmark as well as Bipedal Walker and Enduro dataset, where the proposed method achieves superior final results compared with several prior arts.
strength_weakness: strength_weakness: 1. Presentation: Some parts of this paper are a bit vanilla and takes some efforts to follow. For example, section 3 jumps right into detailing the architectural design. Providing high level overview as well as prioritizing the most important designs could benefit the readability of this work.
2. Analysis. The empirical results suggest that the proposed method performs strongly on high dimensional scenarios, but it seems the paper did not provide the intuition behind why it is the case.
questions: questions: On Bipedal Walker task, the proposed method seems to exhibit substantially higher variance compared with other baselines. Are there any intuitions for this behavior?
limitations: limitations: I did not seem to find any discussion on the limitations.
paper_summary: paper_summary: This paper focus on zero-shot black box optimization. They propose a pretrained optimization model (POM) to pretrain on a training dataset, achieving good results on BBOB benchmark and two robot control tasks. The authors design several parts including LMM and LCM for the POM to achieve the generation of sample strategy. The experiment results are comprehensive with the main experiment and some in-depth experiments.
strength_weakness: strength_weakness: 1. POM contains so many hand-crafted design, but are still under the scheme of mutate and then crossover, what's the motivation to decide this is the best design?
2. From the 'Ablation Study', we learn that the `mask` is the most important design of the POM which is just a random process without any theoretical guarantee. This would raise the concern of the whole design of the framework because those hand-crafted parts aren't even more important than the random mask. Also the NOTRAIN one perform better when one of the core design been discarded, which is weird.
3. More results can be presented. (e.g. there are only results for 100D when BBOB test with optimal solution disturbed)
4. The training dataset is relatively simple, it's easy for a model to overfit to it.
5. Related work can be more comprehensive. For example, for the part of 'LLM for Optimization', there are several works working in resolving black-box optimization problems using LLMs, such as https://arxiv.org/abs/2403.01131, https://arxiv.org/abs/2401.02051...). It would be nice to have a more comprehensive introduction to these related works.
questions: questions: 1. What will the results be when functions in BBOB dataset being rotated and shifted simultaneously? In the main experiment, the optimal points are all located in 0. Although in appendix there are results for shifted version, but rotation is also another usual operation to examine the robustness of some algorithms.
2. How do you implement those baseline methods? It will be nice to mention this.
3. For LES and LGA, was them being trained in the same dataset under the setting in this paper?
limitations: limitations: Please see weaknees and question part.

Comments: title: Reply to new discussion (3/3)
Reply to new discussion (2/3)
Reply to new discussion (1/3)
Response to the Authors
Response to the Authors
comment: I wish to express my sincere appreciation for the invaluable assistance and dedication you have provided during the review process of my paper. Your insightful comments and thoughtful suggestions have significantly contributed to the development and refinement of my work. Your efforts have been of immense help in enhancing the quality and clarity of my research. Thank you for your generous support. It is greatly appreciated.
I thank the authors for the response. Your clarification has addressed my major concern about the conflicting issue. The detail that helped resolve my confusion was that LCM and LLM used the population's ranking and relative performance, which contain information about the task/objective (the same x can be ranked 1st for objective 1 but ranked last for objective 2). I increased my score from 3 to 6.

With regard to writing, I was aware of the differences in some design choices of the proposed method with standard layers, but I still believe these details should be deferred to later sections so that readers can appreciate the main idea.
## Q4: Can the authors explain why POM does not have the conflicting issue I mentioned in the original review...

 Conflicts between optimization objectives do not necessarily lead to conflicts in optimization strategies. POM focuses on learning the mapping from the ranking information of individuals within a population to the search strategies, rather than adopting a fixed optimization strategy to suit all tasks.

Specifically, the LMM module adaptively generates the strategy matrix $\mathbf{S}^t$ using the population's ranking information ($\hat{r}^t_i$) and the information representing their relative advantages $\hat{f}_i^t$ (normalized from $f(\mathbf{x}_i^t)$, representing individual $i$. Note that we can convert all problems to minimization problems by adding a negative sign).

The LCM module uses the population's ranking information ($\hat{r}^t_i$), the relative advantage information of individuals within the population $\hat{f}_i^t$, and the similarity information between parent and offspring $sim_i^t$ to adaptively generate the crossover strategy.

This means that POM is not fitting a specific problem $f$, nor is it searching for an optimal $\mathbf{x}^* \in \mathbb{R}^d$ in its solution space. Instead, it searches for optimal POM strategy parameters $\mathbf{\theta}^*$ in the strategy space through these training tasks. Therefore, even if there are conflicting objective functions, it does not affect the performance of POM. Furthermore, the training algorithm proposed by POM, MetaGBT, integrates gradient information from a set of tasks, making the training process more stable and easier.

Let's simulate the situation you mentioned. Suppose there is a function $f(\mathbf{x})=\sum_i{x_i}$, with task one being $f_1=\mathop{\arg\max}\limits_{\mathbf{x}}f(\mathbf{x})$ and task two being $f_2=\mathop{\arg\min}\limits_{\mathbf{x}}f(\mathbf{x})$. Clearly, these two tasks are in direct conflict.

Assuming that at the current moment, the populations for both tasks are the same, they are in the following state:

$$
\mathbf{X}=
\begin{bmatrix}
1 & 2 & 3 \\\\
4 & 5 & 6 \\\\
7 & 8 & 9 \\\\
1 & 3 & 4 \\\\
\end{bmatrix}
\mathbf{X}_1=
\begin{bmatrix}
7 & 8 & 9 \\\\
4 & 5 & 6 \\\\
1 & 3 & 4 \\\\
1 & 2 & 3 \\\\
\end{bmatrix}
\mathbf{X}_2=
\begin{bmatrix}
1 & 2 & 3 \\\\
1 & 3 & 4 \\\\
4 & 5 & 6 \\\\
7 & 8 & 9 \\\\
\end{bmatrix}
$$

$$
\mathbf{U}_1=\mathbf{CR}\cdot\mathbf{V}_1+(1-\mathbf{CR})\cdot\mathbf{X}_1=
 \left[
 \begin{matrix}
6.96 & 7.96 & 8.96 \\\\
4.54 & 5.55 & 6.55 \\\\
3.25 & 4.79 & 5.79 \\\\
4.51 & 5.65 & 6.64 \\\\
  \end{matrix}
  \right] 
$$

$$
\mathbf{U}_2=\mathbf{CR}\cdot\mathbf{V}_2+(1-\mathbf{CR})\cdot\mathbf{X}_2=
 \left[
 \begin{matrix}
1.02 & 2.03 & 3.03 \\\\
1.14 & 2.89 & 3.89 \\\\
2.68 & 3.83 & 4.83 \\\\
2.27 & 3.54 & 4.54 \\\\
  \end{matrix}
  \right]
$$

Finally, after performing selection between $\mathbf{X}$ and $\mathbf{U}$, the resulting population is:
$$
\mathbf{X}_1'=
 \left[
 \begin{matrix}
7.00 & 8.00 & 9.00 \\\\
4.54 & 5.55 & 6.55 \\\\
3.25 & 4.79 & 5.79 \\\\
4.51 & 5.65 & 6.64 \\\\
  \end{matrix}
  \right] \mathbf{X}_2'=
 \left[
 \begin{matrix}
1.00 & 2.00 & 3.00 \\\\
1.14 & 2.89 & 3.89 \\\\
2.68 & 3.83 & 4.83 \\\\
2.27 & 3.54 & 4.54 \\\\
  \end{matrix}
  \right]
$$

**The average fitness for Task 1 improved from 13.25 to 17.8175, demonstrating an enhancement in the performance of Task 1. Conversely, the average fitness for the population of Task 2 decreased from 13.25 to 8.9025, which also indicates an improvement in the performance of Task 2.**

**This shows that even with conflicting objective functions, Their optimization strategies do not necessarily conflict. This demonstrates that POM learns a universal optimization strategy, and there is no conflict as you claimed.**
## Q2: Moreover, the methodology section is a bit too verbal and mathy, and some equations are unnecessary. I do not see a difference between Eq (6) and (10) and the standard feed forward and attention layers.

In this section, we present detailed formulas to ensure the reproducibility of POM, which are also crucial for elucidating the mechanism of POM. We will then discuss in detail the differences between these formulas and the standard self-attention and feed-forward layers.

### The difference between Eq (6) and standard self-attention

**A. Self-Attention**

1. **Input Transformation**:

   $$ \mathbf{Q}= \mathbf{X}\mathbf{W}^Q, \quad \mathbf{K} = \mathbf{X}\mathbf{W}^K, \quad \mathbf{V} = \mathbf{X}\mathbf{W}^V$$

2. **Attention**:

   $$\text{Self-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}}\right)\mathbf{V}$$

3. **Output**:

   $$\text{Output} = \text{Self-Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})$$

**B. Eq. (6)**
1. **Input Transformation**:
$$
\mathbf{\hat{H}}^t=\text{Tanh}(\mathbf{H}^t\mathbf{W}_{m1}+\mathbf{b}\_{m1}),\quad \mathbf{Q}^t=Tanh(\mathbf{\hat{H}}^t\mathbf{W}\_{m2}+\mathbf{b}\_{m2}), \quad \mathbf{K}^t=Tanh(\mathbf{\hat{H}}^t\mathbf{W}\_{m3}+\mathbf{b}\_{m3})
$$
2. **Attention**:

   $$\text{LMM-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{X})= \text{mask}\left(\text{softmax}\left( \text{Tanh}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}}\right)\right),r_{\text{mask}}\right)\mathbf{X}$$

3. **Output**:

   $$\text{Output} = \text{LMM-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{X})$$

The differences between Eq (6) and standard self-attention are evident as follows:

1. **No Calculation of V**: In Eq (6), we do not compute $\mathbf{V}$; instead, we use $\mathbf{X}$ in the attention calculation. This approach is crucial for achieving good performance in our optimization task, whereas standard self-attention struggles to adapt to our problem scenario.

2. **Different Processing of Input Information**: We introduce the Tanh activation function and a bias term, which helps enhance the performance of POM. This is a significant departure from the standard self-attention mechanism.

3. **Attention Matrix Mapping and Masking**: For the computed attention matrix, we apply the Tanh function to map it to the range $[-1,1]$. Additionally, we use a random mask operation, setting elements to zero with a probability of $r_{mask}$.

In summary, there are several key differences between our approach and standard self-attention. These differences are crucial for the performance of POM, and it is essential to detail them thoroughly. This is not a shortcoming but rather a necessary distinction.

### 2 Eq (10) and feed forward layer
**A. feed forward**
$$\text{Output} = \mathbf{W}_2 \cdot \text{ReLU}(\mathbf{W}_1 \cdot \mathbf{X} + \mathbf{b}_1) + \mathbf{b}_2$$

**B. Eq (10)**
$$\mathbf{cr}^t = \text{sigmoid}\left(\text{layernorm}\left(\tanh\left(\mathbf{Z}^t \times \mathbf{W_{c1}} + \mathbf{b_{c1}}\right) | \mathbf{\tau}\right) \times \mathbf{W_{c2}} + \mathbf{b_{c2}}\right)$$


It is worth noting that, although the two are very similar in form, Eq (10) provides a detailed implementation scheme. Any modifications to Eq (10), such as replacing the sigmoid function with ReLU, result in a significant decline in POM's performance. This is because we rely on the sigmoid function to map the output to the range $[0,1]$, representing probability values. Changes to other activation functions or hyperparameters in Eq (10) also lead to performance degradation.

## Q3: Regarding the baselines, TNP, ExPT, and LICO can all be used for zero-shot optimization because they do not require further fine-tuning after pretraining. However, they do require the new objective to lie in the same dimension/domain as the pretraining tasks.

It is clear that TNP, ExPT, and LICO require retraining in different dimensional scenarios when faced with tasks of varying dimensions. This does not align with the definition of Zero-shot Optimization, which is "an optimizer that can be directly applied to solve a continuous black-box optimization problem $f$ without any tuning."


***"Definition 1 Zero-shot Optimization** : Zero-shot optimization refers to an optimizer that is applied directly to solve a continuous black-box optimization problem $f$ without any tuning. This means that the optimizer does not require any contextual information about 
$f$ and can be directly used to handle problems of any dimensionality."*
First and foremost, I would like to express my sincere gratitude for your comments and suggestions regarding our manuscript submitted to NeurIPS 2024.  We have diligently revised our paper in accordance with your feedback.

We earnestly hope that you will reconsider our revised manuscript and appreciate the improvements made. If you find that our efforts and the results now meet the standards of the conference, we kindly request that you consider increasing the score of our paper. We highly value your assessment and look forward to your final approval. 

Thank you once again for your valuable time and expertise.

## Q1: I still believe the writing can be significantly improved for...

Thank you for your valuable suggestions, which have helped us improve the expression of our paper. To better introduce the background knowledge and clarify the motivation behind the design of POM, we have followed your advice and added the following content in Section 3 before detailing the mechanism of POM (see Section 3.2 for details).

"
***3.2 Classic Population Optimization Algorithm***
*In this section, we use Differential Evolution (DE) as an example to review classic evolutionary algorithms. DE [20,43] is a prominent family within evolutionary algorithms (EAs), known for its advantageous properties such as rapid convergence and robust performance [44,45]. The optimization strategy of DE primarily involves mutation and crossover operations.*

*The classic DE/rand/1 crossover operator is illustrated in Eq. (1) (additional examples are listed in Appendix A.2). Each mutation strategy can be viewed as a specific instance of Eq. (2); Further details are provided in Appendix A.2. Additionally, we represent the mutation strategy in a matrix form, as shown in Eq. (3). The matrix $\mathbf{S}$ evolves with the generation index *t*, indicating that the mutation strategy adapts across different generations. Consequently, we propose a module to enhance the performance of the mutation operation, which leverages the information from the population of the *t*th generation to generate $\mathbf{S}^t$. This serves as the motivation for our design of the LMM.*

$\mathbf{v}\_i^t = \mathbf{x}\_{r1}^t+F\cdot(\mathbf{x}\_{r2}^t-\mathbf{x}\_{r3}^t)$  &nbsp;&nbsp;&nbsp; (1)

*In the crossover phase at step *t*, DE uses a fixed crossover probability $cr_i^t$ ∈ [0,1] for each individual $\mathbf{x}_i^t$ in the population, as shown in Eq. (9). The crossover strategy for the entire population can then be expressed as a vector $\mathbf{cr}^t = (cr_1^t, cr_2^t, ..., cr_N^t)$. Our goal is to design a module that adaptively generates $\mathbf{cr}^t$ using the information from the population. This approach allows for the automatic design of the crossover strategy by controlling the parameter *cr*. This serves as the motivation for our design of LCM.*
I thank the authors for the response. 

I still believe the writing can be significantly improved for better clarity. The main text should provide sufficient background and motivate the approach well before delving into the technical details. Moreover, the methodology section is a bit too verbal and mathy, and some equations are unnecessary. I do not see a difference between Eq (6) and (10) and the standard feed forward and attention layers.

Regarding the baselines, TNP, ExPT, and LICO can all be used for zero-shot optimization because they do not require further fine-tuning after pretraining. However, they do require the new objective to lie in the same dimension/domain as the pretraining tasks. I do not think the other papers the authors mentioned are relevant to this work.

Can the authors explain why POM does not have the conflicting issue I mentioned in the original review? Imagine there are two tasks in pretraining that have opposite objectives, I do not see how POM can learn a general optimization strategy from both tasks that have conflicting gradient information without any contextual information from each task.

I will keep my original score for now.
We hope that our responses have adequately addressed the concerns you previously raised. Your further insights would be greatly appreciated, as they are instrumental in enhancing the quality of our work. We understand that you are busy, and we truly value the time and effort you put into this process. Thank you in advance for your continued support.
We hope that our responses have adequately addressed the concerns you previously raised. Your further insights would be greatly appreciated, as they are instrumental in enhancing the quality of our work. We understand that you are busy, and we truly value the time and effort you put into this process. Thank you in advance for your continued support.
Thank you immensely for your valuable recognition and support towards our work. Your insightful suggestions have not only guided us but also significantly contributed to the enhancement of our paper. We sincerely appreciate your time and effort in helping us refine our research.
Thanks the authors for replying, all of my concern has been addressed. Good work and I would like to raise my score to 7.
**Q1**

We deeply apologize for the oversight in the configuration of our experimental code, which led to an error where the standard deviations were identical in the experimental results provided during the rebuttal period. We have rerun the experiment and obtained the correct mean values (and standard deviations), as shown in the table below. We are pleased to report that the performance of the POM method remains significantly superior to the best baselines, which is consistent with our previous conclusions. We are very grateful for the careful review by the reviewers.

|      **F**       |         **POM**         |       **CMA-ES**       |       **LSHADE**       |
| :--------------: | :---------------------: | :--------------------: | :--------------------: |
|      **F1**      | **1.29E+02(2.60E+00)**  |   6.12E+02(3.15E+01)   |   2.06E+02(2.18E+01)   |
|      **F2**      |   5.75E+00(4.71E-01)    |   6.28E+01(3.25E+00)   | **3.13E+00(5.25E-01)** |
|      **F3**      |   4.72E+03(4.77E+01)    |   8.93E+03(2.57E+02)   | **4.00E+03(1.15E+02)** |
|      **F4**      |   1.02E+04(7.66E+02)    |   2.62E+04(3.08E+03)   | **8.82E+03(5.22E+02)** |
|      **F5**      |   5.46E+03(7.62E+01)    | **1.43E+03(7.16E+01)** |   1.49E+03(6.53E+02)   |
|      **F6**      |   3.18E+05(3.59E+04)    |   8.20E+05(1.09E+05)   | **2.66E+05(3.11E+04)** |
|      **F7**      | **1.13E+03(3.76E+01)**  |   8.06E+03(3.59E+02)   |   1.34E+03(2.63E+01)   |
|      **F8**      | **8.62E+05(2.71E+04)**  |   2.03E+07(3.52E+05)   |   1.79E+06(2.36E+05)   |
|      **F9**      | **3.24E+03(1.72E-01)**  |   1.76E+07(1.87E+06)   |   5.02E+05(3.04E+04)   |
|     **F10**      |   3.53E+06(3.87E+05)    |   5.75E+07(5.25E+06)   | **3.51E+06(4.43E+05)** |
|     **F11**      | **4.25E+02(2.61E+01)**  |   6.48E+03(2.62E+02)   |   5.80E+02(1.90E+02)   |
|     **F12**      | **1.19E+09(6.29E+06)**  |   1.74E+10(1.02E+09)   |   2.28E+09(1.34E+08)   |
|     **F13**      | **1.13E+03(1.48E+01)**  |   2.48E+03(1.37E+02)   |   1.42E+03(4.37E+01)   |
|     **F14**      | **1.02E+01(2.52E-01)**  |   1.14E+02(1.16E+01)   |   1.64E+01(9.05E-01)   |
|     **F15**      | **4.15E+03(9.83E+01)**  |   9.31E+03(1.84E+02)   |   4.75E+03(3.34E+02)   |
|     **F16**      | **4.67E+01(1.61E+00)**  |   6.70E+01(1.73E+00)   |   5.79E+01(2.12E+00)   |
|     **F17**      | **2.18E+00(2.41E-02)**  |   1.06E+01(8.56E-01)   |   3.14E+00(1.84E-01)   |
|     **F18**      | **8.40E+00(8.93E-02)**  |   3.54E+01(2.41E+00)   |   1.29E+01(7.45E-01)   |
|     **F19**      | **7.62E+00(3.10E-01)**  |   1.03E+02(6.85E+00)   |   1.56E+01(8.92E-01)   |
|     **F20**      | **-3.43E+00(3.29E+00)** |   3.41E+05(2.53E+04)   |   5.10E+03(7.19E+02)   |
|     **F21**      |   7.76E+01(2.88E-01)    |   8.61E+01(1.45E-01)   | **7.02E+01(3.09E+00)** |
|     **F22**      |   8.07E+01(3.29E-01)    |   8.46E+01(1.08E+00)   | **7.08E+01(7.65E-01)** |
|     **F23**      | **1.60E+00(7.20E-02)**  |   1.94E+00(1.02E-01)   |   1.64E+00(1.29E-02)   |
|     **F24**      | **7.31E+03(2.04E+02)**  |   1.56E+04(3.21E+02)   |   7.31E+03(4.71E+02)   |
| **win/tie/loss** |          -/-/-          |         22/1/1         |         14/3/7         |

**Q2**

We did not utilize the default projections and rotations provided in the original problem set because we aimed to test the model's performance on a broader range of scenarios on the BBOB benchmark. Consequently, we strictly adhered to the standard BBOB procedure to generate a series of BBOB function parameters, including projections and rotations.
Thanks for the response from the authors, the response addressed most of my questions. 
Two further minor questions are as follows,
1. In the newly provided results on BBOB 500D, data in the columu GLHF seems a bit weird that the optimal solutions and the standard deviation are the same.
2. For the BBOB testsuits, do you mean that you use the default projections and rotations provided in the original problems set?