SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
Authors: Jiacheng Chen
Zeyuan Ma
Hongshu Guo
Yining Ma
Jie Zhang
Yue-Jiao Gong
Venue: ICLR 2024 poster (2024)
Keywords: Black-Box Optimization
Meta-Black-Box Optimization
Deep Reinforcement Learning
Symbolic Equation Learning
Primary area: optimization
TL;DR: We propose SYMBOL, a novel framework that promotes the automated discovery of state-of-the-art black-box optimizers through symbolic equation learning.

Abstract: Recent Meta-learning for Black-Box Optimization (MetaBBO) methods harness neural networks to meta-learn configurations of traditional black-box optimizers. Despite their success, they are inevitably restricted by the limitations of predefined hand-crafted optimizers. In this paper, we present SYMBOL, a novel framework that promotes the automated discovery of black-box optimizers through symbolic equation learning. Specifically, we propose a Symbolic Equation Generator (SEG) that allows closed-form optimization rules to be dynamically generated for specific tasks and optimization steps. Within SYMBOL, we then develop three distinct strategies based on reinforcement learning, so as to meta-learn the SEG efficiently. Extensive experiments reveal that the optimizers generated by SYMBOL not only surpass the state-of-the-art BBO and MetaBBO baselines, but also exhibit exceptional zero-shot generalization abilities across entirely unseen tasks with different problem dimensions, population sizes, and optimization horizons. Furthermore, we conduct in-depth analyses of our SYMBOL framework and the optimization rules that it generates, underscoring its desirable flexibility and interpretability.

Published as a conference paper at ICLR 2024
SYMBOL: GENERATING FLEXIBLE BLACK-BOX OPTI- MIZERS THROUGH SYMBOLIC EQUATION LEARNING
Jiacheng Chen' Zeyuan Ma'.*, Hongshu Guo", Yining Ma², Jie Zhang", Yue-Jiao Gong". 1.1
South China University of Technology 2 Nanyang Technological University Jackchan9345, scut. parazynioolss guohongshu369) gmail 1 com, fyining.mn ma, zhangj]entu. edu. sg gongyue j ji amail.........a com
ABSTRACT
Recent Meta-learning for Black-Box Optimization (MetaBBO) methods hamness neural networks to meta-learn configurations of traditional black-box optimizers. Despite their success, they inevitably restricted by the limitations of predefined hand-crafted optimizers. In this paper, we present SYMBOL, a novel framework that promotes the automated discovery of black-box optimizers through symbolic equation learning. Specifically, we propose Symbolic Equation Generator (SEG) that allows closed-form optimization rules to be dynamically generated for spe- cific tasks and optimization steps. Within SYMBOL, we then develop three distinct strategies based on reinforcement learning, so as meta-learn the SEG efficiently. Extensive experiments reveal that the optimizers generated by SYMBOL not only surpass the state-of-the-ar BBO and MetaBBO baselines, but also exhibit excep- tional zero-shot generalization abilities across entirely unseen tasks with different problem dimensions, population sizes, and optimization horizons. Furthermore. conduct in-depth analyses of our SYMBOL. framework and the optimization rules that it generates, underscoring its desirable flexibility and interpretability.
INTRODUCTION
Black-Bol BBx Optimization (BBO) pertains to optimizing an objective function without knowing its mathematical formulation, essential in numerous domains ranging from automated machine learn- ing (Akiba et al., 2019) to bioinformatics (Tsaban et al., 2022). Due to the black-box nature, op- timizing BBO tasks requires search process that iteratively probes and evaluates candidate solu- tions. While several well-known BBO optimizers, such as Differential Evolution (Storn & Price, 1997), Evolutionary Strategy (Hansen Ostermeier, 2001), and Bayesian Optimization (Snock et al., 2012b) have been proposed to steer the search process based on hand-crafee cand-ed rules, they often necessitate intricate manual tuning with deep expertise to secure the desirable performance for particular class of BBO tasks.
To mitigate the labour-intensive tuning required for BBO optimizers, recent Meta- Meta-Black-Bii Black-Box Op- timization (MetaBBO) methods have turned to meta-learning (Finn et al., 2017) to automate such process. the left portion of Figure 1, we depict existing prevailing BBO (in the first block) and MetaBBO (in the remaining blocks) paradigms. Essentially, MetaBBO follows the bi-level optimization and can be categorized into two branches. The first branch is the MetaBBO for auto- configuration. As shown in the second block, at the meta level, neural policy dictates configura- tions for the lower-level BBO optimizer, and this lower-level optimizer typically employs an expert- derived formulation; Together, they strive optimize meta-objective excel across broader range of BBO tasks. While they offer a pathway to data-centric optimization, the fine-tuned opti- mizer closely follows the designs of the backbone BBO optimizer, thereby inheriting the potential limitations of human-crafted rules. More recently, some MetaBBO methods employ neural net- works directly propose subsequent candidate solutions shown in the third block. however, these models may easily get overfitted and their efficiency may be limited to low- -indeinnsiioal BBOs only.
"Equal Contribution "orresponning Author

---

Published as a conference paper at ICLR 2024
Ganaal weekloow erBBO optizaiza
Traditional 3B0 optimaze
Sangle tokk
Ran lower Jevel optiniaaion
Mita tnis SEG parss
x
curteed
rila
MetBBO Codf-CCCoooinii
Upcas nic
Stupo--
MauRiO Candisbne Solutin Propuoal
Tince Lanisces
Lowerr 20
Nexra
Asobes
Miu JaSerance
Mat pelfi
No
Cambnac
unnal popuamon (0)
Stze repoittatiss
SYNIDOL
auubele splacede
adooek
Necval Recaooll
x" +'-0.18
Cenented (201io))
0.18
0.2xtx x3
WorkDow of ditferent parkligms
SYMBOL as MenBBB limmwork
Figure 1: Comparison of traditional BBO, MetaBBO, and our SYMBOL frameworks. Left: Depic- tion of various BBO methods. Right: A view the bi-level structure of SYMBOL. The lower level optimizes individual BBO tasks, with the SEG generating step-by-step update rules. The meta level compiles performances from the lower level into a meta objective so as to learn the SEG through either the SYMBOL-E (without teacher BBO optimizer), SYMBOL-G, or SYMBOL-S strategy.
Meanwhile, while they attempt to bypass hand-crafted rules, they shift towards becoming "black box" optimization systems, raising concerns about interpretability and generalization.
In this paper, address the above challenges by presenting SYMBOL, novel MetaBBO frame- work designed to autonomously generate symbolic equations for black-box optimizer learning. As shown in the final block on the left side of Figure 1, our proposed SYMBOL considers generating update rules expressed as closed-fore form equations, which not only allows SYMBOL to minimize its dependence
hand-crafted optimizers, enhancing flexibility for discovering novel BBO optimiz- ers, but also to exhibit much better interpretability. Specifically, our SYMBOL framework, illustrated in the right portion of Figure 1, follows the bi-level structure of MetaBBO. It features a Symbolic- Generator (SEG) at the lower level to dynamically generate symbolic update equations for optimizing given BBO task and given optimization step. The generated rules then determine the next generations of candidate solutions (or the next population in BBO parlance). At the meta level, the SEG is trained to maximize the meta-objective across various BBO tasks.
Nevertheless, achieving the above flexible and interpretable framework certainly comes with its challenges. To this end, we introduce the following novel designs: 1) At the lower level, our SEG autoregressively generates closed-form equations with each symbol drawn from basis symbol set rooted in first principles; concurrently, we also design constant inference mechanism for incorpo- rating constants in the equations; 2) At the meta level, put forth three unique SYMBOL training strategies to meta-learn SEG, namely the Exploration strategy (SYMBOL-E), which initiates the learning from scratch; the Guided strategy (S MBOL-..,) which harnesses a state-of-the-art BBO optimizer as a teacher for behavior cloning: and the Synergized strategy (S (SYMBOL-S), which amal- gamates the previous two for effective training: 3) Lastly, to bolster zero-shot generalization, our SEG further combines fitness landscape analysis with robust contextual representation, while our SYMBOL leverages a diverse training set that encompasses various landscape properties.
Through benchmark experiments, we validate the effectiveness of our SYMBOL framework, demon- strating its efficacy in discovering optimizers that surpass the existing hand-crafted ones. Moreover, SYMBOL stands out with its zero-shot generalization ability. matching the performance of well- known SMAC optimizer (Lindauer et al., 2022) but with faster speed on the entirely unseen BBO tasks such as hyper-parameter optimization and protein docking. We also discuss the optimization rules that our SYMBOL generates. Lastly, we conduct extensive analyses of SYMBOL regarding its teacher options, hyper-parameter sensitivities, and ablation studies. Our main contributions are four folds: 1) The introduction of SYMBOL, a pioneering generative MetaBBO framework for automated optimizer discovery; 2) The SEG network, designed to effectively generate closed-form BBO equa- tions; 3) Three training strategies based on reinforcement learning for different application needs; and 4) New state-of of-he-aftt
ahe-art Me performance complemented by in-depth discussions.
2

---

Published as a conference paper at ICLR 2024
2
RELATED WORKS
Traditional black-box optimizer. Traditional black-box optimizers seek global optima without relying on gradient information, with population-based algorithms being particularly dominant. Algorithms like Genetic Algorithm (GA) (Holland, 1992). Differential Evolution (DE) (Storn & Price, 1997), Particle Swarm Optimization (PSO) (Kennedy & Eberhart, 1995) focus on the inner- population evolution, ensuring convergence of individual solutions within the population towards the global optima. Others, like Evolution Strategy (ES) (Hansen et al., 2003), focus on refining the popu- lation distribution based on statistics. To further enhance performance, adaptive strategies have been developed, including dynamic population size strategy (Biswas et al., 2021; Auger & Hansen, 2005), multi-operator selection strategy (Brest et al., 2021), adaptive control parameter strategy (Sarker et al., 2013), etc. Besides population- bbased methods, Bayesian Optimization (BO) (Snoek et al., 2012a; 2015) leverages Gaussian Processes and posterior acquisition to discern optimal solutions. Among these, SMAC (Lindauer et al., 2022) makes advances with a racing mechanism, setting the state-of-the-art performance in autoMI. hyper-parameter optimization (Arango et al., 2021).
Meta-learned black-box optimizer. Initial MetaBBO research utilizes traditional black-box op- timizers at the meta level to determine proper configurations for lower-level optimizers, where a single configuration is usually generated for the entire optimization process (Zhao et al., 2023). Re- cent MetaBBO meta-learns the configurations or update rules of black-box optimizers using a neural network, aiming to generate more flexible update rules dynamically. Typically, they consider fine- tuning the configurations of backbone BBO optimizer, e.g., auto-selecting evolution operators and hyper-parameters for the evolutionary optimizers (Sharma et al., 2019; Sun et al., 2021; Lange al., 2023; Chaybouti et al., 2022; Lange et al., 2022; G.Shala et al., 2020; Li et al., 2023; Wu et al., 2023). Though boosting the performance, they remain constrained by the inherent limitations of the backbone optimizer. Another way is to directly suggest the next candidate solution(s) through neural networks (Chen et al., 2017; TV et al., 2019). These end-to-end models can match the performance of tfat-oof-the-at black-box optimizers in diwdeniniionll tasks (S 6). Yet, for larger scales, their efficiency can diminish. Furthermore, their shift towards becoming "black box" systems raises in- terpretability concerns. Meanwhile, we note that many of the above MetaBBO works may still fail to generalize across different task distributions (Sharma et al., 2019), dimensions (Chen et al., 2017; TV et al., 2019; Chaybouti et al., 2022: Cao al.. 2019), population sizes (Sun et al., 2021). or optimization horizons (Chen et al., 2017: TV et al., 2019). Symbolic-L20 (Zheng et al., 2022) and Lion (Chen et al., 2023), using symbolic learning, show a certain similarity with our SYMBOL in concept. But it focuses on discovering gradient descent optimizers, whereas SYMBOL is designed to devise novel update rules for a range of BBO tasks.
Symbolic equation learning. Symbolic equation learning aims to discover mathematical expres- sion that best fits the given data, which is a core concept in the realm of Symbolic Regression (SR). Traditional SR algorithms utilize genetic programming (GP) to evolve and optimize sym- bolic expressions (Schmidt & Lipson, 2009), whereas recent studies embrace the assistance of neural-guided search methodologies, such utilizing Recurrent Neural Network (RNN) com- bined with reinforcement learning, as in (Petersen et al., 2021). In this approach, RNN is used to autoregressively infer a function f. and the reward could be calculated as the error between the ground truth fxx and the predicted values f (z)). The constants in f are local-searched by the Broy- Fledcher-Gooldfrrbb-------- Shannoo (BFGS) FR.elethher 1970). performing iterative evaluations and adjustments. In our proposed SYMBOL, we favor using RNN to formulate symbolic update rules over resorting to GP-driven search, primarily due to two reasons: 1) the intensive computational demands of GPs, and 2) its pronounced sensitivity to parametr choices (Biggio et al., 2021).
FeenFFeethhr-CCe Goldfarb
METHODOLOGY
Our SYMBOL framework, akin to MetaBBO, follows bi-level optimization. Starting with initial population (a set of solutions), the Symbolic Equation Generator (SEG) at the lower level dynam- ically formulates the update rules, ((1), for advancing the populations (current solutions) from (()) to ((++1) using the formula (!(+1) (() -((), Here, we use represent the optimization step (generation) and we depict the whole process in the bottom right of Figure 2. In the rest of Figure 2, we provide an example of how SEG autoregressively generates such update rules 7(*) based on a basis symbol set & (at the top) and the on-the-fly constant value inference mechanism (highlighted
3

---

Published as a conference paper at ICLR 2024
Dasis symbol 5
operators:
operands:
The generated symtballc the pre-crder Gaversal
Sampled toleen T,
Infor the ernenete
Toko dis- Iribution through
consiant value
interpreted updatt rule 0.8 xp
Noa
LSTM
The luwer level updalle process
PIE
P7S
Pepulbttio
FIA
Symbolic
SEG
expressioon
sull root
exponeor
Jaenesate
0.8
Advance
The Symboole Equation Gensralor (SEG)
population
The illustration of our symbolic equation generator (SEG) at the lower level. Left: An example of generating the update rule via SEG with on- on-the-fly inference of constant (highlighted in red dashed block). Top right: The symbolic expression tree from the pre-order traversal and the interpreted update rule Bottom right: illustration of the overall iterations of SEG.
Figure 2:
in the middle). At the meta level, the SEG is trained via three different strategies of SYMBOL to maximize the meta-objective over task distribution.
3.1 SYMBOLIC EQUATION GENERATOR
Our SEG leverages the structure of symbolic expression trees to represent analytical equations (Pe- tersen et al., 2021), with internal nodes denoting operators (e.g., +) and terminal nodes signifying operands (e.g., constants). Starting with empty expression tree, it involves autoregressively se- lecting a token from a basis symbol sei S, so as to construct the pre-order traversal of this symbolic expression tree. The traversal can then be uniquely interpreted into the corresponding update rule with a closed form. In this paper, we adopt T to denote the generated update rulc, its symbolic expression tree, and the corresponding pre-order traversal, interchangeably. We denote Ti the 7th token of r and the depth of the expression tree as H.
We parameterize the SEG with LSTM, denoted 6, to autoregressively construct update rules token by token. In each construction step, we input the LSTM with Vectorized Tree Embedding (VTE) of the partially constructed expression tree, complemented by features originating from Fit- ness Landscape Analysis (FLA) as the initial cell state of the LSTM so as to capture the current optimization status. The output would be a categorical distribution over all the available tokens from the basis symbol set S, where each token is sampled from the distribution, during both training and inference. This process terminates either when all terminal nodes are assigned as operands or when the depth of the tree, H, exceeds predetermined limit. The likelihood of the pre-order traversal, т, the cumulative product of the probabilities of selecting each token, i.e., P(T = II p(Ta P(Ta/TT:((-1)) 0). More details are provided as follows.
Basis symbol set S. Following the first principle, we identify an essential symbol S, i.c., where we divide them into operators and operands. 1) Oper- ators: + and x. 2) Operands: x- 'candidate solutions', x² and x: 'the best and worst solutions found so far in the whole population', 2; 'the best-so-far solution for the 7th candidate', Ar 'the differential vector (velocity) between consecutive steps of candidates', 2,- randomly selected candidate from the present population of candidates', and e 'constant values". Such basis sym- bol set covers update rules for known optimizers such as DE (Storn & Price, 1997) with mutation strategy (I,ι 2) + (x,,2 Ir3), PSO (Kennedy & Eberhart, 1995) with update formula T cl x Ar + c2 x (x* z) c3 x (xt - .)), etc. More importantly, S could also span beyond the existing hand-crafted space, offering flexibility explore novel and effective rules. Note that the de- sign of S needs to balance representability and learning effectiveness, simply removing/augmenting symbols from/to S may impact the final performance, which is discussed in Appendix C.2.
{+, - x, xx, x=, xx; Ar, cr, ch,
Constant inference. Upon the selection of the constant token for T,, we infer its value immediately by processing the subsequent LSTM hidden states h through two distinct feed-forward layers.
4

---

Published as a conference paper at ICLR 2024
Specifically, any constant is defined as combination of its mantissa w and exponent e, represented as c = w x 10", where we stipulated that e E {0, -11 and w E (-1.0, -0.9, -0.8, ..., 0.8, 0.9. 1.0). The process is shown in Figure 2, highlighted in the red dashed block. Note that such an inference mechanism for constants grants SYMBOL. the versatility to fine-tune the relative emphasis of differ- segments within update rule via sampling from various exponent levels.
Masks for streamlined equation generation. Following past works (Biggio et al., 2021; Valipour al., 2021; Vastl et al., 2022), we introduce several restrictions simplify symbolic equation generation: 1) The children of binary operators must not both be constants as the result is simply another constant; 2) The generated expressions must not include consecutive inverse operation, e.g., +I 2: 3) The children of the operator + must not be the same since it is equivalent to the operator x; 4) The operator x must have one constant child because two-variable multiplication merely appears in any black-hox optimizer; 5) The height H of the symbolic expression tree is empirically set to be between 2 and 5, which aligns with most of traditional black-box optimizers. Given the above restrictions, we mask out the probability of selecting any invalid token O.
Contextual state representation. To inform SEG of sufficient contextual information for gen- erating update rules, our state representation includes two aspects: 1) Vectorized Tree Embed- ding (VTE), which embeds the partially constructed symbolic expression tree created thus far into vector; and 2) Features derived from Fitness Landscape Analysis (FLA), which profiles the opti- mization status for the current optimization step. Our designed FLA features consist of three parts: 1) Distributional features of the current population, e.g. average distance between any pair of indi- viduals, aiding in assessing the distribution and diversity of candidates; 2) Statistic features of the current objective values, e.g, average objective value gap between an individual and the best-so-far solution, providing insights into the quality of optimization progress; 3) Time stamp embedding, e.g. the number of consumed generations, offering temporal perspective on the tasks' evolution. Collectively, FLA features convey the optimization status of current population on the given task, constantly informing the SEG an inductive bias on generating update rule for next generation. Both the leveraged VTE and FLA features are task-agnostic and dimension-agnosse SYMBOL to generalize across diverse tasks. More details on the state representation and the SEG network architecture can be found in Appendix I, and Appendix A.2, respectively.
which allows our
3.2 THREE STRATEGIES FOR TRAINING THE SEG
SYMBOL aims to meta-learn the SEG (parameterized by 8) by op-
timizing the accumulated meta-objective over task distribution D. as shown in Eq. (1). Given a BBO problem f sampled from ID, in the form of the Markov Decision Process, the SEG receives an optimization status as state and generates a symbolic update rule (()) as action to update the candidate population at each gener- ation L. For each generation
Norn. Perfeemacce
0.96
0.30-
0.04
0.30
0.92
Synmor-G
Syamoo E S
measure how well the gener- ated 7(()) performs on the sampled f by meta- veta-performance met- ric R(7((), f) reward. We denote the update rules trajectory generated for I as v (1(1), r(T),, where T denotes the maximum number of generations for the lower-level optimization
Teocher
0.58
20
80
100
Epoch
Figure 3: Performance of three different training strategies.
task, and G))) denote the accumulated meta- meta-performance.
(() E [ EE
Σ R(G
We introduce three SYMBOL training strategies, catering to diverse application needs. Their effec- tiveness illustrated in Figure 3. Each features a distinct reward function R()(",
Exploration learning (SYMBOL-E). Our first proposed strategy SYMBOL-E seeks to directly let the SEG learn to build optimizers from scratch. It utilizes reward function to measure the effectiveness of the generated optimizer (update rules) based on the progress they facilitate:
y op
Resplore (7()) f) (-1) (0) TM
where y (² denotes the minimal objective value during the generations 0 ~ t and y/th denotes the known optimal for the given task (considering minimization). However, BBQ tasks often present
5

---

Published as a conference paper at ICLR 2024
intricate optimization landscapes, leading significant exploration hurdles (Reddy et al., 2019). Consequently, when training the SEG using SYMBOL-E, it might demonstrate competitive outcomes but with relatively slower convergence. This stems from the SEG undergoing numerous trial-and- emror attempts before pinpointing proper update rules, as shown by the orange trajectory in Figure 3.
Guided learning (SYMBOL--).. Our subsequent strategy. SYMBOL-G, endeavours to train the SEG leading black-box optimizer x (teacher optimizer). To this end,
by mimicking the performance let the SEG and the teacher K run in parallel (with identical initial population') ). For each gen- eration in the lower-level optimization process,
compute the meta-performance (r of SEG by measuring a negation of the Earth Mover's Distance (Rubner et al., 1998) between its current population (1()) and the population of the teacher xx
Rguided ())
max
{ min |x
s [2]]
(3)
Tmax
Xiin
i=1
where Tmax and min denote the upper bound and the lower bound of the search space, denotes the size of the population and || ||- //2 denotes the Euclidean distance between two individual solutions. Utilizing the demonstration from the teacher optimizer as a guiding policy, SYMBOL-G efficiently addresses the exploration challenges faced in SYMBOL-E, leading to the fastest convergence as shown in Figure 3. Meanwhile, we note that the final performance of SYMBOL-G is on par with or even surpassing the performance of the teacher, underscoring the desirable effectiveness.
Synergized learning (SYMBOL-S). Our third strategy SYMBOL-S, integrates both the above designs shown in Eq. (4) where À is real-valued weight. Meanwhile, reduce reliance on known optima (in SYMBOL-E), we introduce a surrogate global optimum, gev,, as a dynamic alternative to substitute the static ground truth you in Eq. (2). We refer more details of gopt to Appendix A.4,
Rsynergized (.) Resplure( (.) + X. RRguide(( (-)
(4)
The results, as captured in Figure 3, reveal that SYMBOL-S excels in synergizing exploration and guided learning, resulting steadfast convergence and the best final performance.
Teacher optimizer selection. Choosing a proper teacher optimizer is generally straightforward, in- volving selections from proven performers in BBO benchmarks or saate-of-the-eeee domain literature. For training SYMBOL in this paper, we utilize the MadDE (Biswas et al., 202 I) as the teacher for SYMBOL-G and SYMBOL-S, since MadDE is one of the winners of the CEC benchmark. Note that our SYMBOL is teacher-agnostic framework, which is validated in Section 4.3. For scenarios where teacher selection is challenging, SYMBOL-E offers a fallback, allowing for meta-learning an effective policy from scratch, without relying on any teacher optimizer.
Training algorithm. Upon selecting one of the aforementioned strategies, SYMBOL can be effi- ciently trained using reinforcement learning. In our work, employ the Proximal Policy Opti- mization with a critic network (Schulman et al., 2017). See pseudocode in Appendix A.5.
4 EXPERIMENTAL RESULTS
In this section, we delve into the following research questions: RQ1: Does SYMBOL (in any of the three strategies) attain state-of-the-arr performance during both meta test and meta generalization? RQ2: What insights has SYMBOL gained and how could they be interpreted? RQ3: Does the choice of teacher optimizer in SYMBOL-G and SYMBOL-S has significant impact on performance? RQ4: How do the hyper-parameters and each component of SYMBOL designs affect the effectivvness? Below, we first introduce the experimental setups and then discuss RQI ~ RQ4, respectively.
Training distribution D. Our training dataset is synthesized based on the well-known IEEE CEC Numerical Optimization Competition (Mohamed et al., 2021) benchmark, which contains ten chal- lenging synthetic BBO problems (fi f10). These problems, having a longstanding history in the literature for evaluating the performance of BBO optimizers, show different global optimization
In SYMBOL, the student population size is fixed as 100. However, since different teachers may employ different initial population sizes, it may be impossible for the student to share an identical initial population with the teacher. In such situations, we use stratified sampling to form the student's 's initial population from the teacher's. Details are given in Appendix A.3.

---

Published as a conference paper at ICLR 2024
Table 1: Comparisons of SYMBOL and other baselines during meta test and meta generalization.
Mota Test (D)
Mota Gencralization (HPO-B)
Mota Generalization (Protcin-docking)
10s, #Dim=
50000
#Ps=5, #Dim=2-16,
16,
#FEs=500
#Ps= #Dim 12, #FFs 1000
Basclines
(Sanl
Time
Rank
(Sol))
Time
Rank
(Saa)
Time
Rank
RS
0.932=(0000))
.00003
11
(0908++00.004)
-/0.02s
8
(.906++0.000)
-0.003s
4
DB0
MaiDE
0.940=/000009
(.9024400.004)
.0.2s
0.991 (00.00)
-40.4s
Sep-CMA-ES
0.005=00.017)
-f1.38
9
(0.002))
-10.15
9
0.971 21
-1U.3s
10
ipop CP---MMAEE
0.970=/0.012)
(0.08))
.0.1s
0.996:2000003
-40.3s
SMAC
0.917
-/1.1m
7
(0.005
-10.7m
1
0.998 1:00.000)
-/3.8m
LDE
0.970=(00000)
91/0.9%
fail
fail
DEDDON
(.009=(00007)
38m//1 Im
0.862=(0.026))
-/0.6s
10
0.993
-f1.3s
7
MetaBBO
Meta-
0.936-(0.012)
12h/11.4
(0.092)
10.
3
0.984+00.000)
.0.33s
MELBA
0.0062200012)
4k/2.6m
13
fail
Fail
RNN-Opt
0.923=000010)
11h00.4m
fail
fail
SYMBOL-I
0005
6b50.9%
10
0.920
.0.55
0.9962:000000
-00.5
SYMBOL-G
0.964=000012)
10h/1.0s
(.9401(00001))
-0.55
0.9951/0.000)
-0..5
SYMBOL-S
0.972=(0.01)
10h/1.3 36
(0.06=
-00.76
2
0.999:00.0000
-10.76
Nov For meta generalization, we test on HPO-B and protein-docking jasks featuring ditferent task dimensions (# Dim). population sizes (#),,, and optimization horizons (#F))). Note that several MctaBBO methods fail generalize to these two realistic tasks: RNN-Opt and MELBA are not generalizable across different task dimenssoos: LDE is not generalizable across different population sizes.
properties such as uni-modal, multi-modal, (non-)separable, and (a)symmetrical features. Addi- tionally, they reveal varied local landscape properties, such as different properties around different local optima, continuous everywhere yet differentiable nowhere landscapes, and optima situated in flattened areas. To augment those 10 functions to form a boundless training data distribution for reinforcement learning, we adopt a sampling procedure: 1) setting a random seed; 2) randomly se- lect a batch of N problems from Si ~ S1o (with allowance for repetition), where we set problem dimension to 10 and the searching space to [-100,
10; 3) for each selected problem f, randomly introduce a offset 2 ~ U[-80, 80] 10 to the optimal and then randomly generate a rotational matrix M E IR 10 to rotate the searching space; and 4) yield the transformed problems fM + 2)).
Baselines. We compare SYMBOL with wide range of traditional BBO and MetaBBO optimizers. Regarding the BBO methods, we include Random Search (RS). MadDE (Biswas et al., 2021), strong ES optimizers sep-CMA- (Ros & Hansen, 2008) and ipop-CMA-ES (Auger & Hansen, 2005), and strong BO optimizer SMAC (Lindauer et al., 2022). Regarding the MetaBBO methods, we include DEDDON (Sharma et al., 2019), LDE (Sun et al., 2021), and Meta-ES (Lange et al., 2022) as baselines for auto-configuration RNN-Opt (TV et al., 2019) and MELBA (Chaybouti et al., 2022) as baselines for directly generating the next candidate solutions. All hyper-parameters were tuned using the grid search and we list their best settings in Appendix B.
Training& & code release. We dispatch training details and hyper- ppper-parameter setups of our SYMBOL in Appendix B.2. For MetaBBO baselines, they are all trained on the same problem distribution D as our framework while following their recommended settings. We release the implementation python codes at https: : / /i thub. com com/GMC GMC-DRL/// DEI..DRR..bb/s// Symbol, where we show how to train SYMBOL. with different strategies, and how generalize it to unseen problems.
4.1 PERFORMANCE EVALCATION (RQ1)
To ensure thorough assessment, conduct two sets of experiments: meta test and meta general- ization. The former evaluates the performance of the trained optimizer on unseen, yet in-distribution (w.r.t. ID) BBO problems. The latter evaluates the adaptability of the optimizer on completely novel realistic BBO tasks, characterized by varying task dimensions (#Dim), population sizes (#Ps), and optimization horizons (#FEs). For each method, we report in Table 1: 1) the average of min-max normalized performance with the corresponding standard deviation over 5 independent runs (see Appendix B.3 for normalization details); 2) overall training time and the average solving time per instance, in the format of "training/testing time"; and 3) performance rank among all methods.
Meta test. We meta-test SYMBOL. (trained with the three strategies) on a group of 320 unseen problem instances sampled from D, and compare them with all baselines. For each instance, all participants are allowed to optimize it for no more than x 10a function evaluations (FEs). We note that SMAC is an exception which is given 500 FEs, otherwise it takes several days for evaluating under the same FEs. The results in Table show that: 1) SYMBOL-S stands out by delivering a per- formance that not only surpasses all the baselines but also significantly outpaces its teacher, MadDE; 2) Compared with S SYMBOL-G G which concentrates itself on the teacher demonstration and SYMBOL- E which probes optimal update rules from scratch, SYMBOL-S attain significant improvement by
7

---

Published as a conference paper at ICLR 2024
Table 2: Generated update rules and corresponding frequencies.
Update rule (z
7)
Frequency
x 0.18 xx' 2,)
Tt) I 0.18 x (x" x!)
0.42
42.207%
+ 0.18 (4' 2) 0.42
39,448%
7.448%
x + 0.6 x (x" x.) + 0.6 { x) 0.18 x (x*
2.601 %
T 0.78 x (x' - 2x) 0.42 x (
1.586%
Table 3: 2D Rastrigin.
1-10
1=1>
1-22
1m
5-30
Origndl DE
t-10
2-15
[--2
1-
2-30
CMANDS
Figure 4: Evolution visulization of the optimizers, showing the position of population (red dots) and the global optimal (yellow star). Top: SYMBOL-S; Middle: original DE; Bottom: CMA-ES.
integrating their meta-objectives coordinately; 3) Methods like RNN-Opt and MELBA, which em- ploy coordinate-ban based features, may be prone overfitting, thereby resulting in suboptimal general- ization to unseen instances, even if they are from the same distribution, D. In contrast, our SYMBOL leverages generalizable VTE and VLA features and achieves best performance.
Meta generalization. To test the zero-shot generalization performance on realistic tasks, we con- sider the hyper-parameter optimization benchmark HPO-B (Arango et al., 2021) and the Protein- docking benchmark (Hwang et al., 2010), details of which are claborated in Appendix B.5. The results in Table show that: 1) SYMBOL-S not only achieves the state-of-the-at zero-shot gen- eralization performance against all MetaBBO baselines, also achieves similar performance with the advanced BO tool-box SMAC; 2) While SMAC, developed over decades, takes 42s and 227s for HPO and Protein-docking respectively, the optimizers discovered by our SYMBOL able to achieve comparable results with less than 1s, underscoring the significance of our SYMBOL frame- work; Recent MetaBBO methods DEDDQN and Meta-ES could be severely disturbed by the intricate landscape properties (even worse than exhaustive random search). Instead, SYMBOL with the proposed three strategies present more robust zero-shot generalization performance.
4.2 INTERPRETING SYMBOL (RQ2)
We now examine and interpret the generated closed-form update rules by our SYMBOL-S. Using 2D Rastrigin function (Figure 3), known for its challenging multi-modal nature, we compare the per- formance of the trained SYMBOL-S with original DE (Storn & Price, 1997) and CMA-ES (Hansen et al., 2003). Each optimizer evolves population of 10 solutions over 30 generations. SYMBOL- generates variety of update rules and we detail the top-5 rules based on runs in Table 2. Notably, the two most frequent rules showcase distinct optimization strategies: the first empha- sizes exploration in uncharted territories using z,, while the second prioritizes convergence towards previously identified optimal solutions using the current In Figure 4, illustrate the search be- haviors of our learned SYMBOL-S compared to the baseline. The visualization clearly reveals that SYMBOL-S is able to avoid falling into local optima and successfully locate the global optima by dynamically deciding a proper update rule, while DE and CMA-ES show premature due to their inflexible manual update rules. We validate this conclusion by further investigating the update rules
8

---

Published as a conference paper at ICLR 2024
0.8
DE
PSC
CMA-FS
Norm Peforramee
00.0 IL:--
-10
Tracles SVHRROLS
-1.2
0.00
0,50
FRACLO
Figure Performance of SYMBOL-S with different teachers.
Figure 6: Effects of different A.
generated in different periods of the optimization steps. the later stage of the showcased evolu- tion path (20 < S 30), our SYMBOL-S (the top row) shows two searching patterns: 1) during 20 1 S 25, the population diverges to undiscovered area. We look back at the corresponding gen- erated update rules in this period and find it is 0.18 (a* Ir) + 0.42 x (a x,,), which introduces Ey for exploring potential optimal area; 2) during 25 I t S 30, it prompts the population converging
to some local area. The dominant updated rule in this period is 0.18 (x* 2) 0.42 x (æt x),, accelerating the population to converge to the most likely optima.
4.3 TEACHER OPTIONS (RQ3)
We now investigate more teacher options where we substituted MadDE with the original DE, PSO, and CMA-ES, respectively. Figure 5 presents the normalzzal step-by-step objective values for these variants in comparison to their teacher optimizers. Impressively, SYMBOL-S not only follows the guidance of its teacher optimizer demonstrates the adaptability of our framework, capable of leveraging a wide array of existing and future BBO optimizers as teachers. Comprchensive results, including those of SYMBOL-S on HPO- B and Protein-docking and the outcomes for SYMBOL-G, are detailed in Figure 8, Appendix C.I.
consistently surpasses its performance during training. This
4.4 SENSITIVITY ANALYSIS AND ABLATION STUDIES (RQ4)
Effects of A. In Figure 6, report the normalized performance (5 runs) of different À values HPO-B tasks. When À set too low, behavior cloning may be ineffective, resulting in SYMBOL- performing similarly
SYMBOL-E. On the other hand, a high value of À can potentially cause overfitting. Together with the results on Protein-docking which we leave at Appendix C.2, Figure 10, we suggest setting À
strike good balance between the imitation and the exploration.
Abalation on FLA features and basis symbol set. To verify that the FLA features help SYMBOL at- tain good generalization performance, we conduct ablation studies on SYMBOL-S in Appendix C.2. We find that each element of the FL.A features plays crucial role in ensuring robust generalization capabilities. Besides the FLAA features, the choice of tokens in the basis symbol set may also affect the performance, discussed in Appendix C.2. We find that the key to selecting the basis symbol set is ensuring its alignment with the BBO domain, rather than simply increasing the token count.
5 CONCLUSIONS
In this paper, introduce SYMBOL, novel MetaBBO framework tailored autonomously gen- erate optimizers for BBO tasks. Adopting a bi-level structure, SYMBOL integrates an SEG network the lower level, adept generating closed-form BBO update rules. The SEG then meta-learned via three distinct training strategies: SYMBOL-E, SYMBOL-G, and SYMBOL-S, which possesses the versatility learn update rules either from scratch and/or by leveraging insights from established BBO optimizers. Experimental results underscore the effectiveness of SYMBOL: training on syn- thetic tasks suffices for it to generate optimizers that surpass state-of-the-an BBO and MetaBBO methods, even when applied to entirely unseen realistic BBO tasks. As a pioneer work on genera- tive Me through symbolic equation learning, while our SYMBOL offers several advancements, certain limitations exist. Selecting teacher for new BBO tasks can be challenging: even though SYMBOL-E provides a teacher-fres approach, future work may further enhance its training sample efficiency (as faced by other RL tasks). Additionally, exploration of more powerful symbol sets is another important future work. Furthermore, researching more advanced generators, such as those based large-language models (LLMs)) (OpenAI, 2023), is also promising future direction.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grant 62276100, in part by the Guangdong Natural Science Funds for Distinguished Young Scholars under part by the Guangdong Regional Joint Funds for Basic and Applied Research under Grant 202 IB 1515120078, in part by the TCL Young Scholars Program, and in part by the Singapore MOE AcRF Tier funding (RG13/23).
Grant 1122B151502002
REFERENCES
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama, Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data mining, 2019.
Sebastian Pineda Arango, Hadi Samer Jomaa, Martin Wistuba, and Josif Grabocka. HPO-b: A large-scale reproducible benchmark for black-box HPO based openML.. In Proceedings the 35th Conference on Neural Information Processing Systems, 2021.
Anne Auger and Nikolaus Hansen. A restart ema evolution strategy with increasing population size. In Proceedings of the IEEE Congress of Evolutionary Computation, 2005.
Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascan- dolo. Neural symbolic regression that scales. In Proceedings of the 38th International Conference on Machine Learning, 2021.
Subhodip Biswas, Debanjan Saha, Shuvodeep De, Adam D Cobb, Swagatam Das, and Brian A Jalaian. Improving differential evolution through bayesian hyperparameter optimization. In Pro- ceedings of the IEEE Congress of Evolutionary Computation, 2021.
Janez Brest, Mirjam Sepesy Maučec, and Borko Bošković. Self-adaptive differential evolution al- gorithm with population size reduction for single objective bound-constrained optimization: Al- gorithm j21. In Proceedings of the IEEE Congress of Evolutionary Computation, 2021.
Yue Cao, Tianlong Chen, Zhangyang Wang, and Yang Shen. Learning optimize in swarms, 2019.
Sofian Chaybouti, Ludovic Dos Santos, Cedric Malherbe, and Aladin Virmaux. Meta-learning of black-box solvers using deep reinforcement learning. In Proceedings of the 36th Conference on Neural Information Processing Systems, 2022.
Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xu- anyi Dong, Thang Luong, Cho-Jui Hsich, Yifeng Lu, and Quoe V. Le. Symbolic discovery of optimization algorithms, 2023.
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lilli- crap, Matt Botvinick, and Nando Freitas. Learning learn without gradient descent by gradient descent. In Proceedings of the 34th International Conference on Machine Learning, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning. 2017.
G.Shala, A. Biedenkapp, N.Awad, S. Adriaensen, M.Lindauer, and F. Hutter. Leaminn step-size adaptation in cma-es. Proceedings of the Sixteenth International Conference on Parallel Prob- lem Solving from Nature (PPSN'20), 20), September 2020.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9:159-195, 2001.
Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary computation, 11:1-18, 2003.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning, 2016.
10

---

Published as a conference paper at ICLR 2024
John H. Holland. Genetic algorithms. Scientific american, 267:66-73. 1992.
Howook Hwang, Thom Vreven, Joël Janin, and Zhiping Weng. Protein-protein docking benchmark version 4.0. Proteins: Structure, Function, and Bioinformatics, 78:3111-3114, 2010.
James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of the Interna- tional Conference on Neural Networks, 1995.
Robert Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, and Sebastian Flennerhag. Discovering attention-based genetic algorithms via meta-black-box optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, 2023.
Robert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flennerhag. Discovering evolution strategies via meta-black-box optimiza- tion. In Proceedings of the 11th International Conference on Learning Representations, 2022.
Xiaobin Li, Kai Wu, Xiaoyu Zhang, Handing Wang, and Jing Liu. B2opt: Learning to optimize black-box optimization with little budget, 2023.
Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp. Difan Deng, Car- olin Benjamins, Tim Ruhkopf, René Sass, and Frank Hutter. Smac3: A versatile bayesian opti- mization package for hyperparameter optimization. The Journal of Machine Learning Research, 23:2475-2483, 2022.
Ali Wagdy Mohamed, Anas A Hadi, Ali Khater Mohamed, Prachi Agrawal, Abhishek Kumar, and N. Suganthan. Problem definitions and evaluation criteria for the cee 2021 on single objective bound constrained numerical optimization. In Proceedings of the IEEE Congress of Evolutionary Computation, 2021.
OpenAL. ChatGPT, 2023. URL URL hww.httpps//ppeaa..co////
Brenden K Petersen, Mikel Landajuela Larma, Terrell N. Mundhenk, Claudio Prata Santiago, Soo Kyung Kim, and Joanne Taery Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. In Proceedings of the 9th International Conference on Learning Representations, 2021.
Siddharth Reddy, Anca D Dragan, and Sergey Levine. Sqil: Imitation learning via reinforcement learning with sparse rewards. In Proceedings of the 7th International Conference on Learning Representattinss, 2019.
Flethhrr A new approach to variable metric algorithms. The Computer Journal, 1970.
Raymond Ros and Nikolaus Hansen. A simple modification in cma-es achieving linear time and space complexity. In Proceedings of the 10th International Conference on Parallel Problem Solv- ing from Nature. Springer, 2008.
Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. metric for distributions with applications to image databases. In Proceedings of the 6th International Conference Computer Vision, 1998.
Ruhul Sarker, Saber M Elsayed, and Tapabrata Ray. Differential evolution with dynamic param- eters selection for optimization problems. IEEE Transactions Evolutionary Compulation, 18: 689-707, 2013.
Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data. Science, 324(5923):81-855 2009.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXivv1707.063477 2017.
Mudita Sharma, Alexandros Komninos, Manuel López-lbáñez, and Dimitar Kazakov. Deep inforcement learning based parameter control differential evolution. In Proceedings of the Genetic and Evolutionary Computation Conference, 20199.

---

Published as a conference paper at ICLR 2024
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Proceedings of the 26th Conference on Neural Information Processing Systems, 2012a.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Proceedings of the 26th Conference on Neural Information Processing Systems, 2012b.
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable bayesian optimization using deep neural networks. In Proceedings of the 32nd International Conference on Machine Learning. 2015.
Rainer Storn and Kenneth Price. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, 11:341, 1997.
Jianyong Sun, Xin Liu, Thomas Bäck, and Zongben Xu. Learning adaptive differential evolution algorithm from optimization experiences by policy gradient. IEEE Transactions on Evolutionary 2021.
Computation,
Tomer Tsaban, Julia K Varga, Orly Avraham, Ziv Ben-Aharon, Alisa Khramushin, and Ora Schueler-Furman Harnessing protein folding neural networks for peptide-protein docking. Na- Ture Communications, 13:176, 2022.
Vishnu TV, Pankaj Malhotra, Jyoti Narwariya, Lovekesh Vig, and Gautam Shroff. Meta-learnin for black-box optimization. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2019.
Mojtaba Valipour, Bowen You, Maysum Panju, and Ali Ghodsi. Symbolicgpt: generative trans- former model for symbolic regression. arXiv preprin! arXiv:2106. 14131, 2021.
Martin Vastl, Jonáš Kulhánek, Jiří Kubalík, Erik Demer, and Robert Babuška. Symformer: End-to- end symbolic regression using transformer-basee architecture. arXiv preprint arXiv:2205.15764, 2022.
Kai Wu, Penghui Liu, and Jing Liu. Deen: Automated evolutionary algorithms via evolution inspired deep convolution network, 2023.
Qi Zhao, Qiqi Duan, Bai Yan, Shi Cheng, and Yuhui Shi. A survey on automated design of meta- heuristic algorithms, 2023.
Wenqing Zheng, Tianlong Chen, Ting-Kuei Hu, and Zhangyang Wang. Symbolic learning to opti- mize: Towards interpretability and scalability, 2022.
12

---

Published as a conference paper at ICLR 2024
A TECHNICAL DETAILS OF SYMBOL
A.1 CONTEXTUAL STATE REPRESENTATION
VTE. We first represent each token in & with 4-bit binary code, e.g., 0001 for +. 0010 for etc., listed in in Table 4. During constructing the update rule 7, we embed the corresponding under- constructed symbolic expression tree into fixed-length vector which is called VTE as contextual state to inform the SEG prompting next token for т. We showcase a symbolic expression tree with maximum height H = 3 in Figure 7. In this case, the SEG has generated 4 tokens in the previous steps (+, x², x, c). thus the VTE for the current half-constructee tree is shown in the bottom of Figure 7. For those blank positions in this tree we use 0000 to represent them, while for those positions holding by a token Tr, we use the binary codes in Table 4 to represent them. As we have set maximum height of r to 5 in this paper, the VTE embedding of the symbolic expression tree can be represented by 31 4- bit binary codes, resulting in a vector of {0, 1
FLA. The formulation of the FLA features is described in Table 5. States $(12233) collectively repre- sent the distributional features of the current candidate population. Specifically, state $1 represents the average distance between each pair of candidate solutions, indicating the overall dispersion level. State 82 represents the average distance between the best candidate solution in the current population and the remaining solutions, providing insights into the convergence situation. State 83 represents the average distance between the best solution found so far and the remaining solutions, indicating stage. Then, States 8 $[4,5,) provide information about the statistics of the objective values in the current population and contribute to our framework's 's understanding of the fitness landscape. Specifically, state 81 represents the average difference between the best objective value found in the current population and the remaining solutions, and 85 represents the average difference when compared with the best objective value found so far. State 86 represents the stan- dard deviation of the objective values of the current candidates. Finally, states $(7,8.9) collectively represent the time-stamp features of the current optimization process. Among them, state denotes the current process, which can inform the framework about when to adopt appropriate strategies. States sy and sy are measures for the stagnation situation.
the exploration-exploiaae
Table 4: Binary encoding of tokens in S.
Tokens
Binary code
Tokens
Binary code
0001
x
0010
0011
0100
0101
0110
VTE
x_ 0111
Aw
1000
M001
0110
0010
0000
00OD
0100
0000
Fr
1001
1010
Figure 7: Illustration of VTE transformation.
A.2 NETWORK ARCHITECTURE DETAILS
SEG (Actor Network). The network parameters of SEG 0 includes three components: 1) FLA Em- bedder with parameters {WELAA DFLA 2) LSTM with parameters (W,, 6n, Wi., dyn J: 3) Constant inference decoder with parameters (Wa, bw, W., b.). To generate update rule 5 for the current optimization step (generation t), we first embed the FLA features of the current optimization status SFLA as the initial cell state:
0
CLA = WLLAA + OFLA
(5)
We then initialize the (0) = 0 and VTE(T) (at the very beginning, T is empty) as the other two con- textual states. The LSTM then starts auto-regryssively determining each 7; by iteratively executing the following three steps:
= LLTM(VVTETT)
FLA 1 (i-1)
(6)
FLA.
T ~ (oftmax
(7)
T=TU (T))
(8)
13

---

Published as a conference paper at ICLR 2024
Table 5: Formulations of FLA features.
States
Notes
means,]lax!" =
Average distance between any pair of individuals current population.
Population
Average distance between each individual and the best individual in t generation.
82
mean, |0 -xr -=1111)
Average distance between each individual and the best-so-far solution.
mean;|la?" x=" l|2
Average objective value gap between each individual and the best-so-far solution.
84
mean; ([(n;(((a))) f(x)))
Average objective value gap between each individual and the best individual in gd generation.
Objective
$5
mean, (f(on,(((2)))) - f(x²
Standard deviation of the objective values of population in t generation, value equals denotes converged.
std:(f(x()))
The potion of remaining generations, denotes maximum generations for one run.
57
(T 1)/T
Time Stamp
st denotes how many generations the optimizer stagnates improving.
88
st/TT
if ((x)1)) < fx*)
Whether the optimizer finds better individual than the best-so-far solution.
otherwise
the constant token e inferred hth construction step (T; = c), we activate the constant inference decoder to infer the concrete constant value:
Softmax [ 50
( + bm))
(9)
Softmax T + b.)
(10)
~ 10
(11)
Critic Network. The Critic has parameters Weniics Deritie It receives the FLA features as the input and predicts the expected return value v (baseline value) as follows:
t = WT SPLA + Deritic
critic
(12)
A.3 POPULATION SAMPLING
When SYMBOL-G SYMBOL-S constructs its initial population during the training process, need to ensure SYMBOL. have the same initial population with the teacher optimizer. However, con- sidering that the population size of SYMBOL may be different from the teacher optimizer, and some teacher optimizers even dynamically decay the population size through the optimization process, flexible method is needed initialize SYMBOL'S population. This method should not only adapt to the uncertain population size of teacher but also make the initial population of SYMBOL closely resemble that of the teacher optimizer. Specifically, in the case where the teacher's population size is larger than that of SYMBOL, the population of SYMBOL is stratified sampled from the teacher opti- mizer base on the objective value. In cases where teacher's population size smaller than SYMBOL, SYMBOL firstly replicates the teacher's population and then randomly samples candidates from the search space until the required population size is reached. We illustrate the population sampling method in Algorithm 1.
A.4 SURROGATE GLOBAL OPTIMUM
The teacher optimizer in S SYMBOL-S provides a convenience to reduce the reliance on known optima in SYMBOL-F. We propose surrogate global optimum, jopt, substitute the ground true global
14

---

Published as a conference paper at ICLR 2024
Algorithm Sampling student's s initial population from teacher's population.
input: Teacher's population yds, Teacher's population size Pytes, Student's population size Ps output: Student's population pdth pa t- P t-
2: if PP > P then
3: pau +pxth Up Ps <- Ps
4: while Ps < Ss do
5: Randomly sample solution z, from the searching space
6: Daula 4 pnu U }
7: Pasu +- Py +
8: end while
9:
else
10:
Sort pl upon the objective value
11:
Equally spaced sample # P candidate solutions from pea, store in pample
12: y <- psimple; Py P
13:
end if
optimum. In the very beginning of training process, gesl is determined from the teacher optimizer when it optimizes the corresponding training task. In the rest of training process, 8em is dynamically updated when either the teacher the SEG finds a solution with lowest objective value thus far.
A.5 PSEUDO CODE FOR THE TRAINING PROCESS
Algorithm illustrates the pseudo code for the training process of three strategies of SYMBOL. To execute these strategies, must have access to the training distribution ID and set the learning rate a. Additionally, for SYMBOL-G and SYMBOL-S, the teacher optimizer k is required. First, initialize the SEG with LSTT parameters 0. In each meta-training step, batch of problems are sampled from D. Then, we reset к for SYMBOL-G and SYMBOL-S, initialize its population and evaluate the solutions. The population for student is similarly initialized. For each step in lower-level optimization, we employ к to gain the teacher suggestion, which will be skipped when the training strategy is SYMBOL-E. Next, for all training strategies, we generate the update rule for the student population with our SEG and use the rule to advance the population. The reward will be then calculated according to different training modes as formulated Eq. (2) ~ (4). After lower-level loop concludes, we update the meta-objective and token trajectory, which participant in the gradient calculation PPO manner. Upon the meta-tras veta-ttaiiing loop, the rell-trained SEG with LSTM parameters is returned.
B TRAINING AND TEST SETUP
B. SETTINGS OF TRADITIONAL BASELINES
For the sake of comparison fairness, we conduct grid search on some of the hyper-parameters of baselines the training distribution I. In this section, present the search results of two BBO baselines, MadDE and -MP-EE as they are sensitive to hyper- focus on the following hyper-parameters: the probability of qBX crossover (P,ax), the per- centage of population in p-hest mutation (p), the initial values for scale factor memory (F)) and crossover rate memory (CR,). For Sep-CMA-ES,
values. For MadDE,
consider the following byper-parameters. the initial center (μo) and initial standard deviation (op). The search range and best settings after gird search of the two baseline algorithms are listed Table 6. In the experiments of this paper, for MadDE, we choose PςBx of 0.01, of 0.18, Fu of 0.2, and CR2 of 0.2. For sep-CMA-ES, we use No with a vector of and dy with a vector of 2.
B.2 SETTINGS OF SYMBOL
The neural parameters our framework consist of two parts: 1) The Actor (SEG) includes an FLA embedder for embedding the FLAA features into the cell state of LSTM, which has parameters
15

---

Published as a conference paper at ICLR 2024
Algorithm 2 Meta-learning the SEG with policy gradient.
input: Training distribution ID; Teacher optimizer n;; Learning rate o; Training strategy;
output: Best performing SEG
1: Initialize the SEG with LSTM parameters 0
2: #Meta-level Looping
3: repeat
4: Sample problem f from D
>. See Training distribution part in Section3.2 Skipped if performing SYMBOL-E
5: Reset teacher
6: Meta-objective G + 0: Token trajectory V/ <-
7: Initialize teacher population X,, Y, (()))
Skipped performing SYMBOL-E X,, If SYMBOL-G/S Share the same distribution
8:
Initialize student population X,, Xs, If SYMBOL-E
9:
Y. + f(x,)
Evaluate the candidate solutions
10:
#Lower-level Looping
11: repeat
Xr, Y t x(Xt,Yi) Yi)
Teacher's round, skipped performing SYMBOL-E > Generate update rule for student population Student's round suggest new solutions
13:
T, ( 0) + SEGo(X..
X, X, + + + f(X.)
Rexplore (;,{),
If Training strategy SYMBOL-E
15:
R + Ryaided( (:s f),
If TTaining strategy SYMBOL-G
Roynerizedl J),,
If Training strategy is SYMBOL-S
16:
G
Accumulate rewards
4+ (T}
18:
until Function evaluations run out
g+G Ve log (0)
Compute policy gradients >> Apply policy gradients
20:
8 8 - ag
until Exceed the pre-defined learning steps
Table 6: Grid search for the haper-parameters of MadDE and sep-CMA-ES.
Baselines
Parameters
Scarch range
Chosen values
Paax
00.01,0.55,0 0.5]
0.01
(0.09.0.15 0.18, 0.27, 0.366.36]
0.18
MadDE
Fo
00.1.0.2,..3 0.4)
0.2
00.1, 0.3, 0.4]
0.2
[0, 1. 2,
3
sep-CMA-FS
do
[0. 1, 2,
(WELA E 19 PELA E R16]. The LSTM network for generating symbol distribution has parame- R 124x64 bin E & along with two feed-forward layers for inferring the constants, which have parameters (W, 16 b. E IR2) and ( IR 16x21 b_ R21). 2) The Critic is a feed-forward layer with parameters (Weritcc critic E R Deritic E R!). Our framework is trained by the PPO algorithm, which updates the parameters & = 3 times every sampling step, with a decay parameter 7 = 0.9. The learning rate a is 10 The number of genera- tions (T) for lower-level optimization is 500. The tunable parameter À for SYMBOL-S is set to 1. We simultaneously sample batch of N = 32 problems from problem distribution for training. The pre-defined maximum learning steps for PPO is 5 x 10*. Once the training is done, SYMBOL can be directly applied to meta-test (unseen problems sampled from D) and meta-generalizatiion (unseen tasks out of ID) without the need for teacher optimizer or further fine-tuning. All experiments run on machine with Intel 10-00XE CPU, RTX 3090 GPU and 32GB RAM.
ters (W, E 16x64 by Rd4 Win E
10
B.3 NORMALIZATION OF THE ORJECTIVE VALUES
Considering minimization problem, we first record the worst objective f and the best objective fh. in all runs for the th instance (fft is the fort if the optima is known). For the mht run on the Ath task instance, then record the best-found objective value pm.k Then, the min-max normalized
16

---

Published as a conference paper at ICLR 2024
-070
Teadee
0.5
Log Noem Ob) value 1
Log Norm. Obj volue
1.25
0.-
S
Somme 180 maxy donn
-1.0
150
0.5
0.6
-1.
0.5
0.-
-1.
FE 4)
FEA(10' 2)
F1
(a) SYMBOL-G on
(b) SYMBOL-G HPO-B
(c) SYMBOL-G Prokein-docking
-.0
0.9
Log Noru. Obj. value J.
Tog, Norm. Obj valno.
Narm, Obj. vaine
S13400L-8
PSD
-.00
0.8
-1.5
0.0
CHAMES
FEa 4)
FEa(10 2)
2)
(d) SYMBOL-S on
(e) SYMBOL-S on HPO-B
(f) SYMBOL-S on Protein-docking
Figure 8: The performance of SYMBOL-G, SYMBOL-S and their teachers on the three benchmark
sets: D. HPO-B and Protein-docking.
objective value Obj for a baseline is computed as Σx= M
Σm=
5 - -
where K and normalized
f mm
M is the size of test and the total test runs respectively. The corresponding performance and log-normalized objective value used our experimental results are computed as (1 Obj) and (Obj).
B.4 MORE DETAILS OF TRAINING DISTRIBUTION
Table 7 provides detailed properties of the ten classes of functions in D. These problems in- clude various optimization properties such as unimodal, multi-modal, non-separable, asymmetrical, diverse local global optimization properties and so on. Furthermore, we also visualize these func- tions in 2D configuration in Figure 9 for more intuitive insight. By covering such a wide range of landscape features and incorporating fitness landscape analysis based state representation, SYMBOL is not learning how to solve given task, instead, it learns how to generate different update patterns facing with various landscape feature, which ensures its superior generalization performance. Ad- ditionally, for each selected function, we further apply the random offset and rotating operation to augment the function. This augmentation further enrich the training distribution and may enhance the generalization ability of SYMBOL to unseen tasks.
B.5 TASKS SETS FOR META GENERALIZATION
We provide two challenging realistic task sets to evaluate the zero-shot generalization performance of SYMBOL and other baselines:
HPO-B benchmark (Arango et al., 2021), 1), which includes wide range of hyperparameter optimiza- tion tasks for 16 different model types (e.g.. SVM, XGBoost, etc.). These models have various search spaces ranging from [0. 1 to [0, 1] "6" Each model is evaluated on several datasets. resulting in a total of 86 tasks. For each instance, all baselines are allowed to optimize within 500 function evaluations and with population size of 5 (for baselines that incorporate population of candidate
17

---

Published as a conference paper at ICLR 2024
Table Summary the ten classes of functions in ID)
No.
Functions
Propertics
-Unimodal
Unimodal Function
Bent Cigar Function
-Non separable
-Smooth but narrow ridge
-Muli-modal
Schwefel's Function
-oo separabee
-Local optima's i number is huge
Mulii-modal
Basic
-on-sepaaab
Functions
Ri-Rastrrigin Function
Asymmetricaa
Continuous everywhere yet differentiable onwhere
Rosenbrock's 's plus Griewangk's
Non Non-separable
Function
Optimal point locates
area
Hybrid Function
Mulii-modal or Unimodal, depending on basic function
Hybrid
Hyhrid Function (N 4)
Non separable subcomponents
Functions
Hybrid Function 3 (NV = 5)
-Different properties for different variables subcomponents
Composition
Composition Function
Mulii-modal
Functions
Composition Funciion 2 (N
Non separable Asyymetricall
10
Composition Function 3 (N = 5)
DDifferent propeties around different local optima
Scurch range: [-100, 100]
(a) Function 1
(b) Function 2
(c) Function 3
(d) Function
2395
(e) Function 8
(1) Function 9
(g) Function 10
Figure 9: Fitness landscapes of functions in when dimension set to 2. Functions 5-7 can not be visualized in the 2D configuration due to the inherent constraint from hybrid functions.
solutions). To save evaluation time, we adopt the continuous version of HPO-B, which provides surrogate evaluation functions for time-consuming machine learning tasks.
Protein-docking benchmark (Hwang et al., 2010), where the objective is to minimize the Gibbs free energy resulting from protein protein-proteio protein interaction between a given complex and any other conforma- tion. We select 28 protein complexes and randomly initialize 10 starting points for each complex. All 280 task instances have search space of [-100, 100] 12 For each instance, all basclines are allowed optimize within 1000 function evaluations and with population size of 10.
18

---

Published as a conference paper at ICLR 2024
Table 8: Ablation studies on the feature selection in FLA features.
SYMBOL-S
SYMBOL-S
SYMBOL-S
SYMBOL-S with xy
Task
SYMBOL-S
w/o pop
obj
w/o is
CEC-competition HPO-B1
0.902=(00011)
0.964=(0.011) (.0052(0.015) 0.0079)00009) 0.909=(0.002))
0.910=(0.017) 0.779++0.022) 0..05=(00002))
(.8491 fail
0.963+t(0.006
Protein-dockingt
0.999=(0.000)
(.900=0
0.991=(0.001)
fail
Table 9: Ablation study results for symbols set design.
0366
+
Symbols set SYMBOL-S with S
SYMBOL-S with S+
0362
Task
SYMBOL-S
0.088
055
11.75
1.25
15
CHC-competition HPO-B*
(0.01)(0.011))
0.952+(0.015) 0.037=000009) (0.03)()0006)
0.930++(0.018)
0.910==00.099)
Figure 10: Performance with dif- ferent À on Protein- dootein-docking
Protein-dockingt
0.900=(0.000)
0.905=(0.001) 0.990=(0.002)
C ADDITIONAL EXPERIMENT RESULTS
C.I TEACHER OPTIONS
Additionally, we provide a comprehensive analysis of the potential teacher options for SYMBOL-G and SYMBOL-S. By incorporating different teachers (as described in the main body of this paper), train the resulting variants on ID.. The meta-test results and meta- meta- generalization results are illustrated in Figure 8. The results consistent with the observations made in Section 4.3. SYMBOL-S achieves robust performance improvement, while SYMBOL-G fails to learn from PSO. This issue may be attributed the definitive update rules, which can lead to overfitting. Even for SYMBOL- S. where we integrate exploration behavior and prior demonstrations, our framework encounters difficulties in learning from this type of teacher. We acknowledge this as a potential area for future work and aim to address this limitation.
C.2 SENSITIVITY ANALYSIS AND ABLATION STUDIES
Tunable parameter A. We report the normalized zero-shot performance (5 runs) of SYMBOL-S trained with 7 different À (0,0.20 0.25, 0.5, 0.75, 1, 1.25, 1.5) on the Protein-docking tasks in Fig- ure 10. The results are basically consistent with the observation in Section 4.4. We found that À = a relatively better choice.
Feature selection for state representation. To verify that the FLA features help SYMBOL attain good generalization performance, we conduct ablation studies on SYMBOL-S by omitting various components: the distributional features of the current population (denoted w/o pop). the statistic features of the current objective values (w/o obj). and the time stamp embedding (w/o ts). Addi- tionally, also assessed the performance when replacing FL.A features with the simple coordinate- based feature set {x. } (with xy), which follows the coordinate state representation design in (Chen al., 2017; TV al., 2019; Chaybouti al., 2022). The outcomes, presented in Table 8, confirm that each element of the FLA features plays a crucial role in ensuring robust generalization capabilities. On the contrary, coordinate-basee feature (w/o xy) somehow shows deficiency.
Basis symbol set design. We investigate the effect of the designs of the basis symbol set S on the fi- nal performance of SYMBOL. The investigation is conducted in two directions: subtracting existing symbols from S and augmenting them with new ones. For the former, we subtract three variables T., Ar. and x, from S and denote this subtracted set S For the latter, we add three unary oper- ators sin, cos, and sign (frequently used non-linear functions in recent symbolic equation learning frameworks) to S and denote this augmented set as S+. continue to use SYMBOL-S as a show- case and load it with S+ and S Under identical training and testing settings, we report the ablation results of the two ablated variants in Table 8, which show that: 1) Symbols such as Ty are very important for population-base didate populations. Removing such symbols from the symbol set significantly reduces the learning effectiveness. 2) Augmenting the symbol set certainly enlarges the representation space. However,
black-box optimizers, as they provide exploration behaviors for
19

---

Published as a conference paper at ICLR 2024
this addition may also affect the search for optimal update rules. This opens up discussion for the symbolic equation learning community and ourselves on how to construct domain-specifie symbol sets for extending application scenarios beyond symbolic equation learning itself.
C.3 DISCUSSION PERFORMANCE OF SYMBOL-G
From experiment results in Table and Figure 8, we notice that SYMBOL-G can even outperform teacher optimizers in some cases. Possible explanation are as follows. Firstly, while the reward guides the agent to closely imitate the teacher's searching behavior, our SYMBOT. is not designed to merely replicate their exact update rules but to discover potentially more efficient update rules upon the basis symbol set for handling broad spectrum of BBO task, especially with the help of our robust fitness landscape analysis features for enhanced generalization capability, Secondly, unlike traditional optimizers (e.g., MadDE), which might rely consistent fixed searching rules, SYMBOL-G designed to generate flexible and dynamic update rules for specific optimization steps. Lastly, note that the inherent exploration capability of RL also plays role in the enhanced performance. In contrast to deterministic strategies, the sampling-based strategy in RL provides more diverse actions. Such observations also show up in as the GAIL framework (Ho & Ermon, 2016). These factors collectively contribute to S SYMBOL-G's ability to, in some cases, outperform its teacher optimizer. We note that by further incorporating the exploration reward (S SYMBOL-E)) our SYMBOL-S would consistently surpass the teacher optimizer.
imitation learning frameworks, such
20


Reviews:
[{"review_id": "Z5vpEil2mt", "reviewer": "Reviewer_h4Ki", "summary": "This paper proposes meta-learning for black-box optimization (BBO) methods using symbolic equation learning. The proposed method, termed SYMBOL, trains the neural network model that generates the update rule of the BBO method depending on the task and optimization situation using landscape features as the input of the model. The model for generating the update rule is trained based on the reinforcement learning algorithm. The experimental evaluation using artificial benchmark functions, HPO, and Protein docking benchmarks demonstrates that the optimizer generated by the proposed SYMBOL can beat the existing BBO and MetaBBO baselines.", "questions": "- Many techniques regarding MetaBBO, including meta genetic algorithm and meta evolutionary algorithm, have been developed so far [i]. It might be better to mention the traditional approaches related to MetaBBO.\n- The work of [ii] might relate to this paper. Although it only tunes the step-size adaptation in CMA-ES, the concept and used techniques, such as reinforcement learning-based training and guided policy search, are somewhat similar.\n- Why does SYMBOL-G outperform MadDE in Table 1? As the teacher optimizer of SYMBOL-G is MadDE in the experiment, it seems strange that SYMBOL-G beats the teacher optimizer.\n- Could you describe the relationship between MetaBBO and automatic algorithm configurations?\n\n[i] Qi Zhao, Qiqi Duan, Bai Yan, Shi Cheng, Yuhui Shi, \"A Survey on Automated Design of Metaheuristic Algorithms,\" arXiv:2303.06532\n\n[ii] Shala, G., Biedenkapp, A., Awad, N., Adriaensen, S., Lindauer, M., Hutter, F. (2020). Learning Step-Size Adaptation in CMA-ES. In: Bäck, T., et al. Parallel Problem Solving from Nature – PPSN XVI. PPSN 2020. Lecture Notes in Computer Science(), vol 12269. Springer, Cham. https://doi.org/10.1007/978-3-030-58112-1_48\n\nMinor comments:\n- On page 2, \"At the lower lever\" should be \"At the lower level.\"\n- In equation (3), the redundant right-side parenthesis exists.\n\n\n----- After the rebuttal -----\n\nThank you for the responses. As the responses are convincing, I keep my score on the acceptance side.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed SYMBOL can dynamically change the update rule of solutions depending on the search situation owing to the use of fitness landscape features, which seems to be a technical novelty. In addition, the training strategy of the optimizer generator by mimicking the existing teacher black-box optimizer is reasonable for accelerating the model training.\n- The search performance of the optimizer generated by SYMBOL outperforms other BBO and recent MetaBBO techniques.", "weaknesses": "- It is somewhat unclear to me the key difference and novelty of the proposed SYMBOL because there exists a lot of MetaBBO techniques. It would be very nice if the authors clarified the advantages and key differences of the SYMBOL compared to other existing methods.\n- Although the proposed SYMBOL can generate flexible update rules of solutions, the representation ability of update rules is limited depending on the pre-defined basic symbol set. I suppose that SYMBOL cannot generate the CMA-ES type update rules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "59nYxkx5vr", "reviewer": "Reviewer_VT1T", "summary": "This paper proposes SIMBOL, a framework to learn a black-box optimizer through symbolic equation learning. The paper first presents a symbolic equation generator (SEG) to generates closed-form optimization rule, where such closed-form is found through reinforcement learning. The paper argues the proposed method shows state-of-the-art results on benchmarks as well as showing  strong zero-shot generalization capabilities.", "questions": "I have no expertise in this area, so I will adjust my score based on discussion between authors reviewers, and AC.", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Compared with existing BBO, the proposed method seems technically novel (I am not the expert). \n- The paper is generally well-written and easy-to-follow.\n- The results seem promising compared with existing methods.", "weaknesses": "- There's no strategy in choosing a teacher optimizer ($\\kappa$).\n- Since it requires training based on reinforcement learning, training may require a time for framework compared with existing BBO that does not require any training. I wonder how long it takes for training compared with other MetaBBO method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w6DONacvnr", "reviewer": "Reviewer_ZLbt", "summary": "This paper introduces a novel black-box optimizer (BBO) SYMBOL featured by its symbolic equation learning via meta-learning. Different from previous MetaBBOs which directly output the updated value, SYMBOL meta-learns the updated rule as an equation which achieves a superior performance and better interpretability.", "questions": "I'm not directly working in this field, so I'm a bit curious about how Lion [1] is compared to the symbolic discovery in this work, which I think could be more promising in the optimizer discovery. \n\n[1] Symbolic Discovery of Optimization Algorithms. Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, Quoc V. Le. https://arxiv.org/abs/2302.06675.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Though the idea was also explored in various domains, I think the discoveries from this paper are still interesting. It is great to see the steps for learning more interpretable optimizers.\n2. The paper is well-presented and the idea is easy to follow.\n3. The Synergized learning strategy proposed in this paper is novel and interesting, which I believe could be a promising strategy in the future.\n4 The empirical evaluation shows promising results against baselines. The generalization performance, thanks to the flexibility of its symbolic update rule, is quite impressive against previous MetaBBOs. The authors also carefully ablate the components of their method.", "weaknesses": "1. Though promising, I'm not sure if this work oversimplifies the actual optimization problem in its search space, which makes it not that useful currently. There are only 3 operators and 7 operands in its basic symbol set, is the design of the symbol set relevant to the teacher algorithm?\n2. It is good to see the improvement in the experiment, however, I note SYMBOL-E does not outperform MadDe (2 out of 3 columns). This raises the question of whether the potential of SYMBOL is largely limited by the performance of MadDe. If MadDe does not work well, can the exploration reward actually help the model learn a much better update rule? The authors could try to use a relatively bad baseline as the teacher to verify this and or find a challenging task where MadDe does not work well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E4rH3UU6l8", "reviewer": "Reviewer_hEPw", "summary": "The paper introduces SYMBOL, a novel approach for BBO that leverages a neural network trained via Reinforcement Learning to dynamically predict the explicit expression of the optimization steps for a given task. \n\nSYMBOL is based on a Symbolic Equation Generator (SEG), an LSTM-based model responsible for predicting closed-form (symbolic) optimization updates. SEG can be trained via three different RL strategies, namely Exploration learning (SYMBOL-E), Guided learning (SYMBOL-G) and Synergized learning (SYMBOL-S). While SYMBOL-E does not impose any inductive bias about the sought-for optimizer,  SYMBOL-E explicitly forces the SEG to mimic the behaviour of a given black-box optimizer (teacher optimizer). SYMBOL-S integrates both approaches and regulates their relative importance via a hyperparameter $\\lambda$. \n\nThe model -- in all its three variants -- is extensively tested and compared to multiple state-of-the-art baselines. The results indicate that SYMBOL compares favourably to the considered baselines. In addition, the updates found by SEG can be directly inspected thanks to their symbolic nature.", "questions": "See weaknesses part.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written and the presentation is clear.\n\n- SYMBOL represents an interesting application of deep-learning-based symbolic regression, beyond the standard setting where it is usually applied.\n\n- The experiments compare SYMBOL to multiple baseline methods and show that SYMBOL archives state-of-the-art performance on BBO.\n\n- Contrarily to black box systems, SYMBOL generates interpretable update steps.", "weaknesses": "- Reliance on a specific teacher in SYMBOL-G and SYMBOL-S: how to select a teacher to train the model may not be obvious in real-world applications. This makes the applicability of the aforementioned training strategies relatively limited. \n\n- Reliance on the meta-learning stage: The training of the SEG model requires the selection of a training distribution D. The authors rely on 10 BBO optimization problems to construct D. However, in practice, the obtained distribution might not be large enough to guarantee sufficient generalization to fundamentally different tasks. \n\n- Training SYMBOL is still quite computationally demanding as shown in the meta-test column in Table 1. Is training the model via RL the computational bottleneck?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)