MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning
Authors: Zeyuan Ma
Hongshu Guo
Jiacheng Chen
Zhenrui Li
Guojun Peng
Yue-Jiao Gong
Yining Ma
Zhiguang Cao
Venue: NeurIPS 2023 Datasets and Benchmarks Oral (2023)
Keywords: Black-Box Optimization
Meta-Black-Box Optimization
Reinforcement Learning
Benchmark Platform
TL;DR: A novel, extendable and user-friendly Benchmark Platform for MetaBBO with Reinforcement Learning.

Abstract: Recently, Meta-Black-Box Optimization with Reinforcement Learning (MetaBBO-RL) has showcased the power of leveraging RL at the meta-level to mitigate manual fine-tuning of low-level black-box optimizers. However, this field is hindered by the lack of a unified benchmark. To fill this gap, we introduce MetaBox, the first benchmark platform expressly tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment of the methods. In a bid to illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, we carry out a wide-ranging benchmarking study on existing MetaBBO-RL methods. Our MetaBox is open-source and accessible at: https://github.com/GMC-DRL/MetaBox.

MetaBox: A Benchmark Platform for Meta-Black- Optimization with Reinforcement Learning
Zeyuan Ma', Hongshu Guo', Jiacheng Chen', Zhenrui Li', Guojun Peng',
Yue-Jiao Gong' Yining Ma² Zhiguang Cao
South China University of Technology
National University of Singapore
Singapore Management University
(scut. crazynicolas, guohongshu369, jackchanS345)@gmail.. com,
zhenrui www.otoookk..o..ooo com, (pgj20010419, gongyuejiao)@gma @gmail com, yiningma@u. nus edu, zhiguangca0outlook.coo
Abstract
Recently, Meta-Black-Boo Optimization with Reinforcement Learning (MetaBBO- RL) has showcased the power of leveraging RL at the meta-level mitigate manual fine-tuning of low-level black-box optimizers. However, this field hindered by the lack of unified benchmark. To fill this gap, introduce MetaBox, the first benchmark platform expressly tailored for developing and evaluating MetaBBO- RL methods. MetaBox offers flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment the methods. In bid illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, we carry out a wide- ranging benchmarking study on existing MetaBBO-RL methods. Our MetaBox is open-source and accessible at: https:/ / /githtb....ggggg//ggg//gg..ggh.t com/GMC-DRLL /WMC-DDR//MeaBoo....
Introduction
Black Box Optimization (BBO) is class of optimization problems featured by its objective function that either unknown or too intricate to be mathematically formulated. It has a broad range of applications such as hyper-parameter tuning [1], neural architecture searching [2], and protein- docking [3]. Due to the black-box nature, the optimizer has no access to the mathematical expression, gradients, any other structural information related the problem. Instead, the interaction with the black-box problem primarily realized through querying inputs (i.e., solution) and observing outputs (i.e., its objective value).
Traditional solvers for BBO problems include population- -ased optimizers such as genetic algo- rithms [4], evolutionary strategies [5-7], particle swarm optimization [8. 9], and differential evo- lution [10-13]. The Bayesian Optimization (BO) [14, 15] is also commonly used. However, with limited knowledge of the problem, these optimizers lean carefully hand- hand-crafted designs to strike balance between exploration and exploitation when seeking the optimal solution.
To eliminate the burdensome task of manual fine-tuning, recent research has proposed the concept of Meta- Baaa-BBlac-BBon Optimization (MetaBBO), which aims to refine the black-box optimizers by
Yua Gong the corresponding author.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

Template
Trainer
Logger
Meta-Level RL Agent
Stendardized Metrics
instantiate
Environment
-cal, cai_AElO
Optimizer
Agent
ccll si_MM
Lower-Level Optimizer
cal MTEQ
Problem
Oe
Visualization
Sample
finish learning
plot costeurve()
Testsuites
Truinsel
log
plot scorchist()
-Synihetic
generate table()
-Neisy-Syllhetic
-Protein-ooee Diniin
spiit
Tester
auqua
Testsct
Baseline Library
rest
Baselines
Classic Optimizers
Trained Agent
andicate
to Compare
MetaBBO-RL
Tog
MetaBBO-SL
Test statistics
AEI Scores, MGD, MTE...
Figure I:
Blueprint of our MetaBox platform. MetaBox offers template scripts for quick-start and, once completed by the user, it carries out an automatic Train-Test-Log process on the testsuites for comparison. Results such as performance scores, optimization curves, and comparative tables are automatically generated and made available for review.
identifying optimal configurations parameters that boost the overall performance across various problem instances within a given problem domain. This leads to a bi-level optimization framework, where the meta-level enhances the performance of low-level black-box optimizers. The meta-level approaches include supervised learning (MetaBBO-S)
16,,88, reinforcement learning (MetaBBO- RL) [19-25], and self-referential search (MetaBBO-S) [26, 27]. Among them, MetaBBO-RD models the optimizer fine-tuning as a Markov Decision Process (MDP) and learns an agent to automatically make decisions about the algorithmic configurations. By automating the tuning process, MetaBBO-RL not only significantly reduces the time and effort needed for customizing algorithms to specific unseen problems, but also potentially enhances the overall optimization performance.
Despite the success, MetaBBO-RL calls for unified benchmark platform. Upon examining the recently proposed MetaBBO-RL approaches [19-25], we found that while several approaches claim state-of-the-arr performance, they lack comprehensive benchmark and comparison using a standard- ized, unified testbed. As result, identifying which approach truly excels under specific conditions poses a significant challenge, thereby hindering further progress and advancement in this field.
To bridge the gap, we introduce MetaBox, the first benchmark platform for MetaBBO-PR The blueprint of MetaBox is illustrated in Figure 1, aiming to offer researchers a convenient means to develop and evaluate the MetaBBO-RI. approaches. Specifically, we have made the following efforts:
1. To simplify the development of MetaBBO-RL and ensure an automated workflow: 1) We introduce a MetaBBO-RR Template, depicted at the top left of Figure comprises two main components: the meta-level RL agent and the low-level optimizer, where we provided several implemented works (e.g., [19, 21, 24, 25]) following unified interface protocol. 2)
We automate the Srain-Test-Log procedure in MetaBox, shown in the middle of Figure The users simply need to complete their MetaBBO template, specify the target problem set, and select several baselines for comparison. Initiating the entire automated process is as straightforward as executing the run__xppppe (eeexeerimen() command. This provides users with considerable flexibility in implementing different types of internal logic for their MetaBBO- RL algorithms while benefiting from the automated workflow.
2. To facilitate broad and standardized comparison studies: 1) We integrate a large-scale MetaBox testsuite consisting of over 300 benchmark problems with diverse landscape characteristics (such as single-/multi- modal, non-ill/ill- conditioned, strong/weak global structured, and noiseless/noisy). Showing in the left centre of Figure 1, the MetaBox testsuite inherits problem definitions from the well-known COCO [28] platform and the Protein-Docking benchmark (version 4.0) [29], with several modifications to adapt to the MetaBBO paradigms. 2) We develop Baseline Library, located the bottom left in

---

Figure 1. The library currently encompasses wide range of classic optimizers [6, 8- 14. 30, 31] and up-to-date MetaBBO- RL approaches [19-25]. We additionally integrate a MetaBBO-SI. approach [16] to provide an extended comparison. Notably, all of the baselines are implemented by our MetaBBO-RI. Template. This ensures consistency and allows for fair and standardized comparison among the different approaches.
3. To comprehensively evaluate the effectiveness of MetaBBO-RL approaches: 1) We propose three Standardized Metrics to evaluate both optimization performance and learning effectiveness of MetaBBO- approach, including novel Aggregated Evaluation Indica- tor (AEI) that offers a holistic view of the optimization performance, a Meta Generalization Decay (MGD) metric that measures the generalization of a learned approach across different problems, and Meta Transfer Efficiency (MTE) metric that quantifies the transfer learning ability. 2) We conduct tutorial large-scale comparison study using Baseline Library, evalu- ate them on MetaBox testsuite by the proposed Standardized Metrics. Several key findings reveal that the pursuit of state-of-the-art performance continues to present challenges for current MetaBBO-RL approaches. Nonetheless, our MctaBox platform provides valuable insights and opportunities for researchers to refine and improve their algorithms.
To summarize, MetaBox provides the first benchmark platform for the MetaBBO-RI. community (novel). It is fully open-sourced, offering template scripts that facilitate convenient development, training, and evaluation of MetaBBO algorithms (automatic). Furthermore, MetaBox provides diverse benchmark problems and an extensive collection of integrated baseline algorithms, which will continue to expand through regular maintenance and updates (extendable).
2 Background and Related Work
As shown in 2, MetaBBO methods operate within a bi-level optimization framework designed to automate the fine-tuning process for given BBO optimizer. Distinguishing themselves from conventional BBO techniques, MetaBBO methods introduce a novel meta-level as an automatic decision process. The purpose is to alleviate the need for sabor-intensive manual fine-tuning of low-level BBO optimizers. Typically, they require the abil- ity to generalize behaviors to address previously unseen problems through extensive training in a given problem distribution. Concretely: 1) At the meta level, the meta optimizer (e.g., an RL. agent) dynamically configures the lowelevel optimizer based on the current optimization status at that particular time step. Then, the meta optimizer evaluates the performance of the low-level optimizer over the subsequent optimization steps, referred to as meta performance. The meta optimizer leverages this observed meta performance refine its doccision-maing process, training itself through the maximization of accumulated meta performance, thereby advancing its meta objective. At the lower level, the BBO optimizer receives designated algorithmic configuration from the meta optimizer. With this configuration in hand, the low-level optimizer embarks on the task of opti- mizing the target objective. It observes the changes in the objective values across consecutive optimization steps and transmits this information back to the meta optimizer, thereby contributing to the meta performance signal.
Meta Training
Meta-Level Optimizer
Meta oby
Configuration
Inner Optimization
Low-Level Optimizer
Target obj
Solution
Optimization
Problem
Figure 2: Illustration of the bi-level optimization procedure MetaBBO.
MetaBBO-RL. It leverages reinforcement learning the meta-level to configure the low-level black-box optimizer for boosting meta-performance [26] over problem distribution. It involves three components: an RL agent (policy) 1lg, a backbone black-box optimizer A, and a dataset D following certain problem distribution. The environment Env in MetaBBO-RL is formed by coupling the optimizer with a problem from the dataset, i.c., Enve = (A. f sample(D)), where f represents the sampled problem. Unlike traditional RL tasks [32, 33], the environments in MetaBBO- RL not only include the problem of optimizing but also include the low-level optimizer itself. At each step f, the agent queries the optimization status st, which includes the current solutions produced by A and their evaluated values by f. The policy II, takes 8, as input and suggests an action a; = [*)) to configure the optimizer A. Note that the action space of different MetaBBO algorithms can vary largely, as could involve determining solution update strategy [19, 20, 23], generating hyper- parameter [21, 24, 25], or both of the above two [22]. Subsequently, Env executes a, to obtain the
3

---

Table 1: Comparison to BBO benchmarks. We report #Problem: the number of problems (#synthetic + #realistic); #Baseline: the number of baselines; Template: Template coding support; Automation: automated train/test workflow support; Customization: configurable settings; Visualization: visualiza- tion tools support; and RLSupport: Gym-style [33] RL benchmark.
#Problem
#Boseline
Template
Automation
Customization
Visuolization
Rt.uupoort
COCO [28]
54-0
2
CBC [34, 36]
28+0
0
x
x
10Hprofile [38, 39]
24-0
0
v
x
V
x
Bayesmark [40, 41J
0+228
10
V
x
x
Zigzag [42, 43]
4+0
0
X
x
MetaBox
54+280
19
V
V
next optimization status 8(+1 and the reward ry that measures the meta- peeta-perforancce improvement of A on problem f. The meta-objective of MetaBBO-RL. is to learn a policy Il that maximizes the expectation of the accumulated meta-performance improvement r, over the problem distribution D, E. En= re], where T denotes the predefined evaluation budget, typically referred to the maxFEs parameter as in the existing BBO benchmarks [28, 34].
Other MetaBBO methods. Apart from MetaBBO-RL, two other paradigms include MetaBBO- [16-18 and MetaBBO-SR [26, 27], which all belong to the MetaBBO community but leverage different approaches at the meta-level. Existing MetaBBO-SL methods consider recurrent neural network (RNN) that helps determine the next solutions at each step [16-18]. However, they often encounter a dilemma in setting the horizon of RNN training: a longer length improves the difficulty of training, whereas a shorter length requires breaking the entire optimization process into small pieces, often resulting in unsatisfactory results. MetaBBO-SR utilizes black-box optimizers (such as an evolution strategy [35]) at both the meta and low levels to enhance the overall optimization performance. Since the black-box optimizers themselves can be computationally expensive, the MetaBBO-SR approaches may suffer from limited efficiency due to their inherently nested structure.
Related BBO benchmarks. Existing benchmarks for MetaBBO are currently absent. However, there are several related BBO benchmark platforms, including COCO [28] and 1OHprofiler [38, 39] for continuous optimization, ACLib [44] for algorithm configuration, Olympus [45] for planning tasks, and Bayesmark [40] for hyper-paramete tuning. These platforms provide testsuites, logging tools and some baselines for users to compare or refer to. In addition to these platforms, some BBO competitions provide a series of problems to evaluate specific aspects of algorithms, such as the GECCO BBOB workshop series (based on COCO [28]) the NeurIPS BBO challenge [41] (based on Bayesmark [40]), the IEEE CEC competition series [36], and the Zigzag BBO [42, 43] that consists of highly challenging problems. In contrast, our proposed MetaBox introduces novel benchmark platform specifically tailored for MetaBBO-RL. It provides algorithm development template, an automated execution philosophy, a wide range of testsuites, an extensive collection of baselines, specific MetaBBO performance metrics, and powerful visualization tools. In Table 1, we compare MetaBox and other related benchmarks to showcase the novelty of this work.
3 MetaBox: Design and Resources
3.1 Template coding and workflow automation
The core structure of MetaBox is presented on the left of Figure 3, shown in the form of UML class diagram [46]. Drawing from the recent MetaBBO-RL, we abstract the MetaBBO-RL Template into two classes: the reinforcement Agent and the backbone Optimizer (both marked in pale orange). To develop or integrate new MetaBBO-RE RR approach, users are required to specify their settings in the attributes Agent.connfig and Optimizer.com con fig, and implement the following interfaces: 1) ppent.train_eppe episode( (env) for the training procedure, 2) Aggentrooloutt rowlootecpissode(e(eeeeeeee.e__e rollout procedure, and 3) Optimizer. Spliinzzrr.udulleefee (action,, problem)) for the optimization procedure. Once these interfaces are set, the implemented templates seamlessly integrate with other components, requiring no extra adjustments. Except for the Agent and Optimizer classes, the other classes are hidden from users, simplifying coding tasks and ensuring code consistency.
for the

---

Template Agent
main process loop
Template Optimizer
run experiment()
Problem
inner loop
ccunlig: Dict
config: Diet
Agent.
train opisode(:: Diet
update(): Diet
evall:: Army
Trainer.train))
episode()
rollout episodor): Diet
stops
Env.step(
Trainer
Tester
Env
Optimizer.uddatee
config: Diet
config: Diat
oplimizer: Optimizer
problem: Problem
Tester.estt)
N runs
Agent.rollout,
episode()
agent: Template Agent
agenis:: List
+
trainset: List
optimizer: Optimizer
lestset: List
optimizers: List
+ stepO: Diet
reset(): Array
Env.stepO
train(): Dict
test(): Dict
Loggerlog(
Optimizer.upaate()
UML Class diagram of MetaBox. Users can inherit from MetaBBO-RL templates highlighted in orange to enable polymorphism. Right: The automated Test-Log workflow. The Agent directs the low-level (inner loop) optimization and trains itself on meta-level (outer loop), followed by testing and post- post-processing.
Figure 3:
The core structure and workflow of MetaBox. Left:
Given an implemented approach based on MetaBBO-RL Template, the right side of Figure 3 illustrates the entirely automated Train-Test-Lon workflow in MetaBox. The process initiates when a user launches the function run_ear jin__rperiment() Then, function Trainer ttriin)) executes outer-loop for meta-level iterations, where a runtime RL environment (env) is instantiated by pairing a problem from the trainset with an optimizer instance. The Agent.train Srrrrinnppisddeeeeeeee to start an inner-loop for the BBO iterations, during which the optimizer in env follows the action informed by Agent and applies its updale (action, problem) function to optimize the target BBO problem until the predefined function evaluation budget is exhausted. The meta-iteration ends when the Agent has trained its policy model for M steps. Next, the Tester. (esse) function is called to evaluate the Iearned policy on the testset through N independent runs, recording the statistic results. Finally, the Logger.log() function restores all saved training and testing results. These basic results are further post-processed into comprehensive metries for in-depth analysis. The key to this automation resides in the universal internal-call bridging the MetaBBO RLL Template and the Logt-Lo workflow, thereby obviating the need for users to grapple with complex coding tasks.
function is then called
3.2 Testsuites
The testsuites integrated into MetaBox are briefly described as follows (more details in Appendix A):
Synthetic. It consists of 24 functions in five groups: separable, moderate, ill-conditioned, multi- modal with global structure, and multi-modal with weakly structured. They are collected from the coco:bbob function set in the renowned COCO platform [28].
Noisy-Synthetic. It consists of 30 noisy functions from the coco:bbob-noisy function set [28], by extending the Synthetic set with different noise models and levels. The noises come from three different models: Gaussian noise, Uniform noise and Cauchy noise [47].
Protein- Drreein- is extracted from the Protein-Docking benchmark [29], with 280 instances of different protein-proter ppretenn complexes. These problems are characterized by rugged objective landscapes and are computationally expensive to evaluate.
Besides, we make the following efforts to adapt the above testsuites, while enhancing usability and flexibility: 1) We treat each testsuite as a dataset and split into training and testing sets according to particular proportion, referred to as difficulty. This proportion determines the level of difficulty in generalizing or transferring the learned knowledge to unseen instances during testing. We provide two modes for controlling this aspect: easy mode, allocating 75% of the selected instances for training, and difficuli mode, designating 25% of the selected instances for training. 2) Instead of providing predefined problem dimension candidates, such as 10,20,)) in COCO [28] and (10,30) in CEC2021 [34], we introduce an additional control parameter problem-din to support customized problem dimension. 3) We also recognize that there is potential user group (c.g., those in [16-18])
5

---

who require access to gradients of the objective during the training of their approaches".. To cater to this need, we provide a PyTorch function interface with tensor calculation support, enabling users to incorporate mack-propagation in their methods.
3.3 Baseline library
MetaBox leverages a total of 19 baselines, categorized into three types: 1) Classic Optimizer:
Random Search (RS), Bayesian Optimization (BQ) [14], Covariance Matrix Adaptation Evolution
Strategy (CMA-EES) [6], Differential Evolution (DE) [10] and its self-adaptive variants JDE21 [12],
MadDE [1]], NL-SHADE-LBE [13], Particle Swarm Optimization (PSO) [8] and its self-adaptive
variants GLPSO 191. sDMSPSO [30]. SAHLPSO [31]. 2) MetaBBO-RLE the Q-learning [48-50]
styled approaches DEDDQN [19], DEDQN [20], RLHPSDE [22], QLPSO [23], and the Policy Gradi-
ent [51, 52] styled methdos LDE [21], RLPSO [24], RLEPSO [25]. 3) MetaBBO-SL: RNN-OI [16].
The majority of the classic baselines are reproduced by referring to the originally released codes (when available) or the respective original papers. We meticulously filled the Optimizer template (see Section 3.1) with their specific internal logic. However, there are a few exceptions: CMA-ES, DE and PSO are implemented by calling APIs of DEAP [53] (an evolutionary computation framework with LGPLL-300 license), and BO is implemented by the Scikit-Optimizer [54] (a BO solver set with BSD-3-Clause license). This showcases the compatibility of MetaBox with existing open-sourced optimization codebases. For MetaBBO-RA RL, first encapsulate the internal logic of the RL agent into the Agent template (see Section 3.1), and then encapsulate the internal logic of the backbone optimizer into the Optimizer template. This showcases the case of integrating various MetaBBO-RL methods into MetaBox. In addition, we provide an example of a MetaBBO-SL. approach, RNN-OI, to showcase that MetaBox is compatible with other MetaBBO methods.
3.4 Performance metrics
In MetaBox, we implement three standardized metrics to evaluate the performance of MetaBBO-RL approaches from aspects of both the BBO performance and the training efficacy.
AEL The MetaBBO approaches can be evaluated using traditional BBO performance metrics, includ- ing the best objective value, the budget to achieve predefined accuracy (convergence rate), and the runtime complexity [28, 36, 39]. addition to analyzing them separately, we propose the Aggre- gated Evaluation Indicator (AEI), a unified scoring system to provide users with a comprehensive assessment. Aggregating these metrics together is challenging due to the significant variation in values across different metrics and problems. To address this, the AEI normalizes and combines the three metrics in the following way. We test MetaBBO-RL approach on K problem instances for N repeated runs and then record the basic metrics (usually pre-prose mrepprcesseed by min-max conversion for consistency towards AEI maximization, refer to Appendix B for details): best objective value Vabj" k,n
An consumed function evaluation times fes and runtime complexity v To make the values more distinguishable and manageable, they have been first subjected to a logarithmic transformation:
log(v,:>)
Then, Z-score normalization is applied:
N
kmn
z
(2)
where μt and or are calculated by using RS as a baseline. Finally, the AEI calculated by:
é T'on-- 7ham +
AEI
(3)
K
where Z-scores are first aggregated, then subjected to an inverse logarithmic transformation, and subsequently averaged across the test problem instances. A higher AEI indicates better performance of the corresponding MetaBBO-RL approach.
Imporrant to note that the gradients of the objective during testing not allowed in the context of BBO.

---

MGD. We then introduce the Meta Generalization Decay (MGD) metric, so as to assess the gener- alization performance of MetaBBO-RL for unseen tasks, Given model that has been trained on problem set B and its AEI on the corresponding testset as AETB. we train another model on problem set and record its AEI on the testset B as AETA. The MGD(A, B) is computed by:
AEIA
MGD(A, 100 (1
1%.
AEIB
where smaller (A, B) indicates that the approach generalizes well from A to B. Note that MGD has neither symmetry nor transitivity properties.
MTE. When zero-shot generalization is unachievable due to significant task disparity, the Meta Transfer Efficiency (MTE) metric proposed to evaluate the transfer learning capacity of MetaBBO- RL approach. We begin by maintaining checkpoints of a MetaBBO-RL approach trained problem set B. We locate the checkpoint with the highest cumulative return, recording its index as Tecratch- Next, we pre-train a checkpoint of the MetaBBO- RL approach on another problem set A. then load this checkpoint back and continue the training on the problem set B until reaches the same best-accumulated return, recording the current index finetune- The (A, B) is calculated by:
MTE(A, B) = 100 x (1 Thnetune )%,
(5)
Tecratch
where a larger (A, B) indicates that the knowledge learned in A can be easily transferred to solve B, while an MTE value less than or equal zero indicates potential negative transfer issues. Similar to MGD, the MTE has neither symmetry nor transitivity properties.
4 Benchmarking Study
MetaBox serves as valuable tool for conducting experimental studies, allowing researchers to benchmark specific groups of MetaBBO algorithms, 2) tune their algorithms flexibly, and 3) per- form in-depth analysis on various aspects including generalization and transfer learning abilities. In this section, we provide several examples to illustrate the use of MetaBox in experimental studies.
4.1 Experimental setup
To initiate the fully automated Train-Test-Log process of MetaBox, users need to follow two steps: 1) indicate environment parameters, including problem-type, problem-dim and difficulty, to inform MetaBox about the problem set, its dimension and the train-test split used in the comparison study; and 2) specify the maximum number of training steps (M), the number of independent test runs (N), and the reserved function evaluation times (maxFEs) for solving problem instance. Once these two steps are completed, users can trigger the run experiment() ( to enjoy the full automation process. In the exemplary study below, M, N and maxFEs are set to 1.5 x 10 51, and 2 x 10*, respectively, unless specified otherwise. All results presented are obtained using a machine Intel 19- 10-90980X CPU with 32GB RAM. Note that MetaBox is also compatible with other platforms. We provide default control parameters for each baseline, which are listed in Appendix C.
4.2 Comparison of different baseline (Me methods
We train all MetaBBO baselines (7 MetaBBO-RL and MetaBBO-SL) in the Baseline Library on Synthetic and Protein-Docking testsuites, using both easy and difficult train-test split. We then test these baselines against the classic black-box optimizers available in our library and calculate the AEI of each algorithm. The results are depicted in Figure 4. On the Synthetic testsuites, two MetaBBO- RL methods, namely LDE and RLEPSO, show competitive performance against classic baselines. Specifically, LDE outperforms JDE21, the winner of IEEE CEC 2021 BBO Competitions [34], on the Synthetic- difficult testsuites, while RLEPSO outperforms both JDE21 and SL-SHADE-LBC. testsuites. However,
the winner of IEEE CEC 2022 BBO Competitions [55], on the Synthetic-est for this particular problem set, CMA-ES remains a strong baseline. As we shift to more realistic testsuites (Protein-docking of 280 instances), most baselines show varying degrees of performance drop. Notably, CMA-ES inferior to a number of algorithms such as DEDQN, LDE, RIEPSS,, and GLPSO. In general, we observe that MetaBBO-RL methods more robust than classic hand-crafted
7

---

MrtaRRO-RL
Classic Optimiiser
MrtaBBO-SS.
Syathetie dasy
25
Synrhetioddffiolt
20
10
12.00
1.4
Pricein diffieu:
1.2
0.8
0.4
0.3
DEDDAgS
RLMPSDE
LOE
queso
RLPSO
ALEPSO
DE
JDE
MeiDE
ML-SIADE-LOS Loc
PSO
GLPSO
aUMarad
SAILLSS
CML4-E3
10
RNN
Figure 4: AEI scores of baselines, with error bars denoting the robustness across different tasks. Top: Results on Synthetic testsuites with easy difficult difficulty. Bottom: Results on Protein- -Docking testsuites with easy or difficult difficulty.
The moragg veturn
The normellned cast euree
The AET scores
Logged Numaliod Couto
7713-4
fremm
100
120
2000
11500
300
BR.50
Figure 5: Hyper-tuning for the MetaBBO-RI. approach LDE [21] using a 2 x 2 grid search. Left: The average return during the training over trials. Middle: The normalized cost over 51 independent runs during the test. Right: The corresponding AEI scores during the test.
optimizers when tested on different problem sets. For example, DEDQN generally performs the best for the Protein-Docking problems, ranking first and third in difficult and easy modes, respectively. This suggests that the classic optimizers might be only tailored to a specific family of problems, while MetaBBO-RL learns enhance the low-level optimizer through meta-level learning, thereby exhibiting stronger robustness. Due to the space limitation, we leave additional comparison results, including those on other testsuites, as well the detailed sub-metrics such running time, algorithm complexity, and per-inse instancee optimization results in Appendix D.
4.3 Hyper-tuning a MetaBBO-RL approach
Tuning a MetaBBO-RL requires efforts in configuring both the meta-level RL Agent and the low-level block-box Optimizer. Fortunately, our MetaBox provides Template coding with convenient interfaces to assist users in accomplishing this task. To illustrate this, we conduct 2 x 2 grid search to hyper-tune the MetaBBO-RL. approach LDE [21]. We noticed that LDE achieved lower AEI scores on Syntheticeaasy testsuites than some classic optimizers such as JDE21 and MadDE (see Figure 4). In our study, we investigate both REINFORCE [51] and PPO [52] RL agents for LDE, setting the population size (a
of DE) to either 30 50 (where the original version of LDE uses

---

Zeroscoo Geserulizatin
Fine-tuning, Syrsy-Syrthettie Synthotin
0,302
7,005
6.000
7.
Any. lettrr
2.222
10000
0017
WTE 11, -0
samppis
0.000
-7.00
-0.006
12
pre-train
T
0.300
Syda
Nitey
08
Louning Stepe
Figure 6: Meta performance (MGD and MTE) of LDE [21] across different tasks, tested under the easy mode. Left: Logits i-th and j-th column is the MGD(i. j), the smaller the better. Right: The average return over 10 trials is compared between the LDE models pre-trained and trained from scratch. The annotated MTE score shows that it saves 0.2x learning steps through fine-tuning.
REINFORCE and population size of 50). Figure 5 depicts the average return along the training, the optimization curve during testing, and the AEI scores of these four LDE versions. turns out that LDE with the PPO agent and population size of 30 achieves higher scores and surpasses JDE21 and MadDE. This shows that, by leveraging the capabilities of MetaBox and conducting hyper-tuning studies, researchers able pursue the best configurations for their MetaBBO-RI methods.
4.4 Investigating generalization and transfer learning performance
Existing MetaBBO-RL methods [19-25] rarely assess their generalization and transfer ability, al- though they are two crucial aspects for evaluating the effectiveness of learning-basee method. To bridge this gap, we have introduced two explicit indicators (namely the MGD and MTE) MetaBox to measure the level of learning achieved by MetaBBO-RL algorithm. In Figure 6, we display the values of MGD and MTE for the MetaBBO-RL approach LDE.
Generalization performance. Results in the MGD plot (the left part of Figure 6) show that the pre-trained LDE model possesses strong generalization ability. When the model is pre-trained on the Noisy-Synthetic testsuites, it exhibits a positive MGD on the Synthetic testsuites, with only 2.222% drop AEI, which considered acceptable. Additionally, the LDE model trained on the Betein-Doccknngeeat or Synthetic-easy testsuites both exhibits exceptional generalization to the Noisy-Synthetic-easy testsuites, as evidenced by an MGD of -7000% and -7.049% Such robust generalization abilities are highly desirable in practical applications.
Transfer performance. We fine-tune the LDE pre-trained the Moisy-Synthetic testsuites for the Synthetic testsuites. The progress of this fine-tuning process is depicted in the MTE plot (the right part of Figure 6), showing the average return over 10 trials. The results reveal that the pre-trained LDE requires fewer learning steps to arrive at the peak return level than the LDE trained from scratch, with the MTE value of 20%. Compared with the MGD value of 2. 2.222% when zero-shotting the pre-trained LDE for the Synthetic testsuites, it can be noticed that the MGD and MTE capture two different aspects of the learning effectiveness. By considering both measures, researchers can gain deeper insights into the learning behavior and effectiveness of their MetaBBO-RL. approaches.
Discussion and Future Work
We propose MetaBox, a benchmark platform for Meta-Black Boxxck-Box Optimization with Reinforcement Learning (MetaBBO-RL) It intends to, 1) provide the first unified benchmark platform, 2) simplify coding towards efficient researching, 3) provide broad testsuites and baselines for comprehensive comparison, and 4) provide novel evaluation metrics for in-depth analysis.
Our preliminary experimental investigations lead to the following key observations. First, although hand-crafted optimizers currently outperform learning-based optimizers (e.g., MetaBBO-RL.), they are mostly designed specific problem sets and become less effective when applied other problem domains. In contrast, MetaBBO-RL exhibits adaptability across various optimization scenarios,

---

showing stronger robustness across different problem domains. This is a significant advantage, as it allows MetaBBO-RL to perform well on diverse problem sets without the need for much manual customization. Second, there is still room for further improvement in MetaBBO-RI by discovering more effective designs in both meta-level agents and low-level optimizers. We find that even basic modifications, such as incorporating PPO agent and adjusting the population size, can bring significant performance boost. Third, interpreting the generalization and transfer effects in MetaBBO- RL can be challenging. While we were able to observe positive and negative effects in terms of MGD and MTE, rationally interpreting and understanding them is still difficult. Hence, additional studies needed to interpret the learning effects of MetaBBO-RI R-RL in more comprehensible manner.
While our MetaBox presents a significant advancement, we also acknowledge its potential limitations. It is important to note that the evaluation of BBO performance is not a one-size- mic-size-iits-all endeavour. Different practical applications may have varying preferences and additional concerns, such as solution robustness, solution diversity, parallelization and scalability. These nuances extend beyond the scope of a single evaluation metric. conjunction with the proposed AEI, MetaBox is committed to exploring more pragmatic and advanced evaluation criteria. Moreover, the problem sets in MetaBox will be expanded to include more diverse and eye-catching BBO tasks. Additionally, proactive maintenance and updates to MetaBox, including the extension of the baseline library, will also be pursued to ensure it is up-to-date with the latest methodologies in the field.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Natural Science Foundation of China under Grant 62276100, in part by the Guangdong Natural Science Funds for Distinguished Young Scholars under Grant 0022B151502010 in part by the Guangdong Regional Joint Funds for Basic and Applied and in part by the TCL. Young Scholars Program.
Research under Grant
References
[1] Matthias Feurer and Frank Hutter. Hyperparameter optimization. Automated Machine Learning: Methods, Systems, Challenges, pages 3-33, 2019.
[2] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1):1997-2017, 2019.
[3] Graham R Smith and Michael JE Sternberg. Prediction of protein-protein interactions by docking methods. Current Opinion in Structural Biology, 12(1):28-35, 2002.
[4] John Holland. Adaptation in natural and artificial systems. MIT press, 1975.
[5] Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolu- tion strategies. Evolutionary Computation, 9(2):159-195, 2001.
[6] Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary Computation, 1 11(1):1-18, 2003.
[7] Raymond Ros and Nikolaus Hansen. A simple modification in cma-es achieving linear time and space complexity. In Parallel Problem Solving from Nature-PPST X: 10th International Conference, Dortmund, Germany, September 13-17, 2008. Proceedings 10, pages 296-305. Springer, 2008
[8] James Kennedy and Russell Eberhart. Particle swarm optimization. Proceedings of ICNN'95- International Conference on Neural Networks, pages 1942-1948, 1948. IEEE, 1995.
[9] Yue-Jiao Gong, Jing-Jing Li, Yicong Zhou, Yun Li, Henry Shu-Hung Chung, Yu-Hui Shi, and Jun Zhang. Genetic learning particle swarm optimization. IEEE Transactions on Cybernetics, 46(10):2277-2290 2015.
[10] Rainer Storn and Kenneth Price. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4):341, 1997.
10

---

[11] Subhodip Biswas, Debanjan Saha, Shuvodeep De, Adam D Cobb, Swagatam Das, and Brian A Jalaian. Improving differential evolution through bayesian hyperparameter optimization. In 2021 IEFE Congress on Evolutionary Computation (CEC), pages 832-840. IEFF, 2021.
[12] Janez Brest, Mirjam Sepesy Maučec, and Borko Bošković. Self-adaptive differential evolution algorithm with population size reduction for single objective bound-constrained optimization: Algorithm j21. In 2021 IEEE Congress on Evolutionary Computation (CEC), pages 817-824.
IEEE, 2021.
[13] Vladimir Stanovov, Shakhnaz Akhmedova, and Eugene Semenkin. NI-shade-Ibe algorithm with linear parameter adaptation bias change for cec 2022 numerical optimization. In 2022 IEEE Congress on Evolutionary Computation (CEC), pages 01-08. IEEE, 2022.
[14] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, volume 25, 2012.
[15] Jasper Snoek. Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable bayesian optimization using deep neural networks. In International Conference Machine Learning, pages 2171-21808 2015.
[16] Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap. Matt Botvinick, and Nando Freitas. Learning to learn without gradient descent by gradient descent. In International Conference Machine Learning, pages 748-756. PMLR, 2017.
[17] Vishnu TV, Pankaj Malhotra, Jyoti Narwariya, Lovekesh Vig. and Gautam Shroff. Meta-learning for black-box optimization. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16-20, 2019, Proceedings, Part II, pages 366-381. Springer, 2020.
[18] Hugo Siqueira Gomes, Benjamin Léger, and Christian Gagné. Meta learning black-box population-based optimizers. arXiv preprint arXiv:2103.03526, 2021.
[19] Mudita Sharma, Alexandros Komninos, Manuel López-lbáñez, and Dimitar Kazakov. Deep reinforcement learning based parameter control in differential evolution. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 709-717. Association for Computing Machinery, 2019.
[20] Zhiping Tan and Kangshun Li. Differential evolution with mixed mutation strategy based on deep reinforcement learning. Applied Soft Computing, 111()):00678, 2021.
[21] Jianyong Sun, Xin Liu, Thomas Bäck, and Zongben Xu. Learning adaptive differential evolution algorithm from optimization experiences by policy gradient. IEEE Transactions on Evolutionary Computation, 25(4))666 680, 2021.
[22] Zhiping Tan, Yu Tang, Kangshun Li, Huasheng Huang, and Shaoming Luo. Differential evolution with hybrid parameters and mutation strategies based on reinforcement learning. Swarm and Evolutionary Computation, 75(-):10119)) 1994,, 2022.
[23] Yue Xu and Dechang Pi. A reinforcement learning- based communication topology in particle swarm optimization. Neural Computing and Applications, 32()::0007-11032, 2020.
[24] Di Wu and G Gary Wang. Employing reinforcement learning enhance particle swarm optimization methods. Engineering Optimization, 54(2):329 -448, 2022.
[25] Shiyuan Yin, Yi Liu, GuoLiang Gong, Huaxiang Lu, and Wenchang Li. Rlepso: Reinforcement learning based ensemble particle swarm optimizer* In 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence, pages 1 1-6. Association for Computing Machinery, 2021.
[26] Robert Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, and Sebastian Flennerhag. Discovering attention-based genetic algorithms via meta-black-box optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 929-937, 2023.
11

---

[27] Robert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flennerhag. Discovering evolution strategies via meta-black-box optimization. In The Eleventh International Conference on Learning Representations, 2023.
[28] Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff. Coco: A platform for comparing continuous optimizers in a black-box setting. Optimization Methods and Software, 36(1): 144-144, 2021.
[29] Howook Hwang, Thom Vreven, Joël Janin, and Zhiping Weng. Protein protein docking benchmark version 4.0. Proteins: Structure, Function, and Bioinformatics, 1111513311-3314, 2010.
Protein-protein
[30] Jing J Liang, L Guo, R Liu, and Bo- Yang Qu A self-adaptive dynamic particle swarm optimizer. In 2015 IEEE Congress Evolutionary Computation (CEC), pages 3206-332 3213. IEEE, 2015.
[31] Xinmin Tao, Xiangke Li, Wei Chen, Tian Liang, Yetong Li, Jie Guo, and Lin Qi. Self-adaptive two roles hybrid learning strategies- based particle swarm optimization. Information Sciences, 578:457 0817,44811111 2021.
[32] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 JEEE/RSJ international Conference Intelligent Robots and Systems, pages 0026-5033. IEEE, 2012.
[33] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint a 540, 2016.
[34] Ali Wagdy Mohamed, Anas Hadi, Ali Khater Mohamed, Prachi Agrawal. Abhishek Kumnr, and P. N. Suganthan. Problem definitions and evaluation criteria for the cee 2021 special session and competition on single objective bound constrained numerical optimization. In Tech. Rep. Singapore: Nanyang Technological University, 2021. URL https::/wwwtt ntu. edu. sg/ htm.
home/ hetpe//nnuunnnnnneeeee http/indd___fif (ttele/e//E/22//////0000
[35] Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as scalable alternative to reinforcement learning. arXiv preprint (703.0/770.03355 2017.
[36] Urban Škvore, Tome Eftimov, and Peter Korošec. Cec real-parameter optimization competitions Progress from 2013 to 2018. In 2019 IEEE Congress on Evolutionary Computation (CEC), pages 113363133. IEEE, 2019.
[37] Jing Liang, BY Qu, Ponnuthurai Nagaratnam Suganthan, and Alfredo G Hernández- Díaz. Problem definitions and evaluation criteria for the cec 2013 special session on real-parameter optimization. In Proceedings of the 2013 IEEE Congress on Evolutionary Computation, 2013.
[38] Carola Doerr, Hao Wang, Furong Ye, Sander Van Rijn, and Thomas Bäck. Iohprofiler: bench- marking and profiling tool for iterative optimization heuristics. arXiv preprint arXiv: arXiv:1810.0522 1810.05281, 2018.
[39] Jacob de Nobel, Furong Ye, Diederick Vermetten, Hao Wang, Carola Doerr, and Thomas Bäck. IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics. arXiv e-prints:2111.04007., 2021.
[40] R.Turner and D.Erissson. Bayesmark: Benchmark framework to easily compare bayesian optimization methods on real machine learning tasks, 2019. URL https://gii com/ uber/ lbbr/bbaysmark.
[41] Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, and Isabelle Guyon. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In NewrIPS 2020 Competition and Demonstration Track, pages 3-26. PMLR, 2021.
[42] Jakub Kudela. Novel zigzag-based benchmark functions for bound constrained single objective optimization. 202 IEEE Congress on Evolutionary Computation (CEC), pages 857-862.
IEEE, 2021.
12

---

[43] Jakub Kudela and Radomil Matousek. New benchmark functions for single-objective optimiza- tion based on a zigzag pattern. IEEE Access, (K-)::8262-8278, 2022.
[44] Frank Hutter, Manuel López-lbánez, Chris Fawcett, Marius Lindauer, Holger H Hoos, Kevin Leyton-Brown, and Thomas Stützle. Aclib: A benchmark library for algorithm configuration. In Learning and Intelligent Optimization: 8th International Conference, Lion 8, Gainesville, FL. USA. February 16-21, 2014. Revised Selected Papers 8, pages 36-40. Springer, 2014.
[45] Florian Häse, Matteo Aldeghi, Riley J Hickman, Loic M Roch, Melodie Christensen, Elena Liles, Jason Hein, and Alán Aspuru-Guzik. Olympus: a benchmarking framework for noisy optimization and experiment planning. Machine Learning: Science and Technology, 2(3): 035021, 2021.
[46] Grady Booch. The unified modeling language user guide. Pearson Education India, 2005.
[47] Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger. Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions. PhD thesis, INRIA, 2009.
[48] Volodymyr Mnih, Koray Kavukeuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533 2015.
[49] Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
[50] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine Learning, 8(-):279 292, 1992.
[51] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. The Springer International Series in Engineering and Computer Science, 173:
5-32, 1992.
[52] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint -XXiv1707.00347,, 2017.
[53] Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner Gardner, Marc Parizeau, and Christian Gagné. Deap: Evolutionary algorithms made easy. The Journal of Machine Learning Research, 13(1):2171-2175 2012.
[54] Gilles Louppe and Manoj Kumar. Scikit-optimize, 2016. URL https://github. com/ ceikitt-optimiz/ssciitt optimize.
[55] Abhishek Kumar, Kenneth V. Price, Ali Wagdy Mohamed, Anas A. Hadi, and P. N. Sug- anthan. Problem definitions and evaluation criteria for the cec 2022 special session and competition single objective bound constrained numerical optimization. In Tech. Rep. Sin- gapore: Nanyang Technological University, 2022. URL /aatps://wwwtt ntu. edu, sg/home/ betigganninnd___feee
(EEEC2222/CEC2222 htm.
[56] Yue Cao and Yang Shen. Bayesian active learning for optimization and uncertainty quantification in protein docking. Journal of Chemical Theory and Computation, 16(8):5334-5347, 2020.
[57] lain H Moal and Paul A Bates. Swarmdock and the use of normal modes in protein-protein docking, International journal of molecular sciences, 11(10)33633-368. 2010.
[58] Alex MacKerell Jr, Donald Bashford, MLDR Bellott, Roland Leslie Dunbrack Jr, Jeffrey D Evanseck, Martin J Field, Stefan Fischer, Jiali Gao, H Guo, Sookhee Ha, et al. All-atom empirical potential for molecular modeling and dynamics studies of proteins. The Journal of Physical Chemistry, 102(18):356-33666 1998.
[59] Brian G Pierce, Kevin Wiehe, Howook Hwang, Bong-Hyun Kim, Thom Vreven, and Zhiping Weng. Zdock server: interactive docking prediction of protein-protein complexes and symmetric multimers. Bioinformatics, 30(12): 2014.
13

---

MetaBox: A Benchmark Platform for Meta-Black-Bon Optimization with Reinforcement Learning (Appendix)
A Details of Testsuites
A.1 Synthetic
There are 24 functions in the Synthetic testsuites. These noiseless synthetic functions, established by Hansen al. [47], taken from COCO [28]. The mathematical definitions and formulations
of these functions can be found in Hansen et al. [47]. We have listed these functions Table 2. To provide different levels of difficulty,
have divided the 24 functions into parts: 25% and 75%. The 25% portion includes functions No. 1, 5, 6, 10, 15, and 20, while the remaining 18 functions make up the 75% portion. the easy mode, the 75% portion is used for training, and the 25% portion is used for testing, On the other hand, in the difficult mode, the 25% portion is used for training, and the 75% portion used for testing. The maximum number of function evaluations (maxFEs) for problems in these test suites is set to 2 x 10
Table Summary of the Synthetic test functions.
No.
Functions
Sphere Function
700
Ellipsoidal Function
1900
Separable functions
Rastrigin Function
100
Buche-Rustrign Function
1000
Linear Slope
2000
6
Attractive Sector Function
400
Functions
Step Ellipsoidal Function
300
with low or moderate conditioning
Rosenbrock Function, original
1300
9
Rosenbrock Function, rotated
1300
10
Ellipsoidal Function
600
Functions with high conditioning and unimodal
Diseus Function
900
12
Bent Cigar Function
2000
13
Sharp Ridge Function
2400
14
Different Powers Function
1500
15
Rastrigin Function (non-separable counterpart of F3)
1700
Multi-modal functions with adequate
16
Weierstrass Function
1400
17
Schaffers F7 Function
200
18
Schaffers F7 Function, moderately ill-conditioned
700
global structure
19
Composite
Function F8F2
1700
20
Schwefel Function
2100
Multi-modal functions with weak global structure
Gallagher's Gaussian 101-me Peaks Function
700
22
Gallagher's Gaussian 21-hi Peaks Function
2400
23
Katsuura Function
600
24
Lunacek bi- bi-Rastrigin Function
1200
Default search range: [-5, 5)
A.2 Noisy-Synthetic
There are 30 functions in the Noisy-Synthetic testsuites. These noiseless synthetic functions, es- tablished by Hansen et al. [47], are taken from COCO [28]. The mathematical definitions and formulations of these functions can be found in Hansen al. [47]. We have listed these functions in

---

Table 2. To provide different levels of difficulty, we have divided the 24 functions into two parts: 25% and 75%. The 25% portion includes functions No. 5, 15, 16, 17, 19, 20, 25, while the remaining 22 functions make up the 75% portion. The maximum number of function evaluations (maxFFs) for problems these test suites set to 2 10
Table 3: Summary of the Syythetic test functions.
No.
Functions
Sphere with moderate gaussian noise
700
Sphere with moderate uniform noise
1900
Functions
3
Sphere with moderate seldom cauchy noise
100
with
Rosenbrock with moderate gaussian noise
1000
moderate noise
5
Rosenbrock with moderate uniform noise
2000
Rosenbrock with moderate seldom cauchy noise
400
Sphere with gaussian noise
100
Sphere with uniform noise
2000
Sphere with seldom cauchy noise
2000
10
Rosenbrock with gaussian noise
500
11
Rosenbrock with uniform noise
400
12
Rosenbrock with seldom cauchy noise
700
Functions with
13
Step ellipsoid with gaussian noise
1900
14
Step ellipsoid with uniform noise
800
severe noise
15
Step ellipsoid with seldom cauchy noise
2300
Ellipsoid with gaussian noise
200
17
Ellipsoid with uniform noise
100
Ellipsoid with seldom cauchy noise
200
19
Different Powers with gaussian noise
500
20
Different Powers with uniform noise
1600
21
Different Powers with seldom cauchy noise
1000
22
Schaffer's F7 with gaussian noise
2400
23
Schaffer's F7 with uniform noise
1400
Highly multi-modal functions
24
Schaffer's F7 with seldom cauchy noise
2400
25
Composite Griewank-Rosenbran with gaussian noise
2200
26
Composite
with uniform noise
400
with
27
Composite Griewank-- with seldom cauchy noise
2500
severe noise
28
Gallagher's Gaussian Peaks 101-me with gaussian noise
TO00
29
Gallagher's Gaussian Peaks 101-me with uniform noise
600)
30
Gallagher's s Gaussian Peaks 101-me with seldom cauchy noise
2100
Default search range: [-5, 5)
A.3 Protein-Docking
Ab initio protein docking poses a significant challenge in optimizing a noisy and costly black-box function [56]. The objective of ab initio protein docking, for fixed basis protein conformation 2o, is to minimize the change in Gibbs free energy resulting from protein interaction between Zo and any other conformation æ, denoted as 20) Following the approach of Moal and Bates [57], formulate the objective as follows:
min Eine (I, xv)
Σ
Σ
E(x" xx))
(6)
where E(x",ag) the energy between any pair atoms of and Fo, and is defined as
mill
if Tij <
Exj-
(raft
Piroes +2 Bron)
qus +
Bug
Rug
if 7 r. S 9
(7)
(raf/-ron)*
if Mig
All parameters and calculations are taken from the Charmm19 foree field [58]. switching function employed, between and 9, to disregard long-distance interaction energy. To construct the task

---

instance for protein docking, we select 28 protein prreenn complexes from the protein docking benchmark set 4.0 [29]. Each complex is associated with 10 different starting points, chosen from the top-10 10 start points identified by ZDOCK [59]. Consequently, the Protein-Docking testsuites comprise total of 280 docking task instances. We split these instances into two parts, with approximately 75% and 25% of the instances serving as the difficulty levels within the testsuites. It is important to note that we parameterize the search space as IR 12 which is a reduced dimensionality compared to the original protein complex [56]. This reduction in dimensionality enables computational time savings while retaining the optimization nature of the problem. Due to the computationally expensive evaluations, the maximum number of function evaluations (maxFEs) is set 103, The default search range for the optimization is [-00, oo].
B Details of AEI Calculation
We describe the calculations of AEI as follows. Suppose want (Meta)BBO approach A to obtain its AEI on Testsuites A, i.e., AEI(A,A). Suppose A has problem instances, and we run A to optimize each problem in A for N times. In this testing process, we record kinds of values:
Best objective value. The best objective value a knn denotes the best objective found by A on the k-th problem during the reth test run. To convert its monotonicity, conduct an inverse transformation, Le., k.n
obj
Consumed function evaluations. We let fes denote the number of consumed function evaluations when the approach A terminates the optimization process for the k-th problem during the nith test run, where A would terminate the optimization either upon locating a high-quality solution that fulfils predetermined accuracy or when it reaches the maximum allowed function evaluations, i.e., maxFEs. We then pre-process this value by a mix-max conversion for the consistency towards AEI maximization, i.c., fes
mozFEs
Runtime complexity. The calculation includes three steps following the conventions of IEEE CEC Competitions [34, 55]. We first pre-caleulate To which is basic normalization term, calculating the computation time of some basic numpy operations (e.g., add, division, log, exp) as references. We record the time T for A to evaluate the k-th problem's 's objective during n-th run, and T as total running time for A to solve the k-thh problem's objective during n-th run. The runtime complexity
coo is then calculated by T -
Then we pre-preeoo pre-process this value by a min-max conversion for consistency towards AEI maximization, i.e.,
To
k.n (obj
obj
C
Baseline Setup
Classic optimizer. Random Search does not necessitate any control parameters. We establish the control parameters for CMA-ES, DE, and PSO in alignment with DEAP [53], while the control parameters for BO set in accordance with Scikit-Optimizer [54]. For the classic optimizers implemented by us, the control parameters are delineated and can be accessed at httts://gitthtii WWw.C//R...tett//////...// http:////oobo//ooo/aoeeeaaaaaaat http//rcccccc/// http////cc/c/otooeteeeeooeee
http://ooooooo_paaaeeeeeeeeee pptplpptaraaraaaaee wwwaaaaeteersclassiiiioeeea md.
MetaBBO-SL. We briefly introduce the control parameters of RNN-OI here. It adopts an LSTM network to generate the next solution position on the observation of some present optimization status. This LSTM has input size as problem_pne problem_dine The Adam optimizer is used to train the network with fixed learning rate 10-5 However, can only optimize problem for around mar F.s 100 due to the inefficiency arising from utilizing RNNs to model extended optimization horizon.
+ 2, hidden size as 32, and projection size as
3

---

MicaHBB--RI
Chodde Optimitue
MrtaBBO-SI
Natay-caay
DRIMA
RUIPSII
GE/80
RLASO
REBPU
DiC
inea
ModhE
Pio
GLAS
SARLING
CB1 Rs
Roscol
Figure 7: AEI scores with error bars of baselines on Soisy-Synthetic testsuites.
HeneBB--B
Cleser Optimzer
DEAP DE
1.0
1.0
J0E21
MarDE
FOLOOOO
0.8
DEAP PSO
DEDON
PSO
ALUPRDDE
5BMS, FSO
LDE
SAH
0L-50
DEAP CMAES
ALEPSO
Randdnn SwaThh
2500
20000
13500
FEx
Figure 8: Average cost curve of baselines over Noisy-Synthetic-assy testsuites.
D Additional Results and Comparisons
D.1 Additional results
In addition the AEI scores, MetaBox provides additional experimental results that complement the comprehensive analysis of MetaBBO-RL approaches. These results, along with AEI, are automati- cally generated and presented after the testing process. The currently available results for baselines in Baseline Library can be accessed at https: // /g github. com Com//MC-DDDD DRL/MetaBox//bne /bbb//////////////////////////// hoobmmmmn///// post_pprocessed dat./ www.ddaaattteeeeeeee content. md. We briefly preview these additional results here:
Overall performance table. For each baseline in the Baseline Library, it presents the average final objective value (Obj), the average Gap with respect to the performance of CMA-ES, and the average consumed function evaluation times (FEs). These values are obtained based on 51 independent runs.
Wall-time table. For each baseline in the Baseline Library, it presents the average running time KxN E Ex= Thk and the average algorithm complexity Z-score 1 Ek=1 Z Z These values averaged over all the problems in the testsuites and 51 independent runs.
Cost curve. For each baseline in the Baseline Library, the figure draws the averaged optimization curves (cost curves) over all the problems in the testsuites and 51 independent runs.
In this appendix, as preview examples, we present the experimental results of baselines on the Noisy- Synthetic testsuites, which includes AEI scores with error bars on both the Moisy-Synthettc--es the Noisy-Synthetie Syftheei--difffiuul testsuites in Figure the overall performance table on Noisy-Synthetie easy testsuites in Table 4, the wall-time table on Poiss-Syntheetic-e cost curves of MetaBBO-RL. baselines on miisy-Synthetic-een testsuites in Figure Note that they are partially displayed due to space limitations, without loss of generality.
and
testsuites in Table 5, and the
D.2 Investigation on RLEPSO
Hyper-tuning. Since RLEPSO [25] also shows competitive performance in all the three testsuites. execute the same x 2 grid-search experiment for and measure its MGD and MTE. Similarly,

---

Table 4: Per-instance optimization results of baselines on the first 3 problems in the test set for illustration purpose.
testsuites, showing
Sphere moderate guuss
Rusenbruck moderate umiform
Step- Ellipoiiaal -itypppllioiillacccchyy
Obj
Gup
FEs
Obj
Gap
FES
Obj
Gap
FEs
1
2.0000::44
2.26711:+3
2,000:++4
6.3631:=2
2.0008:444
Random
1.000
1,000
1.000
(2.739E-0)
(0.000E:4)
(1.0233+3)
(0,000E+0)
(2.709E+2)
(0.00085+0)
CMA-FS
(1.844E-9)
7.6891-9
0.000
(1.931E+2)
4
(2.696E+0)
1,1471++0
0.000
(2.174E+3)
L6241:44
(4.048E+1)
3,7431=0
0.000
6.119154
(3.629E3+3)
1,9064+0
2.000E++
2.679E+2
2,000E+4
2,950E+2
2.000E+4
PSO
(8.624E-1)
0.173
(0.000E+0)
(3,578E+2)
0.118
(0.000E+0)
(2,453E+2)
0.459
(0.000
SAHLPSO (-2)
(2.875E-0)
4.477E+0
0:407
(0,000E)))
2.000F+44
(9.798E+2)
9.651F+2
0.426
(0,000++))
2,000F
(2.584E+2)
4.
0.655
(0.000E:+0)
2.000F+44
1.661E40
2,010E+4
1,080E+2
2,010++4
1.395E+2
2,010E+4
sDMSPSO (v1)
(7,380E-1) I)
0.151
(0:000E+0)
(4.728E+1)
0.047
(0.000
(1.922E+2)
0.212
(0.000E+0)
Clussic
GLPSO
(7.4971E-7 7)
1.292E-6
0.000
(0,000E++))
2.000F+44
(8.194I:1)
6.867E+0
(0.000E+D)
2,000F +4
(3.1951+2)
3.322F
0.518
(0,000E+0)
2.000F
DE (rand/1)
(1.636E-9)
8.010E-9
0.000
(L.794E+2)
4.601E+3
(1.373E+0)
5.719E+0
(0 (.0000++))
2,000E+4
(2.041E+2)
1.3739++
0.209
(0.000E+0)
2.00002+4
5.302F-9
6.2831+
6.212E+0
2,001
2.227E+2
2.001E+4
JDE21
(2.4771-9)
-0,000
(1.562I++3)
(5.923E+00)
0.002
(0.0000+0)
(2,44DE++2
0.344
(0,000E:00)
7,826E-9
LA4E++
4,541E+0
2,000E+4
1.8494+2
2.000
NL-SHADE-LBC
(1,494E-9)
0.000
(L1161++2)
(8. 124E-1)
0.001
(0.000E+0)
0.284
(0.0001E+0)
7,988E-9
1994E+
5.361E+0
2,000E+4
1.613E+2
2.000
MadDE
(1.602E9)
0.000
(L-438E+2)
(1.083++0)
0.002
(0.000E+0)
(2.382E+2)
0.247
(0,000E+0))
9. 14442
1 1
4 4,506E+3
1,000E
1.024E+3
1.000++2
Bayesion (BO)
(1.1977-1)
0.008
(0.000F++0)
1,988
(0.000F
(2.678F 1)
1.615
(0.000F++0)
2.568IE+0
2.0001+:4
A467EF2
2,000E
1,4111422
2.000
QLPSO
(2.115E-0)
0.233
(0.000E+0)
(2.788E+2)
0.064
(0.000E+0)
(2.387E+2)
0.215
(0.000E++0)
2.419E-D
2.000E:44
1.607
2,000E+4
1,440E+2
2.000E+44
RLPSO
(1,277E-0)
0.220
(0.000E:+))
(1.164E+2)
0.070
(0.000E+0)
(188433+2)
0.219
(0.000E2+0)
7.521E-6
2.000E+4
7.244E+0
2,000E+4
1,580E+2
2.000E+4
RLEPSO
(1.163E-5)
0.000
70077+00
(2.131F++0)
0.003
(1.473F-0)
(1.824 2)
0,241
(1 (655
3
2.000E++4
2.1522
2.000E+4
1.0971+3
2.000E++4
Learnable
DEDON
(9.225E-0)
3.060
(0.000):++)
(9.220++3)
9.498
(0,000E+))
(4.199H+1)
1,730
(0.000E:+0))
7,907E-9
1 1,692E+
2.913E+0
1.977E+4
1.022E+2
2.000E++
DEDDON
(1,523E-9)
0.000
(5. (5.373E+22
(2,859+00)
0.001
(7.700E 2)
(I.726
0.153
(0.000F+0)
8,127E.9
9
2:681E+0
2,000E++4
1,2921++2
2.000F::44
(1.628E-9)
0.000
(3.920E422
(2. 6399))4)
(0.0000++))
(2.0071E+2)
0.196
(0.000E+0)
L118E-1
2.025E+4
4,248E+1
2,025E+4
6. 6.269E+2
2,025E+4
RLHPSDE
(3.738--2)
0.010
(0.000E+0)
(2. 1.)26E+1)
(0.000E+0)
(3,087E42)
0.985
(0.000E+0)
RNN-OI
(6.824E-1)
6.940F+1
6.304
(0.000E+0)
1.000F++
(6.3471422
3.177
14.021
(0.000E+0)
1.000F+2
(1.1981+1)
1.09F
1.734
(0.000E+0)
1.000F+22
Table 5: Running time and algorithm complexity of baselines on Soiisy-Synthetic-cmm easy testsuits.
Baselines
CMA-ES
PSO
SAIILPSO
sDMSPSO
GLPSO
DE
T2 (ms)
3.337E+02
2.021E+03
4.506E+03
6.484E+01
5,508E+01
8.838E+02
Z-score
9.540E-01
8.093E-01
7.254E-01
1047E+00
100GE+00
8.420E-01
Baselines
JDE21
SL-SHADE-LBC
MadDE
QLPSO
RLPSO
RLEPSO
T2 (ms)
1639E++02
2.036F+02
4.015E+02
4.442F+03
1,3376+04
3.540F+02
Z-score
9.588E-01
9.370E-01
8.827E-01
7.265E-01
6.530E-01
8.918E-01
Baselines
DEDQN
DEDDQN
LDE
RLHPSDE
RNN-OI
BO
T2 (ms)
2,309++02
2.787E+04
3.303B+02
5.867E+02
4,846F+03
1,407++07
Zescore
9.262F-01
6.101E-01
8,972F-01
8.517F-01
7.216F-01
3.542F-01
investigate the combination of different RL methods (REINFORCE or PPO) and different population sizes (100 or 50). The original RLEPSO is implemented in PPO with 100 population size. The grid search results are shown in Figure 9. The RLEPSO with the PPO agent and a population size of 50 has the highest average return during the training, however, this configuration shows the worst optimization results (considering the normalized cost) during the test. On the contrary, RLEPSO with the REINFORCE agent and population size of 50 has the lowest average return during the training and the lowest normalized cost during the test. This may indicate that the design of the RLEPSO reward function may not be appropriate. Mcanwhile, we note that for RLEPSO, changing its RL agent from PPO to REINFORCE may slightly improve the overall performance.
Generalization performance. We also present the MGD of RLEPSO on the testsuites with an easy difficulty level, displayed on the left side of Figure 10. It appears that RLEPSO exhibits poor generalization ability. For instance, when the RLEPSO model is pre-trained on the Protein-Docking easy testsuites, there is a significant 40.312% drop the AET score. These results indicate that while
RLEPSO achieves competitive AEI performance, its generalization ability is much worse compared
5

---

The awernoe Intaum
pommaelloed curve
acme
PrGia
-00
Laornnng Steps
FR
PPO 103
BE. 100
Bi
Figure 9: Hyper-tuning for the MetaBBO-RL approach RLEPSO [25] using a 2 x 2 grid search. Left: The average return during the training 10 trials. Middle: The normalized over 51 independent runs during the test. Right: The corresponding AEI scores during the test.
LDE. The subpar MGD performance of RLEPSO may be attributed to its non-generalizable state design, which directly utilizes the consumed FEs as the state representation.
Transfer performance. We performed fine-tuning the RLEPSO model that was pre-trained on the Seiyy-Sythetti-eeasy testsuites for solving the Synthetic-easy testsuites. The progress of the average return over 10 trials is shown on the right side of Figure 10. However, the pre-trained RLEPSO exhibits negative transferability, with an MTE value of O. This transfer failure suggests that some of the current MetaBBO- methods face overfitting issues, which in the case of RLEPSO may be attributed to its oversimplified state representation or other design elements that lack generalizability.
Zero-shot Generalization
Syrtheeee Symthetie
100
pre-tzain
40 312
4,974
010000
00
Ang Rotezn
10
aperpie
10000
110,05
Syullers
Nrdey Systhette
Farass
Docking
Leuraing
Figure 10: Meta performance (MGD and MTE) of RLEPSO [25] across different tasks, tested under the easy mode. Left: Logits on i-th row and j-th column is the M MG.(i. the smaller the better. Right: the average return over 10 trials compared between the RLEPSO models pre-trained and trained from scratch, with a MTE 0. This is a transfer failure.
E
Used Assets
MetaBox is an open-sourced tool, and it can be accessed at: MetaBox. It is currently licensed under the BSD 3-Clause License. In Table 6, we provide list of resources or assets utilized MetaBox, along with their respective licenses. important to note that adhere strictly to these licenses during the development of MetaBox.
com/GMC- DRL/
Table 6: Used assets and their licenses
Type
Asset
Codebase
License
CMA-ES [5]
DEAP [53]
LGPL-3.0 License
DE [10]
DEAP [53]
LGPL-3.0 License
Baseline
PSO [8]
DEAP [53]
I.PIL-3.00 License
BO [14]
Scikit-Optimizer [54]
BSD-3-Clause License
Synthetic
COCO [28]
BSD 3-clause License
Testsuites
Noisy- Soisy-Synthetic
COCO [28]
BSD 3-clause License
Protein-Docking
Protein- Protein-Docking 4.0 [29]

---

Algorithm 1: MetaBoxTraing Trainer
Input:
User-designed learnable agent A, User-designed optimizer O, User-specified problem set
Punin with configuration Cirain
Output: Trained Agent A, training records Initialize problem set Prain with configuration Crain as the training dataset;
while max learning steps Not reached do for each problem instance ins E Prain do
Construct optimization environment Enw Env Construction( ins); A.train { Enw);
Record training data;
Plot training figures; end end
Summarize and visualize the training records in Logger and return the trained agent;
Algorithm 2: TriioingEpisode
Input: User-designed learnable agent A, Constructed environment Erw
Output: Training records
stote Enw.restt);;
while termination condition Not achieved do
action A. Aeget a
mext_stale, reward, in f Enw chnnssttp(acllon)
Store transition <slate, action, reward, Update agent A;
Record training data and plot figures;
state
end
Summarize training records and return;
Pseudo-code of MetaBox
The pseudo-code of Trainer is shown in Algorithm 1, Given the user-designed learnable agent A and optimizer O, our MetaBox framework firstly initializes the problem instance set (e.g., determines problem types, indexes file directories and loads data to construct problem instances according to user-specified configuration Crmin). Then MetaBox iteratively performs training on each training instance until the max learning step is reached. For each instance, Gym-style environment Env is constructed to merge O and the problem instance together, so as to provide a unified interface. Agent A then calls the train_epp (icin__epood() interface for interacting with Env and performing the actual training. All the generated logs during training are managed by the Logger as depicted in Figure 1.
We now turn to the details of train episode(). Given the variance among numerous learning ap- proaches, such RL and SL, the internal workflow of train tetinnreeeiiiiaeeneee episode() necessitates implementation aligned with specific designs. Within MetaBox, we offer range of examples through the imple- mentation various baselines, serving as guides for users who wish to develop their own interfaces. Meanwhile, we note that our baseline library also covers the implementations of prevalent RI algo- rithms like PPO and REINFORCE. workflow for implementing RL training algorithms within train cecin_eeisode).... Starting from the Env initialization, in each step, the agent A provides Env the action according to the state, receives the next state, reward and other information, and uses them to update the policy. Within the env.step( interface, the action is translated into configurations which are then applied to the optimizer O. The optimizer then performs update rules derive and evaluate new solutions. Following this, rewards and subsequent states are computed, with certain logging information being concurrently summarized.
Algorithm 2, we showcase a straightforward example of the
As for the Tester shown in Algorithm 3, it first initializes the test problem set which is used to evaluate each algorithm in an algorithm set (including several baseline agents for comparison and the user's trained agent). For learning-based based algorithms, respective agents and optimizers are then
7

---

Algorithm 3: MetaBoxTester
Input:
User-specified algorithm set B including baselines and user's trained agent, User-specified problem set Piest with configuration Ciest
Output: Testing results
Initialize problem set Prest with configuration Gest as the test set;
for each algorithm alg E B do for each problem instance ins E Pies do if olg a learning-batt method then Initialize alg.agent and alg aig.optimizer; end
else
Initialize alg. opplimime as an optimizer; end
Construct optimization environment Enw Env oow.. ins); alg.rollout coggrolloo_eeisoeeeeaeeeeee
Record testing data;
Plot testing figures;
end end
Summarize testing results and call Logger for standardized metries and visualization;
initialized, which may involve loading network models and initializing parameters. Conversely, classic optimizers are simply initialized the optimizers. The rollout episode() interface mirrors the train ceiid_eepieee but removes the policy update procedures. After the testing, the recorded testing logs will be organized and presented within the Logger.


Reviews:
[{"review_id": "MKBL68t6ya", "reviewer": "Reviewer_nMBF", "summary": "", "questions": "", "limitations": "Some limitations have been identified in section 5: lack of performance evaluation for BBO, expanding the number of BBO tasks in the platform and proper maintenance of the platform, keeping it up-to-date with latest algorithms from the literature.", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "Comprehensive platform with both a large number of problem instance, a significant number of state of the art optimization algorithms and standardized performance metrics.\n\nThis permits reproducible research in the MetaBBO-RL field.\n\nThe three main contributions constituting the MetaBBO-RL platform, i.e. MetaBBO-RL template approach, a large set of benchmark instances and a representative set of baseline optimization algorithms are clearly introduced and presented in detail in dedicated subsections in section 3.\n\nA comprehensive benchmarking study has been conducted and even if it is not the main purpose of the work, some results did outperform the state-of-the-art, which outlines the potential of the proposed MetaBBO-RL approach.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "Ov8NIRFHXt", "reviewer": "Reviewer_qtiV", "summary": "", "questions": "", "limitations": "The authors acknowledge a potential limitation in their paper, specifically regarding the performance evaluation of BBO. They recognize that this aspect remains an open question, meaning that there might not be a definitive or universally accepted method to assess the performance of Black-Box Optimization algorithms.\n\nHowever, they do not specify further details or elaborate on this limitation in the given text. Depending on the context of the paper and the section where this statement is made, it might be useful for the authors to explain why the performance evaluation of BBO is challenging or open-ended. They could also discuss existing approaches or research gaps in this area. Providing more context and insights about this limitation can help readers understand the scope and implications of their work better.", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The writing is clear; the authors comprehensively describe the benchmarking problems and how the benchmark is composed, both in terms of concepts and code implementation.\n- The benchmark suite is well-designed in terms of code implementation; each module is abstracted effectively, enabling users to focus solely on algorithm implementation without needing to understand the benchmark suite's internals.\n- MetaBox offers a wide range of test suites, including both synthetic and realistic scenarios.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "sdj1UWKJTJ", "reviewer": "Reviewer_D9Jb", "summary": "", "questions": "", "limitations": "The limitations are fairly generic and written as an afterthought. However, this is standard practice in most papers now and therefore, not a criticism but an observation", "rating": 7, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "Exhaustive nature of experiments, the availability of problem instances and the number of problem instances available. These are a big plus.\n\nThe paper is very well written and is clearly readable.\n\nI like the unified benchmark that is provided here.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "YHrxegefum", "reviewer": "Reviewer_t7Zi", "summary": "", "questions": "", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. The benchmark is comprehensive, with numerous instances and baselines for comparison.\n2. The documentation (README) provides sufficient details for users to use the package.\n3. The benchmark provides a standardized way to evaluate the methods in this domain.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper investigates benchmark datasets and methods for meta-black-box optimization with reinforcement learning. The reviewers and I agree that the platform is extensive and covers a large set of relevant problem instances and set of algorithms for solving them. And, the code to solve these is well-designed and will serve as a useful tool for the community. This paper will be a great contribution to this year's program. I recommend it for an oral as these benchmarks and methods are well-executed and worth emphasizing to the community.
Decision: Accept (Oral)