Task-free Adaptive Meta Black-box Optimization
Authors: Chao Wang
Licheng Jiao
Lingling Li
Jiaxuan Zhao
Guanchun Wang
Fang Liu
Shuyuan Yang
Venue: ICLR 2026 Oral (2026)
Keywords: Meta Black-box Optimization
Evolutionary Algorithms
Primary area: optimization

Abstract: Handcrafted optimizers become prohibitively inefficient for complex black-box optimization (BBO) tasks. MetaBBO addresses this challenge by meta-learning to automatically configure optimizers for low-level BBO tasks, thereby eliminating heuristic dependencies. However, existing methods typically require extensive handcrafted training tasks to learn meta-strategies that generalize to target tasks, which poses a critical limitation for realistic applications with unknown task distributions. To overcome the issue, we propose the Adaptive meta Black-box Optimization Model (ABOM), which performs online parameter adaptation using solely optimization data from the target task, obviating the need for predefined task distributions. Unlike conventional metaBBO frameworks that decouple meta-training and optimization phases, ABOM introduces a closed-loop adaptive parameter learning mechanism, where parameterized evolutionary operators continuously self-update by leveraging generated populations during optimization. This paradigm shift enables zero-shot optimization: ABOM achieves competitive performance on synthetic BBO benchmarks and realistic unmanned aerial vehicle path planning problems without any handcrafted training tasks. Visualization studies reveal that parameterized evolutionary operators exhibit statistically significant search patterns, including natural selection and genetic recombination.

Published as a conference paper at ICLR 2026
TASK-FREE ADAPTIVE META BLACK-BOX OPTIMIZA- TION
Chao Wang', Licheng Jiao',
Lingling Li Jiaxuan Zhao?, Guanchun Wang', Fang Liu'
Shuyuan Yang' Xidian University Tianjin Research Institute for Water Transport Engineering, M.O.T. xiaofengxde 126 com, {111i, wanguuacchun, wtwpr///essaaaaaooeiiiiieeeeedde en, jixxuanzhao@stu xidiani edu. en, f631iu8163.com
edu.
ABSTRACT
Handcrafted optimizers become prohibitively inefficient for complex black-box optimization (BBO) tasks. MetaBBO addresses this challenge by meta-learning to automatically configure optimizers for low-level BBO tasks, thereby eliminat- ing heuristic dependencies. However, existing methods typically require exten- sive handcrafted training tasks to learn meta-strategies that generalize to target tasks, which poses a critical limitation for realistic applications with unknown task distributions. To overcome the issue, we propose the Adaptive meta Black- Optimization Model (ABOM), which performs online parameter adaptation using solely optimization data from the target task, obviating the need for prede- fined task distributions. Unlike conventional metaBBO frameworks that decouple meta-training and optimization phases, ABOM introduces closed-loop adaptive parameter learning mechanism, where parameterized evolutionary operators con- tinuously self- self-update by leveraging generated populations during optimization. This paradigm shift enables zero-shot optimization: ABOM achieves competitive performance on synthetic BBO benchmarks and realistic unmanned aerial vehi- cle path planning problems without any handcrafted training tasks. Visualization studies reveal that parameterized evolutionary operators exhibit statistically sig- nificant search patterns, including natural selection and genetic recombination.
INTRODUCTION
Black-box optimization (BBO) problems arise in diverse machine learning applications such as neu-
roevolution Stanley et al. (2019) Miikkulainer (2025), hyperparameter tuning Bai & Cheng (2024), neural architecture search Wang et al. (2023); Salmani Pour Avval et al. (2025), and prompt engi- neering Pareeess et al. (2024); Wang et al. (2025a). In these scenarios, the objective function accessible solely through expensive evaluations f(x). with derivative information like gradients Hessians inherently unavailable. Evolutionary algorithms (EAs) Eiben & Smith (015); De Jong (017) address this challenge by iteratively updating populations through derivative-free heuristic operators, including selection, crossover, and mutation, to explore complex fitness landscapes. Re- cent advances in computational infrastructure have enabled EAs to generate robust solutions for increasingly complex BBO problems Mükkulainen Forrest (202T).
The "No Free Lunch" (NFL) theorem Wolpert & Macready (2002) establishes that no optimization algorithm universally outperforms others across all problem domains. To enhance cross-domain ap- plicability, numerous adaptive mechanisms have been designed Bäck Bäck & Schwefel (1993),
(1993); [ Brest et al. (2021): Li al. (013):; Hansen (2016); Tao al. (202T) that leverage optimization data generated during the search process to dynamically select operators or adjust parameters. Although these adap- tive methods achieve strong performance on standard benchmarks, they require specialized expertise in optimization theory and problem characteristics Ma et al. (2024). Meta Black-Box Optimization (MetaBBO) addresses this limitation by automating meta-level strategies Ma et al. (2025b), such algorithm selection Tian al. (2020): Guo al. (2024), algorithm configuration Lange et al. (2023 a); Guo et al. 2025), solution manipulation Li et al. (2024: 2025), and generative design
Chen al. 2024 Yang al. (2024), through meta-learning (Fig. Left). Yet existing MetaBBO

---

Published as a conference paper at ICLR 2026
methods require training handcrafted task distributions F or prior knowledge for generalization to new domains, Since such distributions are often inaccessible in practical scenarios (c.g. when the target task is unique data-sc this dependency severely limits real-worid deployment.
Meta Black-Box Optimization
Our Framework
Selection/
Configuration/
Solve
Generate
Manipulation/ Genoration
Optimization Task
Popuintion
Solve
Optimization Task
Sample
Evolutionary Algoritham
Machine Learning
Evolationary Algor thm
Learning
Optimization
Mete Linta-Leari Learning
Adaptive Parameter Leurning
Data
Task Distr Bution
Figure I: Conceptual comparison: (Left) MetaBBO methods learn meta- meta-strategies from task distri- butions but depend on handcrafted training tasks; (Right) Our framework performs adaptive param- eter learning using self-generated optimization data, eliminating task distribution dependency.
To address this limitation, propose the Adaptive meta Black-box Optimization Model (ABOM), a task-free meta-optimizer that adaptively learns parameters using only self-generated data (Fig. D Right). ABOM's distinguishing feature is an end-to-end differentiable framework that parameter- izes evolutionary operators as learnable functions (Fig. 2 Inspired by EA dynamics, it employs attention mechanisms to separately model relationships among individuals, fitness landscapes, and genetic components, thereby replicating selection, crossover, and mutation as differentiable oper- ations. Crucially, ABOM updates its parameters during optimization by aligning the generated offspring population with elite archive of high-quality solutions, bypassing the need for meta- training on task distributions. This design yields two key contributions:
Task-free adaptation: The parameters of ABOM are updated via adaptive learning using optimization data from the target task, eliminating the reliance on handcrafted training tasks heuristic rules. Theoretically, ABOM guarantees convergence to the global optimum.
Intrinsic interpretability: Attention matrices provide quantifiable insights into search pat- terns, such as selection bias toward high-fitness individuals and consistent genetic interac- tion patterns during mutation. Moreover, ABOM supports GPU acceleration out of the box, without requiring changes to standard EA infrastructure.
RELATED WORKS
Evolutionary Algorithms. EAs, such as genetic algorithms (GA) Holland ( (1962), evolution strate- gies (ES) Rechenbbbr (1984), particle swarm optimization (PSO) Kennedy & Eberhart (1995), and differential evolution (DE) Storn& & Price (1997), are widely adopted for BBO tasks due to their derivative-free nature. These methods manipulate populations via heuristic operators but often suf- fer from inefficiency and fragility when applied to new tasks, as they require labor-intensive manual parameter tuning. While ABOM draws inspiration from EA dynamics, it eliminates manual tuning by enabling adaptive parameter learning directly from optimization data.
Adaptive Optimization. To improve cross-domain generalization, adaptive EA variants employ dynamic operator selection or parameter adjustment such as CMAES Hansen (216)); Ollivier et al. (2177) SAHLPSO Tao et al. (20TT), JDE21 Brest al. (202T)). These methods achieve otate-of- the-art results standard BBO benchmarks but demand deep expertise in optimization theory and often require problem-specific GPU acceleration for scalability. In contrast, ABOM adheres to
unified deep learning architecture, replacing heuristic rules with adaptive parameter learning and reducing deployment barriers.
Meta Black-Box Optimization. MetaBBO techniques leverage meta-learning to automate meta- level strategies for solving lower-level BBO tasks Ma et al. (2025b); Wang et al. (2025b); Yun et al. (2025), thereby reducing the need for expert intervention. Common paradigms include algorithm selection Tian et al. (2020): Guo et al. (2024). which chooses from a predefined pool of operators; algorithm configuration Lange et al. (2023b a): Guo al. (2025), which tunes hyperparameters via
2

---

Published as a conference paper at ICLR 2026
meta-strategies; solution manipulation Li al. 2024
2025) which integrates meta- mtrttsstateggies di-
rectly into the optimization process; and algorithm generation Chen et al. (2024); Yang et al. (2024), which synthesizes entire optimization workflows. Despite their promise, these methods enttccllyy depend on manually designed components, such as discrete algorithm search spaces A, state fea- ture spaces, meta-objectives, and training task distributions F. The dependency on handcrafted F hinders real-world applicability when task distributions are unavailable. ABOM addresses this limi- tation by unifying evolutionary operators into a continuous, differentiable parameter space, enabling adaptive parameter learning without requiring F or discrete algorithm search spaces.
3
ADAPTIVE META BLACK-BOX OPTIMIZATION MODEL
3.1 PROBLEM DEFINITION
target BBO task is defined as:
min fr((x, xER
(1)
where is the solution vector in a d-dimensional search space. MetaBBO methods formalize the automated design of optimizers as a triplet 7 := (A, R, F), with the discrete algorithm search space
A, the performance metric R, and the training task distribution F. The meta-optimization objective maximizes expected performance Ma et al. (2025b):
J(0) max y m0
no,
where the meta-strategy Ho selects the algorithm (or configuration) a A for each task f. The Eq. (2) needs to be designed manually F. To mitigate the need for F, we define adaptive MetaBBO Tadaprive : (A, R, fr), operating directly the target task fr. Using cumula- tive optimization knowledge M = (x()))))))) where X² (x(1), ..... x(0) (solutions) and
(+)) (f(x("),..., ((x(()) (evaluations) during optimization, the Eq. becomes the follow-
ing:
e0o J(O) = max (R, M
with 0 updated online using M However, and xo still require expert-crafted components.
To address this limitation, ABOM replaces the discrete meta-optimizztion framework (A,, xa) with a single, differentiable optimizer ma parameterized by 0. The final objective
(()) iaax (()))
(4)
where is updated only using (( from fr, thereby eliminating the need for manual design of F, discrete search spaces A, and expert-dependent feature engineering. The Eq. (4) establishes end-to-end differentiable framework where adaptive parameter learning occurs through continuous feedback from M(1)
3.2
META-STRATER ARCHITECTURE
ABOM implements differentiable meta-strategy Po()) learns evolutionary operators via attention mechanisms Vaswani et al. (2017). At generation 4, the population ((t)
(P(t), F(O) (Fig. Bottom) that
P (
(OT Nxd represents N candidate solutions in the
:...: Px
search space, where the individual p E Rd is solution vector. The fitness values F(!)
f (r(p RN are scalar evaluations fr(p.") obtained via black-box queries the target objective fr(-), with lower values indicating better solutions. Given the population-fitness pair, ABOM generates offspring through three unified modules:
Selection. The selection matrix A (t) N is computed jointly model relationships in the solution space and among fitmess values via attention:
(P + (F QFF) (F(()) W W
(t) softmax
(5)
Vda
3

---

Published as a conference paper at ICLR 2026
ER aa Fg
frik Rf -
20:17
Best teditteall
BLJKS
35.20
p'
Oycuulantis Tesk
Saaeple 70 Radtigia
g
Made
Adaptation
Meu-SStraogy Archiretuze
PM my
Seletiio
Cremone
Muturce
Popubroon ptl e glr
Fieeou g R
Pepabaionpli ge
Indirizal
batsidad
Lincer
Lincar
Lincer
Lincer
Libir
Libsrr
Linear
Linear
MatMaleSaale
MasMelitcck
Sehmas
Sodtnas
Saftnax
P&Droppst
Mi ghic
Maxsire Klzzia
Mirsicel
Matii,
M ge
Martill
MatMial
Selection
Mavis
Pepulsiorp a g
MI.PADeeperr
Cencal
MiPdDDesjes
Al gaaa
OGiiiing
Cttipeta Populatios
pil food
Offiginng M Px.
Figure 2: Workflow of ABOM: (Top) Adaptive optimization loop: Initialization, reproduction, eval- uation, elitism, and parameter adaptation; (Bottom) Meta-strategies for reproduction: Attention- based evolutionary operators, including selection, crossover, and mutation.
where Q W Bddxd project solution features, and W W E RIxda process fitness values. The first term captures spatial relationships in the solution space, while the second term en- codes ttmess-driven selection pressure. This dual-path design ensures that recombination prioritizes solutions based on both their search-space positioning and fitness ranking, rather than fitness alone.
Crossover. The intermediate population p((() is generated by:
p*(1 = p + MLP'. (A (P(!))))
(6)
where ML.P
D. (z) tanh(2 b W2 2 + b2 with W1 E RRdddar b Rdd,, W₂ E Re xd b2 E gd Dropout with rate pc is applied to the hidden layer during both adaptive parameter learn- ing and inference. The mechanism ensures persistent exploration through controlled randomness and is consistently maintained across all stochastic operations in ABOM. The term (t)(t) com- putes an adaptive recombination pool: each row Σ
represents context-awarr blend of parent solutions, where weights A A,j dynamically balance proximity in solution space and fitness- driven selection pressure.
Mutation. For each individual p, (()) E Rd in p((t), offspring p.
(1) is generated via:
(p, ((0) WKMJT
D Pj (1)) - -MLP... 8m (M. Dj 1 M softmax (p.
Vda
where WQM WKM E BlxdA, and MLP (z) tanh + ba) W4 + b4 with W E Bixds, by E Rdw, Wa E Rdar, b, E R. Following the same exploration principle as crossover, dropout with rate PM is applied during inference to maintain persistent exploration. The mutation matrix M Iddd dynamically models gine-wise dependencies: each entry M( quantifies the interac- tion strength between the j-th and k-th dimensions, enabling context-aware perturbations. Finally, offspring are concatenated as:
p(0
(1) : .... PN
(I)
E B-d
(8)
The set 0 containing all parameters is:
OP KPP OF KF OM W Um..
(9)
with 0, = (W,, b, W. b2). 6m = (W,, by, W4, ba). Note that Pc and PM are hyperparam- eters that govern the intensity of exploration. All modules share attention dimension d and MI.P
4

---

Published as a conference paper at ICLR 2026
hidden dimension da- The parameterization transforms evolutionary operators into stochastic yet differentiable functions, where structured randommess maintains exploration without compromising gradient-based adaptation.
3.3 ADAPTIVE PARAMETER LEARNING
As shown in Fig. (Top), ABOM's optimization loop comprises: 1) Initialization: The initial population (()) is randomly generated by Latin hypereube sampling: 2) Reproduction: Offispring p(O) are generated via ((t) (P(t)) F()); ; 3) Evaluation: Fitness values F(()) computed for p(0; 4) Elitism Deb et 2002): The clite archive E(4) Rxxd formed by the top N individuals from PO)) P(O) and their fitness values F E RR,, are carried over to the next generation; 5) Parameter adaptation: 0 is updated via adaptive parameter learning. The pseudocode of ABOM can be found in the Appendix (Alg. []. Crucially, ABOM performs adaptive parameter learning by minimizing the distance between offspring and the clite archive:
min (()) |P() - E
(10)
where, E(!) denotes the elite archive. The objective refines evolutionary operators using task-specific
knowledge from M From learning perspective, adaptive parameter learning operates in supervised paradigm. Gradients of L with respect to 0 are computed, and 0 + - Vo(( is
updated via gradient-based optimizer (e.g., Adam Loshchilov & Hutter (019 ) The process ensures continuous udaptation to the target task without handcrafted training tasks.
Discussion. ABOM introduces three algorithmic properties that enhance its suitability for BBO: (1) Learnable operators: evolutionary mechanisms are parameterized and adapted online via gradient- based learning, reducing reliance on hand-designed heuristies; (2) GPU-parallelizable design: neu- ral computation enables efficient batched execution on GPU, reducing wall-clock time per iteration; and (3) Interpretable dynamics: learned selection and mutation matrices reveal structured patterns in solution-fitness interactions and dimensional dependencies.
3.4 COMPUTATIONAL COMPLEXITY AND CONVERGENCE ANALYSIS
The computational cost of ABOM is primarily dominated by the selection, crossover, and mutation. The selection matrix (Eq. 5 incurs complexity O(Ndd, + A where N is the population size, d the search space dimension, and d the attention dimension. The MLP of the crossover (Fq. 6 contributes ONNAdds + Nd with dy the hidden dimension of the MLP. The mutation (Eq. contributes dd + ddadM). Summing these, the total complexity is:
O(Ndd + N + NdAda + Nd + dd + ddAdss.
(11)
Assuming dA dı = d for simplicity, the formulation typical high-dimensional optimization (N < d), the leading term is (dd*), indicating that compu- tational cost is primarily governed by the problem dimension. Note that d and dı can be adjusted in practice balance expressivity and efficiency. Next, we establish that ABOM achieves global convergence under the following assumption:
reduces to O(Nd + N + d In
The search space X C Rdd is compact, the objective fr continuous with global minimizer x" in the interior of X, and ABOM uses tanh-activated MLPs (dss 2 1) with dropout rates (0 PC.PM PM <) during inference (operator execution).
Assumption
Let ft minxe fx(x) denote the best objective value the elite archive. The filtration F. = (P))) p(t) g(o) gio)..... g(t) captures all algorithmic history up to generation t. ABOM
preserves non- non-vanishing probability of generating offspring (t) near the global optimum:
Corollary (Exploration Guarantee) For any 8 > 3 > such that Vt 2 0.
PP3:: IDD x11 < F
>) 0.
(12)
5

---

Published as a conference paper at ICLR 2026
Let f =x(x") be the global optimum value. Corollary establishes a positive drift condition: when f/ > f + e, the expected improvement is strictly positive.
Corollary (Progress Guarantee) For any > 3m(e) such that Vt 2 0.
Llsi Ji+1 Ft, Jt > f + <] 2 n(e).
(13)
Combining these properties, we have:
Theorem 3.1 (Global Convergence) Under Assumption ABOM converges to the global optimum almost surely:
ft
a.s.
f
as t c0.
(14)
All proofs are provided in Appendix
EXPERIMENTS
In this section, we address the following research questions: RQI (Performance Comparison): How does ABOM compare against classical and state-of-the-art BBO baselines on both synthetic and real-world benchmarks? RQ2 (Visualization Study): What statistical patterns emerge in ABOM's selection and mutation matrices? RQ3 (Ablation Study): Are all components of ABOM neces- sary for achieving competitive performance? RQ4 (Parameter Analysis): How sensitive is ABOM's performance to its key hyperparameters? We first describe the experimental setup and then system- atically address RQI-RQ4.
4.1 EXPERIMENTAL. SETUP
BBO Tasks. We evaluate ABOM on the advanced MetaBox Benchmark Ma et al. (2023 2025a)
comprising both the synthetic black-box optimization benehmark (BBOB) Hansen et al. (2021) l) and the realistic unmanned aerial vehicle (UAV) path planning benchmark (Shehadeh & Küdela (2025). )
The BBOB benchmark suite, widely adopted for evaluating black-box optimizers, comprises 24 con- tinuous functions that exhibit diverse global optimization characteristies, including unimodal, mul- timodal, rotated, and shifted structures, with varying properties of Lipschitz continuity and second- order differentiability. We set the search space to 100, 100)* with d = 30/100 00/100/500. The UAV benchmark provides 56 terrain- based problem instances for path planning in realistic landscapes with cylindrical threats. The objective is to select a specified number of path nodes in 3D space to minimize the total flight path length while ensuring collision-free navigation. The maximum func- tion evaluations for BBOB and UAV are set to 20,000 and 2,500, respectively. All experiments are conducted on Linux platform with an NVIDIA RTX 2080 Ti GPU (12 GB memory, CUDA 11.3). Detailed task contigurations and other experimental results are provided in Appendices
and
Baselines. We compare ABOM against three categories of baselines: (1) Traditional BBO meth-
ods: Random Search (RS) Bergstra Bengio (212)) PSO Kennedy Eberhart (1995). and DE
Storn & Price (1997); (2) Adaptive optimization variants: SAHLPSO (advanced adaptive PSO variant) Tao et al. (2021). JDD21 (advanced adaptive DE variant) |Brstt et al. (202T), and CMAES
(state-of-the-ane adaptive ES variant) Hansen (2016): Ollivier et al. (2017;; (3) MetaBBO meth-
ods: GLEET (advanced MetaBBO for PSO) Ma et al. (2024). RLDEAFL (advanced MetaBBO for DE) Guo et al. (2025), LES (advanced MetaBBO for ES Lange et al. (2023b), and GLHF (advanced Me for solution manipulation) Li et al. (2024) All baselines follow the config- urations outlined in the original papers. For all MetaBBO methods, we train them in the same problem distribution as ALDEAFL Guo et al. (2025) using the recommended settings. For BBOB, out of the 24 problem instances are used as the training set, and the remaining 16 instances (f4, So ~ /18 S20, J22 ~ f24) serve as the test set. For UAV, the 56 problem instances are evenly divided into training and test sets, with a partition of 50% / 50%. All parameter configu- rations are provided in the Appendix G
4.2 PERFORMANCE COMPARISON (RQ1)
Results on BBOB We evaluate ABOM against the baselines on the BBOB suite with d 30/100/500. Tabless [ and 8 (See Appendix
K) show the mean and standard deviation over

---

Published as a conference paper at ICLR 2026
The comparison results of the baselines on the BBOB suite with d 500. All results are reported as the mean and standard deviation (mean + std) over 30 independent runs. Symbols "-", "x=" and "+" imply that the corresponding baseline is significantly worse, similar, and better than ABOM on the Wilcoxon rank-sum test with 95% confidence level, respectively. The best results are indicated in bold, and the suboptimal results are underlined.
Table 1:
Traditional BBO
Adaptive Variants
MetaBBO
Ours
ID
PSO
DE
SAILPSO
JDE21
CMAES
GLEET
RLDEAFL
LES
GLIU
ABOM
f4
#1 192294
3,700e45
115525044
8.863e44
+ 4..506
3.160e+5
22418p44
2.947e45
2..886+4
7876644
27.80-4
1.47644
22.175--4
2.056445
11.079
4.573644
44007544
2.363645
4 57.512
2.524e45
25.389++2
1.215es4
f6
#6.317613
1.529e47
#5.262013
5.206e16
11.390
1.200e+7
1.026e47
+ 9.051e15
4.399e46
#6.814c33
d.24099e-
9.870046
#23555e5
1.875666
9.253646
9.194646
224104
2..526e+2
6.201e43
4
+1094e13
393264
+23309113
2.811e14
+2
3.360e14
#2.240e13
2.774c44
+ 2,522433
1856644
#58885e42
1:289
de 1.62255ed
2.525644
+10057ee
1.81e44
2.285e14
2.173444
2.250ee
2,432443
6.82248
4,083488
4.153e48
2.194448
1.33248
2.276455
1.183848
5.007447
5.08e47
5.55647
8.866444
22354e77
333477977
+4.952e+7
#2.342207
+ 2.257ee7
20.292294
+1227ee+7
1.228777
2
4 1540e
1.267ees
19
+1.30er7
4.159c+8
+27700777
1,719++8
+:3.0te17
2.002
+1,00++7
5.151e+7
+ 1.2258++7
3.326e-7
$3.4450e4
2333e45
4:.238c-66
1443e+7
43.538cc6
1.145++7
3
4,548c+3
4-5.600e-2
3,243++3
1.792c+5
110
+13366e77
2.832++8
49458016
8.026++7
+:32177e+7
2.490c+8
#2.573e77
2.229cr8
#: 1.667e77
5.459=77
#2885ce6
1.580e+7
#:218
2.07c+88
#3:291ee6
2.232e+7
4.4324ee6
2,085c+8
454466
2,002++8
55916ee15
5,95e+66
f1
11.991e+2
5.881++3
17
+7.006+2
6,187e+3
+.5091e+2
1,999c+3
#8.
18.1170+2
4,835e+3
15.691e+3
4.371e=3
#22251e+3
1.248=+1
1:1.75e--1
3.722++3
+2.52le=2
3,259c+3
18488e++
5,107++3
1.3248e+1
2.539c+3
1
13.19
5,392c+3
f12
12.04649
3.015++10
11.40
L414e+10
1: 4.4662
1.055e+10
#3.633e+9
1665e+10
19
4,800++9
-9.841e+8
12.256e+7
1.32=88
12.046e99
1:33e++00
13 +3.899
2.819++9
4
1.08e+110
1.6 844444
9,757c+9
14,993++
2.733e+7
f13
1.444:14
1,319c44
1.334e14
1,197e44
3.94
2.363043
1.116044
7.073643
1,255ex4
1.024c44
1.231e43
11618e+2
#277542
14,15
#2.8570+2
26.370++2
11,33=3
13.338e-3
13.426e42
+:5007e+1
4.5 299
13010++2
f1
12241e+1
6.644412
1442966+1
4.834442
1
5.081e42
44277e+
4.208042
22.99e++
1842242
1..55
2.494
12.764e-1
3.231042
#: 112116e=1
1.157642
16 108c+0
14S8e43
18 222e00
2.542442
2
1,487e+1
fi
#5025e40
1428e12
+9.614e00
1.017442
+8.207e00
1.093442
#7.383e40
9.814+1
+8.102240
7.074c41
# 1066ee1
1.100c43
#8.36640
8.539
14.191240
460626-1
3.74e422
+ 4222240
6.875e41
4.07500
3.792e+1
f1
1604les1
200043
21.488112
9.012+42
+1.70ee12
9.60K++2
2.764442
4.88%csl
15885
1.772+42
1
1.374p+1
# 1666--1
8.479641
# 1
K.767s4
1
3,502443
2.504c11
1813e+1
120
#1022e15
3,233646
+23300445
2.574
#:2800045
2.432246
#1945e13
1.471e46
= 1.566e4
8884
# +1.002243
3802243
# L.09es
9.466045
#6.060444
2.136045
46088e43
3.772645
46417643
3,753e45
11,35tes3
2,56ee22
f32
#19426-2
8.36e+1
494326-2
8.609c+1
11.371e-1
8.61e+1
122400-11
8.542e+1
= +1.33e00
8.003e+1
33.8526-1
2831e-T
+2.4%--1
8478e+1
7.159e+1
24.3946-2
1.184e+3
47.5406-2
8.356e+1
1..520e10
4,971e+0
f23
+35518-2
1.622+00
444291622
1661e+0
4
1.69K+0
13.7906-2
1668c+0
66.784622
15866=0
33.44-22
3999t-1
444296-2
1669040
11,525-11
1.559e40
43.532622
1,200e+3
43359962
1.60e+00
43.446-22
L656e+0
f24
12.555e+2
2,089++
15.679442
1.80e+1
19,073+22
1.60c+4
14.7He+2
1,223644
19.58le+2
L198c44
13.357e-1
4.96644
14.16e+2
1010e+4
16.456-2
9.e43
1:3.922++1
8.822+43
17.572++1
7,437e+3
3.2668++2
8,09c43
-12/4
15/10
15/100
14/11
14/11
14/111
150/11
14/1/1
14/11
14022
11/1/4
30 runs for each baseline. Convergence curves of average normalized cost across all cases are pro- vided in the Appendix K ABOM matches or outperforms all baselines, achieving state-of-the-art performance, which validates the effectiveness of the proposed method. Both ABOM and adaptive optimization methods adjust the parameters online using optimization data. ABOM's parameterized operators offer greater flexibility than the fixed adaptation rules in the variant, leading to stronger performance. Compared to existing metaBBO algorithms, ABOM's improvements highlight the importance of parameter adaptation in enabling effective meta-optimization across diverse prob- lem instances. The results suggest that adaptive mechanisms can support competitive performance without relying on handcrafted training tasks.
Results on UAV. We evaluate ABOM on 28 UAV benchmarks to validate its practical effectiveness. Fig. 3 shows the convergence of the normalized cost and runtime, ABOM converges fastest under limited evaluations and achieves the lowest normalized cost. Unlike metaBBO-basee methods (e.g., GLHF, GLEET, RLDEAFL, LES), ABOM and adaptive optimization methods eliminate the need for training on hand-crafted tasks and associated overhead. Through GPU-accelerated evolution and adaptive parameter learning, ABOM achieves significantly faster runtime than most baselines.
4.3 VISUALIZATION STUDY (RQ2) AND ABLATION STUDY (RQ3)
We visualize the learned selection and mutation matrices of ABOM on three BBOB functions (f4. fui /24. d = 30) in Fig. 4 In all cases, the matrices develop structured statistical patterns as optimization proceeds. The selection matrix shows row similarity, indicating that ABOM learns to generate offspring from small subset of individuals. This behavior resembles the difference vector mechanism in DE Stom & Price (1997). which reflects the strong expressive capacity of the learnable operator. Individuals with higher fitness are preferentially selected, in line with the
7

---

Published as a conference paper at ICLR 2026
10
0.87
Optmbtiion Time
60003
42003
64
SAHUPID
B822
Cleet
GUPT
RLOBAF
AROW
30E
1206
X00
2533
(8) Convergencee curve
(b) Runtime comparison
Figure 3: Performance on 28 UAV problems: (Left) Convergence curve of average normalized cost across all problems. Costs (lower is better) are min-max normalized for each case. Detailed results shown the Appendix K (Right) Average runtime (GPU seconds) 30 independent runs.
Selection Marris
Mistrionn Matrix
fu
fxx
Ozz
900
1000
Gma
Figure 4: Learned selection and mutation matrices of ABOM on BBOB functions f1. f11 and f24 (d 30) Generation 1, 500, and 1000. For the selection matrix, axes represent individuals ranked by their fitness values (0 is the best). For the mutation matrix, axes represent gene (variable) indices.
principle of survival of the fittest. The best individual not always selected, which may help preserve population diversity. The mutation matrix evolves from random initialization to ordered structure, suggesting that mutation follows consistent patterns adapted to the problem. These results demonstrate that ABOM provides greater interpretability than the BO methods, which directly map neural networks to solution manipulation Li et al. (2024 2025).
Table 2: Ablation study of ABOM's s key components on the BBOB suite with d = 30.
No (mean ad)
Mutation (mean = sid)
No Adaptation
ABOM
(mean
4.2e+033
3.02++03
1.0le000 1: 444--02
2.58x+04
1.67=01
2.95e-02
fg
1.22e04
1.55e+04
1.10+04
1.95e-04
4.5ee00
3.61es0t
2.60e102
2.64c+02
f7
6.39e+03
6,91e+03
L18e+00
1 6.86e-02
1.7e+04
Lleee00
5,58e+02
2.77e+02
1.50+03
2.91e+03
1,94e403
1 3.62--03
1.0e408
2.68408
1.15e402
1.56e-02
Fg
7.022+044
3.15e-03
5.94ce06
F10
1.
7.11c+06
1,07<007
3.48e-05
3.65e+07
1.60e+07
9,72e+05
7,38e+05
f11
105e+05
= 3.19
1.00+005
+ 2.68e-04
8.0c+04
=: 2.20e=04
2.6e4444
1.01e=04
1.12e+09
3.12c+09
2.30c+07
5.74e-077
1.3es10
L61ce110
5.25c+07
1.45c-08
fi5
7.5te+01
7.71c+01
3.93-001
f14
843es101
7.23ex01
9.29cH00 + 2.49e-01
9.28ex022
3.96cs00
3.46c-02
3.39e-02
f18
1.18e033
à 2.21ex03
7.02>x02
+: 2.46e-02
494c+42
4: L.f6ce02
5.12eH02
1.37e+02
f18
2335+01
3.45e+01
1.23++01 1 1.23e-01
1.6e+002
3.27e+02
248e-01
1.1e-43
f20
6.8ex01
3.53ee00
-5822+01
4.19<x00
3.80e=00
f22
8.66+101
0,00e+00
8.66++01
1: 0.00e-00
8.50e0i
0.00g+000
866e101
0.00c=000
f2a
3.03+400
1: 5.33e-01
3.12e+00
1: 4.54c-01
3.20++00
2: 4.25e-01
301e-1
2.19e-01
292er02
2.40ev022
2.30ex02 531e-001
3.41ea0
2.4e102
2.37c+01
-/
13/300
10/33
14/111
8

---

Published as a conference paper at ICLR 2026
We conduct ablation study comparing the proposed ABOM with variants that disable specific mechanisms, including no crossover, no mutation, and no parameter adaptation. Table 2 presents the mean and standard deviation over 30 runs the BBOB suite with d 30. Table 2 illustrates that both crossover and mutation are crucial components, as their removal individually causes sig- nificant performance deterioration. Furthermore, the variant without parameter adaptation performs significantly worse than ABOM, underscoring the critical importance of the adaptive mechanism for achieving robust and high-quality optimization.
4.4 PARAMETER ANALYSIS (RQ4)
Fig. 5 illustrates ABOM's hyperparameter sensitivity on the BBOB suite (d = 30). population size 20 proves sufficient for robust performance across most functions within 20,000 evaluations. Similarly, hidden dimension dıs smaller than d (e.g.. dı = 16) often achieves competitive results. practice, das should be carefully configured to balance computational efficiency and optimization quality effectively. The parameters PC and Par exhibit optimal performance at 0.95, indicating that higher values increase stochasticity and exploration. Nevertheless, setting either parameter eliminates beneficial randomness, degrading performance. Thus, controlled stochasticity is crucial for maintaining the balance between exploitation and diversity.
Norm Stred heengee Chuctive Salur
69
61
fa
G
6x
G2
6x
65
10
50
0.0
32
64
128
Population Size N
Hiddan Dimirsioo
(a) N
(b) d.r
8.0
deppect riten
(d) PC
(d) PM
Figure 5: Sensitivity analysis of key hyperparameters the BBOB suite with d = 30: Algorithm performance across different settings for population size (N), hidden dimension (dmr). crossover dropout rate (Pc), and mutation dropout rate (PM). The learning rate analysis is in Appendix
5 CONCLUSION AND DISCUSSION
Summary. We present a task-free adaptive metaBBO method, ABOM. which eliminates depen- dency on handcrafted training tasks by performing online parameter adaptation using only opti- mization data from the target task. Unlike conventional MetaBBO methods that require offline meta- training, ABOM integrates parameter learning directly into the evolutionary loop, enabling zero-shot generalization. Our framework parameterizes evolutionary operators as differentiable modules, up- dated via gradient descent to align offspring with elites, thereby eliminating the need for pretraining or heuristic design. Empirical results in synthetic and realistic benchmarks demonstrate that ABOM matches outperforms advanced baselines without prior task knowledge. Attention visualization reveals interpretable search behaviors with consistent structural patterns. Thus, ABOM establishes aask-free paradigm for metaBBO, where learning and search co-evolve in real time.

---

Published as a conference paper at ICLR 2026
Limitations and Future Work. Current limitations motivate several promising directions: (1) Ad- dressing the cubic computational bottleneck (O(d)) through sparse or low-rank attention mecha- nisms to reduce ABOM's complexity; (2) Dynamically adapting population size and model capacity during optimization; (3) Conducting a convergence rate analysis grounded in the theoretical exam- ination of adaptive parameter learning in ABOM; and (4) Exploring hybrid training paradigms that integrate pretraining on prior knowledge with online adaptation, thereby enhancing optimization efficiency and bridging the gap between task-agnostic adaptation and cross-task generalization.
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of (2in5(N..22576264), Project supported by the National Science and Technology Major Project of the Ministry of Science and Technology of China Noo.20257D05515000 5000, No. No.2025ZD0551 502) the Key Project of National Natural Science Foundation of China (62431020,62231 the Joint Fund Project of National Natural Science Foundation of China (No.U22B2054), the Fund for Foreign Scholars in University Research and Teaching Programs (the 111 Project) (No. NN7.0074 the Postdoctoral Fellowship Program of China Postdoctoral Science Foundation (CPSF) (No.GGCC00232033), the Natural Science Basic Research Program of Shaanxi (Program the China Postdoctoral Science Foundation (Certificate Number:
No. 2025JC-YBQN-799 0025T180431 and 2025M771550), the Program for Cheung Kong Scholars and Innovative Re- search Team in University (No.RTT 15R53), the Key Scientific Technological Innovation Research Project by Ministry of Education and the National Key Laboratory of Human--Maching Hybrid Augmented Intelligence, Xi'an Jiaotong University (N0.HMHAl-202404, No. HMHAI-202405).
REFERENCES
Thomas Bäck and Hans-Paul Schwefel. An overview of evolutionary algorithms for parameter optimization. Evolutionary computation, 1(1):1-23, 1993.
Hui Bai and Ran Cheng. Generalized population-based training for hyperparameter optimization in reinforcement learning. IEEE Transactions on Emerging Topics in Computational Intelligence, (5):3450-3462, 2024.
James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(10):281-303 2012.
Janez Brest, Mirjam Sepesy Maučec, and Borko Bošković. Self-adaptive differential evolution al- gorithm with population size reduction for single objective bound-constrained optimization: Al- gorithm j21. In 2021 IEEE Congress Evolutionary Computation, pp. 817-824, 2021.
Jiacheng Chen, Zeyuan Ma, Hongshu Guo, Yining Ma, Jie Zhang, and Yue-Jiao Gong. SYMBOL: Generating flexible black-box optimizers through symbolic equation learning. In The Twelfth International Conference on Learning Representations, 2024.
Kenneth De Jong. Evolutionary computation: unified approach. In Proceedings of the Genetic and Evolutionary Computationn Conference Companion, GECCO '17, pp. 138-388, New York, NY, USA, 2017. Association for Computing Machinery.
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2): 2002.
Agoston E Eiben and Jim Smith. From evolutionary computation to the evolution of things. Nature, 521(7553):46--882, 2015.
Hongshu Guo, Yining Ma. Zeyuan Ma, Jiacheng Chen, Xinglin Zhang, Zhiguang Cao, Jun Zhangg and Yue-Jiao Gong. Deep reinforcement learning for dynamic algorithm selection: A proof-of- principle study on differential evolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 44(7):4247-444 4259, 2024.
Hongshu Guo, Sijie Ma, Zechuan Huang, Yuzhi Hu, Zeyuan Ma, Xinglin Zhang, and Yue-Jiao Gong. Reinforcement learning- based self-adaptive differential evolution through automated land- scape feature learning. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '25, pp. 1117-1126. New York, NY, USA, 2025. Association for Computing Machinery.
10

---

Published as a conference paper at ICLR 2026
Peter Hall and Christopher C Heyde. Martingale limit theory and its application. Academic press, 2014.
Muqi Han, Xiaobin Li, Kai Wu, Xiaoyu Zhang, and Handing Wang. Enhancing zero-shot black-box optimization via pretrained models with efficient population modeling, interaction, and stable gra- dient approximation. The Thirty-nimi Annual Conference on Neural Information Processing Systems, 2025.
Nikolaus Hansen. The cma evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772. 2016.
Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff Coco: platform for comparing continuous optimizers black-box setting. Optimization Methods and Software, 36(1) 36(1):114-144) 2021.
Jun He and Xin Yao. Drift analysis and average time complexity of evolutionary algorithms. Artifi- cial intelligence, 127(1):57-885 2001.
John H. Holland. Outline for logical theory of adaptive systems. J. ACM, 9(3):97-314,, July 1962.
J. Kennedy and R. Eberhart. Particle swarm optimization. In Proceedings ICNN'95 -nternnttaii Conference on Neural Networks, volume 4, pp. 1942-1948 vol.4, 1995.
Robert Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, and Sebas- tian Flennerhag. Discovering attention-based genetic algorithms via meta-black-bor optimiza- tion. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '23, pp. 929-937, New York, NY. USA. 2023a. Association for Computing Machinery.
Robert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flennerhag. Discovering evolution strategies via meta-black-tor optimiza- tion. In The Eleventh International Conference on Learning Representations, 2023b.
Ke Li, Alvaro Fialho, Sam Kwong, and Qingfu Zhang. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolu- tionary Computation, 14-300, 2013.
Xiaobin Li, Kai Wu, Yujian Betterest Li, Xiaoyu Zhang, Handing Wang, and Jing Liu. Pretrained optimization model for zero-shot black box optimization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Xiaobin Li, Kai Wu, Xiaoyu Zhang, and Handing Wang. B2opt: Learning to optimize black-box optimization with little budget. Proceedings of the AAAI Conference on Artificial Intelligence, 39 117::18502-1810, Apr. 2025.
Fei Liu, Xi Lin, Shunyu Yao, Zhenkun Wang, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang- Large language model for multiobjective evolutionary optimization. In Hemant Singh, Tapabrata Ray, Joshua Knowles, Xiaodong Li, Juergen Branke, Bing Wang, and Akira Oyama (eds.), Evolu- tionary Mulli-Criteriae Optimization, pp. 178-191, Singapore, 2025. Springer Nature Singapore.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019.
Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Zhenrui Li, Guojun Peng, Yue-Jiao Gong, Yining Ma, and Zhiguang Cao. Metabox: A benchmark platform for meta-black-bor optimization with re- inforcement learning. In Thirtyseventth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
Zeyuan Ma. Jiacheng Chen, Hongshu Guo, Yining Ma, and Yue-Jiao Gong. Auto-configuring exploration-exxloittaio tradeoff in evolutionary computation via deep reinforcement learning. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '24, pp- 1497-1505, New York, NY, USA, 2024. Association for Computing Machinery.

---

Published as a conference paper at ICLR 2026
Zeyuan Ma, Yue-Jiao Gong, Hongshu Guo, Wenjie Qiu, Sijie Ma, Hongqiao Lian, Jiajun Zhan, Kaixu Chen, Chen Wang, Zhiyang Huang, Zechuan Huang. Guojun Peng, Ran Cheng, and Yining Ma. Metabox-v2: A unified benchmark platform for meta- weta-blackk-ox optimization. In The Thirty- ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025a.
Zeyuan Ma, Hongshu Guo, Yue-Jiao Gong, Jun Zhang, and Kay Chen Tan. Toward automated algo- rithm design: A survey and practical guide to neta-black-box box-optimization. IEEE Transactions on Evolutionary Computation, pp. 1-1, 2025b.
Risto Mikkkulainen. Neuroevolution insights into biological neural computation. Science, 387 (6735))e 2025.
Risto Müikkulainen and Stephanie Forrest. A biological perspective on evolutionary computation. Nature Machine Intelligence, 3(1):9-15, 2021.
Yann Ollivier, Ludovic Arnold, Anne Auger, and Nikolaus Hansen. information-geometri opti- mization algorithms: A unifying picture via invariance principles. Journal of Machine Learning Research, 18(18):1-65 2017.
Ingo Rechenberg. The evolution strategy. a mathematical model of darwinian evolution. In Syner- getics- Microscopic to Macroscopic Order: Proceedings of the International Symposium on Synergetics at Berlin, July 4-8, 1983, pp. 122-132. Springer, 1984.
Bernardino Romera-Paredes, Mohammadanin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 62797595):468- 475, 2024.
Sasan Salmani Pour Avval, Nathan D Eskue, Roger M Groves, and Vahid Yaghoubi. Systematic review on neural architecture search. Artificial Intelligence Review, 58(3):73, 2025.
Mhd Ali Shehadeh and Jakub Küdela. Benchmarking global optimization techniques for unmanned aerial vehicle path planning. Expert Systems with Applications, pp. 128645, 2025.
Kenneth O Stanley, Jeff Clune, Joel Lehman, and Risto Mikkkulainen. Designing neural networks through neuroevolution. Nature Machine Intelligence, 1 1(1):24-35, 2019.
Rainer Storn and Kenneth Price. Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J. of Global Optimization, 11(4):341-35 December 1997.
Xinmin Tao, Xiangke Li, Wei Chen, Tian Liang, Yetong Li, Jie Guo, and Lin Qi. Self-adaptive two roles hybrid learning strategies-based particle swarm optimization. Information Sciences, 578: 457-481, 2021.
Ye Tian, Shichen Peng, Xingyi Zhang, Tobias Rodemann, Kay Chen Tan, and Yaochu Jin. recom- mender system for metaheuristic algorithms for continuous optimization based on deep recurrent neural networks. IEEE Transactions on Artificial Intelligence, 1(1):5-18, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Chao Wang, Licheng Jiao, Jiaxuan Zhao, Lingling Li, Xu Liu, Fang Liu, and Shuyuan Yang. Bi-level multiobjective evolutionary learning: A case study on multitask graph neural topology search. IEEE Transactions on Evolutionary Computation, 28(1):208-25 222, 2023.
Chao Wang, Jiaxuan Zhao, Licheng Jiao, Lingling Li, Fang Liu, and Shuyuan Yang. When large lan- guage models meet evolutionary algorithms: Potential enhancements and challenges. Research, 8:0646, 2025a.
12

---

Published as a conference paper at ICLR 2026
Chen Wang, Zeyuan Ma, Zhiguang Cao, and Yue-Jiao Gong. box optimization through latent space reverse engineering. arXiv preprint arXiv:2509 15810, 2025b.
Instance generation for meta-black-
David H Wolpert and William Macready. No free lunch theorems for optimization. IEEE trans- actions on evolutionary computation, 1(1):67-82, 2002.
Xiaoming Xue, Cuie Yang, Liang Feng, Kai Zhang, Linqi Song, and Kay Chen Tan. A scalable test problem generator for sequential transfer optimization. IEEE Transactions on Cybernetics, 55(5): 2110-2123, 2025.
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In B. Kim, Y. Yue, Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference Representation Learning, volume 2024, pp. 12028-12068, 2024.
Taeyoung Yun, Kiyoung Om, Jaewoo Lee, Sujin Yun, and Jinkyoo Park. Posterior inference with diffusion models for high-dimensional black-box optimization. In Forty-second International Conference on Machine Learning, 2025.
Zhi-Hon Hua Zhou, Yang Yu, and Chao Qian. Evolutionary learning: Advances in theories and algo- rithms. Springer, 2019.
13

---

Published as a conference paper at ICLR 2026
A REPRODUCIBILITY STATEMENT
To ensure the reproducibility, we have taken the following measures:
Source Code: We provide complete implementation of the Adaptive meta Black-box Optimization Model (ABOM) at the following repository: https://githh:: com/
xiaofangxd/ABok,
Algorithm Specification: ABOM is fully detailed in Section 3 including the mathemati- cal formulations for the selection, crossover, and mutation operators. The complete pseu- docode for the optimization loop is provided in Algorithm (Appendix C
Experimental Setup: All baselines are implemented and evaluated using the state-of-the- art MetaBox benchmark platform Ma et al. (2023) 2025a). This ensures standardized, fair, and reproducible comparison. Appendix describes the BBOB and UAV benchmarks, including their characteristics, search space dimensions, and evaluation budgets. Appendix G details the hyperparameter settings for all baselines as specified the original papers and implemented in MetaBox.
By providing the code, algorithmic descriptions, and experimental configurations the unified MetaBox platform, we aim to enable other researchers to fully reproduce and build upon our results.
B USE OF LARGE LANGUAGE MODELS
Large Language Models (LLMs) were used solely as general-puree purposs writing assistance tool. Their role was limited language polishing, grammatical refinement, and improving the clarity and flu- ency of the paper. LLMs did not contribute to the conception of research ideas, experimental design, data analysis, or interpretation results. All intellectual contributions, including the formulation of the problem, methodology, and conclusions, were made entirely by the human authors.
C PSEUDOCODE OF ABOM
Algorithm Adaptive meta Black-box Optimization Model (ABOM)
Input:
Target black-box optimization task fr, population size N, max generations T, crossover dropout rate Pc, mutation dropout rate Pas, learning rate 7, attention dimension dA, and MLP hidden dimension dys
1: Initialize (0) via Latin hypercube sampling:
2: Evaluate: (0) + (P())
3: for = to T 1 do
4: Generate offspring: ((1) +- no (P(1), F(t); dA. dMM.dM.P..P.. PM:P
5: Evaluate: (() + fz (P());;
6: Form elite archive: E F(t) + tOpN (P(t) ((())
ttoN F(t) UUUW)
7: Update parameters by Adam W: - V p(0) E(0)
8: Elitism: ((++1) B(0 F(++1) ((()
9: end for
Output: Optimal individual (solution) p arg minee fr
D CONVERGENCE ANALYSIS OF ABOM
This section presents convergence analysis of ABOM under some assumptions. We rigorously prove that ABOM converges with probability to the global optimum of the objective function. Let X' Ed be compact search space and fr X + E be continuous objective function with global minimum f fr(x"). ABOM maintains population ((t) E NR generation
14

---

Published as a conference paper at ICLR 2026
with corresponding fitness values F(O, For the convergence analysis, make the following assumptions:
Assumption The following conditions hold:
(i) The global optimum x" lies in the interior of X.
(ii) Dropout rates satisfy PC-PM 1.
(iii MLP hidden dimension das with tanh activation.
Define f7 min (x), where E(1) is the elite archive containing the top N individuals from be the filtration representing all information
U (P()) Let F; = (P! p(() 010) , ,(()) up to generation f. The elitism mechanism ensures fi+1 I f almost surely, implying [E S Since fr is bounded on the compact set X. the sequence [f forms lower-bounded supermartingale. By the martingale convergence theorem Hall & Heyde (2014), fž converges with probability to the random variable f 2 f". To establish global convergence, we need to prove fx = f with probability 1.
F,)
Lemma Under Assumption for any 8 > 0, there exists 7 > such that:
P(3i: lb x < Fi) 2 1 - (1 - 7) > 0.
(15)
Proof 1 For the crossover operation, consider any parent (1) E X and let v x" -p; Define
the MLP configuration with W 0, b₁ 0, Wa 0, and b2 v. Then:
MLP 0g (A (4) p. (8)
= tanh A (1) P W1 + by W2 + b2 =V.
(16)
Consequently:
p + MLP 0: (A (A =
P
(17)
By continuity the MLP (as composition of continuous functions), there exists all Og E (N)):)):
0 such that for
Ip (4) MLPP, (A P (1)
-x11 - 0/2.
(18)
Define pe P(( E N((())) Fi). Given the parameter update d((+1) = o.!) - -n + g())
with stochastic perturbations g(t) from dropout patterns ((t) Bernoulli(1 - dM) which have minimum probability mass:
min ((D =D) = =minn(p, Pc)) > 0,
(19)
and since the conditional distribution of of has positive density, there exists et > such that:
Me (min(pe, - -ppc)d cz >
(20)
(()) and let = x' -p, Define
(())
For the mutation operation, consider any intermediate solution p.
the MLP configuration with W = 0, by 0, W = O, and b w. Then:
MLP (M()), m
((0) (W; by) W. by w. P; (()) tanh(M (0) p,
(21)
Consequently:
p 1() + MLPe: (M,)) p (1)
=
(22)
By identical continuity properties, there exists c > O such that for all fm E Nc
Ip, (0) + MLP,,., (M. (0))
(0) x 6/2.
(23)
Define pm TP(O! (Ott Fi). By analogous reasoning to the crossover operation:
pm > (min(pas; -pas)das c.. > 0,
(24)
15

---

Published as a conference paper at ICLR 2026
where d₁ > 0 is determined by mutation parameters.
The probability that a single offspring Di falls within è xf
x" satisfies:
P(IP: (6) -x <
Fi) 2 MeHm-
(25)
Let Maffm > which is positive constant independent of and depends only hyperparam- eters pc, PM, dm. and the adaptive parameter learning. Finally, for N independent offspring:
N (1 - P(IIP) (9 - xx < F.)) 2 1- 111111111)0 0. (26)
P(3:: - xx1x <
II mm1
Define the distance function 1/2 fr f 0. Using Lemma we establish the following drift condition He & Yao (200T): Zhou et (2019)
Lemma Under Assumption for any O, there exists rs(e) > such that:
EVV, - Ve+1 Fr, V3
2 nTe) 0.
(27)
2roof Define the event A, =Bi: lIb(9) x || < 63. where ô chosen such that for all
E X' with |x- x"| < 6, we have fr(x) < f + cc22 (which exists by the continuity of fr and
Assumption i)). By Lemma ! there exists 7 > such that:
P(A Fe. V1 <) 2 1 11 (1-11))
(28)
When A, occurs, the elite archive at generation t ++ contains at least one solution finees value less than f + c/2, so:
V4+1 f f < <//2.
(29)
When A, does occur, the elitism mechanism ensures Vi+1 S Vi (since the elite archive preserves the best solutions). Therefore, the expected drift can be decomposed as:
VVV
Ve+1
Fr, V1
= EV,
>
V4+1
Fr 1/2 > c, A,] P(A, F,, V; > c)
(30)
+ E[V - Ve+1 F,, V, > 6, A7] P(A Fr, V1 > 6).
(31)
For the first term, using Eq.
29 and the condition V/ > e:
V V4+1 Fe, V₁ > e, Ar] 2 V - E/2
(32)
E c/2 c/2.
(33)
For the second term, since V4+1 S Vz by the elitism mechanism:
VVV Vi+1 Fr, > 49 2 0.
(34)
Combining these results with Eq. 28
E[V, V(+1
Fe, V2
2 (e/2). (1 (1-(1-)))) ) +0 P(A Fe, V/
(35)
2 ( (1-(177)
(36)
Setting n(c) = (c/2) (1- (1 > 0 completes the proof.
With the positive drift condition established, we can prove global convergence.
Theorem D.1 Under Assumption
ABOM converges with probability to the global optimum:
f a.s, f" as 1 0o.
(37)
Proof By the martingale convergence theorem Hall & Heyde (2014), ft converges with proba- bility I to some random variable f > f Assume for contradiction that f% > I" with positive
16

---

Published as a conference paper at ICLR 2026
probability. Then there exists 0 such that Vi >> e eforr all sufficiently large t. Define the stopping time Tj = it 2 k V₁ c}.
Consider the value function change from time k to Tk-
TK-
V
VTL Σ (V-V--
(38)
Taking expectations and applying the law of iterated expectations:
BlV2 1 E
ElV,
Ve+1 Fi)
(39)
t=
Fort t < The we have V; > so by Lemma 2
E
V Ve+1 Fi)
2 2/(c) El[r - k).
(40)
t=
Thus:
[[V]] [[[ ] 2 mT(e)) BBTT- - kJ.
(41)
Since V. S 6, we have:
[VV ncc)
E(Tx-
x] S
< co.
(42)
This implies P(rk < 0c) = 1, contradicting the assumption that Ve for all sufficiently large Therefore, fo = f with probability 1.
Theorem D. establishes that ABOM converges with probability to the global optimum under Assumption The persistent application of dropout during inference, coupled with the adaptive pa- rameter learning mechanism. ensures that there exists a positive probability of generating offspring within an arbitrarily small neighborhood of the optimum at each iteration.
Theoretical Limitation. Theorem D.I establishes asymptotic convergence but not polynomial tomomiill-tmee convergence. Convergence rate analysis (expected hitting time) for specific problems is of the important research directions for the future. It is worth noting that our convergence analysis does not directly apply in cases where the global optimum lies on the boundary or where constraint handling results in a discontinuous feasible region.
Theoretical Contributions. Our work establishes theoretical contributions for meta black-box optimization: 1) We prove novel exploration guarantee (Lemma [ showing that attention-bart MLP parameterization with dropout maintains persistent exploration capability. 2) We prove global convergence of ABOM with adaptive parameter learning (Theorem D.I). Crucially, this demon- strates that self-supervised parameter adaptation does not compromise convergence guarantees. This stands in contrast to existing neural fetwork-parameteriae methods such GLHF Li et al. (2024) and B20pt Li et al.] (2025), which lack rigorous convergence analysis despite empirical success. These theoretical foundations provide ABOM's reliability while preserving the flexibility of adap- tive optimization.
E CONVERGENCE ANALYSIS OF PARAMETER ADAPTATION
Fig. showw the loss curves of parameter adaptation the BBOB suite with |-00 100]500 De- spite the large search space and high dimensionality, the loss of parameter adaptation consistently decreases and converges across all test functions, with minimal variance across 30 independent runs. This empirical evidence validates our theoretical assumption of local convergence for parameter adaptation (Eq. (8). demonstrating that the self-supervised learning paradigm with AdamW opti- mizer remains stable even in challenging high-dimensional optimization scenarios. The consistent convergence behavior aligns with standard machine learning practices for training iitentiin-based MLP architectures using gradient-based optimization, confirming the practical viability of our adap- tive parameter learning framework.
17

---

Published as a conference paper at ICLR 2026
Buchs Raitros
Httactive Sector
Step pilpes dal
Rocembrudk origial
0:000
140:00
10000
200000
10000 Faiitinn Exalationn (FESt
Faneman Evll iatiann (FF4
Fanttiin Exnkortion giF
Runet Palivater
110100
10000
20080
100000
19805
Vola
Vilio
Liite
1ain
1967
1986
Fanetion Eullartann (WE4
Fanttiun Buok 9114
Ravition Exsliation (FES)
Tunnt on Eodiaat ond 1FE4
Shary, Xidge
Cettorent Powers
Schafters tigh aond
Compusite Sela,
105/0
30080
100000
10026
10800
30080
10000
1980
Hella
Schwefhl
Go Gallggee 21Pasc
catsarr
Lutscilc Di, Partrigh
150004
100000
100000
30800
54000
24040
50000
Fonerian Euil wallaaiann
Fanmian Evokorion
Fantionn Exaluttion
Figure 6: Loss curves of parameter adaptation on the BBOB suite with d = 500. Each subplot de- picts the mean loss across 30 independent runs, with shaded regions representing standard deviation.
Fig. 7 presents the convergence behavior of ABOM
1.0
ABOM
Normalized Costs
ABOM NPA
and ABOM-NPA (no parameter adaptation). Both meth- ods demonstrate convergence in practice, which empiri- cally confirms that the convergence guarantee stems from the core mechanisms of elite preservation and dropout- enabled exploration rather than solely depending on pa- rameter adaptation. While our theoretical analysis (The- orem D.I. formally establishes global convergence for ABOM with parameter adaptation, the experimental com- parison reveals that the fundamental convergence proper- ties are maintained even without this component, suggest- ing that the theoretical framework could be extended to cover variants without parameter adaptation. This empir- ical observation aligns with the martingale convergence argument in Theorem D.I] where the supermartingale property E [F]] S ft is primarily ensured by the elitism mechanism rather than parameter adaptation.
0.8
0.6
0.4
0.2
0.0
5000
10000
15000
20000
Function Evaluations (FEs)
Figure 7: Convergence comparison be- tween ABOM and ABOM-NPA on BBOB suite with d 500, which shows normalized costs against function eval- uations over 30 independent runs.
F TASK CONFIGURATION
Table 3 presents 24 instances of synthetic black-box optimization benchmarks (BBOB) with diverse characteristics and landscapes. Following the standard protocol of the benchmark platform Ma
et al. (2023 2025a). functions fi, f2, Sa, Ss. S1s. f16. f17, and faı are designated as training
18

---

Published as a conference paper at ICLR 2026
Table 3: Overview of the BBOB suites.
ID
Function
Characteristic
Usage
h
Sphere
Train
Ellipsoidal
Train
Separable
Rastrigin
Train
fa
Linear Slope
Train
fIs
Rastrigin (non-separable)
Train
fas
Weierstrass
Multi-modal with adequate global structure
Train
f1r
Schaffers F7
Train
fz1
Gallagher's Gaussian 101-me Peaks
Multi-modal with weak global structure
Train
f4 Buche-Rastrigin
Separable
Test
Attractive Sector
Test
fr
Step Ellipsoidal
Test
Low/moderate conditioning
fa
Rosenbrock, original
Test
fo
Rosenbrock, rotated
Test
f10 Ellipsoidal
Test
fn
Discus
Test
f12
Bent Cigar
High conditioning, unimodal
Test
f13
Sharp Ridge
Test
f14
Different Powers
Test
f18
Schaffers F7, ill-conditioned
Test
Multi-modal with adequate global structure
f19
Composite Griewank-Rosennron F8F2
Test
$20
Schwefel
Test
f22
Gallagher's Gaussian 21-hi Peaks
Test
Multi-modal with weak global structure
f2a
Katsuura
Test
f24
Lunacck bi- bi-Rastrigin
Test
functions, while the remaining functions serve as test instances, ensuring a balanced distribution of optimization difficulty between the training and test sets. The maximum number of function evaluations is set to 20,000. All functions are defined over [-100, 100 d = 10/10005000
The UAV benchmarks comprise 56 terrain-based scenarios that represent realistic unmanned acrial vehicle path planning problems, each with 30 dimensions. The scenarios are divided into training and test sets of equal size (28 instances each), with test instances corresponding to even- wven-numbered indices (0, 2, 4, 54). Following the standard protocol of the benchmark platform Ma et al. (2023) 2025a), the maximum number of function evaluations set to 2,500.
G BASELINES
Since our ABOM is an evolution-basee meta-black-bee box optimization (metaBBO) algorithm, we re- strict comparisons exclusively evolution-based methods, excluding non-evolution-basee methods such as Bayesian optimization. Furthermore, we omit LLM-based metaBBO methods Liu et al. 2025 Komera-Paredes et al. (2024): Yang al. (2224) from our baselines, as they are tailored for specific task types and are not directly comparable to evolution-based general-purposs frameworks.
To ensure fair and reproducible comparison, all baselines are implemented using the source code provided by the official MetaBox platform Ma et al. (2023) 2025a). Detailed hyperparameter con- figurations for baselines are provided Table 4] while the configuration of our proposed ABOM summarized in Table All results reported the mean and standard deviation over 30 indepen- dent runs, with a fixed population size of across all trials.
For traditional BBO methods, we adopt the hyperparameter settings recommended in the original pa- per, rather than performing grid search or manual tuning. The design choice aligns with core mo- tivation of adaptive optimization and metaBBO methods: to reduce the reliance labor-intensive hyperparameter tuning. By using default settings, we ensure a fair and meaningful comparison that highlights the intrinsic advantages of adaptive strategies.
19

---

Published as a conference paper at ICLR 2026
Table 4: Detailed hyperparameter configurations of baselines. ub and lb denote the upper and lower bounds of the search space, respectively. randn(d) denotes sampling a dddimensional vector from standard normal distribution. All MetaBBO methods are trained on the same synthetic problem
distribution as RLDEAFL Guo et (025)..
Baseline
Parameter
Setting
Traditional BBO methods
Uniform sampling within [tb, ub]* Bengio (012))
Bergstra &
RS
Linearly decreased from 0.9 0.4 itera- tions tions Kennedy & Fherhart (1995
PSO
Inertia weight w
Coefficients ea /
2.0/2.0 Kennedy & Eberharl 1995
DE
Mutation factor F
0.5 Storn Price 1997
Crossover probability CR
0.5
Storn & Price 1997
Strategy
DE/rand/ 1/bin Storn & Price 1997
Adattive optimization variants
Parameter ranges follow those specified in the original paper Tao et al. 2021).
SAHLPSO
Adaptive parameters
Parameter ranges follow those specified in the original paper Brest et al. (2021)
JDE21
Adaptive parameters
CMAES
Initial step size
0.3 x (ub 16) Hansen 2016 l = lb + randn(d) x {u0 1b]
Initial mcan
Hansen 2016
MetaBBO mechiossTraining on BBOB or UAV training set)
Parameter configurations are consistent with the original paper Ma et al. (2024).
GLEET
Training parameters
Parameter configurations are consistent with the original paper Guo et al. 2025
RLDEAFL
Training parameters
Parameter configurations are consistent with the original paper] [Lange et al. (2023b).
LES
Training parameters
Parameter configurations are consistent with the original paper Li (224).
GLHF
Training parameters
Table 5: Hyperparameter configuration of ABOM.
Parameter
Setting
Crossover dropout rate Pc:
0.95
Mutation dropout rate pM
0.95
Learning rate y AdamW x 10
Attention dimension d
MLP hidden dimension dar
(4)]
20

---

Published as a conference paper at ICLR 2026
H COMPARISON WITH EPOM
This section conducts performance comparison tween method ABOM and EPOM, recently proposed meta black-box optimization method that represents the current optimization Han et al. (2025). EPOM operates as a pre-trained optimization model that learns a gener- alizable mapping from tpec-fpe features to op- timization strategies, thereby enabling zero-shot op- timization capabilities on previously unseen black- box problems. We evaluate ABOM and EPOM on the Bipedal Walker task, which requires optimizing a fully-connected neural network policy with 874 parameters over k 800 timesteps to enhance robotic locomotion control performance. To ensure a fair and reproducible comparison, to the experimental protocol and parameter settings established in the original paper [Han al. (2025). ABOM utilizes the identical hyperparameters
1.2
Normalized Reward
1.0
of the art in zero-shot
0.8
0.6
0.0
ABOM
200
800
1000
Function Evaluations (FEs)
Figure 8: Convergence comparison between
strictly adhere
ABOM and EPOM on Bipedal Walker task.
as those employed in our prior experiments (refer to Table 5 for details), maintaining consistency
across evaluations. As demonstrated in Fig. g ABOM achieves significantly faster convergence to
high-quality solutions, while EPOM exhibits premature convergence, underscoring the robustness
and effectiveness of our method challenging optimization scenarios.
SENSITIVITY ANALYSIS OF LEARNING RATE
0.8
Normaltzed Avarage objective Value
Normallized Loss
n=1x10
08
0.6
n=1x10-1
n=1x102
Learning Rate
1x
0.4
1x10
1x10
0.4
0.2
02
0.0
Function ID
5000
10000
15000
20000
Function Evaluations (FEs)
Normalized Average Objective Value
Normalized loss convergence
Figure Sensitivity analysis of learning rate 7 on the BBOB suite with d 30.
Fig. 9 presents the sensitivity analysis the learning rate of Adam for ABOM's parameter adapta- tion. The heatmap (Fig. (a)) shows optimization performance across different learning rates, while the loss curves (Fig. (9()) demonstrate the convergence behavior of the parameter adaptation.
The loss exhibits stable convergence across the evaluated learning rate spectrum (Fig, 9()).. pirically validating our theoretical assumption of local convergence for parameter adaptation (Eq- 18). However, shown in Fig. (a) optimization performance varies significantly across learning rates. 7 = x 10 achieves the best balance between convergence speed and solution quality. This demonstrates that while convergent behavior of the loss is necessary for stable parameter adaptation, it does not guarantee optimal optimization performance. The choice of learning rate remains crucial for effective parameter adaptation.
PRELIMINARY GENERALIZATION ANALYSIS OF ABOM
This section preliminarily explores the generalization capability of ABOM through pre-training the STOP benchmark suite Xue et al. (2025). The STOP suite comprises 12 sequence transfer optimization problems, where each problem contains series of source optimization tasks and one
21

---

Published as a conference paper at ICLR 2026
target optimization task. Based the similarity between source and target tasks (measured by fitness landscape overlap and optimal solution alignment), the 12 problems are categorized into three groups: high similariiy (STOP1-4), mixed similarity (STOP5-8), and low similarity (STOP9- 12). Detailed properties of the optimization tasks are provided in Xue et al. (2025). For experimental evaluation, each problem is instantiated with 10 source tasks under maximum evaluation budget of 5,000 per task. We treat these source tasks as the training set and the target task as the test set. This setup spans diverse similarity scenarios between training and test sets, enabling a more comprehensive evaluation of ABOM's generalization performance.
We introduce ABOM-PT, pre-trained variant of ABOM, where meta-optimization knowledge is distilled from the training set. Specifically, ABOM is executed on 10 source tasks for T 250 iter-
ations per task, generating 2500 prior training samples M {(PP)) E(()) P( E R F(
RN A 1,.... K, I 1,.... T). The pre-training objective minimizes the prediction error of popu- lation evolution:
K T-1
((++1) ABOM (P . //
min Cpx W
(43)
[=1
where ABOMW predicts the next-generation population Pk (t+1) from current population-fitnes pairs. We optimized the Eq. (43) using AdamW with a learning rate of 1 x 10 and batch size of 256. The hyperparameters for parameter adaptation are consistent with those in Table
Table 6 presents the experimental results of ABOM and ABOM-PT on the STOP benchmark suite, revealing four key insights: 1) ABOM-PT outperforms ABOM in 9 of 12 problems, confirming the generalization capability of our method; 2) Under high-similarity conditions (STOP1-4), ABOM-PT achieves substantially better performance by effectively leveraging optimization knowledge from training tasks to the test task; 3) ABOM-PT underperforms on some mixed-similarity problems (such STOP8), revealing limitations in handling complex task relationships; 4) Surprisingly, pre- training on low-similarity tasks (STOP9-12) consistently improves performance on the test task, demonstrating that even dissimilar training tasks contain valuable optimization knowledge that en- hances generalization capability.
Table Performance comparison of ABOM vs. ABOM-PT on the STOP suite over 30 independent runs, reported the mean and standard deviation of objective values (lower is better).
Problem
Similarity
ABOM
ABOM-PT
(mean + std)
(mean + std)
STOP1
High
1.08c+0 + 4.42e-1
4.73e-1 + 1.97e-1
STOP2
High
1.92c-1 = 7
2.61e-2 + 7.45e-4
STOP3
High
1.20e+0 = 4.87e+0
1.71e-1 8.04e-3
STOP4
High
2.52e-1 = 6.50e-3
2.08c-1 2.72c-3
STOP5
Medium
2.79c+0 = 1.21c+1
1.28e-2 9.88e-6
STOP6
Medium
1.06++2 = 9.62e+2
1.01c42 3.74c+2
STOP7
Medium
5.27e-2 = 1.28c-2
6,84c-3 + 7.83e-5
STOP8
Medium
3.60c+0 + 1.22c+1
5.39c+0 + 2.90e+1
STOP9
Low
1.75e-2 = 1.50e-4
3.84e-3 + 4.32e-6
STOP10
Low
3.87c+1 5.49c+1
2.69c+1 + 4.61e+1
STOP11
Low
5.02c+0
3.97c+
6.20e-1 1.20e-1
STOP12
Low
9.52e+2 = 1.38e+6
8,55c+1 1.20c+4
1//+
9/3/0
K EXPERIMENTAL RESULTS
Tables and show the mean and standard deviation over 30 runs for each baseline on the BBOB suite with d 30/100. The convergence curves of the average normalized cost across all test functions for the BBOB suite, with dimensions 100/1005500 500, are presented in Fig. 10 based on 30 independent runs.
22

---

Published as a conference paper at ICLR 2026
The convergence eurves of cost (log scale) for the 28 UAV problems 30 independent runs shown in Fig.
Fig. 12 and Fig. 13
The boxplots of cost (log scale) over 30 independent runs for the 28 UAV problems are shown in Fig.
and Fig. 16
Table 7: The comparison results of the baselines on the BBOB suite with d == 30. All results are reported the mean and standard deviation (mean + std) over 30 independent runs. Symbols "-", "RJ", and "+" imply that the corresponding baseline is significantly worse, similar, and better than ABOM the Wilcoxon rank-sum test with 95% % confidence level, respectively. The best results indicated in bold, and the suboptimal results are underlined.
Traditional BBO
Adaptive Variants
MetaBBBO
Ours
ID
RS
PSO
DE
SAHLPSO
JDE21
CMAES
GLEET
REDEARL
LES
GLHF
ABOM
f4
$1.25e44
5.17c+5
# 13877444
1.28045
+ 10000
3.84++
#2.45e45
2.04c45
5. 5.58e+3
34.39
1.685440
3.25e+1
# 2.44444
6.6-+4
13899x3
6.18e+3
23.00445
188e-6
24 08---
6 36644
22.95
5.45++2
f6
27.85046
5.42+77
# 22,55-45
2.49c45
27.08433
6.12+
+1.050+7
5.11
#1.1904
2.00+4
24428e44
5.35++0
27.82444
3.76c=4
#1 24044
2,00+4
28. 28.00446
8.01e+7
11588-7
661e47
22.64
2,80++7
47
24.35644
2.68++5
# 1.85544
6,70 70c+4
+4 34.99
2.4e+4
#5.23e44
5.67c44
3 366+4
3,500
+7 27.00
3.26++5
25.56
8.11
18.1
6.28++3
136204
3.54c+5
24.79e-4
2.59.45
2
5.58c+2
fa
#2.94e49
10
#5 5,32248
1:23c49
32.66043
1.27e43
#2.53e48
2.46e+8
#1 1.226044
7.5e13
+1 1,777
5.5ee42
26 76777
1.29e47
#1 98.98
1.0cs33
44.30448
382c49
55450-8
4
+ 11,56442
1.15ex2
f0
22.13
10
2,27744
886048
+ 11.33044
1.50es4
#3.65047
3.82c77
#5.40044
2.34444
1.5644
2.48c
25.28444
2.03444
+ 47645
4.30
22.12:40
L49ca3
11,60-00
185e12
+ 30e4
2.35c43
147e48
143848
1.11es8
2.1cs3
1.73e47
8:99466
1.5447
742216
181cs9
68%e18
9.72e45
+1,6er8
# 6.28ce+
+2.29ee7
+3.13er8
+1.22077
+4.44
#2.44e77
+3.12e46
+4,37e+8
+301e-88
47.38e15
f1
+122eed
1.12445
+ 288244
882644
1.04e15
#2.5er4
9.90e44
+2,54e44
7.32e44
+290ee
3.55e44
1.04er
3.23c44
#2888e44
8.57e44
+1.71e66
8.99c45
+7.71e+3
672614
+ 1.01es4
2.61e44
f1
13
15
#2.12ee11
1. 188411
+4.35er8
1.05499
11,40c+19
18
+2224ce9
1.52e19
+2.44c0
143cx0
3.6e-10 10
3.221e+0
+1.99ee9
645es8
+7
8.23ex19
45.162-17
2.5es177
+ 1.45e
5,28c47
f13
+236e13
4.3e+1
+ 2,49c+3
2.e7
+111ee22
6.96+22
+5.51er3
1.5cel
+4489c+2
5.18
1145ce0
1,0c+00
#3,03ee3
3.57e+3
+181e43
7.7l++2
+1,00+3
3.97el
+241e-3
3.78ed
+3.07e+1
7.25++
f14
15.40
3.5e+l
# 186c-3
3.77
17 40.00+2
3.76++3
16. 19c+3
5.07e+3
11.91e=2
3.72e+2
16 73.73
3,99c+0
4..4e++2
5.69e+2
18.73e41
1.9e+2
22,17
9.99cm
1.75ee-
3.30e
13.99-2
3.6e-1
f18
11.0e4
1,42++3
# 2.09--
9.43++
10e4
6.00++2
11.19
8.89c43
11.26e+2
5.6e+2
12 11
3.36e=12
1 E1,62+2
5.79e+2
+ 18.28+
521e+2
11.74
2.48=66
14.1
5.18++5
11.37++
5.12e+2
f19
11.8004
1.01e66
# 1,98c-4
7.82+44
13.78
1.22++1
12.84e+3
1.92++3
1.97ee
2.0e+1
17740e
6.76c+0
1..3
1.37e+
13.03++
2.87+1
+1.27
1,30e3
1591e--0
9.9e+0
Lele
2.8e-1
220
12.75e+6
1.24c+7
26 6.889
9.79645
22.75e++0
-388%41
1 61040
4.04c43
+2.77=00
6.20041
19.73e40
14 +..98
3397641
15.06
-5.00e+1
+2,37e40
1.4943
1..77-00
1.100
13.00+
f22
10.00+0
-0000--0
866041
10000+0
8.60e41
10000.+0
20.00>00
10.00.40
10,00.+0
1000>+0
8.6ee1
14.55e-13
1.19s33
20.00-00
10.0+00
8.6641
f23
+399e-1
3.20c40
3382e-1
3.3640
1:4.66-11
3.18e40
#4494--1
#3.22-1
+3.08-11
+3.34e-1
2.98a0
d:5.37e-1
3.3ee40
#3.11e-1
1,30043
33.70e-1
2.11--11
3.1c-1
f21
11.7es4
1,4e455
#
384544
#5.03e+1
3.23442
1,22644
28.02c92
5.70642
+1.4ee11
#7.1ee
3.9844
2
7.08442
22.34
5.35e44
42770-3
2
2.44.12
1510
15/1/
151100
1518
15/100
10/15
15/1,)
157100
150/1
14/1/1
23

---

Published as a conference paper at ICLR 2026
Table 8: The comparison results of the baselines on the BBOB suite with d = 100. All results are reported as the mean and standard deviation (mean + std) over 30 independent runs. Symbols "-", "z"" and "4" imply that the corresponding baseline is significantly worse, similar, and better than ABOM on the Wilcoxon rank-sum test with 95% confidence level, respectively. The best results are indicated in bold, and the suboptimal results underlined.
Adaptive
MetaBBO
DE
SAHLPSS
JDE21
CMAES
GLEET
RLDEAFL
GLHE
ABOM
9,66ce6
1.3e+6
7.23cr5
2.49c46
2.82c+5
3.72++3
3.77e-5
2.9c+5
1i4e=7
9.00e16
9.72c+3
11.21e+6
42.1
15.83e
18.00+5
1.477
6,11ee8
4,36e17
1033+6
4.50e+7
7.16crd
2.15er4
4.27e-5
196e+5
5.71ee8
5.45e88
7.92c44
1.381et7
1229ee77
1:9.2++
13.58c+7
1.00e66
1:9666c
19.09ee
1:66e++7
12.78c=7
1.349c+4
L,75e66
6.23e15
4,96215
4.36e15
9,32ce4
8Ster3
7,92c-4
1.17e+5
1.13c66
1.0e16
6.23e+3
19,77ee1
16.59e-4
2.285e
12.9763
11,50e+3
4.33-111
8.7ee100
2,67e48
2660410
4.60x49
5.16443
1.41e-8
3.70649
5,242-10
5.00+
3,44433
13.31e+10
1:1599+10
15005e+7
11.53e=10
14.18et9
15.73e43
13.6fc+9
2876=11
7.20e400
1.53+49
1.34+49
4,97448
5.3644
6,376-7
341c48
2.96-+
6.43642
1,13c45
12.82e+10
1:121e+10
1:445e+8
15.55e+8
1:221c+7
12.71e+8
14.10e1
1.279e+5
f10
17,0048
7.86+9
43.14
1.97499
#2.58e48
2.30++9
29.32c48
1.88+49
11.37e18
4,37c48
3.66647
1.30+88
2,55--8
28 93.93
2/7048
23.1944
6,5c49
28.8845
563649
11.8es7
6.65647
4.30e45
4 12044
4.35c-5
2.99e45
3.78e45
9 9.004
1.97445
2.0045
1.1546
2.7045
3.8k45
#2.62444
15.51e44
13.3es44
25.28e44
14.01e14
33.3es44
1491044
114066
2884043
16.24e44
f12
22.126422
184c=22
13.776416
700e115
16.93+13
6,78c+13
2.77e422
2804es12
5.29c+12
5.66cc48
8.66118
11.10
284c+11
1681e411
4.7ieell
15.86122
6.99-222
29.78
375e+21
11.31e
397ee9
f1
LüBke5
1461e43
6.0es4
2.66
11.44
3.28144
1600043
2.32004
2.355s41
1.09r12
1385343
Lo9e-4
#4 78.3
231er4
1..5e4
8,37ee4
1..7744
8.14
11.44
3.0c92
f14
13.66rr1
3.10ce5
1985e+3
4333e4
6.79cr4
1644ee3
#5.22ce
534e14
8.14c13
+3.75e+2
1.33913
4.62+-3
1.90e3
11.88e+
4.39113
1262et1
2.60eed
13.70ce4
2 16e45
1 1.Me+2
3.33ev2
fi
#284e+?
581ce7
18933+3
6.42e13
1.59s4
8445+3
#3.12c48
7:27e+7
16.33e+3
33600c+3
2X2e13
1.679
4.63--3
44.78eed
2.8er4
11.1ett
2.49ce7
66,30c+6
1. 1.12
2.46ev2
14.1e++1
f19
15,66r3
2.48e66
42800=5
1.77e16
# 7.9661
3.42e44
# L.422e
2.77e44
1.,00++
LiKee4
4:554e+00
2.3es1
7.4ee2
Lode-3
16.13ee3
6.4413
1.,6000
5.06ee2
L.1cel
1.5e91
17.6e
2.50c-1
L16c=8
3 44c47
3.05c=4
2.33e45
7.12c+4
-3.04e40
-1.92e41
7.90e4
2.10c=3
9186-1
-.600=11
220
15.88c
16,40c-6
3.00er
#2888cc55
d 114ee+
1.72e+
5,28c++0
1.51es
1..78e
15.31e
13336r0
f2a
+0.00.0
8.66c=1
20000=00
8.66+1
20.0e00
8.66c+1
20.00+40
3.66c+1
10.0000
8.66-=1
20.00e40
8.66441
10.0000
8,66c-1
20 0leee
8.66e+1
14.55p-11
1.29c=
20,00a0
6.6641
10.00++0
8.66++
/23
#3.86e-1
485c=0
1
4.87e+0
#3.876-1
4.90c+0
=4.32e-1
4.30e00
#2.276-1
4.85c40
26.07-11
7.1e+00
#344e-1
4.86--0
26.00-1
4 1.66+40
12.96-11
2.1le=3
#3.51e-1
4 4.80+0
14.1ee-1
4.6l+00
S24
#3.0ee44
9.1ee45
13666444
4.3e445
21.36e44
2.03e+4
12.12
1.58e45
#1 1..53
3.36+4
#7.02641
1.2ee4
11.8443
9,89e-3
#1 64044
366644
23.334
2.26c45
11.9044
2.35445
1..56441
1.5e+3
-/olt
14/20
14/200
14/20
13/2/1
1420
9/2/5
13/2/1
13/2/1
158V1
12/2/2
24

---

Published as a conference paper at ICLR 2026
1.0
GLHF
GLEET
RUDEAFL
LES
0.8
ABOM
CIMESS
Normalized
0.6
PSO
JDF21
Random_ search
2580
5000
7500
10000
12500
15600
17500
20000
Function Evaluations (FEx)
1.0
GLHF
RLDEAFL
0.8
ABOM
SAHLPSO
Nomaltzed
0.6
DE
Random, search
Evaluations
(b) 100
GLHF
GLEET
LES
0.8
ABOM
CIMESS
SAHLPSO
Nomalized
0.6
DE
JDE21
Random, search
0.4
0.2
2500
5000
7500
10000
12500
15000
17500
20000
Function Evaluations
(0) 500
Figure 10: Convergence curves of the average normalized cost across all test functions in the BBOB over 30 independent runs. The cost values are min-max normalized suite, with d comparability. Each subplot displays the performance trend, highlighting the function to ensure algorithm's ' scalability as the dimensionality increases.
25

---

Published as a conference paper at ICLR 2026
Galf
QJF
GUEAT
11.4
ADCH
CMAES
ONKES
SARUISC
21.0
EEE
Banidaon seard
30.00
16.1
36.25
Fis
FEs
(a) Terrain 2
(h) Terrain 4
GJIf
LES
RLDEATL
11.
LES
FLEEKLL
ADOM
13.0
21.2
SARUSSO
DATILSSO
PSO
5 16.4
FSC
Konvier Coarch
Rewdinn seach
2622
20.2
36.2
13:00
2000
2300
(c)
Terrain
GUIF
GLIB
GUEET
GLEST
ADCH
ADOM
CHAES
CHANES
220.0
DE
0.3
0.3
5.2
0.2
11x
(e) Terrain 10
(f) Terrain 12
GLIf
13.75
31.5
FLEEKL
13.50
LES ADOM
AZON
13.25
EAMIESS
33.00
PSD
16.75
Rondinm seach
16.50
10.23
5.5
382
2010
1300
2002
2000
100
1000
15000
2000
3500
(g)
(b)
Figure 11: Convergence curves of cost (log scale) for UAV problems (Terrain 2 to 16).
26

---

Published as a conference paper at ICLR 2026
13.4
GLIE
GuAAT
GLSET
ALDGNE
13.5
ABCH
OMAES
RBOM
CHAEE
10.3
IPE23
220.2
17EZI
DE
26.4
20.2
72x
FEa
(a) Terrain 18
(b) Terrain 20
GLI
13.75
QLEET
KLDEALL
13.50
LES
ASON
13.25
SAHEPSO
$3.00
31.0
PSO
Rander
30.0
Rondonn Seachh
30.5
20.23
36.4
16.00
100
3300
11000
2032
740
110
11003
1100
2000 7500
(c) Terrain 22
(d) Terrain 24
13.75
CLEET
13.50
11.50
AGOM
13.25
23.00
PSD
11.40
CLEET
26.75
16.50
CHAES
16.50
16.25
30.00
10021
162
3300
1500
2002
2500
500
1000
15:90
2000
(e)
26
(l)
GUSET
31.4
GLSET
11.2
ABCH
RBOM
CMALS
CHANES
33.0
JOEZL
Runddor Jearc
20.4
363
20.2
Fix
Fix
(g) Terrain 30
(b) Terrain 32
13.0
12.00
0UP
GLEET
BLDDAYL
LES
31.75
11.50
AROM
SAHUSO
PSO
PSD
23.0
$1.00
16.3
Rander SoaCe
Banidar.ssaca
30.25
20.4
36.0
20.25
572
1000
13:00
2000
200
506
3200
15:30
2000
2003
Terrain
(i) Terrain
Figure 12: Convergence curves of cost (log scale) for UAV problems (Terrain 18 to 36).
27

---

Published as a conference paper at ICLR 2026
13.75
CLEET
OJP
QLEET
13.50
11.50
AGOM
(SOM
13.29
11.25
11.00
20.75
10.50
30.00
30.25
566
2300
1500
2002
2500
500
1000
1250
2000
(a) Terrain 38
(b) Terrain 40
GLSET
KLDEAE
ABON
CHANES
REDEAR
LES
30.3
JOEZI
263
Funddrr search
SAH PSD
9.5
9.2
kandem, srarhh
Fix
(c) Terrain 42
(d) Terrain 44
11.5
LG5
SAHUSO
PSD
ELH
FLEET
233.
SOAEE
26.4
578
1000
1300
2000
2300
(e) 46
(l) Terrain
GUSET
ABCH
13.0
CMAES
RLDEAL
SARIPID
9.5
5.7
Fix
FEx
(g) Terrain 50
(b) Terrain 52
GLVE
31.5
LES
10.5
DL
IREET
FSC
SAMEPSO
280
ARTY
OHEES
36.5
Bunderr seanch
3.3
572
1000
13:00
2000
2500
15003
2500
Terrain
(i) Terrain
Figure 13: Convergence curves of cost (log scale) for UAV problems (Terrain 38 to 56).
28

---

Published as a conference paper at ICLR 2026
1000
(a) Terrain 2
Terrain
1000
(c) Terrain 6
Terrain
(e)
(f) Terrain
(g) Terrain
(b) Terrain
1000
700m
Terrain
(i Terrain
Figure 14: Boxplots of cost (log scale) over 30 runs for UAV problems (Terrain 20).
29

---

Published as a conference paper at ICLR 2026
2000
(a) Terruin 22
(b) Terrain 24
1800
(c)
(d)
(e)
(i) Terrain
(g) Terrain 34
(h) Terrain
1900
Terruin 38
Terrain
Figure 15: Boxplots of cost (log scale) over 30 runs for UAV problems (Terrain 22 to 40).
30

---

Published as a conference paper at ICLR 2026
(a) Terrain 42
(b) Terrain 44
(c)
(d)
Terrain 50
(D Terrain
3400
Terrain 54
(b) Terrain 56
Figure 16: Boxplots of (log scale) over 30 runs for UAV problems (Terrain 42 to 56).
31


Reviews:
[{"review_id": "0uEKJild3C", "reviewer": "Reviewer_dnfT", "summary": "This paper introduces a novel learning-assisted approach for optimization. The authors propose ABOM framework to address the time-consuming training in existing meta-black-box optimization approach. ABOM inherits the concept of  learnable evolutionary operators from previous GLHF framework, where the iterative reproduction & selection operations in an evolutionary algorithm are abstracted as s group of neural networks, and hence bridge efficient gradient decent with learning for optimization. The most important component in ABOM is its online parameter adaption mechanism, where the up-to-date elite solutions are recorded along the optimziation progress, and at each step, the solution population output by the neural network-based evolutionary operators is self-supervised by minimizing the distance with elite solutions at hand. Through this way, ABOM could adapts itself to novel optimization tasks during the online evolution process, without tailored pre-training, The authors provide an intuitive theoretical proof on the convergence of ABOM and validated ABOM's performance on both synthetic and realistic testbeds. The results show that online adaption could deliver comparable performance to existing MetaBBO approaches, while mitigating the training burden.", "questions": "See Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I appreciate the novelty of this paper. Since existing MetaBBO approaches require a pre-defined problem distribution and corresponding pre-training to make the meta-level policy generalizable, the idea in this paper (online adaption through self-supervision) enlights more efficient MetaBBOs.", "weaknesses": "1. Theoretical perspective: I have say that though the authors provide a intuitive proof on what they have claimed (Collary 1, 2, Theorem 3.1), a strong assumption (may not happen in real optimization problem) makes me suspect the rationale behind the proof. This assumption is: \"the elite solution is ϵ-suboptimal\". I wonder how to guarantee such assumption since we face randomized (stocastic) optimization here. This question becomes more obvious when we consider Eq. (10), what if the elite information prematures? What is exact contribution of ABOM to promote the evolution stepping out such premature? Is the crucial component the Dropout in mutation and crossover neural network? Is this indicating that ABOM is actually a random search with adaptive local search?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CgjQYBMrjE", "reviewer": "Reviewer_RRGJ", "summary": "This paper introduces ABOM, By eliminating the dependency on handcrafted training tasks, ABOM performs adaptive parameter learning using only optimization data from the target task itself. This enables zero-shot optimization, where ABOM adapts evolutionary operators during optimization to improve performance dynamically. The paper evaluates ABOM on both synthetic benchmarks (BBOB) and a real-world unmanned aerial vehicle (UAV) path planning problem, demonstrating competitive performance across multiple scenarios. The results are promising, showing that ABOM outperforms existing baseline methods.", "questions": "Refer to the above weakness part.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Sound Method Design: The parameterization of evolutionary operators (selection, crossover, mutation) into differentiable, attention-driven modules is well-designed. The online update of optimizer parameters is achieved by using population data generated during the target problem's runtime, with the objective of minimizing the distance between offspring and the elite archive  as the loss function.\n2. Theoretical Guarantee (to a degree): The paper provides a convergence proof under idealized assumptions.\n3. The experimental evaluation is thorough, and the results are positive.", "weaknesses": "1. The paper needs to explain the differences between the proposed method and other, such as GLHF and B2OPT, in detail, especially regarding the parameterization of evolutionary operators. If prior methods were adopted, proper citations are required.\n2. The loss function $\\min _\\theta||\\hat{P}^{(t)}-E^{(t)}||_2$ encourages the offspring population $\\hat{P}^{(t)}$ to be close to the current elite archive $E^{(t)}$, essentially using the current local optima to guide optimizer updates. This is a form of \"bootstrapping,\" which introduces search bias and may theoretically lead to getting trapped in local optima.\n3. While the authors attempted a theoretical proof of global convergence, it is presented in an overly absolute manner. The resolution of the aforementioned bias relies on persistent exploration via Dropout. However, this theory has a problem: its global convergence is inherently \"unbounded,\" meaning there's no guarantee of achieving convergence within a bounded number of iterations. To illustrate, even a completely random algorithm, when combined with a selection operator, can be proven to converge globally. Therefore, the significance of this convergence proof is limited.\n4. Furthermore, if the global optimum lies on the boundary of the search space, or if constraint handling cuts the feasible region into discontinuous parts, the convergence proof will not hold. In summary, attempting to prove convergence is commendable, but the conclusions should not be presented too absolutely or rashly. Further refinement and discussion are needed.\n5. This paper effectively performs On-Instance Training. Although there's a meta-behavior of \"learning optimization strategies,\" it lacks cross-task generalization, which deviates from the traditional definition of meta-learning. This needs clarification in the introduction.\n6. The limitations of On-Instance Training need to be discussed. For example, a lack of adaptation to cold-start scenarios with limited evaluation counts, and the inability to transfer knowledge between tasks (each task requires independent optimization, unable to leverage similar problems for acceleration).\n7. A very detailed parameter sensitivity analysis is needed for On-Instance Training. It's recommended to include the learning rate and dropout rate, as they determine the exploration & exploitation tradeoff and are critical to the proposed method.\n8. The experimental section lacks in-depth analysis of results. While the performance of ABOM is compared to several baselines, there is limited interpretation of the underlying reasons for ABOM's success.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hwvfdn0pzj", "reviewer": "Reviewer_kZrh", "summary": "This paper proposes an end-to-end optimization model, ABOM, which employs attention mechanisms to perfrom parent selection, crossover and mutation operators on the population to generate offspring. A loss function based on the distance between offspring and the elite archive is introduced to guide the model to improve the offspring. Experimental results validate ABOM's advantage over traditional BBO and MetaBBO methods.", "questions": "1. In Figure 4, the selection often focuses on very few individuals (sometimes only one). Does this imply the model is performing a greedy local search around a limited set of points?\n\n2. Are the population sizes and hidden dimensions sensitive to the problem dimension? Figure 5 (a) & (b) indicate that different functions have different optimal settings (e.g., f4 prefers a larger population while f24 prefers a smaller one). Is it feasible to adapt population sizes and hidden dimensions dynamically to improve performance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Unlike existing MetaBBO methods based on deep neural networks, ABOM does not require pre-training, saving substantial time and computational resources. Additionally, the model takes the population itself as input and produces offspring in an end-to-end manner, reducing the human effort required to design optimization states and model actions. Although this end-to-end approach is also adopted in other MetaBBO methods such as RNN-OI and GLHF, ABOM's loss function avoids requiring the gradient of the target problem, which appears novel.\n\n2. The end-to-end neural computation enables efficient batched execution on GPUs, as validated in experiments, but it also imposes higher hardware requirements, especially for high-dimensional problems.\n\n3. ABOM demonstrates superior performance compared to BBO and MetaBBO baselines on both BBOB and UAV path planning tasks, confirming its effectiveness.", "weaknesses": "1. The loss function is calculated as the distance between offspring and the elite archive. Given that the search range and problem dimensionality can be large (e.g., [-100, 100] range and 500 dimensions for BBOB in this paper), the scale of the loss and gradients could be large, potentially causing unstable training. Furthermore, the loss function encourages the model to generate offspring that completely surpass the parent population, representing a greedy strategy that may reduce exploration.\n\n2. For each problem instance, model parameters are initialized and updated from scratch. While this enables task-free adaptation, it disregards experience and knowledge gained from optimizing previous problems. Leveraging learned knowledge to enhance current optimization is a key advantage of MetaBBO.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vng04fhMYs", "reviewer": "Reviewer_tkpd", "summary": "This paper proposes ABOM, a black-box optimization algorithm which adaptively adjust parameters based on experience from function evaluations. ABOM parameterizes the selection, crossover and mutation module in evolutionary algorithm via MLP with attention mechanism, and train these parameters by approximating to the elite selection procedure. The authors provide the global convergence guarantee of ABOM. The experimental results over both synthetic and UAV benchmarks demonstrate that ABOM matches or exceeds commonly used BBO baselines.", "questions": "1. According to Algorithm 1, is there only one gradient update in each sampling iteration?\n\n2. How does ABOM handle the boundary issue, since MLP parameterized policy may generate samples outside the input bound.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed MLP parameterized algorithm can benefit from GPU parallel computation.\n\n2. ABOM demonstrates superior empirical optimization performances on both synthetic and UAV benchmarks.", "weaknesses": "1. The theoretical and empirical algorithm analysis does not explain the reason of using attention-based MLP to parameterize the evolutionary modules. From the algorithm design side, the \"optimal\" policy corresponds to a elite-selection style EA algorithm. In the theoretical analysis, the main results are derived by setting all parameters except the last layer's bias to 0. The global convergence guarantee is not finite-time, which seems to hold simply because of the introduced randomness from dropout. I think other EA based on elite selection might also fit to this infinite-time convergence guarantee. \n\n2. The proposed ABOM also introduces additional hyperparameters such as dropout rate and hidden dimensions, which is sensitive according to figure 5 and non-trivial to tune like other BBO algorithms such as CMA-ES.\n\n3. The target problem of the proposed method should be single-task black-box optimization problem, where algorithm sample strategies and parameters are usually adjusted based on the sampled data (e.g., MLE-based parameter fitting in Bayesian optimization, and step-size adjustment in CMA-ES). Therefore, I think meta BBO problem and methods are not much related to the main research topic of this paper.\n\n4. I think adding more recent EA-based or NN-based BBO baselines such as [1, 2] and more challenging high-dimensional benchmarks used in these works can better assess the algorithm performance.\n\n5. (minor) The presentation can be improved to be clearer. (1) The citation format should be \\citep or \\citet instead of \\cite to compatible with ICLR templates. (2) The population size and elite number both use notation $N$.\n\n[1] Zhang, Zhengfei, Yunyue Wei, and Yanan Sui. \"An invariant information geometric method for high-dimensional online optimization.\" 6th Annual Learning for Dynamics & Control Conference. PMLR, 2024.\n\n[2] Yun, Taeyoung, et al. \"Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization.\" Forty-second International Conference on Machine Learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)