Optformer: Beyond Transformer for Black-box Optimization
Authors: Xiaobin Li
Kai Wu
Xiaoyu Zhang
Handing Wang
Jing Liu
Venue: Submitted to ICLR 2023 (2023)
Keywords: Transformer
Black-box optimization

Abstract: We design a novel Transformer for continuous unconstrained black-box optimization, called Optformer. Inspired by the similarity between Vision Transformer and evolutionary algorithms (EAs), we modify Tansformer's multi-head self-attention layer, feed-forward network, and residual connection to implement the functions of crossover, mutation, and selection operators. Moreover, we devise an iterated mode to generate and survive potential solutions like EAs. Optformer establishes a mapping from the random population to the optimal population. Compared to baselines, such as EAs, Bayesian optimization, and the learning-to-optimize method, Optformer shows the top performance in six black-box functions and one real-world application. We also find that untrained Optformer can also achieve good performance.

Under review as a conference paper at ICLR 2023
BOPTFORMER: BEYOND TRANSFORMER FOR BLACK- BOX OPTIMIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
We design novel Transformer for continuous unconstrained black-box optimiza- tion, called BOptformer. Inspired by the similarity between Vision Transformer and evolutionary algorithms (EAs), we modify Tansformer's multi-head self-attention layer, feed-forward network, and residual connection to implement the functions of crossover, mutation, and selection operators. Moreover, we devise an iterated mode to generate and survive potential solutions like EAs. BOptformer learns the optimization strategies from the target task automatically without human interven- tion, which addresses the poor generalization of human-designe EAs when given a new task. Compared to baselines, such as EAs, Bayesian optimization, and the (L20) method, BOptformer shows the top performance in six black-box functions and two real-world applications. We also find that untrained BOptformer achieve good performance
the simple tasks. Deep BOptformer performs better than shallow ones. We bring a new and efficient Transformer-bart black-box optimization framework for the L20 and EA communities.
INTRODUCTION
Many tasks, such as neural architecture search (Elsken et al. 2019) and hyperparameter optimization (Hutter et al. 2019 Golovin et al., 2017), can be abstracted as black-box optimization problems, which means that although we can evaluate fxx for any x E X, we have no access to any other information about f. such as the Hessian and gradients. A series of desi-dd algorithms, such as evolutionary algorithms (EAs) (Mitchell 1998 Khadka & Tumer 2018 Zhang & Li, (007). Bayesian optimization (Snoek et al. 2012, Mutny & Krausel 2018: [.i et al. [007] Kandasamy
et al. 2015 Balandat et 2020), and evolutionary strategies (ES) Wierstra al.) 2014] Hansen
Ostermeier 2001: |Auger& & Hansen 2005 Salimans et al. 2017). have been designed to solve black-box optimization.
Recently, the learning to optimize (L20) framework (Chen et al. 2022) gives an new insight on optimization by leveraging the recurrent neural network (RNN), long short terrtterrmme memory architecture
(LSTM) (Chen et al.) 2020 Andrychhwicc et al.] 2016; Chen et al.] 2017 Li & Malik, 2016
Wichrowska et [ 2017] Bello et al.. 2017) multilayer perceptron (MIP) (Metz al. (019)) as the optimizer to develop optimization methods, aiming at reducing the laborious iterations of hand engineering (Sun al. 2018 Vicol et al. 2021; Flennerhag al. 2021 Li & Malik 2016 Sun ct al. 2018). They don't concentrate on issues with black-box optimization. The core of L20 is constructing strong mapping from the initial solutions to the optimal solution. Although several efforts like (Cao et al., 2019: Chen et al.] 2017) have coped with the black-box problems, their effectiveness may hindered by the limited representational capabilities of RNN, LSTM, and MLP.
In EAs, the hand-designed crossover, mutation, and selection operators make the initial population move near the optimal solution. This updated model has stood the test of time. Because the evolutionary operators must be modified to maximize their performance the target task, human- designed EAs have a low generalization ability to a new black-box problem. Most notably, the limited of target function information in EA design due expert knowledge limitations makes it difficult to adapt to the target task. Learning the optimization strategies from the taget task is the key step to overcome this limitation.
This paper designs a novel L20 framework based on the advantages of Vision Transformer (Doso vitskly al. 2021) and EAs to overcome the above limitations, termed BOptformer. Moreover,

---

Under review as a conference paper at ICLR 2023
Transformer (Han et al. 2022 owns strong representation ability, and there is currently no work to use Transformer for optimization. Inspired by the similarity of EAs and Transformer (Zhang et al.) 2021 2022), BOptformer revised the critical part of Transformer realize the mapping from the ran- dom and optimal populations. To generate potential individuals to approach the optimal solution, we first design self-attention (SA)-hased crossover module (SAC) to simulate the crossover operator of EA, and then the output of this module is input into the proposed feed-forward network (FFN)-based mutation module (FM) to perform mutation. Moreover, the residual and selection module (RSSM) is designed to survive the fittest individuals. RSSM is a pairwise comparison between the output of SAC, FM, and the input population regarding their fitness. We design an BOptformer Block (OB) consisting of SAC, FM, and RSSM. Finally, we construct BOptformer by stacking OBs to simulate generations of EAs.
Moreover, to cope with black-box optimization, we establish function set to train BOptformer under an unsupervised mode. We construct a set of differentiable functions with similar properties to the targeted black-box optimization problems. This training set contains the pair of the initial population and the designed function. Thus, we can use gradient-based methods to train BOptformer.
We tested BOptformer on six standard functions, the protein docking (Cao & Shen 2020) problem, and the planar mechanic arm problem (Wang et al. 202Ip. The experimental results demonstrate the top rank of BOptformer and the strong representation compared with three population- based baselines, Bayesian optimization, and one learning-to-ppe to-opiimize method (Cao et al. 2019). Moreover, also analyze the effect of learning rate, deep structure, and weight sharing between OBs. The highlights of this paper are summarized as follows:
1) We propose a solid Transformerr-aaeed L20 framework addressing black-box problems to the L20 community. We have demonstrated its benefit when compared with standard black-box optimization methods, particularly for the L20-based method.
2) BOptformer efficiently uses the target black- bock-box function's 's information to aid in the development of the optimization strategy. Compared to the human-designed EA, BOptformer has substantially greater degree of task fit.
2 RELATED WORK
Transformer Transformer structure achieves significant progress for machine translation task (Vaswani et al. 2017), computer vision task (Dosovitskiy et al. 202T), time series task (Zhou et al. 2021), and on. Many improved models are proposed and obtain great achievements (Han et al. 2022). There are no Transformer-bas
efforts for handling optimization problems, which is crucial in the machine learning community. (Vaswani et al. 2017) proposed the meta-learning hyperparameter optimization framework with Transformers to learn both policy and function priors from data across different search spaces. However, the BOptformer proposed this paper expands the application scope of Transformer and can effectively deal with this case. The basic modules of Transformer are shown in Appendix A.I.
Evolutionary Algorithm Inspired by the evolution of species, EAs have provided surprising performance for black-box optimization (Mitchell, 1998). The basic modules of EAs are shown in Appendix A.2. Many influential variants have been proposed to deal with different problems (Das) & Suganthan| 2010 Wu & Liu 2019), but at their core they are: 1) recombination and mutation, how to produce the excellent solution; 2) selection, how to choose the best individuals between the parents and offspring. Thus, many algorithmic components have been designed for different tasks. The performance of algorithms varies towards various tasks, as different optimization strategies may be required given diverse landscapes. Current methods manually adjust genetic operators' hyperparameters and design the combination between them (Kerschke et al., 2019 Tian al., 2020) to map the random population to the optimal solution. We require an expert to design or choose the evolutionary operations when given a new black-box optimization task to maximize its performance on the target task, which negatively impacts generalization ability. Most notably, the limited use of target function information in EA design due to expert knowledge limitations makes it difficult to adapt to the target task. The suggested BOptformer uses Transformer framework instead of the manually designed crossover, mutation, and selection operators. The genetic operator is then designed automatically by the built Transformer rather than by human designer. BOptformer efficiently
2

---

Under review as a conference paper at ICLR 2023
uses the target black-box function's 's information to aid in developing the optimization strategy. In comparison to the human-designed EA, BOptformer has a substantially greater degree of task fit.
BOPTFORMER
3.1 PROBLEM DEFINITION
A black-box optimization problem can be transformed as minimization problem, shown in Equation (T. and constraints may exist for corresponding solutions:
min f(z), s.t. 2; [4 u.]
where a = (2,, Id) represents the solution of optimization problem f. the lower and upper bounds
(11, 11.12..... ,'a) and u (n1, U2. ,dd, and d the dimension (12,1. ,X1,)) =
of æ. Suppose n individuals of one population be X1 (X2,1, X2,,, X2,a). X, = (X,,1, Xn.2, ,,,a). then BOptformer are required to find the population near the optimal solution á. We suppose that X is the initial population and X is the output population.
3.2 SELF-ATTENTION CROSSOVER MODULE
Similar to the crossover operator in EAs, we propose new module based on SA to generate the potential solutions by maximizing information interaction among individuals in population. The crossover operator generates a new individual by Σi= X;W; (Zhang et al. 2021). W is the diagonal matrix. If W is full of zeros, the ith individual has contribution. Suppose population X is arranged a non-descending order fitness, and F R this module be represented follows:
x be the fitness matrix of X. Then,
X = SAC(X,
where X the output population of the proposed SAC module.
Since the object processed by BOptformer is the population, and the order of individuals in the population does affect the population distribution, SA does not require position coding. Standard SA projects the input sequence X into a d-dimensional space via the queries (Q), keys (K), and values (V). These three mappings enable the SA module to capture better the characteristics of the problems encountered during training. In other words, these three mappings strengthen the ability of SA to focus on specific problems but do not necessarily make SA have good transferability between different problems. Therefore, we consider removing these three mappings for enhanced transferability, and X AX. A |R"x" is self-attention matrix that can learned to maximize inter-in inter-indiviee individual information interaction based individual ranking information. This is why the population needs to be sorted in nonddescn order.
However, designing crossover operations based solely population ranking information is coarse- grained approach. Because this method only considers the location information of individuals the population, but does not consider the fitness relationship between individuals. Therefore, we further introduce fitness information to assist in learning crossover operators:
A = SA(F) = Softmaz (FW (FW*KFWWK)) / /sqrt(dk))
Thus, X AX + AF X.
To better balance the roles of A and AF, we introduce two learnable weights W x and W E 1 Therefore, the final crossover operation is shown as follows:
X tile Wf) @ (AX) + tile W (AFX) X)
where X E Ra is the population obtained by X through the SAC module; @ represents Hadamard product; the tile copy function extends the vector matrix.
3.3 FFN-BASED MUTATION MODULE
The mutation operator brings random changes into the population. Specifically, individual X; in the population goes through the mutation operator to form the new individual X,, formulated
3

---

Under review as a conference paper at ICLR 2023
Input Population X
Output Population X
OB;
Populatio Sotting
Matrik
X
RSSM
Ordand Pepalationn
Nx
Figure 1: Overall architecture of BOptformer and OB. Nx stands for BOptformer composed of Ne stacked OBs. These OBs can be set to share weights with each other or not share weights with each other.
X; W. W the diagonal matrix. In Transformer, each patch embedding carries on directional feature transformation through the FFN module. We take one linear layer as an example: each embedding
X XW where W is the weight of the linear layer, and it is applied separately and identically. This equation and the mutation operator have the same formula format, which inspires us to design learnable mutation module FM based on FFN with ReLU activation function:
X (M = (ReLU W + W. + b2
where X is the population after the mutation of X W.P and W represent the weight of the second layer of FFN and the weight of the first layer of FFN, respectively. b2 and b₁ represent the bias of the second layer and the first layer of FFN, respectively.
3.4 SELECTION MODULE
The residual connection the transformer can analogized to the selection operation in EA (Zhang et al. 2021). We combine the residual structure and selection module (SM) (Anonymous, 2023 to design a learnable selection module RSSM. The RSSM generates the offspring population according the following equation:
X RRSS X X")
(5)
Sort((SM tile(Wi)) & X tile(W?)) @ X tile (W3) O X)))
where X is the fittest population for the next generation; the learnable weights Wi Rex1 W RTx1, and Wj gax1 are the weights for X, X and X",, respectively. Sort(X) represents that X sorted in non-descending order of fitness. We use quicksort sort the population. These three learnable weight matrices realize the weighted summation of residual connections, thereby simulating a learnable selection strategy. Meanwhile, the introduction of residual structure also enhances the model's representation ability, enabling BOptformer to form a deep architecture.
SM updates individuals based on a pairwise comparison between the ofispring and input population regarding their fitness. Suppose that X and X' are the input populations SM. We compare the quality of individuals from and X' pairwise based on fitness. A binary mask matrix indicating the selected individual can be obtained based on the indicator function 4z>0 (z). where (x>0(())) 1 if
and (z) 0 if
SM forms new population X by employing Equation 66.
X = tile(Iasal (M MF)) @ X + tále(1 - l2>0(Mrr MM)) O X
where the tile copy function extends the indication vector a matrix, (F(MF) denotes the fitness matrix of ((X')
3.5 STRUCTURE OF BOPTFORMER
BOptformer comprises basic BOptformer blocks (OBs), and parameters be shared among these t OBs or not. The overall architecture of BOptformer and OB is shown in Figure Each OB consists of SAC, FM, and RSSM. X Raxd represents the initial population input into BOptformer, which
4

---

Under review as a conference paper at ICLR 2023
needs to be sorted in non-descending order of fitness. In Equation 7 x is fed into OB, to get X where E [1. fl. BOptformer realizes the mapping from the random initial population to the target
population by stacking OBs.
X = OB(X X = SAC(X
X FM(X);; X = RSSM(X Xx"
3.6 TRAINING OF BOPTFORMER
Training Dataset Before introducing the details of the training dataset, fidelity (Kandasamy et al. 2016) is defined as follows: Suppose the differentiable surrogate functions fi. ... fm the continuous exact approximations of the black-box function f. We call these approximations fidelity, which satisfies the following conditions: 1) fi.... Sú, fm approximate |l filloo I Sm- where the fidelity bound Çi > S2 > Sm- 2) Estimating approximation fü is cheaper than estimating f. Suppose the query cost at fidelity is A, and λ X2 ....
Training data is crucial factor beyond the objective functions. This paper establishes the training set by constructing dataset only contains (Xo. fa (z)))) the initial population and objective function, respectively. The variance of w causes the shift in landscapes. The training dataset is designed follows: 1) Randomly initialize the input population Xo; 2) Randomly produce a shifted objective function fa(w)) by adjusting the parameter w; 3) Evaluate Xo by fz (w); 4) Repeat Steps 1)-3) generate the corresponding dataset. We show the designed training and testing datasets as follows:
of differentiable functions related to the optimization objective. This training
ptrain
train); ".. J' ( (ml (Si (ar|w,
where Aroin represents the ith different values of w in math function fm-
Loss Function BOptformer attempts search for individuals with high quality based on the available information. The loss function tells how to obtain the parameters of BOptformer to generate individuals closer to the optimal solution by maximizing the difference between the initial population and the output population of BOptformer. The following loss function is employed (Anonymous: 2023
Σ f((x)w) Ex(xO)
Σ
f((x|w)
X
4(X0, f(=lw)
xxE,
Σ x
fa(l)w)
X
where denotes parameters of BOptformer (E). Equation (9) calculates the average fitness difference
between the input and output, further normalized within [0. To encourage BOptformer to explore the fitness landscape, for example, the constructed Bayesian posterior distribution over the global optimum (Cao & Shen 2020) can be added to Equation (9). Since the derivatives of functions in the training dataset are available, we can obtain the gradient information of Equation (9) for the training process. Also, we can employ REINFORCE (Williams) 1992) to approximate these derivatives.
Training BOptformer We then train BOptformer under a supervised mode. Since the gradient unnecessary during the test process, BOptformer solve black-box optimization problems. To prepare BOptformer to learn balanced performance upon different optimization problems, we design loss function formulated follows:
la
Σ 4(X", f((alw X
(10)
We employ Adam (Kinmma Ba 2014) method with a minibatch Q train BOptformer upon the constructed training dataset.
The goal of the training algorithm is to search for parameters 0* the BOptformer. Before training starts, BOptformer is randomly initialized to get initial parameters A. Then the algorithm will perform the following three steps in a loop until the training termination condition is satisfied: Step randomly initialize minibatch Ω comprised of K populations X
Detailed Training Process
5

---

Under review as a conference paper at ICLR 2023
Step 2, for each f. Ftrain, given training data (X", fr), update 0 by minimizing the Ini Step 3, given X update by minimizing -1/m E, lg. where m is the number of functions in ptrain After completing the training process, the algorithm will output 8*.
4 EXPERIMENTS
4.1
EXPERIMENTAL SETUP
4.1.1 DATASETS
Synthetic Functions This paper first employs nine commonly used functions to show the effective- ness of the proposed BOptformer. The characteristics of these nine functions are shown Tables and (Appendix). Here, BOptformer is trained on ptrain is generated based on functions in Table 6 and the target functions are shown Table 7 (Appendix). Here, {10, 100).
Protein Docking We also handle the problem of Ab initio protein docking (Cao & Shen 2020),
which optimizes noisy and costly function in a high-dimensional conformational space. Mathemati- cally, this problem is formulated as optimizing the Gibbs binding free energy f(x) for conformation a. We calculate the energy function in a CHARMM 19 force field as in (Moal & Bates, 2010) and shift it that fa) 0 the origin of the search space. f(x) is differentiable when parameterize the search space as R 12 (Smith & Sterbberg, 2002). Here, only 100 interface atoms are considered. The details of this problem can be found in Appendix A.8.
We further evaluate the performance of the proposed scheme on the planner mechanic problem, which has been widely used to evaluate the performance of the black-box optimization algorithms (Cully et al. [2015] Vassiliades et al. 20018) Vassiliades & Mouret 2018 Mouret & Maguire| 2020). The optimization goal of this problem is to minimize the distance from the top of the mechanic arm to the target position by optimizing a set of lengths angles. The detailed problem can be found in Appendix A.5. represents the distance from the target point the origin of the mechanic arm, as shown in Fig. (Appendix).
Planner Mechanic Arm
Table I: The compared results on six functions.
f
DE
ES
CMA-ES
L20-swarm
Dragonfly
BOptformer
F4 0.13(0.06)
0.22(0.30)
4.2e-4(3.5e-4)
16.92(2.10)
1.3e3(1.3e3)
1.2e-4(5e-5)
FS
(.99().24)
0.55(0.37)
0.03(0.01)
2.97(0.01)
48.4(9.58)
8e-3(2e-3)
F6
210.2(49.7)
60.02(48.28)
61.90(96.25)
26.83(21.48)
3.8e8(1.4e8)
8.93(0.03)
10
F7
17.83(3.59)
51.53(8.05)
45.74(17.02)
4.88(3.55)
81.1(24.0)
0.01(0.03)
F8
0.21(0.07)
0.26(0.18)
7.6e-3(0.01)
1.02((14e-3)
35.4(22.6)
le-5(2e-5)
F9
1.90(0.3))
20.56(0.03)
0.04(0.02)
9.06(0.67)
16.2(3.64)
0.01(3.4e-3)
F4 8.283(3.8e2)
8.9e4(9.5e3)
7.8e3(..233)
0.32(0.03)
11200(3750)
0.11(0.09)
F5
28.2(0.61)
10.
78.3(9.18)
0.28(1.3e-3)
50(0)
0.14(0.15)
F6
2.48((2.3e7)
2.5e10(4.5e9)
3.3e8(8.777)
692(108)
99(0)
129(346)
100
F7
9410K548)
8.9c4(
8050(775)
85.7(18.6)
144(13.1)
24.1(13)
F8
3.04(0. 14)
23.0(3.00)
3(0.22)
0.16(2.4c-4)
125(11.3)
0.02(0.03)
P9
14)
21.4(0.02)
21.4(0.04)
2.59()))
10.5(0.32)
0.15(0.05)
4.1.2 BASELINES
BOptformer is compared with standard EA baselines, such as DE(DE/rand///bin) (Das & Suganthan 2010), ()-(E)))EES and CMA-ES, where DE and ES are implemented based on geatpy (Jazzbin 2020 and CMA-ES is implemented by pymoo Blank & Deb 2020). L20-swarm (Cao al. 2019) is a representative L20 method for black box optimization. Moreover, Dragonfly (Kandasamy et al. 2020). representative algorithm for Bayesian optimization, is employed as reference. We design three BOptformer models, including 3 OBs with WS (weight sharing), OBs without WS, and 30 OBs with WS. The parameters of these methods are shown in Appendix A.4.

---

Under review as a conference paper at ICLR 2023
4.2 RESULTS
Synthetic Functions The results on six functions are provided in [ BOptformer outperforms three EA baselines, Dragonfly, and L20-swarm in all cases, but loses once to Dragonfly in F6 with = 100. These cases also show the excellent generalization ability of BOptformer on more tasks unseen during the training stage. We think the transferability of BOptformer is proportional to the fitness landscape similarity between the training set and the problem. Although new problem attributes are not available in the training set, BOptformer can still perform better. However, this conclusion only holds when the similarity between the problem and training dataset is high. We plot the convergence curves of BOptformer (10 OBs with WS), ES, DE, and CMA-ES on F7 (see Appendix A.7, Figure 5). BOptformer converges quickly and can obtain better solutions. BOptformer can only iterate ten times to get the best solution relative to EA baselines. ES and DE converge around 100 generations, and CMA-ES shows slow convergence rate.
Protein Docking We also test the performance of BOptformer the problem of protein docking.
Table The results on the problem of protein docking.
Methods
IATN_7
2JEL_1
7CEI_1
The experimental results is shown L.20swarm
2090(25.08)
2765(24.80)
1689(23.64)
in 2
The performance of BOpt- 30OBs with WS
-.03e3((15)
-6.03e3(127)
-5.97e3(125)
former exceeds that of 1.20-swarm.
50Bs without WS
6.01e3(146)
-6.02e3(125)
-.1983(131)
During training, L20-swarm does not converge. At the same time, we find that better solutions exist in the initial population than those found by L20-swarm. However, during testing, L20-swarm lost these good solutions.
Planner Mechanic Arm The detailed experimental results are given in Tables 3 BOptformer selects 5 OBs without WS as the example, which evolves only five generations. Untrained represents the untrained BOptformer. DE, ES, and CMA-ES are tested when the maximum generations set to 100. EA baselines have 100/5 times many function evaluations as BOptformer. However, even in this unfair situation, BOptformer achieves the best results. We have observed that BOptformer can achieve better results with deeper architectures. However, it is currently difficult for us to train deep BOptformer. Moreover, as far as we know, the use of ES to optimize deep models has been studied a lot (Vicol et al. 202T), which will be essential research prospect in the future.
Table 3: The results of planar mechanical arm. Simple Case (SC): searching for different angles with the fixed lengths. Complex Case (CC): searching for different angles and lengths.
DE
ES
CMA-ES
L20-Swarm
BOptformer
Untrained
100
1.2000.64)
10.6(5.58)
1.36(0.35)
40.4(3.89)
0.30(0.18)
243(238)
SC
300
1.38(0.71)
44.9(43.3)
1.38(0.41)
69.5(3.77)
0.48(0.37)
1210(820)
1000
93.8(137)
183(239)
43.7(110)
176(7.20)
26.6(57.4)
5070(2770)
100
0.81(0.47)
8.95(6.42)
0.76(0.20)
31.9((.78)
0,06(0.05)
243(238)
CC
300
6.15(12.2)
47.8(56.0)
0.87(0.37)
89.1(1.96
0.50(0.79)
1210(820)
1000
232(233)
251(258)
88.4(158)
262(2.99)
25.0(55.8)
5070(2770)
4.3 PARAMETER ANALYSIS
We consider the performance of different BOptformer architectures. Here. d = 10. The experimental re- sults are shown in Table 4 We find that they were sorted from good worst by their performance, and the result is 30 OBs with WS>5 OBs with- out WS>3 OBs with WS. Deep archi-
Table 4: The performance of different BOptformer structures.
f
Untrained
5 OBs
30 OBs
OBs
without WS
with WS
with WS
F4
0.28(0.09)
0.08(0.03)
1.2e-4(5e-5)
8.16(3.44)
F5
0.37(0.05)
0.15(0.03)
0.008(0.002)
1.47(0.40)
F6
45.8(16.9)
15.43(2.34)
8.93(0.03)
1891(1396)
F7
1.08(0.72)
4.43(1.82)
(.01
35.72(8.52)
tectures have better representation ca-
F8
0.69(0.09)
0.06(0.03)
1e-5(2e-5)
0.82(0.10)
pabilities and also lead to better per-
P9
0.85(0.20)
0.29(0.07)
0.01().003)
3.28(1.00)
formance. However, it is challenging to train non-weight sharing BOptformers with more layers due to the difficulty of training deep architectures. Untrained represents that the parameters of 5 OBs without WS are randomly initialized. The results show that 5 OBs without WS outperforms Untrained, which demonstrates the effectiveness of the designed training process.
7

---

Under review as a conference paper at ICLR 2023
Table S: The results of ablation study. d= 10.
f
Not SAC
Not FM
Not RC
Not RSSM
BOptformer
F4
52.7(15.0)
23.3(6.68)
50.5(6.81)
271(346)
3.03(5.82)
F5
5.08(0.98)
2.94(0.41)
3.75(0.09)
(159(
1.02(0.20)
F6
1.25c5 2664)
1.10e4(1.44e)
5.22e4(1 (1.28c))
1.9966(1.21e)))
4.03e3(6.9445)
F7
47.2(5.11)
19.7(4.27)
63,4(8.60)
10,6(8,10)
44.8(8.34
F8
1.18(0.04)
1.19(0.07)
0.87(0.07)
3,28(1.14)
0.60(0.17)
F9
4.49(1.09)
3,42(0.78)
8.36(0.43)
3.56(0.72)
3.03(0.70)
We also find an interesting phenomenon: OBs without WS outperforms OBs with WS in all cases. Our untrained deep architecture, 5 OBs without WS, can achieve good results simple cases, which shows that BOptformer retains the advantages of Transformer architecture and has strong generalization ability. We use the untrained OBs with WS to test on the complex plannar mechanic arm problem and find that it performs poorly.
We train BOptformer the F1-F3 function set with different learning rates (Ir) and then test them on the F4-F9 function set. The experimental results are shown in Table (Appendix A.6). For OBs without WS, setting (r 0.01 achieves the relatively best performance. Using (r 0.0001 would be a good choice for 30 OBs with WS and 3 OBs with WS.
4.4 ABLATION STUDY
This section considers the performance impact of different parts in BOptformer. We take BOptformer with 3 OBs and weight sharing as example, which is trained on Fi-F3 and tested on F4-F9. We remove SAC, FM, RSSM, and RC in BOptformer, respectively, and denote them as Not SAC, Not FM, Not RSSM, and Not RC. The experimental results are shown in Table 5 When their results were sorted from good to worst, the rank is BOptformer >> Not FM >> Not RC z Not SAC z Not RSSM. The role of FM is slightly weaker than that of the other three modules. Taken as whole, the parts of SAC, RSSM, and RC are of equal importance. The absence of these core components can seriously affect the performance of BOptformer. At the same time, also shows the effectiveness of the proposed four modules. The removal of any one of the modules in the crossover, mutation, and selection of EAs will degrade the performance of EAs. This shows that BOptformer implements a learnable EA framework that does not require human-designed parameters.
Crossover Strategy Leamedd by OB 1 Crossover Strategy Learned by 08 2 Crossover Strategy Learned by OB
6:00
individual's strategy
Indiviouals strategy
indivioualss strategy
50th individual's strategy
The individual's strategy
The individua's strategy
The 100th ind Viduals strabegy
The 100 to individulls sitateyy
The 100th indoidaal's sitategy
6:000
63124
60600
0.109
6.0m
Crossover Strategy Learned by OB 4
Cressover Strategy Learned by OB
60600
individual's strulegy
1302
The Lut individual's strutegy
0.008
The s0th individual's stratagy
The S0th individal's strategy
The 100th individualss stratogy
The 100th individuals crobogy
6.0m
2354
0307
5/00
1303
Figure 2: Crossover Strategy learned by BOptformer.
4.5 VISUALIZATION ANALYSIS
The tested model is 5 OBs with WS trained on F1-F3 with d 100. The population size is 100.
Visual Analysis of SAC The crossover strategies learned by the five SAC are shown in Fig. 2 For the presentation, we select individuals with fitness rankings 1st, 50th, and 100th. The horizontal axis
8

---

Under review as a conference paper at ICLR 2023
represents the fitness ranking of individuals, and the vertical axis represents the attention (weight when performing crossover) on these individuals. OB1 tends to crossover with lower-ranked individuals, showing a preference for exploration. From OB1 to OB5, the bias of SAC gradually changes from exploration to exploitation.
Visual Analysis of FM We test 5 OBs with WS on F4 with d = 2. The mutation strategies learned by the five OBs are shown in Fig. 3 Input and output represent the input and output populations of the FM module, respectively. OBI tends to explore broad solution space, and the next OBs gradually shift from searching the vast space to searching the space near the input population. The strategies learned by FM and SAC modules show a common feature: the preference for generating solutions gradually shifts from exploration to exploitation as the population converges.
OB
2
OB
Input
20000
20000
15000
13000
25020
10000
10300
10030
5030
5200
5020
-100
100
-100
-50
-50
-50
50
55
20
1.00
120
120
202
130
OB
input ourput
output
20000
15030
15003
L0000
10300
-100
50
100
100
120
100
Figure 3: Mutation strategy learned by BOptformer.
CONCLUSIONS
We successfully designed the Transformer-based L20 framework for black-box optimization, which does not need hand-designed operators. The better performance than that of EA baselines, Bayesian optimization, and the L20 method demonstrates the effectiveness of BOptformer. Moreover, BOpt- former can be well adapted to unseen black-box optimization. Meanwhile, we experimentally demonstrate that the proposed three modules have positive effects. BOptformer still has room for improvement.
1) Our scheme is not limited to black-box optimization. Similar to the LSTM architecture, our scheme can directly optimize differentiable functions. However, the architecture of BOptformer does not directly involve the gradient information of the optimization target, which makes BOptformer inferior to existing L20 schemes. In future work, we will design information of the optimization target;
module that embeds the gradient
2) In the loss function, did not effectively consider the diversity of the population, and the population can be regularized in the future;
3) The training set seriously affects the performance of BOptformer. If the similarity between the training set and the optimization objective is low, it will cause the performance of BOptformer to degrade drastically. Building the dataset relevant to the target as possible is essential.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Marcin Andrychowiez, Misha Denil, Sergio Gomez, Matthew W Hoffmman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, pp. 09.13998-99 2016.
Anonymous. DECN:
Evolution inspired deep convolution network for black optimization. In Submitted to The Eleventh International Conference on Learning Representations, 2023, URL httts::/openreviiee net/ taree/forunee wowommiddurrreeeeee goR6 9R, under review.
A. Auger and N. Hansen. restart ema evolution strategy with increasing population size. In 2005 IEEE Congress Evolutionary Computation, volume 2, pp. 1769-1776. 2005.
Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524 23:25824-21538,, 2020.
Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoe V Le. Neural optimizer search with rein- forcement learning. In International Conference on Machine Learning, pp. 459-468. PMLR, 2017.
Blank and K. Deb. pymoo: Multi-objective optimization in python. IEEE Access, 8:89497-89509, 2020.
Yue Cao and Yang Shen. Bayesian active learning for optimization and uncertainty quantification in protein docking. Journal of Chemical Theory and Computation, 16(8):5334-5347, 2020.
Yue Cao, Tianlong Chen, Zhangyang Wang, and Yang Shen. Learning to optimize in swarms. Advances in Neural Information Processing Systems, 32: 12:15444-50044 2019.
Tianlong Chen, Weiyi Zhang, Zhou Jingyang, Shiyu Chang, Sijia Liu, Lisa Amini, and Zhangyang Wang. Training stronger baselines for learning to optimize. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020.
volume 33, pp. 333--333.
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: primer and benchmark. Journal of Machine Learning Research, 23:1-59 2022.
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap. Matt Botvinick, and Nando Freitas. Learning to learn without gradient descent by gradient descent. In International Conference on Machine Learning, pp.
PMLR, 2017.
Antoine Cully, Jeff Clune, Danesh Tarapore, and lean-Baptiste Mouret. Robots that can adapt like animals. Nature, 521:503-507, 2015.
Swagatam Das and Ponnuthurai Nagaratnam Suganthan. Differential evolution: A survey of the state-of-the-art, IEEE transactions on Evolutionary Computation, 15(1):4-31 2010.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations, 2021.
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1):1997-2017, 2019.
Sebastian Flennerhag, Yannick Schroecker, Tom Zahavy, Hado van Hasselt, David Silver, and Satinder Singh. Bootstrapped meta-learning, arXiv preprint arXiv:2109.05004 2021.
Daniel Golovin, Benjamin Solnik. Subhodeep Moitra, Greg Kochanski, John Karro, and David Sculley. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD infernational conference knowledge discovery and data mining, pp. 1487-1495, 2017.
10

---

Under review as a conference paper at ICLR 2023
Kai Han, Yunhe Wang. Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer. IEEE transactions on Patter Analysis and Machine Intelligence, 2022.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 122:159-195, 2001.
Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. Automated machine learning: methods, systems, challenges. Springer Nature, 2019.
Howook Hwang, Thom Vreven, Joël Janin, and Zhiping Weng. Protein pestein-protein docking benchmark version 4.0. Proteins: Structure, Function, and Bioinformatics, 78 18(15):311-3314 2010.
ct.al. Jazzbin. gealpy: The genetic and evolutionary algorithm toolbox with high performance in python, 2020.
Kirthevasan Kandasamy, Jeff Schneider, and Barnabás Póczos. High dimensional bayesian opti- misation and bandits via additive models. International conference on machine learning, pp. 295-304. PMLR, 2015.
Kirthevasan Kandasamy, Gautam Dasarathy, Junier Oliva, Jeff Schneider, and Barnabás Póczos. Gaussian process bandit optimisation with multi-fidelity evaluations. Advances in neural informa- fion processing systems, 29, 2016.
Kirthevasan Kandasamy, Karun Raju Vysyaraju. Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, and Eric P. Xing. Tuning hyperparameters without grad students: Scalable and robust bayesian optimisation with dragonfly. Journal of Machine Learning Research, (168)):1-77, 2020.
Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann. Automated algorithm selection: Survey and perspectives. Evolutionary Computation, 27(1):3-45, 2019. doi: 10.1162/
evco_a_00242.
Shauharda Khadka and Kagan Tumer. Evolution-guided policy gradient in reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6900 69800, 2014.
Cheng Li, Sunil Gupta, Santu Rana, Vu Nguyen, Svetha Venkatesh, and Alistair Shilton. High dimensional bayesian optimization using dropout. IJCA/' 17. AAAI Press, 2017.
Ke Li and Jitendra Malik. Learning to optimize. arXiv preprint arXiv: 2016.
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, Daniel Freeman, and Jascha Sohl-Dickstein. Understanding and correcting pathologies in the training of leameed optimizers. In International Conference on Machine Learning, pp. 4556 4565. PMLR, 2019.
Melanie Mitchell. An introduction to genetic algorithms. MIT press, 1998.
lain Moal and Paul A Bates. Swarmdock and the use normal modes in protein-protein docking. International Journal of Molecular Sciences, 11(10):36233360 2010.
Jean-Baptinn Baptiste Mouret and Glenn Maguire. Quality diversity for multi-task optimization. Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 2020.
Mojmir Mutny and Andreas Krause. Efficient high dimensional bayesian optimization with additivity and quadrature fourier features. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (cds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Brian G Pierce, Kevin Wiehe, Howook Hwang. Bong-Hyun Kim, Thom Vreven, and Zhiping Weng. Zdock server: interactive docking prediction of protein complexes and symmetric multimers. Bioinformatics, 17111::771-1777, 2014.

---

Under review as a conference paper at ICLR 2023
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and llya Sutskever. Evolution strategies scalable alternative to reinforcement learning. arXiv preprint arXiv:/703.03864, 2017.
Graham R Smith and Michael JE Sternberg. Prediction of protein-proteli interactions by docking
methods. Current opinion in structural biology, 12(1):28-35 35, 2002.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25, Curran Associates, Inc., 2012.
Haoran Sun, Xiangyi Chen, Qingjiang Shi, Mingyi Hong, Xiao Fu, and Nicholas D. Sidiropou- los. Learning to optimize: Training deep neural networks for interference management. IEEE Transactions on Signal Processing, 46(20):5438-5455 2018. doi: 10.1199/SP2201828866333
Ye Tian, Shichen Peng, Xingyi Zhang, Tobias Rodemann, Kay Chen Tan, and Yaochu Jin. A recommender system for metaheuristic algorithms for continuous optimization based on deep recurrent neural networks. IEEE Transactions Artificial Intelligence, 1(1):5-18, 2020. doi: 20.1009/AA1.2220302222
Vassilis Vassiliades and Jean- Cann-Bapistte Mouret. Discovering the elite hypervolume by leveraging interspecies correlation. Proceedings of the Genetic and Evolutionary Computation Conference, 2018.
Vassilis Vassiliades, Konstantinos Chatzilygcroudis, and Jean- cann-Bapisste Mouret. Using centroidal voronoi tessellations scale up the multidimensional archive of phenotypic clites algorithm. IEEE Transactions on Evolutionary Computation, 22:623-65 630, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwantthan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Paul Vicol, Luke Metz, and Jascha Sohl-Dickstein. Unbiased gradient estimation in unrolled computation graphs with persistent evolution strategies. In International Conference on Machine Learning, pp. 10553-10555 PMLR, 2021.
Chao Wang, Jing Liu, Kai Wu, and Zhaoyang Wu. Solving multi-task optimization problems with adaptive knowledge transfer via anomaly detection. IEEE Transactions Evolutionary Computation, 2021.
Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo. Misha Denil, Nando Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and generalize. In International Conference Machine Learning, pp. 3751 PMLR, 2017.
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. Journal of Machine Learning Research, 15(1):949-995 2014.
Ronald J Williams. Simple statistical gradient-follle following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229 2266..9-256. 1992.
Kai Wu and Jing Liu. Classification-based optimization with multi-fidelity evaluations. 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 1126-1131 IEEE, 2019.
Jiangning Zhang, Chao Xu, Jian Li, Wenzhou Chen, Yabiao Wang. Ying Tai, Shuo Chen, Chengjic Wang, Feiyue Huang, and Yong Liu. Analogous to evolutionary algorithm: Designing unified sequence model. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (cds.), in Neural Information Processing Systems, volume 34, pp. 26674-2668 Curran Associates, Inc., 2021.
Jiangning Zhang, Xiangtai Li, Yabiao Wang, Chengjie Wang, Yibo Yang. Yong Liu, and Dacheng Tao. Eatformer: Improving vision transformer inspired by evolutionary algorithm. arXiv preprint arXiv:2206.0983 09325, 2022.
12

---

Under review as a conference paper at ICLR 2023
Qingfu Zhang and Hui Li. Moea/d: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 11(6:712-731, 2007.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng.
Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11106-11111 2021.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 VISION TRANSFORMER
We mainly introduce the core part of Vision Transformer, such as the multi-head self-attention layer (MSA), feed-forward network (FFN), layer normalization (LN), and residual connection (RC).
MSA MSA fuses several SA operations to handle the queries (Q), keys (K), and values (V) that jointly attend information from different representation subspaces. MSA is formulated follows: Multi Head Q, K, V) = Concat H1, H2. H₂) W where Concat means concatenation operation. The head feature H; can be formulated as:
H. SA(QW KW VW.)
Softmax QQW: (KW: /qqt(dd)) VW.- AVW!
where W.O Rdmxd, W Rddx and W.V [Rdmxxd are parameter matrices for queries, keys, and values, respectively; wo RRdd xdm maps each head feature H; to the output. Moreover, dm is the input dimension, while dg. dr, and d, are hidden dimensions the corresponding projection subspace; / is the head number. A E R!xi is the attention matrix of /th head, is the sequence length.
FFN FFN employs two cascaded linear transformations with ReLU activation to handle X, which shown as: FFN(X) = mar (0, XW + b1 W2 + b2, where W, and W2 are weights of two linear layers, and b and b2 corresponding biases.
LN LN is applied before each layer of MSA and FFN, and the output of LN calculated by X [MSAFFN((LN(()),
A.2 PRELIMINARY EAS
The crossover, mutation, and selection operators form the hasic framework EAs. EA starts with randomly generated initial population. Then, genetic operations such as crossover and mutation will be carried out. After the fitness evaluation of all individuals in the population, a selection operation is performed to identify fitter individuals to undergo reproduction generate offispring. Such an evolutionary process will be repeated until specific predefined stopping criteria are satisfied.
The crossover operator generates a new individual X, by Equation 11 and er is the probability of the crossover operator.
Crossover
X J. X k.
rand(0, 1) <
(11)
otherwise
where &s E [1.... d]. This operator is commonly conducted individuals. After expression expansion, we re-formulate Equation [1] as Σ X,W (Zhang et al. 2021). W is the diagonal matrix. If We is full of zeros, the ith individual has contribution.
The mutation operator brings random changes into the population. Specifically, an individual X; in the population goes through the mutation operator to form the new individual X,, formulated as follows:
Mutation
(rand(le, ux)
rand(0, 1) < mr otherwise
(12)
2.k
X i
where mr is the probability of mutation operator and E [1....,d] ,d.. Similarly, Equation 12 can be ce-fommulaedd X,W where W.nh is the diagonal matrix.
14

---

Under review as a conference paper at ICLR 2023
We introduce the binary tournament mating selection operator in Equation 13 The selection operator survives individuals of higher quality for the next generation until the number of individuals is chosen.
Selection
P {
f(x)) < [(x)) f(X,) > ) (x) X,
(13)
where reflects the probability that X; is selected for the next generation, and (X,, X&)) in Equa- tion 13 are randomly selected from the population X U X
A.3
SYNTHETIC FUNCTIONS
Table 6: Training functions.
ID
Functions
Range
Fl
Σ, ww,sin(;; b.)l
[-10, 10), E [-10, 10
F2
Eg [2; bil
x -10, 10 E
-10, 10
F3
E, (r: bi) + (0i+1 be+1)l + E, l2;
b,
x E 10, 10). E
10, 10
Table 7: Testing Functions.
ID
Functions
Range
F4(Sphere)
E = by max 21), 1 Si D).
100, 100), b E [-50, 50 100, 100), b E 50, 50
F5
F6(Rosenbrock)
(100(2) - #111)² 3 (z - 1)²); = 22 b.
x [- 100, 100), b E -05 50)
F7(Rastrigin)
(2] - -10coo((22z) + 10), xz - b,
x [-5, 5] b b | 2.5, 2.5]
F8(Griewank)
IT: cos( 3 + =
E [- 600, 600 b 300, 300]
4000
(9(Ackley)
20 e00e t E 2??)
x E (-3, 32,32 32), b [616, 16]
exp( D DD cos(2 20 exp(1),
I
A.4 PARAMETERS
BOptformer. For example, 30 OBs with WS contains 30 OBs, and each OB consists of 1 SAC, 1 FM, and RSSM. In 30 OBs with WS, these 30 OBs share parameters. 5 OBs without WS has 5 OBs, and parameters shared among them. During the training process, BOptformer is iterated for 1000 epochs. The initial learning rate (Ir) was set to 0.01 and Ir Ir x 0.9 each 100 cycles. The 2-norm of the gradient is clipped that is not larger than 10. The bias of the function is regenerated each epoch, and a new batch of random initial populations is generated.
Baselines. The number of generations of the reference algorithms is of ES, DE, and CMA-ES is set to 100. For all cases, we choose the optimal hyperparameters. To ensure validity, all experimental results are averaged over 10 runs. All experiments were performed on a Ubuntu20.04 PC with Intel(R) Core 17 (TM) 13-8100 CPU at 3.60GHz and NVIDIA GeForce GTX 1060.
100. The population size
A.5 PLANNER MECHANIC ARM PROBLEM
The optimization goal of this problem is to search for set of lengths L = (L1, Lg, L.,) and set of angles a = (a), a22 ... a,)) so that the distance f(L, a, p) from the top of the mechanic arm to the target position p the smallest, where represents the number of segments of the mechanic
15

---

Under review as a conference paper at ICLR 2023
arm, and L, (4,, u.) and oş (I), represent the length and angle of the ith mechanic arm, respectively. Typically, d is calculated as follows:
2
(LL,,
Σ cos(ai))i Px
+ Σ
sin(a;) Py
(14)
where Ps. and Py represent the x-coordinate and y-coordinate of the target point, respectively.
Here, n 100, 1; = O and = 10. We design two groups of experiments. 1) Simple case. We fixed the length of each mechanic arm as 1 = 10 and only searched for the optimal a. 2) Complex case. We need to search for 7. and simultaneously. We randomly selected 600 target points within the range of 1000 to form a set S. where r represents the distance from the target point to the origin the mechanic arm, as shown in Fig. 4 During the training process of BOptformer, sample point set is re-extracted from S for training every T training cycle. In the testing process, we extracted 128 target points (Stest) in the range of 100, s 300, and S 1000, respectively, for testing. The purpose of testing in three different regions to explore the generalization performance of BOptformer further.
We evaluate the generalization ability of the algorithm by Steat
f(L, 8)) / //Stear).
a2
4
L
a3
L
a1
a 4
d
Figure 4: Planar Mechanical Arm.
DE, ES, and CMA-ES are tested when the maximum generations Margen set to 10, 50, and 100, respectively. We find that BOptformer outperforms all baselines.
A.6 EFFECT OF LEARNING RATE ON BOPTFORMER
We train BOptformer on the F1-F3 function set with different learning rates, and then test it on the F4-F9 function set. The experimental results are shown in Table OBs without WS and 30 OBs with WS perform poorly when the learning rate is 0.1, which may be because the learning rate is too large, which affects the convergence of BOptformer during the training process. For 5 OBs without WS, setting the learning rate to 0.01 achieves relatively best performance. Using learning rate of 0.0001 would good choice for 30 OBs with WS and OBs with WS. However, our experiments are coarse-grained. The learning rate has greater impact on BOptformer. Then using Auto-ML to search for the optimal hyperparameter combination of the model is expected to achieve better performance.
A.7 CONVERGENCE OF BOPTFORMER
We plot the convergence curves of 30 OBs with WS, ES, DE, and CMA-ES on F7. BOptformer converges quickly and can obtain better solutions. BOptformer can only iterate ten times get the best solution relative to EA baselines. ES and DE converged around 100 generations, and CMA-ES showed slow convergence rate.
16

---

Under review as a conference paper at ICLR 2023
Table The results planar mechanical arm on searching for different angles with the fixed lengths.
Maxgen=10
r
DE
ES
CMA-ES
250-Swarm
BOptformer
100
2.96(1.63)
11.2(4.70)
236(46.8)
40.4(3.89)
0.30(0.18)
300
11.3(14.7)
45.3(43.3)
243(125)
69.5(3.77)
0.48(0.37)
1000
227(226)
257(246)
397(321)
176(7.20)
26.6(57.4)
Maxgen=50
r
DE
ES
CMA-ES
L20-Swarm
BOptformer
100
1.28(0.60)
10.7(5.91)
2.42(0.65)
40.4(3.89)
0.30(0.18)
300
1.54(0.89)
42.0(41.0)
4.06(6,54)
69.5(3.77)
0.48(0.37)
1000
110(152)
193(235)
103(182)
176(7.20)
26.6(57.4)
Maxgen=100
DE
ES
CMA-ES
L20-Swarm
BOptformer
100
1.20(0.64)
10.6(5.58)
1.36(0.35)
40.4(3.89)
0.30(0.18)
300
1.38(0.71)
44.9(43.3)
1,38(0.41)
69.5(3.77)
0.48(0.37)
1000
93.8(137)
183(239)
43.7(110)
176(7.20)
26.6(57.4)
Table 9: Ablation study learning rate.
5 OBs withou WS
Ir
F4
F5
F6
F7
F8
F9
0.1
0.93(7.42)
0.31(0.49)
2.04e7(2.03e8)
15.3(6.4)
0.36(0.16)
0.61(0.18)
0.01
0.01(0.003)
0.05(0.02)
9.57(0.22)
1.62(0.60)
0.03(0.01)
0.06(0.03)
0.001
0,88(3.41)
0.36(0.12)
226(1750)
6.18(2)6
0.56(0.16
1.36(0.36)
0.0001
0.06(0.03)
0.13(0.03)
13.6(2.11)
0.83(0.50)
0.17(0)44)
0,28(0.10)
30 OBs with WS
0.1
1.64(1.19)
0.85(1.59)
493(3110)
28.4(4.35)
0.47(0.11)
2.82(0.5)
0.01
0.05(0.30)
0.09(0.03)
39.2(240)
1.05(1.40)
0.01(0.06)
0,28(0.07)
0.001
1.01e-300.00))
0.02(0.01)
9.03(0.18)
0.03(0.02)
0.003(0.001)
0.03(0.01)
0.0001
1.50e-3(0.00))
0.016(0.004)
9.01(0.13)
0.02(0.02)
0.006(0.002)
0.02(0.0.01)
3 OBs with weights sharing
0.1
3.04(5.55)
0.98(0.4)
1150(7930)
43.3(8.87)
0.64(0.12)
2.43(0.98)
0.01
29.9(47.7)
2.68(1.30)
6.24c4(4.87c5)
40.4(8.52)
1.05(0.06)
4.45(0.85)
0.001
1.82(1.20)
0.76(0.32)
654(4780)
7.00(7.11)
0.79(0.13)
1.91(0.53)
0.0001
0.39().21)
0.33(..07)
46.8(79.4)
2,22(2241)
0.66(0.09)
0.59(0.19)
17

---

Under review as a conference paper at ICLR 2023
1600
80ptformerlwsa0)
CMA-ES
800
600
400
200
0
25
5.0
75
100
125
150
175
200
Figure 5: Convergence curves of Boptformer and EA baselines. The figure shows the convergence curve of these algorithms F7 in Table 7.
A.8 THE DETAILS OF PROTEIN DOCKING
Protein Docking We also handle the problem of Ab initio protein docking (Cao & Shen 2020).
which optimizes noisy and costly function in high-dimensional conformational space. Mathemati- cally, this problem is formulated as optimizing the Gibbs binding free energy f(x) for conformation æ. We calculate the energy function in a CHARMM 19 force field as in (Moal & Bates 2010) and shift it so that f f(x = 0 at the origin of the search space. f(x) is differentiable when parameterize the search space as 12 (Smith & Sternberg 2002). Here, only 100 interface atoms are considered.
Training dataset. 25 protein protein complexes (see Appendix A.8) from the protein docking benchmark set 4.0 [Hwagg et al. 2010),, each which has 5 starting points (top-5 models from ZDOCK (Pierce al. 2014)).
Testing dataset. Three complexes (with one starting model each) of different levels of docking difficulty are selected, including 1ATN. 7, 2JEL_1, and 7CEI_1
25 Protein-protein Complexes The training dataset contains 25 protein proteinnonennnnni complexes from the protein docking benchmark set 4.0 (Hwang et 2010). The detailed information shown as follows: 1ATN, 1AVX, IAY7, 1BJ1, IBVN, ICGI, IDFJ, TEAW, 1EWY, 1EZU, 1GRN, 1IBR, 1JJ,, IIQD, 1JPS, 1KXQ, IM10, IMAH, IN80, IPPE, IROR, IXQS, 2B42, 2COL, and 2HRK.
18


Reviews:
[{"review_id": "t-xyDZTjt_G", "reviewer": "Reviewer_xa7p", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "# Summary \nAt a high level, the paper in order can be summarized as:\n\n1. Operations (e.g. crossover) in evolutionary algorithms over continuous search spaces, can be seen as specific matrix multiplications over a population of suggestions $X$.\n2. This implies a form of learnability over these matrix multiplications, and a modified variant of the Transformer block can be interpreted as a large, learnable population mutator, mapping an initial population $X_{t}$ to an intermediate population $X_{t+1}$. These blocks can be stacked together to obtain population chains $X_{0} \\rightarrow X_{1} \\rightarrow X_{2} ...$. Note that then the output of this \"modified Transformer\" will be the final population.\n3. The model is trained directly with a basic normalized performance loss, over **differentiable** functions. \n4. Experiments are conducted to check the end-to-end performance of the method, which seems to outperform standard evolutionary algorithm baselines. Ablations are also conducted to show:\n    * Deeper models perform better.\n    * A deep randomly initialized model \"nws5-r\" outperforms a trained shallow model \"ws3\"", "summary_of_the_review": "Overall, while the approach and overall theme are solid, the execution of the paper is still currently flawed and needs much polishing (both in the writing/presentation, in addition to needing more experimental studies). Thus I currently recommend rejection.", "strength_and_weaknesses": "# Strengths\n* The overall approach (i.e. a Transformer block has an analogy to an evolutionary algorithm population update) is interesting. If the weaknesses below are resolved, this could be a solid contribution to the field of evolutionary algorithms.\n* The baselines used in the experiments (e.g. ES, CMA-ES, Dragonfly) are quite comprehensive. This at least currently establishes that the method has some promise in end-to-end performance.\n\n\n# Weaknesses\n* Experimental results do not present how the intermediate layers work, but only present the end result. This makes it hard to know if the method is doing what it's supposed to, especially given the large claims about the model's analogies to evolutionary algorithms. For example, some lingering questions exist, and can be resolved with more ablations:\n    * What kinds of population updates / cross-over techniques is each block learning? Are they similar or different from baselines? \n    * \"Perhaps using ES is a good training strategy\": Is the untrained block performing some sort of local or global mutation?\n* Currently, the paper only presents work on continuous search spaces. However, one big advantage of using regular evolutionary algorithms are their flexibility in dealing with combinatorial search spaces. Could the method be modified to approach combinatorial / categorical spaces? \n* Large amounts of unnecessary notation make it very difficult to comprehend what is going on. In general, it's not hard to convey my summary written above with fewer notation in general, and this could significantly improve the paper's quality and presentation. For example:\n    * Section 3.2 can be first introduced by stating the intent (i.e., showing that crossover / mutations can be represented as matrix multiplications).\n    * Section 3.3 can be moved to the Appendix or deleted altogether since they simply describe the original Transformer module. \n    * It would also make a huge difference in readability, if e.g. a diagram was made to represent Sections 4.1-4.3 (e.g. see Figure 1 in the original Transformer paper [1]). \n    * Too many abbreviations (e.g. \"nws5\") in the experimental section in Section 5. These could simply be written explicitly, e.g. \"Trained\"/\"Untrained/\"5 layers\".\n* (Minor): It seems that a previous paper [2] has already claimed the \"OptFormer\" name. I would suggest changing \"OptFormer\" in this paper to a different name.\n\n\n\n[1] Attention is all you need. https://arxiv.org/abs/1706.03762\n\n[2] Towards Learning Universal Hyperparameter Optimizers with Transformers. https://arxiv.org/abs/2205.13320", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "kpBfHzAF6y", "reviewer": "Reviewer_YunP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors proposed an interesting approach based on what I can tell to be a novel connection between vision transformers and evolutionary algorithms. The paper then modifies the transformer architecture to implement evolutionary-specific operations like crossover, mutation and selection. The authors then demonstrate their method on six black-box functions and show superior performance to baselines. ", "summary_of_the_review": "In general, I think this is an interesting approach. In its current state, however, I can not recommend acceptance. I am of course willing to change my score if the authors convince me in the rebuttal phase. ", "strength_and_weaknesses": "Strengths: \n- Interesting approach with potential impact on black-box optimisation \n- Favourable results in application domains considered \n\nWeaknesses: \n\n- Paper writing: I have found the paper hard to follow and not clearly written. Of course, I feel with the authors who had to introduce many related concepts before proposing their method. Maybe the authors could consider moving their contributions before page 4 to make it easier to grasp the novelty of the paper. \n\n- There does not seem to be regard to sample efficiency in the proposed approach. Are the authors not worried about expensive of evaluating black-box functions? I ask since the domains considered in the experiments are relatively toy and running the experiments in a real-world problem beyond Ab initio docking. Looking at the literature on high-dimensional Bayesian optimisation (which needs to be cited) can help in designing such experiments.  \n\n- I am also a bit confused about the experimental results, which seem to somewhat contradict the notion of black-box optimisation. To elaborate, it seems that the training set's similarity to the black box is a critical factor in the approach's success. If that is the case, how can we define this notion? If the target task (so to say) is a black box then what does it mean for us to be able to pick similar training data? What does similarity mean in this context? \n\n- Why haven't the authors compared to state-of-the-art bayesian optimisation solutions in their experiments? As far as I can tell, the dimensions vary from 10 to 100 d which SOTA BO should be able to handle. \n\n- What motivated the choices of the training and testing functions? After reading the paper, they seemed to be arbitrary. Maybe the authors can help me understand? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "eR9iqScj2Yu", "reviewer": "Reviewer_qcoV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new approach based on Transformer, namely Optformer, to optimize unconstrained black-box optimization problems. The key idea is based on an existing observation that Vision Transformer is similar to the evolutionary algorithm (EAs). Therefore, the paper aims to modify the components of the Transformer to implement the evolutionary algorithm's operations such as crossover, mutation, and selection. The proposed model Optformer then creates a mapping between a random population and the optimal solution of the black-box optimization problem. Experiments are conducted on 6 black-box functions and one real-world problem to show the efficacy of the proposed method.", "summary_of_the_review": "Overall, I think the key idea of the paper, which is to propose a transformer-based model to solve the unconstrained black-box optimization problems is very interesting and novel. The proposed method is sound to me. It is also shown to perform well on various problems. The paper's writing is generally clear. But I still have various concerns as listed in my questions. If these concerns are addressed, I can increase the score for the paper.", "strength_and_weaknesses": "Strengths:\n\nOverall, I think the key idea of the paper, which is to propose a transformer-based model to solve the unconstrained black-box optimization problems is very interesting and novel.\n+ The key idea of using the similarity between the Vision Transformer and evolutionary algorithms (EAs), and then developing a new model by modifying the Transformer's components to implement the EA operations seems to be sound and novel to me.\n+ The paper's writing is generally clear and easy to understand, although there are some parts that I feel I need to understand more (please see my questions below).\n+ The experimental evaluation shows the strong performance of the proposed method. It also includes some ablation study and sensitivity analysis to further understand the behavior of the proposed method.\n\nWeaknesses:\n+ Using a very deep learning model to solve the black-box optimization problems will introduce a lot of extra hyper-parameters that may significantly affect the performance of the proposed method. I found it hard to believe that an untrained model could still yield good results. Perhaps the test problems are not complex enough?\n+ The experimental evaluation only includes one real-world problem (protein docking). For these types of work, I expect more real-world problems to be included in the experimental evaluation.\n\nQuestions:\n+ In Section 4.5, I feel very unclear about how the training dataset is generated. The paper states that the method to generate the training dataset is to \"randomly produce a shifted objective function by adjusting the location of the optima\", but we do not know the objective functions (they're black-box functions) so how can we generate these shifted objective functions? In the experimental evaluation, I notice that the paper uses 3 new functions to create the training datasets, so how are these functions chosen? Do they need to share the similarity with the objective functions that we need to optimize?\n+ For the loss function in Eq. (12), a simple model that minimizes this loss function is the model where E_{\\theta}(X0) = X(0), but if this happens, then we can never find the optimal solution set of the black-box optimization problem. Will this case ever happen for the proposed method?\n+ The paper always states that X0 needs to be sorted in non-descending order of fitness (e.g., in the first sentence of page 6), but it never describes how we sort this set X0. Does the paper mean to say that the proposed Optformer will actually perform this ordering of fitness?\n+ In the experiments, the number of generations of all the reference algorithms is set to 100. This seems to be quite a small value given that some objective functions have the dimensions of 100. Plots showing the optimal values found by all the algorithms w.r.t. the number of generations (and the number of generations is much larger than 100) might be helpful.\n+ Is there any guideline on how to choose the hyper-parameters of the Optformer? E.g., learning rate, number of layers, etc.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d1TY_uqqnP", "reviewer": "Reviewer_eoPi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel method for continuous unconstrained black-box optimization problems, called Optformer. It is claimed that the Optformer is inspired by the similarity between Transformer and EA. Six simple benchmark functions and a real-world problem are used to verify the performance of Optformer. In general, this paper is poorly written. I could not find the scientific question therein and there are many errors. I think the contribution of this work is rare. ", "summary_of_the_review": "In summary, I think this paper should be considerably re-organized and carefully improved in terms of at least motivation, related work, and experimental studies.  ", "strength_and_weaknesses": "1. The English writing is poor. Too many long sentence making it difficult for understanding. \n2. In the first paragraph, I do not think neural architecture search can be said that \"we have no access to any other information about f\". And it is not generally correct to say any hyperparameter optimization problems that \"we have no access to any other information about f\".\n3. The second sentence, \"evolutionary strategies\" should be \"evolution strategies\". Also, Evolution strategies (ES) is a type of evolutionary algorithms(EAs), if the authors list EAs, it is no need to list ES.\n4. What is \"hand engineering\"? And do not list 11 references at the same time.\n5. \"The rarely focus on black-box optimization problems\", this is too strong and should be clearly justified. As they aim to solve general optimization problem, why cannot they solve black-box optimization problems.\n6. \"Although several efforts like Cao et al. (2019); Chen et al. (2017) have coped with these problems, they have limited performance due to poor representation ability of RNN, LSTM, and MLP.\" this is too strong and should be clearly justified.\n7. The major motivation of this work is said as \"Inspired by the similarity of EAs and Transformer\", but there is no discussion about what is the similarity inbetween. And to the reviewer, EAs look not similar to Transformer.\n8. For the experimental studies, the comparisons are too simple. First, the benchmark contains only six very simple functions. It is not enough to demonstrate the advantages of Optformer on these kinds of problems. Second, the real-world problem of Ab initio protein docking lacks of descriptions. It is unclear what the problem it is essentially. Third, the compared algorithm of DE look outdated as it is 12 years ago, and it is unclear what exact algorithms/versions of ES and CMA-ES are as there are references.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject