Generative Evolutionary Strategy For Black-Box Optimizations
Authors: Changhwi Park
Seong Ryeol Kim
Young-Gu Kim
Dae Sin Kim
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Optimization
Blackbox
Generative model
Evolution

Abstract: Many scientific and technological problems are related to optimization. Among them, black-box optimization in high-dimensional space is particularly challenging. Recent neural network-based black-box optimization studies have shown noteworthy achievements. However, their capability in high-dimensional search space is still limited. This study proposes a black-box optimization method based on evolution strategy and generative neural network model. We designed the algorithm so that the evolutionary strategy and the generative neural network model work cooperatively with each other. This hybrid model enables reliable training of surrogate networks; it optimizes multi-objective, high-dimensional, and stochastic black-box functions. In this experiment, our method outperforms baseline optimization methods, including , including evolution strategies, and a Bayesian optimization.

The estimating process of Bayesian optimization is very efficient when the number of search points is Generative Evolutionary Strategy For Black-Box Optimizations
Anonymous Author(s)
Affiliation Address
email
Abstract
Many scientific and technological problems are related to optimization. Among them, black-box optimization in high-dimensional space is particularly challenging. Recent neural network- black-box optimization studies have shown notewor- thy achievements. However, their capability in nigh-dimensional search space is still limited. This study proposes a black-box optimization method based on evolu- tion strategy and generative neural network model. We designed the algorithm that the evolutionary strategy and the generative neural network model work coop- eratively with each other. This hybrid model enables reliable training of surrogate high-dimensions
3
networks; it optimizes mulli-objectivn box functions. In this experiment, our method outperforms baseline optimization methods, including evolution strategies, and Bayesian optimization.
and stochastic black-
10
11
1 Introduction
13 Optimization is one of the most crucial issues in science and technology. Various simulations and experiments work black-box functions, and there have also been innumerable optimization studies.
15 Gradient-based optimization methods are easy choices for differentiable functions or simple convex
18 functions. However, black-box functions are often non-differentiable and non-convex. Furthermore.
17 they can be multi-objective and stochastic. A simple description of multi-objective black-box optimization follows
18
Optimize( f")
19
X R
The search space is defined in real space R, where N is the dimension. f is single-objective function, which can be stochastic.
21
22 The evaluation of electronic device designs is practical problem of the black-box optimization.
23 Since many device simulators have time-sequential input-output structures, it seems like they can
24 be solved in reinforcement learning. However, if the observation cost is too high, it will be almost
25 impossible to observe time-sequential data. Instead, the only information which we can observe the final score. Therefore, the evaluation problem is defined black-box optimization, in this case.
27 For practical purposes, researchers have studied optimization methods in various ways. [4] [S]
28 [6]. Typically, Bayesian optimization [1][2][3] and evolutionary strategies [7]-[20] are widely used. Notably, the Bayesian optimization is advantageous when the cost of the target function is high and
29
30 the number of function calls is limited.
31
32 small. On the contrary, it becomes inefficient when the number of search points increases. Therefore,
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

a)
b)
Generator Pool
Stadie region
Generstor
Butfer menorry
Network
Dnserator
Score 2
Gonemstor
Score 3
Evolution Strategy
Generative N.N. model
Generstor HD
Canentor jo
vertuble budi
Gonerater
Score
Black Box (F)
muf
Score
Optinad mutation
Figure I: Schematic figures of GEO. a) GEO full algorithm. 1: Random sampling of generators. 2: Mutation (backpropagation).
Variables (x) generation. 4: Black-box run. Data save and repeat 1-4. 6: Generator sorting with Pareto efficiency. 7: Storing search history in the buffer memory. 8: Critic network training. b) A simple description of the cooperative workflow
33
Bayesian optimization may not be appropriate in high-dimensional problems that require a large number function calls.
34
35
Meanwhile, evolution strategies (ES) can be better choices for aforementioned cases. In most evo- lution strategies, the computational cost does not drastically increase according the number of search points. Nevertheless, it does not mean that they can avoid the curse of dimensionality. The optimization performance of the evolutionary strategies also decreases rapidly the dimension the black-box increases. Although SEP-CMA[55] VD-CMA[56], and LM-CMA(57) have shown opti- mization capability in high-ddimensional space of convex functions, optimization of high-dimensional non-convex problems still seems difficult.
37
38
39
40
41
Generative neural network-based models are recent approaches. They show noteworthy performance in test function optimizations. but their capability seems to be limited to single-objective functions and to 100-dimension [21][28]. We present GEO: Generative Evolutionary Optimization, method for general black-box optimizations. It is designed to optimize stochastic, multi-objective, and high- dimensional black-box problems. We show that GEO outperforms baseline methods in finding Pareto [42], Ackley [39], Rastrigin [36][37][38], Rosenbrock [40][41], ZDT1, ZDT2, and ZDT3 [43] functions. Also, by converting Cartpole-VI [44] to high-dimensional
43
45
47
fronts Styblinski-Tann
48
49
black-box problems, we show that GEO can be used in sequential problems. We also tested LeNet-5
50
[45] to see how it generates sub-manifold structures.
Related works
51
GEO is related to Evolutionary Generative Adversarial Networks (EGAN) [22], and Local Generative
53
Surrogates Optimization (L--GSO [21]. This section briefly introduces them.
54
2.1 L-GSO
L-GSO is a surrogate network model based black-box optimizer. has a surrogate network and a 55
generator network. The main idea of L-GSO is that the surrogate network only estimates local 57 shape of the objective function. Since the stabilization of the surrogate network is difficult, they suggest only to surrogate local region. Also, the optimizer be used in stochastic environments since it works in the neural network.
59
It is shown that L-GSO outperforms baseline optimizers in dimension = 10 and sub-manifold dimension 100 problems. However, due to the limitation of the local sampling method, L--GSO applies only to single-objective function.
61

---

Algorithm GEO
Require: Initial generator pool {(Gi, s1), (G2, (2), ..., (Gps 8p) initial critic networks {Ct, , (constant or random variable). The multi-objective function is defined as (f) , F. while iteration do while < NN do Sample C in {C*, C ,,,,,.,CN
buffer memory ((x1, 81); 82), ...)..... (xD, }n),, input seed
> Critic network training
- 9c = VV= Mj Σド= [1))) - f" (zy))l, x E buffer
C t Optimize ( 9c)
end while
while < N do
D Generator network mutation (multi-objective)
Sample C in (C',C,,... ,,,,., C.
C,,
while m M2 do RandomSample G3 in (G1,,1), (G2, $2). (Gpp 8p)) +: minimize, -:maximize
D M2; the number of mutations
9g Vol+C"M (Gi,o(=))
G. Optimize Ggi99)
Gi(z)
8; = F(x,)
(uffer.appendd(2;; s:)
pool.append((G,, 8;))
end while
end while
pool
ParetoEfficieeccy(poo))
Non-dominated sorting > (optional)
pool +- AgeEvolution(pool)
end while
63
2.2 EGAN
64
Evolutionary Generative Adversarial Networks (EGAN) combines Generative Adversarial Network (GAN) [30]-[33] and evolution strategies. The core idea of EGAN is that the evolution strategy can assume backpropagation as mutation. has one discriminator and multi generators. Generators of the evolution pool are mutated for each iteration, and they are sorted by fitness scores.
85
66
By comparing the mode collapsing results, the study shows that the evolution strategy efficiently complements the GAN algorithm. EGAN is not an optimizer. Nevertheless, we expected that combination of an evolution strategy and a GAN could be adopted in our black-box optimization algorithm.
89
70
71
72
2.3
Other approaches
Global Topology Optimization network (GLOnet) [23] is method for electromagnetic device designs. is an advanced study the previous research, adjoint-base based applogy-oppimiinn (ABTO) [24]-[27]. GLOnet increases optimization performance by adding generator networks ABTO. GLOnet is not a black-box optimizer because the gradient is given directly from the target simulator: However, we discover essential role of the generator network for better optimization.
74
76
77
78
GNN-ES (Evolutionary Strategies with Generative Neural Networks) [28] a combined method of bijective neural networks and evolutions. GNN-ES assumes latent space and bijective Generative Network (GNN) x = g(2), z = h(z). It optimizes latent space and a bijective network. The update of latent space carried out by evolution strategies. The study shows that GNN-ES can optimize test functions in dimension = 10. However, GNN-ES is not surrogate basedd optimizer and is restricted to bijective networks.
80
82
84 Conservative Objective Models (COMs) [29] is surrogate model-based black-box optimizer. The key idea of COMs is regularizing the loss function of a surrogate model in training. Along with 86 standard supervised regression, adds COMs-regularizers to prevent erroneously large predictions of
the trained model.

---

Generator Network
Critic Network
200
Atteition ook
Attertion
ADD
Uiias Dopputt Act
Figure 2: A trunk-branch network structure. The two-level structure is an ad hoc method to reduce memory overuse of attention networks.
88
Surrogate assisted evolution models are also related [58]. However, they do not guarantee O(n) computational complexity. This can be a weakness in high-dimensional problems.
89
Methods
90
91
GEO consists of stages: the evolution and network training. The evolution pool maintains certain number of generators on the basis of fitness scores. The evaluation and sorting of multi- objective scores is determined by Pareto-efficiency.
92
93
94
generator training is also mutation in the evolution. 1. A generator is randomly sampled from the pool. 2. The critic network trains the selected generator (using backpropagation, to increase or 3. The trained generator suggests a new variable ==G(=). 4. Check
96
decrease a prediction of a critic). Fcor 5. Sort new (G, score) pair in the pool.
96
(x, score) pairs are stored in buffer memory and they are used to train critic networks. Each critic network is trained to surrogate corresponding black-box object.
100
3.1
Generative model
GEO consists of pool of generator networks and critic networks that create backpropagations. For each iteration step, generators are randomly sampled to make mutated generators. N-critic networks are prepared make N-objective surrogate model, each critic network corresponds to single objective function. The mutated generators create variables x Gz(z), and N-objective
102
103
104
scores s¹ = (x) are measured. The set of scores (I, 8), = (s* s² ..... ) is stored in the buffer memory. Training of the critic network carried out using the buffer memory. After training, critic
105
106
107
networks mutate the generator networks in the next iteration step. The backpropagation serves optimal mutations by increasing predictions of critic networks.
108
MG (G) EC!....,
Usually, traditional GAN generators feed random latent variables through the input layer, while some GAN algorithms separate latent variables from the input feeds [33]. Because GEO does not need inferences, do not (Figure 5) and constants (Figure 4) as input feeds
110
latent vector. We experimented with both random variables
112
Since each critic network has corresponding objective, it must be trained separately using its corresponding objective function. We used LI loss with a single objective function fo and a critic network C"), The loss function is defined as follows
114
Las Eenp, (1C(()))()))))))))))))))()) (x)1l
(
117 The critic network learns variable in a global region. Global training is essential for multi- dimensional Parcto front searches. (See 3.4)

---

8)
b)
18
14
660
107
KSCA3
RS242
1071
HSGA2
39
107
CHAR52
OPAR52
50.0
3300
500
10030
1000
3000
5008
10090
50.0
2000
Ackley
Resenbrock
StybinoiiTong
23.
Get Bin (200
100
2.0
12.
10
29339 32000
Senter
Figure 3: Performance comparisons of GEO and baseline optimizers in oingle-objective test functions. a) Optimization performances from dimension to 8, 8,192 dimension (Styblinski- Tang test function). b) Computational time in 8,192 dimension (real time). c) Optimization performances of GEO and LSM in single-objective functions. LSM is a modification of L-GSO.
119
3.2
Evolution strategy
Each generator is stored the pool with a corresponding fitness score = (s).....)) = F(ay). Score data &z)..))... sy)...} (for minimization cases)
120
121
are sorted by Pareto efficiency. The Pareto efficiency is defined as follows
ViE L,.,, N: f((x") flz). I...,, N: f)())) fzx)
123
then, x² E where X and i the Pareto efficiency. Pareto efficiency can be ranked order Pi Pi=Pattto(X)) P3=Pareto(XX - P1), ..., they are calculated by non-dominated sorting methods. A more detailed explanation provided in the supplement.
124
125
126
Optionally, age evolution can be added in the sorting part. The age evolution removes the oldest elements from the pool. Thereby, it prevents "the high score due stochasticity" from surviving in the pool. A pool-refresh method is another option, but it makes the calculation time almost doubled.
128
In GEO, evolution strategy is not just an auxiliary tool. Without an evolution strategy, the training of networks can be unstable, which leads to the divergence. We discuss details in section 3.4.
130
131
3.3
Neural networks
Any kinds of neural networks, including Recurrent Neural Network (RNN) [47][48][49], Convo- lutional Neural Network (CNN) [46], and Full Connected (FC) network can be used as generator networks and critic networks. We chose convenience. Figure shows the self-an self-attention network we used. The overall structure is modified from the original transformer model.
133
settt-head--ef--ttennle network [50] for operational
135
137
Because the attention network consumes gigantic memory size, may cause GPU out-of-memory. was significant problem when we optimized high-dimensional functions. When using NVIDIA Tesla V100 32G GPU in variable space of dimension d > 211, the total required memory exceeds the network structure to solve this problem.
139
available memory size. We devised an ad hoc
5

---

a)
Stylktrciiling.co AcMey
Trng, Rastiggn
HODADARC
MOCAO
IEGAD MAGA
NSAO LIC.
MORAATI
Skyal
b)
HSDADARC
SESADLINC
POEAO
MULAN
MAKA UN
MODAD
Figure 4: Optimization results of GEO, MOEA/D, NSGA-IIII and NSGA-II in non-stochastic-mute objective functions. Plots show optimization results in the 8, 192 dimension, after 100, 000 function calls (1 (1,000 iterations). LHC indicates the latin hyper cube initial variable points, and / indicates the initial points which are obtained from GEO's initial points.
141
This structure has one trunk network and several branch networks, and each branch network extends from the trunk network. Branches have an identical structure with each other, and the length of the output tensor is defined by Neabber = nhar Dbranchee.
142
143
144
The split branch trick serves a memory-efficient structure, but it could be detrimental to optimization performance. Therefore, implement the trunk-branch structure only for the sake of memory efficiency of GPU.
145
148
The baseline attention network structure includes dropout layers [51], and the dropout layers' random- ness makes the generator stochastic. A random input feed is also a source of stochastic behavior. We experimented with both stochastic generators (Figure 5) and non-stochastic generators (Figure 4). However, for critic networks, we maintained the dropout layers as non-zero.
148
150
151
3.4
A complementary strategy of generative network and evolution
Figure shows the full algorithm of GEO. Training a surrogate model (critic network) is the essential part of a surrogate model-based optimizer, but it is also the trickiest part. The point is that the training data (the true data in GAN concept) is not prepared, and the data can be only acquired through on-the-fly searches. Without prepared data, training can be unstable since outbreaks of new data make the training region fluctuate. In this case, the algorithm diverges for the following reasons:
153
154
155
156
157
The generator suggests an input variable x in a wrong direction. 2. The critic network trained with input variable but has information of Pareto front. 3. The critic network trains the generator, but it does not give meaningful information.
159
In short, the divergence is result of evil cycles of two networks.
The local sampling of L-GSO seems be simple stabilization method. case of N=1, L-GSO samples data in a local region, where the center is a current point. The current point is the Pareto-front, in this case.
162
163
164
However, it can be challenging in N-objective functions (in cases of N>1). Since the Pareto- front is R~ surface (not single point), there will be lot of centers of sampling. Then, it is not local anymore. The local sampling method cannot be used multi-objective problems.
186
Therefore, we need to devise stabilizing method for multi-objective functions. We suggest that the evolution strategy can be good solution. The role of an evolution pool to trap G and corresponding near the Pareto-front (rank 1). At the same time, the data that is far from the
188

---

STACTASTIC 2012
Figure 5: Optimization results of GEO, MOEA/D, NSGA-III, and NSGA-II in stochastic-mas multi- objective functions. Plots show optimization results in the 8, 192 dimension, after 100,000 function calls (1,000 iterations).
Pareto-front is discarded. Also, it slows down the fluctuation of the training data. As result, the training data region stabilized around the Pareto-front.
171
172
We expect the stabilization method to serve as an anchor, which prevents the training data from floating, At the same time, a properly trained critic network provides better mutation strategies. The interdependent cooperation of an evolution strategy and surrogate models is our main idea.
173
174
4
Experimental results
178
In this section, we compare the black-box optimization results of GEO with baseline optimizers. We tested single and multi-objective functions, stochastic and non-stochastic functions. The baseline optimizers are Bayesian optimization (Gaussian process), NSGA-II [7] (GA in 1-object), NSGA-III [15], MOEA/D [18], and CMA-ES [9] evolution algorithms.
177
179
180
4.1
Single objective functions
Figure 3a) shows performances of optimizers according dimensions. For the single-objective test function, we used Styblinski- function. At dimension = 2, Bayesian optimization shows the best performance. However, the dimension increases, GEO outperforms baseline optimizers. At dimension = 8, 192, baseline optimizers rarely finds the global minimum, while the GEO shows better performances.
182
184
185
186
To see how the depth of the generator network affects the performance, we also tested GEO with a single-layer generator. The single-layer generator appears to have little optimization capability. Even in the dimension = problem, shows a considerably slow optimization.
188
189
Bayesian optimization is a powerful method for high-cost black-box problems, but its performance can be weakened when problem requires numerous function calls. The computational complexity of Bayesian optimization is known to be (nz3) [3]. Figure 3b) shows the computational time of GEO, NSGA-II, and Bayesian optimization. Like most evolution strategies, GEO designed have
190
192
computational complexity of On)).
194
Figure 3c) shows performance comparisons L-GSO algorithm. LSM follows the general outline of L-GSO GAN implementation but adopts the self-attention network used in GEO. In the Styblinski- Tang function, LSM shows worse performance than in other functions. Also, the performance of LSM rapidly decreases the dimension increases. the other hand, GEO shows better optimization performance under various conditions.
Local Surrogate Model (LSM), modification the
185
196
197
198
199
4.2
Multi objective functions
In this section, show optimization performance comparisons in multi-objective problems. We only compare GEO and evolution strategies because the digh-dimensional problems require a lot of
201
202
function calls.
203
Figure shows optimization results in the won-stocaatiii-ultiiooiieee functions. Fach figure result after 100,000 function calls. For the multi-objective test functions, ZDT functions and
7

---

600
RSGA2
NSG43
WODAD
39 SLB
KLD
Figure 6: Optimization results of GEO, NSGA-II, NSGA-III and MOEA/D according to dimensions. (After 100,000 function calls with a ZDT3 test function.)
Table I:
Black-box optimization of Cartpole-V VV. Scores are measured in relative scale (max score 1.0, max steps 500) after 50, 000 function calls.
Sequence length
256
512
1024
2048
GEO
0.598 + 0.05
0.305 + 0.03
0.310 +: 0.02
0.323 + 0.03
CMAES
0.583
0.09
0.292 + 0.05
0.263 + 0.02
0.280 + 0.02
NSGA2
0.243 0.01
0.120 + 0.01
0.124 + 0.01
0.121 + 0.01
combined functions = 1 (f:: Styblinski-Tang, Ackley and Rastrigin function) were used. Latin Hyper Cube (LHC) [53] method is a good guess for initial states. However, GEO cannot implement LHC because it generates initial points through the neural network. We gave two initial
206
208
states in the baseline optimizer to control performance according to the initial state. NSGA2-1 uses GEO's initial distribution G(z) its initial points, while S2 uses LHC-initial points.
210
In the high-dimension, GEO outperforms baseline optimizers. The choice of initial points for NSGA- rarely affects final results. A slightly different result appears in ZDT2. ZDT2, GEO finds an optimal point, but it fails to find a global shape of the Pareto-front. Figure 6 shows the result of ZDT3 optimization according to dimensions. Classical ES algorithms significantly reduce performance in high-dimensional space, while GEO shows more robust performance in high-dimensional space.
212
214
216
Figure shows optimization results in the poochatic-multi-ojjectii functions. The optimization of stochastic functions is defined as follows
217
x² ([( F(x) = (Ji, f - N
218
219
220
N random normal distribution, and we set μ 0.0 and o 1.0. GEO outperforms baseline optimizers even in stochastic environment, but it still has single-point collapsing problem in the ZDT2 function.
222
223
4.3
Cartpole-vl
224
For the variety test functions, we optimized the OpenAl [54] Cartpole-VI by converting it into bla problem. also a simple toy model of sequenequlntall input-output (I/O) problems.
225
226
Cartpole-V is a test package that mainly used in reinforcement learning [34][35]. Reinforcement learning requires series of I/O structure. However, in the black-box problem, the entire input sequence is assumed as one large input, and only the final score is measured without observing the intermediate rewards and states. Since this experiment is toy model of real-world problems which have stochastic environments, we kept the Cartpole-V V1 stochastic.
228
230
The final score is measured in relative score to the sequence length. (If the Cartpole is alive for m- in n-sequences, the score is m m/n.) n.) Therefore, the maximum and minimum score set to 1.0 and 0.0. GEO outperforms others from 256 to 2048 dimensions in the experiment (Table l).
232

---

T-
je 0.000
Figure 7: Black-box optimization of LeNet-5 (MNIST-trained), Each corresponds prediction of LeNet-5 for a target number. After optimizations, scores get close to the maximum score (1.0). 0: 0.999, 1: 0.959, 2: 0.999, 3: 0.999, 4: 0.999.
234
Cartpole-V V1 is difficult to solve with black-box optimization approach due to the stochastic change of the initial state, but it shows clear performance difference between the black-box optimizers.
236
4.4
LeNet-5
We also optimized LeNet-5, which is trained with the MNIST dataset. The optimization goal is to generate an image that makes the LeNet-5 predict a target number with a maximum score (maximum
238
239
prediction score 1.0). The LeNet-5 is regarded as a non-differentiable black-box. After 50, 000 function calls, the final scores of generated images reach very close to the maximum score (Figure 7).
240
In the related experiment, L-GSO, generative models appear to be better at finding the local optimum in manifodssssds. For the same reason, we expected to see the sub-manifold structure the generated image, but the generated image does not seem intuitive.
242
Discussion
244
Often, a neural network's 's learning mechanism is likened to learning manifolds in high-dimensional space. Similarly, we guess that the critic network in GEO learns low-dimensional manifolds in the high-dimensional space. Therefore, we expect that finding global or local optima would be easy if the optima are low-dimensional manifolds. Also, we consider that the reason why the depth of generators is important.
246
248
Meanwhile, guess that the collapse problem of ZDT2 caused by concave shape of its Pareto- front. This is because, when the data is formed as concave shape, non non-dominated sorting selects the edge state first. We have yet to find clear solution to solve the collapse problem without compromising performance.
252
253
Conclusion
254
255
We have described method for caitbastic-mlti-obbieee interdependent cooperation method of generative neural networks and the evolution strategy. The evolution strategy provides a stable training region for critic networks, and the critic networks provide efficient mutations the evolution strategy. As our design intent, GEO seems work appropriately in stochastic and high-dimensional multi-objective test functions.
black-box optimization. GEO is an
257
259
Mcanwhile, the Pareto-froo front collapsing problem, shown in ZDT2, is important issue to be dealt with. Another limitation of GEO is the GPU memory consumption problem. The excessive memory consumption of attention networks limits its search space to around 10, 000 dimensions. In future researches, we can study other memory-efficient networks to solve this problem.
261
263
264
GEO is designed for optimization in extremely high- didddmiioins.. However, the performance at lower dimensions is not guaranteed (see supplement). We think that [1,000 < 10,000] the practical range of use of GEO, unless we improve the efficiency of network structures. In addition, the mutation of generators concentrates on an exploit, the explore strategy could be weak. the next study, strong exploit & explore strategy should be added to improve optimization performance.
265
257

---

References
289
[1] Snock, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms, Advances in neural information processing systems, 25.
272
[2] Frazier, P. (2018). A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811.
273
[3] Lan. G., Tomczak. J. M., Roijers, D. M., & Eiben, A. E. (2022). Time efficiency optimization with a bayesian aayesian-evolutionayy algorithm. Swarm and Evolationary Computation, 69, 100970.
274
[4] Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The computer journal, 7(4), 308-313.
276
277
[5]
Kennedy, J., & Fberhart, R. (1995, November). Particle swarm optimization. In Proceedings of ICNN'95- international conference on neural networks (Vol. 4, pp. 1942-1948). TEEF.
278
279
[6] Hooke, R., & Jeeves, T. A. (1961)."Direcl "Direct Scarch""olution "Solution of Numerical and Statistical Problems. Journal of the ACM (JACM), 8(2), 212-229.
281
77] Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. A. M. T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-IL JEEE transactions on evolutionary computation, 6(2). 182-197.
282
283
88] Deb, K., & Sundar, J. (2006, July). Reference point based multi-objecttvve optimization using evolutionary algorithms. In Proceedings of the 8th annual conference on Genetic and evolutionary computation (pp. 635-642).
284
285
206
99
Hansen, N., & Ostermeier, A. (2001). Completely derandomized self-adaptation evolution strategies.
287
Evolutionary computation, 9(2), 159-195.
[10]
Hansen, N. (2006). The CMA evolution strategy: comparing review. Towards a new evolutionary computation, 75-102.
289
290
Price, K., Stor,, R. M., & Lampinen, J. A. (2006). Differential evolution: a practical approach to global optimization. Springer Science & Business Media.
292
[12]
Runarsson, T. P., & Yao, X. (2000). Stochastic ranking for constrained evolutionary optimization. IEEE Transactions evolutionary computation, 4(3), 284-294.
294
[13] Runarsson, T. P.& & Yao, X. (2005). Search biases in constrained evolutionary optimization. IEEE Transac- tions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 35(2), 233-243.
295
296
[14] Deb, K., & Jain, H. (2013). An evolutionary mbje-objective optimization algorithm using reference-point based nondominated sorting approach, part I: solving problems with box constraints. IEEE transactions on evolutionary computation, 18(4). 577-601.
297
298
299
[15] Blank, J., Deb, K., & Roy, P. C. (2019, March). Investigating the normalization procedure of NSGA-III. In International Conference on Evolutionary Multi-Criterion Optimization (pp. 229-240). Springer, Cham.
300
[16] Seada, H., & Deb, K. (2015). A unified evolutionary optimization procedure for single, multiple, and many objectives. IEEE Transactions on Evolutionary Computation, 20(3), 358-369.
302
303
[17] Vesikar, Y., Deh, K., & Blank, J. (2018, November). Reference point based NSGA-III for preferred solutions. In 2018 IEFE symposiun series on computational intelligence (SSCI) (pp. 1587-1594). IEEE.
304
305
[18] Carvalho, R. D., Saldanha, R. R., Gomes, B. N., Lisboa, A. C., & Martins, A. X. (2012). A multi- objective evolutionary algorithm based on decomposition for optimal design of Yagi-Uda antennas. JEEE Transactions on Magnetics, 48(2), 803-806.
307
308
[19] Li, K., Chen, R., Fu, G., & Yao, X. (2018). Two-archive evolutionary algorithm for constrained multiob- jective optimization. IEEE Transactions on Evolutionary Computation, 23(2). 303-315.
310
[20] Panichella, A. (2019, July). An adaptive evolutionary algorithm based on non-Euclidean geometry for
311
many-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 595-603).
312
313
Shirobokov, S., Belavin, V., Kagan, M., Ustyuzhanin, A., & Baydin, A. G. (2020). Black-box optimization [21]
314
with local generative surrogates. Advances in Neural Information Processing Systems, 33,
315
[22]
Wang, C., Xu, C., Yao, X., Tao, D. (2019). Evolutionary generative adversarial networks. JEFE Transactions on Evolutionrr Computation, 23(6), 921-934.
316
317
[23] Jiang, J.& & Fan, A. (2019). Global optimization of dielectric metasurfaces using a physics-driven neural network. Nano letters, 19(8). 5366-5372.
318
[24]
Yang, J.. Scll, D., & Fan, J. A. (2018). Freeform metagratings based on complex light scattering dynamies for extreme, high efficiency beam steering. Annalen der Physik, 530(1), 1700302.
320
321
[25] Hughes. T. W., Minkov, M., Williamson, I. A., & Fan, S. (2018). Adjoint method and inverse design for nonlinear nanophotonic devices. ACS Photonics, 5(12), 4781-4787.
10

---

323
[26] Jensen. J. S., & Sigmund, O. (2011). Topology optimization for nano-photonics. Laser & Photonics Reviews, 5(2), 308-321.
325
[27] Molesky, S., Lin, Z., Piggott, A. Y., Jin, w., Vucković, J., & Rodriguez, A. W. (2018). Inverse design in nanophotonics. Nature Photonics, 12)1 659-670.
326
[28] Faury, L., Calauzenes, C., Fercoq. O., & Krichen, S. (2019). Improving evolutionary strategies with generative neural networks. arXiv preprint urXiv.1901112211
328
329
[29]
Trabucco, B., Kumar, A., Geng, X., & Levine, S. (2021, July). Conservative objective models for effective
330
offline model-based optimization. In International Conference on Machine Learning (pp. 10358-10368). PMLR
332 [30] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014).
333
Generative adversarial nets, Advances in neural information processing systems, 27.
[3] Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint ar0iv:1710.1011:
335
336
[32] Karras, T., Laine, & Aila, T. (2019). A style-hased generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4401-4
339
[33] Donahue, J., Krähenbühh,, P., & Darrell, T. (2016). Adversarial feature learning. arXiv preprint 1605,
340
334
Mnih, V.. Kavukeuoglu, K., Silver, D., Graves, A., Antonoglou, L., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint urXiv:1312.5602.
342
343
[35
Mnih, V. Badia, A. P., Mirza, M., Graves, A., Lillicrap. T., Harley, T., & Kavukcuoglu, K. (2016, June). Asynchromous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928-19377 PMLR.
344
346
[36]
Rastrigin, L. A. (1974) Systems of extremal control. Mir, Moscow.
347
[37]
Hoffmeister, F., & Bäck, T. (1990, October). Genetic algorithms and evolution strategies: Similarities and
differences. In International Conference on Parallel Prublem Solving from Nature (pp. 455-469). Springer,
349
Berlin, Heidelberg.
350
[38]
Mühlenbein, H., Schomisch, M., & Born, J. (1991).. The parallel genetic algorithm as function optimizer. Parallel computing, 17(6-7), 619-632.
352
[39]
Ackley, D. (2012). connectionist machine for genetic hillclimbing (Vol. 28). Springer Science Business Media.
353
354
[40] Rosenbrock, H. (1960). An automatic method for finding the greatest or least value of a function. The
355
Computer Journal, 3(3), 175-184.
356
[41] Dixon, L. C. W., & Mills, J. (1994). Effect of rounding errors on the variable metric method. Journal of Optimization Theory and Applications, 80()), 175-179.
357
[42] Styblinski, M. A., Tang, T. S. (1990). Experiments in nonconvex optimization: stochastic approximation with function smoothing and simulated annealing. Neural Networks, 3(4), 467-483.
359
380
[43] Deb, K., Thicle, L.., Laumanns, M., & Zitzler, E. (2002, May). Scalable multi-objective optimization
test problems. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No. 02TH8600) (Vol. I, pp. 825-830). IEEE
382
363
44
Kumar, S. (2020). Balancing a CartPole System with Reinforcement Learmii A Tutorial, arXiv preprint arXiv:2006.04900
384
365
[45]
LeCun, Y., Bottou, I.., Bengio, Y., & Haffner, (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11). 2278-2324.
366
387
[46]
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E.,, Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, I(4), 541-551.
389
[47] Rumelhart, D. E.. Hinton, G. E., Williams, R. (1986). Learning representations by back back-propagating errors. nature, 323(6088). 533-536.
370
[88]
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-
372
[49]
Cho, K. Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint 140.::114991153
373
[50] Vaswani, A., Shazcer, N., Parmar, N., Uszkoreit, J., Jones, L.,, Gomez, A. N., & Polosukhin, I. (2017).
Attention is all you need. Advances in neural information processing systems, 30.
376
[SI] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929. 1958.
377
11

---

379
[52]
Blank, J., & Deb, K. (2020). Pymoo: Multi-objective optimization in python. IEEE Access, 8, 89497- 89509.
381
[53]
Iman, R. L., Davenport, J. M., & Zeigler. D. K. (1980). Latin hypercube sampling (program user's 's (LH,, in FORTRAN] (No. SAND-79-1473) Sandia Labs.. Albuquerque, NM (USA).
382
383
[54]
Brockman, G., Cheung. V., Pettersoo,, L.. Schneider, J., Schulman, J., Tang. J., Zaremba, W. (2016).
384
Openai gym. arXiv preprint arXiv: 1606.01540
[55] Ros, R., & Hansen, N. (2008, September), A simple modification in CMA-ES achieving linear time and space complexity. In International conference on parallel problem solving from nature (pp. 296-305). Springer, Berlin, Heidelberg.
386
387
388
[56]
Akimoto, Y., Auger, A., & Hansen, N. (2014, July). Comparison natural gradient optimization in high dimension. In Proceedings of the 2014 Annual Conference Genetic and Evolutionary Computation
389
390
(pp. 373-380).
381
[57] Loshchilov. L. (2017). LM-CMA: An alternative to L-BFGS for large-scale black box optimization. Evolutionary computation, 25(1), 143-171.
393
[58]
Blank, J. & Deb. K. (2022). pysamoo: Surrogate-Asiinn Multi-Objective Optimization in Python. arXiv preprint arXiv:2204.05855
394
395
[59]
Tian, Y., Wang. H.. Zhang, X., & Jin, Y. (2017). Effectiveness and efficiency of non-dominated sorting for cvolutionary multi-and many-ohjective optimization. Complex & Intelligent Systems, 3(4), 247-263.
396
[60] Long, Q. Wu, X., & Wu, C. (2021). Non-dominated sorting methods for multi-objecttvve optimization: review and numerical comparison. Journal of Industrial & Management Optimization, 17(2), 1001.
398
Checklist
399
400
1. For all authors...
401
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
403
(b) Did you describe the limitations of your work? [Yes] See discussion and conclusion
404
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
405
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes]
407
2. If you are including theoretical results...
408
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
409
(b) Did you include complete proofs of all theoretical results? [N/A]
410
3. If you ran experiments..
411
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] See supplemen- tary for implementation details
412
413
414
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See supplementary for implementation details
416
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See supplementary for experimental details
417
418
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See supplementary for experimental details
419
420
421
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
422
(a) If your work uses existing assets, did you cite the creators? [Yes] See references
423
(b) Did you mention the license of the assets? [Yes] See supplementary
424
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes]
426
427
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes]
428
5. If you used crowdsourcing conducted research with human subjects...
12

---

430
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
431
432
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount
435
spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "mQZJRhuqww8", "reviewer": "Reviewer_wE4h", "summary": "This paper proposes a combination of deep generative model and evolution strategy, targeting high-dimensional multi-objective black-box optimization. The approach consists of N neural networks, called critic networks, where N is the number of objective, and p generative models, from which solutions to the original problem are generated by giving latent vectors z. The critic networks are trained to approximate the objective functions and the generative models are trained on the critic networks to generate a better solution.", "questions": "The algorithm description looks strange to me. What do you mean by “while n = N do” and “while m = M do”?\n", "limitations": "See the comment above. The lack of literature review and the lack of experimental comparison.", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 1, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Literature Review:\n\nThough this paper targets high-dimensional black-box optimization, there is almost no literature review on this topic. For example, in evolution strategies, which is the basis of the proposed approach, different approaches have been proposed for high-dimensional problems, such as SEP-CMA [1], VD-CMA[2], LM-CMA[3], etc. \n[1] A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, PPSN 2008\n[2] Comparison-Based Natural Gradient Optimization in High Dimension, GECCO 2014\n[3] Lm-cma: An alternative to l-bfgs for large-scale black box optimization, Evolutionary Computation (2017)\n\nMoreover, though one of the contributions of this paper is a surrogate-assisted evolution strategy, no literature review has been performed on this line. See for example [4, 5] and references therein.\n[4] A global surrogate assisted CMA-ES, GECCO 2019.\n[5] pysamoo: Surrogate-Assisted Multi-Objective Optimization in Python, arxiv (2022)\n\nMulti-objective optimization has also not been reviewed. \n\nExperimental Evaluation:\n\nThe baseline approaches are very limited, as the literature review has not been performed well. NSGA-II is a traditional GA based approach for mainly bi-objective optimization. A lot of improvements are reported. Surprisingly, it has been applied to a single objective optimization in Figure 3, which is not understandable. CMA-ES is basically an approach for single objective optimization, and there are several extensions of CMA-ES to multi-objective optimization, but it has not been explained which algorithm is used. \n\nNo surrogate-assisted evolution strategies or other surrogate-assisted approaches are compared in the experiments. No state-of-the-art evolutionary multi-objective optimization approaches such as MOEA/D or NSGA-III are compared. I suggest you to look at BBOB BiObjective Testbed and algorithms reported there (https://numbbo.github.io/data-archive/bbob-biobj/).\n\nBecause of the lack of the literature review and comarison with the state-of-the-art approaches, I can not judge the goodness of the proposed approach.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "bUbqRPGHKq", "reviewer": "Reviewer_bPMJ", "summary": "This work proposed a new Generative Evolutionary Optimization method combined Evolutionary Strategies and Generative Neural Network, aiming to address stochastic multi-objective high-dimensional optimization problems. The proposed GEO is verified on a set of benchmark optimization functions (both single-objective and multi-objective), and a black-box optimization of neural networks in reinforcement learning and image recognition. ", "questions": "1. Please name a few real-world applications involving stochastic multi-objective high-dimensional optimization problems. The current experiment studies seem to consider 4 benchmarks each touching one or two aspects of the above problem features. It is interesting to see an empirical case study on the stochastic multi-objective high-dimensional optimization problem.\n\n2. How Evolution Strategies contribute to GEO? Section 3.4 states that ES can help stablize the training, but why is this case? Please clarify this part. Besides, the ablation studies of verifying the importance of ES in GEO is missing. \n\n3. How generative model generates candidate solutions? The review suggests first introducing the general framework of GEO at the begining of section 3, so readers can easily get how GEO works. For the current manuscript, it is hard for those whoe do not farmiliar with L-GSO and EGAN to understand the GEO, even for the basic idea of it.\n\n4. What is network structure and hyper-parameters of the generative models and critic models?\n\n5. For Cartpole-v1, why not compare GEO with RL methods if RL methods perform better? Also, it would be much better to briefly introduce what is Carpole-v1 scenario.\n\n6. It is not intuitive what does section 4.4 want to show about. \n\n\n\n", "limitations": "No. ", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The reviewer did not see similar work for addressing the stochastic multi-objective high-dimensional optimization problem. Though through related works, there are L-GSO, EGAN and other works, the authors clarify their differences inbetween.\n\nThe organization of the technique part of this manuscript should be considerably improved. For the reviewer, it is quite difficult to get how GEO exactly works and the introduction of sub-routine techques are quite vague, e.g., how generative model generates candidate solutions. Besides, the empirical studies section is also problematic. It is unclear why Cartpole-v1 and LeNet-5 are used for verification and how to fairly assess the quality of the GEO based on them. For example, why not compare GEO with RL methods on Cartpole-v1? What is Cartpole-v1? How representative it is? Why we have to regard it as a black-box optimization problem if some RL methods can solve it better with MDP structure. And it is also unclear what information has been delivered to the readers in section 4.4.\n\nThough this work focuses on the  stochastic multi-objective high-dimensional optimization problem, it is not clear what real-world problem is with these features and there seems no empirical studies on the problem with all these features. Thus, it is difficult to judge how significant the proposed GEO is.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RItkzVUquQ_", "reviewer": "Reviewer_uC1V", "summary": "The paper presents a new black-box optimization method called GEO. It is designed for solving stochastic and multi-objective black-box large-scale optimization problem. \n\nGEO has two stages: evolution and surrogate/critic network training. The evolution pool stores a number of generators ranked by fitness scores. The backprogragation from the critic network is the mutations generators. \n\nWhen comparing with baseline methods including L-GSO, BO,  CMA-ES and NSGA-II, GEO shows competitive results in optimization functions, finding Pareto fronts, etc. ", "questions": "I had a hard time to understand the Algorithm 1 on page 3. What is N and M? Should \"while n = N \" be \"while n < N\"?\n\nIn Figure 3, what is BO package used? \n\nIn Figure 5, what does The noise is restored for visualization mean?\n\nWhy does GEO fail to find the Pareto front for ZDT2?\n\nWhy the generated images can not be recognized but still have high scores?\n", "limitations": "I think the authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strengths of the paper include:\n1) It proposes an interesting method that utilizes generative neural networks and evolution strategy. Evolution helps to stabilize the training of critic networks while the critic networks provide efficient mutations. \n2) Experiment results are better than baseline methods in many cases.\n\nThe weaknesses of the paper include:\n1) For generative neural networks, stabilizing training often is very challenging, and training can be expensive. \n2) Due to the complexity of generative neural networks, it will be difficult for the proposed method to scale to handle high dimension problems. \n3) The organization of the paper can be better. The pictures and algorithm should be close to their description pages. The pictures' quality is not high. I failed to print the paper using two different printers. There are something wrong with the format to this paper. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QKlduAyTPTW", "reviewer": "Reviewer_uzBL", "summary": "The authors introduce a new black-box optimization technique that works for multi-objective, high-dimensional functions. They do this by learning a critic function and a population of \"generator\" functions. The outputs of the generator are used to train the critics. The generators backpropagate through the critics in order to improve their scores. The use of evolution strategies is key to stabilizing the training of the critics.", "questions": "1. Line 107 the authors write: \"We experiment with both random variables and pseudo-random constants as input feeds z\". Where are the results? If it is in a plot or figure, please reference it (I could not find which one it is referring to). If they are not, please at least mention what the results of the experiments were. What do you mean by \"pseudo-random constants\"? How is it different from a \"random variable?\" Are the random variables somehow not generated by a \"pseudo-random\" number generator? If $z$ is constant throughout training, then is it really a \"generative\" method? How would optimizing $G$ be any different than optimizing \"x\" directly?\n\n2. Line 140 the authors write: \"We experimented with both stochastic and non-stochastic generators\". Again, I'm not sure where in the paper the results are. In particular, this is an interesting experiment: If non-stochastic generators worked, then this wouldn't really be a generative process. Can you get away with not using generators and instead just having a large population?\n\n3. How do you measure pareto efficiency in the paper? In the pseudo-code \"ParetoEfficiency\" is written. In the paper, on Line 117 the authors write that data are \"sorted and selected by Pareto efficiency\". How exactly is this calculated?\n\n4. Similar to question 3, there are simply many parts of the paper that are poorly explained, so much so that I cannot list out every question that I have. Here are a few:\n\nA. Line 19: I understand that there are multiple objectives to optimize, but how are these objectives combined? The authors simply write \"Optimize\" without explanation. Do you take the min? Do you take the sum? If they're combined, how is it different from a single objective? Or instead, are you looking for multiple $X$ / trying to find the best $X$ for each of the $m$ functions?  I assume you take the min, but this is not made clear in the paper.\n\nB. Line 95: \"GEO uses only searched points without prepared data\". I think I have a good idea of what this means, but it is not made explicitly clear at all. What do you mean by \"prepared data?\"\n\nC. Line 113: \"The critic network learns variable $x$ in a global region. Global training is essential\". What does this mean? What would be the alternative approach? Again, while I think I have a decent guess of what the authors are trying to convey, it is not completely clear.\n\nD. In Cartpole-V1 is the random seed fixed? My understanding is that cartpole's initialization is random, so observing the state is required.", "limitations": "The authors seem aware of the limitations of the work. They mention the \"Pareto front collapsing problem\" and the \"GPU memory consumption problem\".", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality:\n\nThe paper seems to be original, though I'm not extremely familiar with the SOTA on multi-objective high-dimensional blackbox optimization. \n\nQuality:\n\nThe paper has interesting experiments. However, there are many simple baselines that seem relevant for blackbox optimization of high-dimensional algorithms. In particular, some well-known works include:\n\nSalimans, Tim, et al. \"Evolution strategies as a scalable alternative to reinforcement learning.\" arXiv preprint arXiv:1703.03864 (2017).\n\nSuch, Felipe Petroski, et al. \"Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning.\" arXiv preprint arXiv:1712.06567 (2017).\n\nSehnke, Frank, et al. \"Policy gradients with parameter-based exploration for control.\" International conference on artificial neural networks. Springer, Berlin, Heidelberg, 2008.\n\nIn particular, OpenAI ES and PGPE are similar to this approach in that they estimate a gradient. These papers could be a more fair comparison since the baselines used in the paper are notoriously poor at high-dimensional optimization. Furthermore, the methods are fairly unsophisticated, making them easy to compare to.\n\nFurthermore, I'm not sure what the purpose of the LeNet-5 experiments is, seeing as there is no baseline comparison. \n\nClarity:\n\nThe paper is poorly structured and written to the point where it is quite difficult to understand. See the \"Questions\" section for the many points of uncertainty. \n\nThere are numerous grammatical errors that make it difficult to read. In particular, the paper often either misuses or is missing articles (words like \"the\" and \"a\").\n\nThe pseudocode uses \"while n=N\", which is not at all clear. I'm assuming that they mean \"for n $\\in$ N\".  \n\nThe authors often use imprecise language, which can cause confusion. For example, in Line 119 they write about \"accidentally-high-scoring elements\". I'm assuming that they are referring to points that are high due to stochasticity -- however, calling those \"accidental\" is imprecise/incorrect.\n\nSignificance:\n\nThe approach is interesting and seems to perform well -- however, with the missing baselines and lack of clarity on the details, it is very difficult to evaluate its significance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject