Title: Generative Evolutionary Strategy For Black-Box Optimization
Venue: NeurIPS 2023
Accepted: no
Keywords: Non-convex optimization

black-box optimization
Abstract:  Numerous scientific and technological challenges arise in the context of optimization, particularly, black-box optimization within high-dimensional spaces presents significant challenges. Recent investigations into neural network-based black-box optimization have shown promising results. However, the effectiveness of these methods in navigating high-dimensional search spaces remains limited. In this study, we propose a black-box optimization method that combines an evolutionary strategy (ES) with a generative surrogate neural network (GSN) model. This integrated model is designed to function in a complementary manner, where ES addresses the instability inherent in surrogate neural network learning associated with GSN models, and GSN improves the mutation efficiency of ES. Based on our experimental findings, this approach outperforms both classical optimization techniques and standalone GSN model

Abstract

Numerous scientific and technological challenges arise in the context of optimization, particularly, black-box optimization within high-dimensional spaces presents significant challenges. Recent investigations into neural network-based black-box optimization have shown promising results. However, the effectiveness of these methods in navigating high-dimensional search spaces remains limited. In this study, we propose a black-box optimization method that combines an evolutionary strategy (ES) with a generative surrogate neural network (GSN) model. This integrated model is designed to function in a complementary manner, where ES addresses the instability inherent in surrogate neural network learning associated with GSN models, and GSN improves the mutation efficiency of ES. Based on our experimental findings, this approach outperforms both classical optimization techniques and standalone GSN models.

1 Introduction

Black-box optimization plays a vital role in both science and technology; however, it has long been an unresolved problem particularly for high dimensional problems. While low-dimensional problems, which have dimensions lesser than 100, can be optimized easily, high-dimensional optimization problems pose more significant challenges. In specific cases, such as convex functions, classical algorithms such as Evolution Strategies (ES) [7-20] and others [4-6] can efficiently tackle highdimensional optimization problems. However, their efficiency tends to decline rapidly when faced with general black-box problems characterized by high dimensionality and non-convexity.

Furthermore, in high-dimensional optimization problems, the number of function calls inevitably grows proportionally with the dimension size. Consequently, maintaining O ( N ) time complexity is crucial in preventing the optimization process from failing owing to rapidly increasing computation time. In this context, algorithms such as the Bayesian optimization [1-3], which exhibit non-linear complexity, are at a significant disadvantage. Conversely, neural networks offer a promising solution to this problem. The field of artificial intelligence has demonstrated their considerable benefits in managing data within high-dimensional spaces, such as images or language models, while preserving linear time complexity.

Recently, GSN-based approaches, inspired by Generative Adversarial Networks (GANs) [30-32], 29 have emerged to tackle the black-box optimization problem, offering a novel solution for high30 dimensional, non-convex problems. However, in contrast to GANs, Generative Surrogate Neural 31 networks (GSNs) face a significant challenge with learning stability in the surrogate model, and the 32 performance of GSN-like algorithms remains limited to just hundreds of dimensions. 33

Hence, addressing the training instability problem is crucial for enhancing the performance of GSN34

based algorithms[21, 29]. Our research aligns with this perspective, and in this work, we introduce 35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

a method called Generative Evolutionary Optimization (GEO). GEO arises from the cooperative interaction between two linear complexity algorithms, ES and GSN. Furthermore, ES contributes to

the stability of the surrogate network training for GSN while GSN enhances the mutation efficiency of ES, leading to their complementary functioning.

In this study, we designed an algorithm to accomplish five goals: optimizing non-convex, highdimensional, multi-objective, and stochastic target functions, while preserving O ( N ) complexity.

In the following chapters, we explore GEO's design and the methodologies used in addressing its most significant challenge: training instability. The Related works chapter discusses two prior works, Local Generative Surrogate Optimization (L-GSO) [21] and Evolutionary Generative Adversarial Network (EGAN) [22], which serve as the foundation for GEO's core concepts. In addition, emphasis on the importance of the generator network is also discussed. The Methods chapter, we combines ideas from L-GSO and EGAN studies to clarify the aspects of GSN that require improvement and how they can be addressed. The Results chapter presents the findings from the test function experiments. Because GEOis a combination of ES and GSN, we assume a close relationship with ES, and therefore compare it against non-convex test functions commonly used in ES, such as ZDT [42], Styblinski-Tang [41], Rosenbrock [39, 40], Rastrigin [35-37], and Ackley [38] test functions. The experimental results show that GEO outperforms traditional ES and GSN as dimensionality increases, thus enabling optimizations of approximately 10,000 dimensions.

As mentioned earlier, we excluded non-linear algorithms from the comparison because our aim is to maintain O ( N ) complexity. This makes a direct comparison with Bayesian optimization, another significant branch of black-box optimization, difficult. Consequently, the issue is revisited in the Conclusion chapter.

2 Related works

Before delving into the structure of GEO, we will first introduce some related works, focusing on GSN and GAN algorithms. By examining earlier optimization approaches that utilized neural networks, we can gain valuable insights. As GEO is inspired by an L-GSO study (a type of GSN) and an EGAN study (a type of GAN), we will provide a brief overview of both algorithms, discussing their strengths and weaknesses for a better understanding.

2.1 L-GSO

Local Generative Surrogate Optimization (L-GSO) is a type of GSN that approaches the problem using a local surrogate network and a local gradient. To better understand this, let's think of a situation where we need to optimize a target function F ( x ) within an optimization space x ∈ X and find the best point x . If we identify x 0 as the optimal point at some stage, L-GSO samples the local space around x 0 and calculates pairs [ x ′ , F ( x ′ )] , where x ′ = x 0 + ϵ and | ϵ | << 1 . It is important that ϵ is sufficiently small so that we only sample within a space close to x 0 . From this data, L-GSO trains a surrogate network, C , which acts as a local surrogate model with information around x 0 and can generate a local gradient. After training the surrogate network, the generator network G is trained using C . Let p = C ( G ( z )) , where x = G ( z ) and z is an input seed. G is trained with a loss function ∓ p that either increases or decreases the prediction p . Finally, the trained generator suggests a search point x , and the iterative process continues.

This way, G is trained to be a generator that produces optimal (or near-optimal) points x , assuming that C simulates the local distribution accurately enough. Meanwhile, as the data points are focused within a localized area, the surrogate network can benefit from a stable training data region, which enables accurate gradient prediction. Another advantage is that the data information is retained in the surrogate network C , allowing lesser amount of data generation required when predicting the local gradient at new points.

Although the approach of utilizing GSNs in L-GSO is quite innovative, it does have some limitations. 82 The primary constraint is that it is only applicable to single-objective function problems. The optimal 83 point x 0 mentioned earlier is just a single point. However, if the optimal points (the Pareto front) 84 consist of multiple points, the distance between these optimal points and the data sampling space 85 (which the surrogate network must learn) will no longer be local. This undermines the fundamental 86 premise of L-GSO. Hence, the challenge is that the optimal point of an n -objective function features 87

a Pareto front in n -1 dimensions. For example, in a two-objective function, the Pareto front 88 forms a line, typically containing infinitely many non-dominated points that are distant from one 89 another. Consequently, the locality of L-GSO becomes unsuitable for n -objective problems that 90 require non-dominated sorting [58, 59]. 91

92

93

94

95

96

97

98

99

100

101

The second problem with this algorithm is that its performance may be significantly influenced by the interaction between the hyperparameters and the test function. Specifically, the combination of the sampling hypercube size ϵ and the test function characteristics can significantly impact the algorithm's optimization performance. For instance, if the test function is convex, estimating the local gradient with any ϵ is generally not a problem; however, this can often be challenging for non-convex functions. Around the local optimum, if ϵ is smaller than the size of the localized well, the algorithm can hardly escape the local optimum. Conversely, if ϵ is too large, the local gradient cannot be accurately estimated. The interplay among the type of test function, the location of the local optimum point, and ϵ is substantial, making it difficult to determine the appropriate value of ϵ for the test function in advance; thus posing a disadvantage for black-box problems.

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

Conclusively, L-GSO effectively handles high-dimensional spaces using surrogate networks and improves the stability of surrogate network training through locality. However, it is clear that nondominated sorting for multi-objective problems is not feasible, and the relationship between the hyperparameter ϵ and the test function might be too strong.

2.2 EGAN

The Evolutionary Generative Adversarial Network (EGAN) integrates ES to improve GAN performance. Although this method does not focus on black-box optimization, its algorithmic structure is similar to GEO.

Usually, GANs are trained with one generator and one discriminator (critic) network, which alternate. In EGAN, a scenario is considered where there is only one discriminator network; however, with multiple generator networks. The main idea of this study is to rank the generator networks using an evolutionary strategy and keep only the suitable networks. By using the prediction of discriminator C as the fitness score, the generators G that can increase C ( G ( z )) at each iteration survive. This process incorporates ES into the GAN, and it has been established that the introduction of ES reduces mode collapsing, which is a common issue in GANs.

Although EGAN is not directly related to optimization problems, we can gain valuable insights into improving GSN based on EGAN's concepts. Because the generator and surrogate network structure in GSN is similar to the generator and discriminator network structure in GAN, we can adopt EGAN's strategy in GSN. This forms the core basis of our research, GEO. However, the main difference between the two algorithms is that GAN operates without evolution, whereas GSN typically diverges and fails when it consists only of a single generator and a surrogate network. In the Methods section, we discuss how the working mechanism of GSN can change owing to the introduction of ES.

2.3 GLOnet

Global Optimization of dielectric metasurfaces using a physics-driven neural network (GLOnet) [2327] is a study to optimize devices in electromagnetic systems. The study investigates the optimization of device structures within a specific electromagnetic system using neural network techniques.

In this study, x serves as device design parameters, and the goal is to find the optimal value of x that maximizes the objective value F ( x ) of the simulator F . The algorithm finds x = G ( z ) , where z is the input seed, and the generator G is trained by backpropagating the gradient from the simulator. Technically, this is not a black-box optimization, as it receives the analytic gradient information directly from the simulator. Therefore, a surrogate network is not required.

An important implication of the GLOnet study is the necessity for a deep generator network. One might assume that since we have a gradient, we can optimize x through direct gradient descent. However, this study demonstrates that employing a generator network is more advantageous than updating x directly without a generator. In fact, this also holds true for GSN studies. The importance of deep generators is not emphasized in many GSN studies; however, it can be argued that it is omitted because they are inspired by GANs, and the significance of a deep generator is implicitly understood.

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

a)

b)

GAN

GSN

Generator

True

data

Fake

data

Critic

Generator

Search

points

Critic

Black

box

Near-optimal region

Suboptimal region

Divergence direction

Generator

Critic

Next search points

(Suboptimal)

Gradients

(Divergence direction)

c)

(Fixed)

3 Methods

In the previous chapters, we examined L-GSO, EGAN, and GLOnet. Our goal is to adopt the surrogate network training strategy of L-GSO while enabling multi-objective optimization, which cannot be implemented in the local surrogate model, and also reducing the excessive dependency between the hyperparameters and the test functions. Meanwhile, we adopt the evolution strategy of EGAN; however, unlike GAN, GSN has an inherently unstable structure; hence, we need to consider how to address this instability. Additionally, EGAN assumes a single-objective fitness score; hence, we need to modify it to work with multi-objective functions.

3.1 A training instability

First, let's explain why surrogate training in GSNs is particularly unstable. The structure of GSNs is almost identical to GANs, hence it might seem that, just as GANs are highly successful in the fake data generation task, GSNs should also be successful. However, in practice, when we run a GSN model with just one generator network and one surrogate network, without employing any special tricks, we encounter a situation where they diverge to infinity almost immediately after starting, in most test cases. This is neither the desired result nor can we consider it being optimized. This is why GSN studies incorporate some kind of special trick.

In addition, the crucial difference between GANs and GSNs lies in the presence or absence of true data. GANs rely on fixed true data; for instance, if the task is to generate images of human faces, a large dataset of real human face photos is required. Discriminator networks possess a significant amount of fixed true data initially and learn additional fake data during training, providing a stable training region. Conversely, GSNs, which are used for black-box optimization, do not have true data. Consequently, we must start from scratch, with no prior information. The first Pareto front points that have been explored, and the surrounding points, might correspond to true data. However, the amount of data near the first Pareto front is insufficient because the first Pareto front in the objective problem is constantly changing and data is collected on-the-fly. To draw an analogy, the GAN represents a situation in which the target is anchored by true data, whereas the GSN represents a situation where the target is floating without an anchor.

This is why the critic network (discriminator network) in GANs and the critic network (surrogate network) in GSNs have similar learning structures; however, they yield completely different results. To summarize the erroneous learning process of GSN's critic network, we describe its workings as follows:

The critic network begins training with insufficient data.

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

The training data has minimal information about the Pareto front, which prevents the network from learning anything meaningful.

As the critic network is trained improperly, the gradient it provides fails to guide x towards the Pareto front.

The generator, using incorrect gradient information, produces a point unrelated to the Pareto front when suggesting the next x .

The critic network is then retrained using the poorly generated x .

This leads to a vicious cycle between the generator and the critic network, a situation that does not occur in GANs owing to the presence of true data.

One solution to this problem is to create a data region that corresponds to the true data region found in GANs. In optimization problems, the most crucial information is concentrated at the first Pareto front. Hence, it is necessary to sample the area around the first Pareto front and use it as training data for the critic network. L-GSO addresses this issue by intensively sampling only the local region around x 0 , representing the zero-dimensional first Pareto front, and supplying that data to the critic network. However, as mentioned in the related works section, the local sampling method cannot be applied to n -objective functions with n > 1 . Consequently, we need an approach that is similar to the local sampling method and can also accommodate the multi-objective target functions.

Hence, we suggest using ES. The idea is to keep generators G that produce x values close to the first Pareto front and discard those that are farther away. By selecting G through a non-dominated sorting process, we ensure that only [ x, G ] pairs generating search points near the Pareto front survive at each iteration. Even if the critic network initially learns the incorrect region, as iterations progress, the probability of generating a search point x near the Pareto front increases. Eventually, the critic network will have a stable training data region near the Pareto front; thus breaking the vicious cycle and achieving our desired outcome: stability in critic network training for multi-objective targets.

GEO was developed with the notion that ES complements GSN; however, this idea can also be reversed. As previously mentioned, from the ES perspective, the algorithm consists of mutation and (non-dominated) sorting based on fitness scores, with generator networks being the targets of mutation. In this case, mutation occurs through backpropagation from the critic network, which can be more efficient because the neural network has learned information about the Pareto front. Therefore, from the ES perspective, GSN serves as an auxiliary means to enhance the mutation efficiency of ES.

Conclusively, GEO functions in such a way that GSN complements ES, and ES complements GSN. By doing so, it integrates the strengths of both GSN and ES while mitigating their weaknesses, particularly effectively addressing the instability problem of GSN. Moreover, since both GSN and ES are O ( N ) algorithms, GEO is able to maintain O ( N ) complexity.

However, it is important to emphasize that both GSN and ES are exploitation-oriented algorithms in terms of exploit and explore strategies. Bayesian optimization, another significant branch of black-box optimization, offers a powerful feature by estimating uncertainty and incorporating it into the next step search. From that perspective, both ES and GSN lack the uncertainty estimation aspect, and even with the addition of supplementary exploration strategies, they may not reach the same level of robust exploration performance as the Bayesian optimization. Consequently, GEO, similar to GSN and ES, cannot guarantee global optimization even though the number of function calls N approaches infinity. Nevertheless, GSN, ES, and GEO are free from the non-linear complexity problem that Bayesian optimization encounters. Hence, it is clear that Bayesian optimization and GEO have distinctly different optimization goals and conditions in which they are best applied.

3.2 Operation steps

The following are the operation sequences for GEO in the context of n -objective black-box optimization F = ( f 1 , f 2 , ..., f i , ..., f n ) . G : generator network, C i : i th Critic network, z : input seed of generator, x : search point of search space X .

Pretraining steps:

Prepare a set of generators and n critic networks. Each critic network predicts one corresponding objective. The networks have not been trained yet.

224

225

226

227

228

229

230

Generator 1

Score 1

Generator 2

Score 2

Generator 3

Score 3

Generator i

Score i

Generator N

Score N

Sampled

Generator (G)

Trained

Generator (G

ʹ

)

Input

variable

x=G

ʹ

(z)

x

Score

Score

Bu

ff

er

memory

Critic

Network (C)

Trained

Generator

Black

Box (F)

Score

s=F(x)

1

2

3

4

5

6

7

8

Evolution

Strategy

Generative

N.N. model

Stable training region

E

ffi

cient mutation

a)

b)

ES cycle

GSN cycle

Prepare the [ x, F ( x )] training set for initializing the critic network. The Latin Hypercube method[52] is generally recommended as the initial sampling method for search point x . However, the generator network can also be initialized using weight initialization techniques for neural networks, such as Xavier or He initialization.

Store the initialized [ x, F ( x )] in buffer memory, which has a maximum length.

Pretrain the critic network using the data stored in buffer memory. For example, C i is trained with [ x, f i ( x )] pairs.

Main iteration steps: 231

232

Randomly sample a few generators from the generator set (evolution pool).

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

Train the generators through backpropagation with the critic network C i . The loss function is -C i ( G ( z )) for the maximization problem. The training of the generators also serves as mutation from the ES perspective. Because there are n critic networks, the single-objective mutation is repeated n times; thus implying that for each sampled G , we make n mutants.

Generate new x ′ = G ′ ( z ) points from the mutated generators G ′ .

Evaluate F ( x ′ ) from the new x ′ and store the pair [ x ′ , F ( x ′ )] in the buffer memory. If the buffer memory's maximum length is exceeded, delete the previously stored data.

Train the critic networks using the data in the buffer memory.

Save the [ G ′ , F ( x ′ )] pair back to the generator set. F ( x ′ ) corresponds to the multi-objective fitness score.

The number of generators in the evolution pool has increased with the newly stored mutated G ′ . Perform a non-dominated sort on them based on their fitness scores. Predetermine a maximum number for the generator set and retain only the generators with high fitness (top Pareto-front data).

Repeat the iteration.

The configuration of the critic network might have been designed to allow a single critic network to predict multiple target objectives. However, we separated the critic network independently for each objective to minimize correlation, under the assumption that it is more common for black-box problems to have independent objective targets. By dividing the critic network into n parts and applying backpropagation separately, it behaves as if the single-objective problem is being run n times independently. Nevertheless, it can be used for multi-objective optimization because the results are combined and ranked using non-dominated sorting.

In addition, if the target function is stochastic, age evolution can be incorporated into the nondominated sorting step. In age evolution, we can store the time order information of generators together and remove a few of the oldest generators before performing non-dominated sorting.

a)

b)

GEO

BO

GA

GEO single layer

CMAES

GEO

BO

GA

GEO single layer

CMAES

GEO

BO

GA

GEO single layer

CMAES

GEO

BO

GA

GEO single layer

CMAES

The provided explanation outlines the basic algorithmic structure of GEO. As described, the operation 258 of GEO is achieved when both the ES-direction cycle and the GSN-direction cycle work together 259 simultaneously. 260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

4 Results

Previously, we mentioned that L-GSO has a particularly strong correlation with the hyperparameter set and test function. However, all black-box optimization problems exhibit a significant correlation between the algorithm type, hyperparameter set, and target function, thus leading to entirely different results with even slight changes. Comparing the performance of optimization algorithms can be challenging precisely because of this reason. It is impossible to prove which hyperparameter set is optimal for a specific test function. Even if the optimal hyperparameter set for a particular test function is found through repeated experimentation, it would constitute overfitting to that specific test function and would no longer be considered black-box optimization. Therefore, a straightforward performance comparison between algorithms for the final results is susceptible to cherry-picking problems. Therefore, this study focuses on describing how the trend of optimization performance depends on the dimension, rather than simply comparing the values of the final results.

In the single-objective function, it is crucial to compare GEO with L-GSO. To ensure a fair comparison, we matched L-GSO's network configuration with GEO's, referring to it as Local Surrogate Model (LSM). Examining the performance changes of LSM and GEO as dimensions increase, LSM is more efficient at smaller dimensions. However, its performance declines significantly as the dimensions grow larger. This situation is also evident in classical ES and Bayesian optimization (based on Gaussian process). Although GEO is less efficient at lower dimensions, its efficiency increases as the dimensions grow, outperforming the other methods.

In the related works chapter, we discussed the importance of deep generators. To investigate this 280 further, we conducted an experiment with GEO using a single-layer FC network as the generator. 281 (However, the critic network remains a deep neural network.) The experiment demonstrated that the 282 shallow layer GEO experienced a significant decrease in optimization performance. 283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

a)

b)

We also conducted high-dimensional experiments in multi-objective functions. According to the experimental results, as the dimension increases, the performance of the comparison group declines rapidly, whereas the performance of GEO is relatively well maintained. By the time it reaches 8192 dimensions, there is a significant difference in the final results. The comparison group tends to get trapped in local optima easily; however, GEO manages to escape local optima and makes considerable progress.

However, a limitation of GEO can be identified in one of the experimental results. The ZDT2 function has a concave Pareto-front shape. In this case, although GEO succeeds in optimization, it fails to find the entire shape of the Pareto-front and tends to collapse toward one side. We also experimented the stochastic functions by adding normal random noise to the ZDT test functions. The results for the stochastic multi-objective functions exhibit similar characteristics. Here, the performance of GEO appears to be better compared to the baseline methods; however, it also shows a similar tendency to collapse toward one side while optimizing the ZDT2 function. In the case of ZDT1 and ZDT3, although some lines of the Pareto-front are found, the collapsing tendency is stronger compared to non-stochastic functions.

Summarily, as we intended, GEO successfully overcomes the difficulties of critic network training 299 in GSN and demonstrates stable performance. Because it does not assume locality, it shows excel300 lent performance in multi-objective functions and operates effectively in stochastic environments. 301

Although in cases with low dimensions, GEO's performance is lower compared to traditional algo302 rithms, possibly due to too large neural network size we used; however, as the number of dimensions 303 increases, it shows better performance than other algorithms. 304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

5 Conclusion

We can observe that GEO successfully accomplishes our five primary objectives: optimizing nonconvex, high-dimensional, multi-objective, and stochastic target functions while maintaining O ( N ) complexity.

In the Related works chapter, we examined insights into L-GSO and EGAN and combined them to create the foundational concept behind GEO. GSN has problems with unstable critic network training, and to address this, methods that focus on sampling around optimal values can be considered. Although L-GSO employs such an approach, introducing locality to implement it limits the algorithm to single-objective functions and results in excessive sensitivity to hyperparameters. Hence, we introduced ES combination strategy to create a stable training data region near the Pareto-front. This method can be used for multi-objective problems and also resolves the hyperparameter sensitivity problem because it does not assume a separate local ϵ size. Moreover, GSN and ES work complementarily, enhancing each other's strengths and compensating for weaknesses, leading to improved efficiency.

As indicated in the Results chapter, GEO appears to be more effective in high dimensions rather than low dimensions. For instance, ES is an algorithm that is advantageous in low dimensions when a large number of function calls is available, whereas Bayesian optimization is favorable in low dimensions with limited number of function calls. Conversely, GEO might have an advantage when many function calls are possible in high dimensions. This observation suggests that GEO has the potential to address optimization areas not covered by existing algorithms.

Because GEO is specialized for high-dimensional non-convex functions, it is worth considering its potential applications in other areas of machine learning. For example, some research trains reinforcement learning (RL) neural networks through black-box optimization [60]. As these techniques require high-dimensional black-box optimization, GEO, which specializes in high-dimensional optimization, could be a viable option.

As previously mentioned in the Results section, one difficulty in black-box optimization research 330 could be the variability in the performance of algorithms. The performance can vary greatly depending 331 on the combination of algorithm type, hyperparameters, and test function type. A certain algorithm 332 and hyperparameter set might be highly effective when targeting a specific test function; however, 333 it may yield poor results for a different test function. Hence, this can lead to biased preparation 334 toward specific test functions, making it easier to cherry-pick results. In the worst-case scenario, one 335 could develop an algorithm specialized for a target test function and fine-tune the algorithm through 336 repeated experiments to obtain good results. Consequently, these results would not be considered 337 genuine black-box optimization outcomes because they utilize prior knowledge gained through the 338 iterative experiments. Therefore, it is challenging to determine the state-of-the-art (SOTA) status in 339 black-box optimization research. Hence, although GEO demonstrates outstanding performance in 340 this study, further research is necessary to determine its performance across various environments 341 and to identify its limitations. 342

343

References

Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning 344 algorithms. Advances in neural information processing systems, 25 . 345

Frazier, P. I. (2018). A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811. 346

Lan, G., Tomczak, J. M., Roijers, D. M., & Eiben, A. E. (2022). Time efficiency in optimization with a 347 bayesian-evolutionary algorithm. Swarm and Evolutionary Computation, 69, 100970. 348

Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The computer journal, 349 7 (4), 308-313. 350

Kennedy, J., & Eberhart, R. (1995, November). Particle swarm optimization. In Proceedings of ICNN'95351 international conference on neural networks (Vol. 4, pp. 1942-1948). IEEE. 352

Hooke, R., & Jeeves, T. A. (1961). 'Direct Search'Solution of Numerical and Statistical Problems. Journal 353 of the ACM (JACM), 8 (2), 212-229. 354

Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. A. M. T. (2002). A fast and elitist multiobjective genetic 355 algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6 (2), 182-197. 356

357

358

359

Deb, K., & Sundar, J. (2006, July). Reference point based multi-objective optimization using evolutionary algorithms. In Proceedings of the 8th annual conference on Genetic and evolutionary computation (pp. 635-642).

360

361

Hansen, N., & Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9 (2), 159-195.

Hansen, N. (2006). The CMA evolution strategy: a comparing review. Towards a new evolutionary 362 computation , 75-102. 363

Price, K., Storn, R. M., & Lampinen, J. A. (2006). Differential evolution: a practical approach to global 364 optimization. Springer Science & Business Media. 365

Runarsson, T. P., & Yao, X. (2000). Stochastic ranking for constrained evolutionary optimization. IEEE 366 Transactions on evolutionary computation, 4 (3), 284-294. 367

Runarsson, T. P., & Yao, X. (2005). Search biases in constrained evolutionary optimization. IEEE Transac368 tions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 35 (2), 233-243. 369

Deb, K., & Jain, H. (2013). An evolutionary many-objective optimization algorithm using reference-point370 based nondominated sorting approach, part I: solving problems with box constraints. IEEE transactions 371 on evolutionary computation, 18 (4), 577-601. 372

Blank, J., Deb, K., & Roy, P. C. (2019, March). Investigating the normalization procedure of NSGA-III. In 373 International Conference on Evolutionary Multi-Criterion Optimization (pp. 229-240). Springer, Cham. 374

Seada, H., & Deb, K. (2015). A unified evolutionary optimization procedure for single, multiple, and many 375 objectives. IEEE Transactions on Evolutionary Computation, 20 (3), 358-369. 376

Vesikar, Y., Deb, K., & Blank, J. (2018, November). Reference point based NSGA-III for preferred 377 solutions. In 2018 IEEE symposium series on computational intelligence (SSCI) (pp. 1587-1594). IEEE. 378

379

380

381

Carvalho, R. D., Saldanha, R. R., Gomes, B. N., Lisboa, A. C., & Martins, A. X. (2012). A multiobjective evolutionary algorithm based on decomposition for optimal design of Yagi-Uda antennas. IEEE Transactions on Magnetics, 48 (2), 803-806.

382

383

384

385

386

Li, K., Chen, R., Fu, G., & Yao, X. (2018). Two-archive evolutionary algorithm for constrained multiobjective optimization. IEEE Transactions on Evolutionary Computation, 23 (2), 303-315.

Panichella, A. (2019, July). An adaptive evolutionary algorithm based on non-Euclidean geometry for many-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 595-603).

Shirobokov, S., Belavin, V., Kagan, M., Ustyuzhanin, A., & Baydin, A. G. (2020). Black-box optimization 387 with local generative surrogates. Advances in Neural Information Processing Systems, 33 , 14650-14662. 388

Wang, C., Xu, C., Yao, X., & Tao, D. (2019). Evolutionary generative adversarial networks. IEEE 389 Transactions on Evolutionary Computation, 23 (6), 921-934. 390

Jiang, J., & Fan, J. A. (2019). Global optimization of dielectric metasurfaces using a physics-driven neural 391 network. Nano letters, 19 (8), 5366-5372. 392

Yang, J., Sell, D., & Fan, J. A. (2018). Freeform metagratings based on complex light scattering dynamics 393 for extreme, high efficiency beam steering. Annalen der Physik, 530 (1), 1700302. 394

Hughes, T. W., Minkov, M., Williamson, I. A., & Fan, S. (2018). Adjoint method and inverse design for 395 nonlinear nanophotonic devices. ACS Photonics, 5 (12), 4781-4787. 396

Jensen, J. S., & Sigmund, O. (2011). Topology optimization for nano-photonics. Laser & Photonics 397 Reviews, 5 (2), 308-321. 398

Molesky, S., Lin, Z., Piggott, A. Y., Jin, W., Vuckovi´ c, J., & Rodriguez, A. W. (2018). Inverse design in 399 nanophotonics. Nature Photonics, 12 (11), 659-670. 400

Faury, L., Calauzenes, C., Fercoq, O., & Krichen, S. (2019). Improving evolutionary strategies with 401 generative neural networks. arXiv preprint arXiv:1901.11271 . 402

Trabucco, B., Kumar, A., Geng, X., & Levine, S. (2021, July). Conservative objective models for effective 403 offline model-based optimization. In International Conference on Machine Learning (pp. 10358-10368). 404 PMLR. 405

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). 406 Generative adversarial nets. Advances in neural information processing systems, 27 . 407

408

409

Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 .

410

411

412

413

414

415

416

417

418

419

420

421

422

423

Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4401-4410).

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 .

Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... & Kavukcuoglu, K. (2016, June). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928-1937). PMLR.

Rastrigin, L. A. (1974) Systems of extremal control. Mir, Moscow.

Hoffmeister, F., & Bäck, T. (1990, October). Genetic algorithms and evolution strategies: Similarities and differences. In International Conference on Parallel Problem Solving from Nature (pp. 455-469). Springer, Berlin, Heidelberg.

Mühlenbein, H., Schomisch, M., & Born, J. (1991). The parallel genetic algorithm as function optimizer. Parallel computing, 17(6-7), 619-632.

Ackley, D. (2012). A connectionist machine for genetic hillclimbing (Vol. 28). Springer Science & Business 424 Media. 425

Rosenbrock, H. (1960). An automatic method for finding the greatest or least value of a function. The 426 Computer Journal, 3 (3), 175-184. 427

Dixon, L. C. W., & Mills, D. J. (1994). Effect of rounding errors on the variable metric method. Journal of 428 Optimization Theory and Applications, 80 (1), 175-179. 429

Styblinski, M. A., & Tang, T. S. (1990). Experiments in nonconvex optimization: stochastic approximation 430 with function smoothing and simulated annealing. Neural Networks, 3 (4), 467-483. 431

Deb, K., Thiele, L., Laumanns, M., & Zitzler, E. (2002, May). Scalable multi-objective optimization 432 test problems. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No. 433 02TH8600) (Vol. 1, pp. 825-830). IEEE. 434

Kumar, S. (2020). Balancing a CartPole System with Reinforcement Learning-A Tutorial. arXiv preprint 435 arXiv:2006.04938 . 436

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document 437 recognition. Proceedings of the IEEE, 86 (11), 2278-2324. 438

LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). 439 Backpropagation applied to handwritten zip code recognition. Neural computation, 1 (4), 541-551. 440

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating 441 errors. nature, 323 (6088), 533-536. 442

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9 (8), 1735-1780. 443

Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine 444 translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 . 445

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). 446 Attention is all you need. Advances in neural information processing systems, 30 . 447

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple 448

way to prevent neural networks from overfitting. The journal of machine learning research, 15 (1), 1929449 1958. 450

Blank, J., & Deb, K. (2020). Pymoo: Multi-objective optimization in python. IEEE Access, 8 , 89497451 89509. 452

Iman, R. L., Davenport, J. M., & Zeigler, D. K. (1980). Latin hypercube sampling (program user's 453 guide).[LHC, in FORTRAN] (No. SAND-79-1473). Sandia Labs., Albuquerque, NM (USA). 454

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016). 455 Openai gym. arXiv preprint arXiv:1606.01540 . 456

Ros, R., & Hansen, N. (2008, September). A simple modification in CMA-ES achieving linear time and 457 space complexity. In International conference on parallel problem solving from nature (pp. 296-305). 458 Springer, Berlin, Heidelberg. 459

Akimoto, Y., Auger, A., & Hansen, N. (2014, July). Comparison-based natural gradient optimization in 460 high dimension. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation 461 (pp. 373-380). 462

Loshchilov, I. (2017). LM-CMA: An alternative to L-BFGS for large-scale black box optimization. 463 Evolutionary computation , 25(1), 143-171. 464

Blank, J., & Deb, K. (2022). pysamoo: Surrogate-Assisted Multi-Objective Optimization in Python. arXiv 465 preprint arXiv:2204.05855. 466

Tian, Y., Wang, H., Zhang, X., & Jin, Y. (2017). Effectiveness and efficiency of non-dominated sorting for 467 evolutionary multi-and many-objective optimization. Complex & Intelligent Systems , 3(4), 247-263. 468

Long, Q., Wu, X., & Wu, C. (2021). Non-dominated sorting methods for multi-objective optimization: 469 review and numerical comparison. Journal of Industrial & Management Optimization, 17 (2), 1001. 470

Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017). Evolution strategies as a scalable 471 alternative to reinforcement learning. arXiv preprint arXiv:1703.03864. 472

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a Generative Evolution Optimization (GEO) algorithm to black-box optimization, introducing. The GEO algorithm is claimed to combine the strengths of Evolution Strategy (ES) and Generative Surrogate Network (GSN) to address the limitations of Bayesian optimization and other existing methods. Some benchmark functions are tested to verify the performance of GEO.\n\nSTRENGTHS:\nOriginality: The paper introduces GEO which combines the strengths of L-GSO and Evolutionary Generative Adversarial Networks (EGAN). I did not see such method before.\nQuality: The authors provide explanations of the GEO method, including its foundational concepts, operation steps, and algorithmic structure. Some benchmark functions have been tested.\nSignificance: The paper addresses a significant challenge in the field of black-box optimization, e.g., the optimization of non-convex, high-dimensional, multi-objective, and stochastic target functions.\n\nWEAKNESSES:\nSome claims and concepts are not adequate, like the O(N) complexity. Without the target of finding global optimal, we can design various methods that achieve O(N) complexity easily.\nSome related works are not cited adequately, like Xavier and He initialization.\nThe experiments seem to be limited to specific test functions. Performance on such few benchmark functions are not convincing\nThe paper discusses the potential application of GEO in other areas of machine learning, such as reinforcement learning. However, it does not provide any empirical evidence or case studies to support these claims. Including real-world applications or case studies could strengthen the paper's significance and practical relevance.\nThe paper does not clearly outline the limitations of the GEO method, which could be beneficial for future research and application of the method.\nThe paper mentions the tendency of GEO to collapse towards one side while optimizing certain functions, but it does not delve into why this happens or how it could be mitigated. A more in-depth analysis of this issue could improve the paper's quality.\n\nQUESTIONS:\nThe paper mentions the tendency of GEO to collapse towards one side while optimizing certain functions. Could you provide more insight into why this happens?\nThe paper mentions that GEO is more effective in high dimensions rather than low dimensions. Could you provide more insight into why this is the case?\nThe paper discusses the potential application of GEO in other areas of machine learning, such as reinforcement learning. Could you provide any empirical evidence or case studies to support these claims?\nWhat are the experimental settings? Why we choose those benchmark functions? What are the measurements of the performance? What are the parameter settings for the algorithms? How many extra computational costs are introduced in the optimization procedure? Those are all unclear.\n\nLIMITATIONS:\nSocietal impact of the work is not discussed in this paper. Furthermore, limitations of the proposed technique are not discussed clearly.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates a new integrated optimization method targeting at black-box optimization within high-dimensional spaces scenario, called Generative Evolutionary Optimization (GEO).\nDifferent from the popular black-box optimization method - Bayesian optimization, GEO exhibits a linear time complexity.\nIntrinsically, Geo a black-box optimization method that combines an evolutionary strategy with a generative surrogate neural network (GSN) model,\nand the two basic components could function in a synergetic manner.\nSpecifically, evolutionary strategy helps to deal with the stability of the surrogate network training for GSN,\nwhile GSN improves the mutation efficiency (sample efficiency) of the former.\nSince the fitness results are combined and ranked using non-dominated sorting in GEO, it can be applied to multi-objective scenario.\nBesides, the age evolution strategy is leveraged to dominated sorting step when the target function is stochastic.\nFinally, the experimental findings reveal that GEO can accomplish the mentioned objectives: optimizing convex, high-dimensional, multi-objective, and stochastic target functions while maintaining O(N) complexity.\n\nSTRENGTHS:\n1. This paper is well-written and easy-to-follow, and the following parts are highlights: technique explanation, limitation analysis, high-level summary.\n2. The technical design (GSN, ES, training stability) is reasonable, and the experimental evaluation is clear.\n3. The key design in the cooperative framework is novel, which integrates the strengths of both GSN and ES while mitigating their weaknesses.\n\nWEAKNESSES:\n1. The specific parameter settings are not clear.\n2. The reviewer suggests that in the discussion chapter, the related multi-objective high-dimensional solutions from Bayesian optimization community could be analyzed in terms of time complexity or efficiency if possible.\n3. The test function in the experimental evaluation is limited, and this hamper the evaluation confidence. As mentioned by the authors, more test functions from different domains (maybe a fair benchmark) should be included to evaluate the performance of GEO.\nIn addition to Ackley, Rosenbrock and Styblinski-Tang, there are many objective function family, including CONSTR, SRN and so on.\n\nQUESTIONS:\n1. How many initial points are sampled in the latin hypercube stage.\n2. What is the setting for the parameters in GEO, e.g., the number of generator. These greatly affect the reproducibility.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
In this paper, a black-box optimization approach is proposed that combines an evolutionary strategy (ES) with a generative surrogate neural network (GSN) model. This integrated model is designed to function in a complementary manner, where ES addresses the instability inherent in surrogate neural network learning associated with GSN models, and GSN improves the mutation efficiency of ES. From the overall view of this paper, the authors basically expressed clearly the point of innovation and the proposed algorithm.

STRENGTHS:
The organization of this paper and the technical details of the proposed method are clear and easy to follow.

WEAKNESSES:
Theoretical derivations and proofs are lacking, and the validity of the method is difficult to be supported.

QUESTIONS:
1.In the introduction, there is no summary of the contribution, and I suggest that the contribution of this work be further emphasized.
2.This paper lacks experimental results and analysis of some arguments. For example, please explain why GEO is less efficient at lower sizes.
3.The writing of the paper could be improved for better description and clarification.
4.The selected algorithms for comparison are not new and are not state-of-the-art, please add the recently proposed multi-objective evolutionary approach for comparison.

LIMITATIONS:
The relevant limitations are described, but not in depth and specific enough.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new method called Generative Evolutionary Optimization (GEO) that aims to address the challenges of black-box optimization in high-dimensional problems. The authors highlight that existing algorithms, such as Evolution Strategies (ES) and Bayesian optimization, have limitations when it comes to optimizing high-dimensional, non-convex problems while maintaining linear time complexity. They propose GEO as a combination of ES and Generative Surrogate Neural networks (GSNs) to achieve better performance in terms of stability, mutation efficiency, and optimization in high dimensions. The paper outlines the goals of GEO, discusses related works (L-GSO and EGAN), presents the methodology, and provides experimental results showing GEO's superiority over traditional ES and GSN in higher dimensions.\n\nSTRENGTHS:\n- The paper addresses an important problem in black-box optimization: optimizing high-dimensional, non-convex problems while maintaining linear time complexity\n- The introduction provides a clear overview of the challenges faced by existing algorithms and the potential benefits of using GSN-based approaches like GEO\n- The goals of GEO are well-defined, and the paper sets the stage for discussing the methodology and experimental results\n- Combining EA with GSN is novel and interesting\n\nWEAKNESSES:\n- Some simple ES algorithms, such as OpenAI-ES [1], can optimize about 100k parameters in their paper; it is used to optimize the weight of the policy network. Although the idea of this paper seems novel and interesting, I am not sure that the 10k params can be called high-dimensional. \n- The main results are shown in Figure 3, but it is unclear which function is used for 3-a), and the figure is not explained in the manuscript. \n\n[1] Salimans, Tim, et al. \"Evolution strategies as a scalable alternative to reinforcement learning.\" arXiv preprint arXiv:1703.03864 (2017).\n\nQUESTIONS:\n- I think the authors should provide more results compared with the previous methods similar to Figure 3-a), with various functions and algorithms. If there is no space for plotting all results, I think the authors should summarize them in a table. In the evosax [2], an open-source ES library, there is an example code for testing many ES algorithms in the ES benchmark function like Rosenbrock. \n\n[2] https://github.com/RobertTLange/evosax/tree/main\n\nLIMITATIONS:\n- In the single-objective function, I think more algorithms should be compared to the proposed algorithm.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}