Original: Generative Evolutionary Strategy For Black-Box Optimizations
Revised: Generative Evolutionary Strategy For Black-Box Optimizations

Original: Abstract
Revised: Abstract

Original: Many scientific and technological problems are related to optimization.
Revised: Many scientific and technological problems are related to optimization.

Original: Amongthem, black-box optimization in high-dimensional space is particularly challenging.
Revised: Amongthem, black-box optimization in high-dimensional space is particularly challenging.

Original: Recent neural network-based black-box optimization studies have shown noteworthy achievements.
Revised: Recent neural network-based black-box optimization studies have shown noteworthy achievements.

Original: However, their capability in high-dimensional search spaceis still limited.
Revised: However, their capability in high-dimensional search space isstill limited.

Original: This study proposes a black-box optimization method based onevolution strategy and generative neural network model.
Revised: This study proposes a black-box optimization method based on evolution strategy and generative neural network model.

Original: We designed the algorithm so that the evolutionary strategy and the generative neural network modelwork cooperatively with each other.
Revised: We designed the algorithm sothat the evolutionary strategy and the generative neural network model work cooperatively with each other.

Original: This hybrid model enables reliable training ofsurrogate networks; it optimizes multi-objective, high-dimensional, and stochastic black-box functions.
Revised: This hybrid model enables reliable training of surrogatenetworks; it optimizes multi-objective, high-dimensional, and stochastic blackbox functions.

Original: In this experiment, our method outperforms baselineoptimization methods, including evolution strategies, and  Bayesian optimization
Revised: In this experiment, our method outperforms baseline optimizationmethods, including evolution strategies, and a Bayesian optimization

Original:  Introduction
Revised:  Introduction

Original: Optimization is one of the most crucial issues in science and technology.
Revised: Optimization is one of the most crucial issues in science and technology.

Original: Various simulations andexperiments work as black-box functions, and there have also been innumerable optimization studies.
Revised: Various simulations andexperiments work as black-box functions, and there have also been innumerable optimization studies.

Original: Gradient-based optimization methods are easy choices for differentiable functions or simple convexfunctions.
Revised: Gradient-based optimization methods are easy choices for differentiable functions or simple convexfunctions.

Original: However, black-box functions are often non-differentiable and non-convex.
Revised: However, black-box functions are often non-differentiable and non-convex.

Original: Furthermore,they can be multi-objective and stochastic.
Revised: Furthermore,they can be multi-objective and stochastic.

Original: A simple description of multi-objective black-boxoptimization is as follows
Revised: A simple description of multi-objective black-boxoptimization is as follows

Original: Optimize (
Revised: Optimize (

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The search space is defined in real space R , where N is the dimension.
Revised: The search space is defined in real space R , where N is the dimension.

Original: f i is a single-objectivefunction, which can be stochastic.
Revised: f i is a single-objectivefunction, which can be stochastic.

Original: The evaluation of electronic device design is a practical problem of the optimization.
Revised: The evaluation of electronic device design is a practical problem of the optimization.

Original: Since thedevice simulators have time-sequential input-output structures, it seems like they can be solved inreinforcement learning.
Revised: Since thedevice simulators have time-sequential input-output structures, it seems like they can be solved inreinforcement learning.

Original: However, if the cost of simulator is too high, it is almost impossible to observetime-sequential data.
Revised: However, if the cost of simulator is too high, it is almost impossible to observetime-sequential data.

Original: Therefore, the evaluation problem is defined as a black-box optimization, inthis case.
Revised: Therefore, the evaluation problem is defined as a black-box optimization, inthis case.

Original: For practical purposes, researchers have studied optimization methods in various ways.
Revised: For practical purposes, researchers have studied optimization methods in various ways.

Original: Typically, Bayesian optimization [1][2][3] and evolutionary strategies [7]-[20] are widely used.
Revised: Typically, Bayesian optimization [1][2][3] and evolutionary strategies [7]-[20] are widely used.

Original: Notably, the Bayesian optimization is advantageous when the cost of the target function is high and the number of function calls is limited
Revised: Notably, the Bayesian optimization is advantageous when the cost of the target function is high and the number of function calls is limited.

Original: The estimating process of Bayesian optimization is very efficient when the number of search points is small.
Revised: The estimating process of Bayesian optimization is very efficient when the number of search points is small.

Original: On the contrary, it becomes inefficient when the number of search points increases.
Revised: On the contrary, it becomes inefficient when the number of search points increases.

Original: Therefore,
Revised: Therefore,

Original: Evolution Strategy
Revised: Evolution Strategy

Original: Generative N.N. model
Revised: Generative N.N. model

Original: Stable training
Revised: Stable training

Original: [Figure]
Revised: [Figure]

Original: Bayesian optimization may not be appropriate in high-dimensional problems that require a largenumber of function calls.
Revised: Bayesian optimization may not be appropriate in high-dimensional problems that require a largenumber of function calls.

Original: Meanwhile, evolution strategies can be better choices for aforementioned cases.
Revised: Meanwhile, evolution strategies can be better choices for aforementioned cases.

Original: In most evolutionstrategies, the computational cost does not drastically increase according to the number of searchpoints.
Revised: In most evolutionstrategies, the computational cost does not drastically increase according to the number of searchpoints.

Original: Nevertheless, it does not mean that they can avoid the curse of dimensionality .
Revised: Nevertheless, it does not mean that they can avoid the curse of dimensionality .

Original: The optimization performance of the evolutionary strategies also decreases rapidly as the dimension of theblack-box increases.
Revised: The optimization performance of the evolutionary strategies also decreases rapidly as the dimension of theblack-box increases.

Original: Although SEP-CMA[55], VD-CMA[56], and LM-CMA[57] have shown optimization capability in high-dimensional space of convex functions, optimization of high-dimensionalnon-convex problems still seems difficult.
Revised: Although SEP-CMA[55], VD-CMA[56], and LM-CMA[57] have shown optimization capability in high-dimensional space of convex functions, optimization of high-dimensionalnon-convex problems still seems difficult.

Original: Generative neural network-based models are recent approaches.
Revised: Generative neural network-based models are recent approaches.

Original: They show noteworthy performancein test function optimizations, but their capability seems to be limited to single-objective functionsand to 100-dimension [21][28].
Revised: They show noteworthy performancein test function optimizations, but their capability seems to be limited to single-objective functionsand to 100-dimension [21][28].

Original: We present GEO: Generative Evolutionary Optimization, a methodfor general black-box optimizations.
Revised: We present GEO: Generative Evolutionary Optimization, a methodfor general black-box optimizations.

Original: It is designed to optimize stochastic, multi-objective, andhigh-dimensional black-box problems.
Revised: It is designed to optimize stochastic, multi-objective, andhigh-dimensional black-box problems.

Original: We show that GEO outperforms baseline methods in finding
Revised: We show that GEO outperforms baseline methods in finding

Original: Pareto fronts of Styblinski-Tang [42], Ackley [39], Rastrigin [36][37][38], Rosenbrock [40][41],
Revised: Pareto fronts of Styblinski-Tang [42], Ackley [39], Rastrigin [36][37][38], Rosenbrock [40][41],

Original: ZDT1, ZDT2, and ZDT3 [43] test functions.
Revised: ZDT1, ZDT2, and ZDT3 [43] test functions.

Original: Also, by converting Cartpole-V1 [44] (and LeNet-5[45]) to high-dimensional black-box problems, we show that GEO can generate optimized sequences(and images) without observing intermediate states (and without the direct gradient flow from targetfunctions).
Revised: Also, by converting Cartpole-V1 [44] (and LeNet-5[45]) to high-dimensional black-box problems, we show that GEO can generate optimized sequences(and images) without observing intermediate states (and without the direct gradient flow from targetfunctions).

Original:  Related works
Revised:  Related works

Original: GEO is related to Evolutionary Generative Adversarial Networks (EGAN) [22].
Revised: GEO is related to Evolutionary Generative Adversarial Networks (EGAN) [22].

Original: Like Local Generative
Revised: Like Local Generative

Original: Surrogates Optimization (L-GSO) [21], surrogate model-based optimizations are also related to GEO.
Revised: Surrogates Optimization (L-GSO) [21], surrogate model-based optimizations are also related to GEO.

Original: This section briefly introduces the network-based black-box optimizations and EGAN to show how
Revised: This section briefly introduces the network-based black-box optimizations and EGAN to show how

Original: GEO can adapt them.
Revised: GEO can adapt them.

Original: 2.1 L-GSO
Revised: 2.1 L-GSO

Original: The black-box optimization with gradient descent method is a tricky task since the black-box doesnot provide gradient information directly.
Revised: The black-box optimization with gradient descent method is a tricky task since the black-box doesnot provide gradient information directly.

Original: The L-GSO study shows that a local surrogate modelcan be a good solution for it.
Revised: The L-GSO study shows that a local surrogate modelcan be a good solution for it.

Original: In this study, the surrogate network estimates a local shape of theobjective function, and the locality is a crucial idea of L-GSO.
Revised: In this study, the surrogate network estimates a local shape of theobjective function, and the locality is a crucial idea of L-GSO.

Original: By using local sampling of ε -cube (orball) around a currently-optimized point, a stable surrogate model provides local gradients.
Revised: By using local sampling of ε -cube (orball) around a currently-optimized point, a stable surrogate model provides local gradients.

Original: Anotheradvantage of L-GSO is that the neural network-based surrogate model naturally works in stochastic environments.
Revised: Anotheradvantage of L-GSO is that the neural network-based surrogate model naturally works in stochastic environments.

Original: Because surrogate networks estimate a distribution probabilistically, the model cansuggest gradients even in a stochastic function.
Revised: Because surrogate networks estimate a distribution probabilistically, the model cansuggest gradients even in a stochastic function.

Original: It is shown that L-GSO outperforms baselineoptimizers (Bayesian optimization and CMA-ES [9][10] in dimension = 10 and sub-manifolddimension = 100 problems).
Revised: It is shown that L-GSO outperforms baselineoptimizers (Bayesian optimization and CMA-ES [9][10] in dimension = 10 and sub-manifolddimension = 100 problems).

Original: However, due to the limitation of the local sampling method, L-GSOapplies only to a single-objective function.
Revised: However, due to the limitation of the local sampling method, L-GSOapplies only to a single-objective function.

Original: Algorithm 1 GEO
Revised: Algorithm 1 GEO

Original: Require:
Revised: Require:

Original: Initial generator pool { ( G 1 , s 1 ) , ( G 2 , s 2 ) , ..., ( G p , s p ) } , initial critic networks { C 1 , C 2 , ..., C N } , a buffer memory { ( x 1 , s 1 ) , ( x 2 , s 2 ) , ..., ( x B , s B ) } , an input seed z (pseudo random constant or random).
Revised: Initial generator pool { ( G 1 , s 1 ) , ( G 2 , s 2 ) , ..., ( G p , s p ) } , initial critic networks { C 1 , C 2 , ..., C N } , a buffer memory { ( x 1 , s 1 ) , ( x 2 , s 2 ) , ..., ( x B , s B ) } , an input seed z (pseudo random constant or random).

Original: The multi-objective function is defined as ( f 1 , ..., f N ) =
Revised: The multi-objective function is defined as ( f 1 , ..., f N ) =

Original: F .
Revised: F .

Original: while iteration do while n < N do ▷ Critic network training
Revised: while iteration do while n < N do ▷ Critic network training

Original: Sample C n in { C 1 , C 2 , ..., C N } g c = ∇ θ 1 M (cid:80) Mj
Revised: Sample C n in { C 1 , C 2 , ..., C N } g c = ∇ θ 1 M (cid:80) Mj

Original: =

Revised: =

Original: || C nθ
Revised: || C nθ

Original: ( x j ) − f n ( x j ) || , x ∈ buffer
Revised: ( x j ) − f n ( x j ) || , x ∈ buffer

Original: C nθ ← Optimize
Revised: C nθ ← Optimize

Original: ( C nθ , g c ) end while while n < N do ▷ Generator network mutation (multi-objective)
Revised: ( C nθ , g c ) end while while n < N do ▷ Generator network mutation (multi-objective)

Original: Sample C n in { C 1 , C 2 , ..., C N } while m < M do ▷ M: the number of mutations
Revised: Sample C n in { C 1 , C 2 , ..., C N } while m < M do ▷ M: the number of mutations

Original: RandomSample G i in { ( G 1 , s 1 ) , ( G 2 , s 2 ) , ..., ( G p , s p ) } g g = ∇ ϕ
Revised: RandomSample G i in { ( G 1 , s 1 ) , ( G 2 , s 2 ) , ..., ( G p , s p ) } g g = ∇ ϕ

Original: [ − C n ( G i,ϕ ( z ))]
Revised: [ − C n ( G i,ϕ ( z ))]

Original: G i ← Optimize ( G i , g g ) x
Revised: G i ← Optimize ( G i , g g ) x

Original: i =
Revised: i =

Original: G i ( z ) s i
Revised: G i ( z ) s i

Original: = F ( x i ) buffer.append( ( x i , s i ) ) pool.append( ( G i , s i ) ) end while end while pool ← ParetoEfficiency(pool)
Revised: = F ( x i ) buffer.append( ( x i , s i ) ) pool.append( ( G i , s i ) ) end while end while pool ← ParetoEfficiency(pool)

Original: ▷ evolution pool ← AgeEvolution(pool)
Revised: ▷ evolution pool ← AgeEvolution(pool)

Original: ▷ (optional) end while 2.2 EGAN
Revised: ▷ (optional) end while 2.2 EGAN

Original: Evolutionary Generative Adversarial Networks (EGAN) is a method to increase Generative Adversarial Network (GAN)
Revised: Evolutionary Generative Adversarial Networks (EGAN) is a method to increase Generative Adversarial Network (GAN)

Original: [30]-[33] performance by using the evolution strategy.
Revised: [30]-[33] performance by using the evolution strategy.

Original: The core idea of EGAN isthat the evolution strategy can assume a backpropagation as a mutation.
Revised: The core idea of EGAN isthat the evolution strategy can assume a backpropagation as a mutation.

Original: It has one discriminator andmulti generators.
Revised: It has one discriminator andmulti generators.

Original: Generators of the evolution pool are mutated for each interation, and they are sortedby fitness scores.
Revised: Generators of the evolution pool are mutated for each interation, and they are sortedby fitness scores.

Original: By comparing the mode collapsing results, the study shows that the evolution strategy efficientlycomplements the GAN algorithm.
Revised: By comparing the mode collapsing results, the study shows that the evolution strategy efficientlycomplements the GAN algorithm.

Original: EGAN cannot be regarded as a black-box optimization becausethe discriminator must be trained by prepared true data (image dataset).
Revised: EGAN cannot be regarded as a black-box optimization becausethe discriminator must be trained by prepared true data (image dataset).

Original: Nevertheless, we expectedthat a combination of evolution strategy and GAN could be adopted in our black-box optimizationalgorithm.
Revised: Nevertheless, we expectedthat a combination of evolution strategy and GAN could be adopted in our black-box optimizationalgorithm.

Original: 2.3 Other approaches
Revised: 2.3 Other approaches

Original: Global Topology Optimization network (GLOnet) [23] is a method for designing optimized electromagnetic devices.
Revised: Global Topology Optimization network (GLOnet) [23] is a method for designing optimized electromagnetic devices.

Original: It is an advanced study from the previous research, adjoint-based topologyoptimizer (ABTO)
Revised: It is an advanced study from the previous research, adjoint-based topologyoptimizer (ABTO)

Original: ABTO finds the refractive index (target variables) directly from thegradient given by a device (simulator).
Revised: ABTO finds the refractive index (target variables) directly from thegradient given by a device (simulator).

Original: GLOnet increases optimization performance by addinggenerator networks on ABTO.
Revised: GLOnet increases optimization performance by addinggenerator networks on ABTO.

Original: GLOnet is not a black-box optimizer because the gradient is givendirectly from the target simulator.
Revised: GLOnet is not a black-box optimizer because the gradient is givendirectly from the target simulator.

Original: However, we can discover an essential role of the generator networkfor better optimization.
Revised: However, we can discover an essential role of the generator networkfor better optimization.

Original: GNN-ES (Evolutionary Strategies with Generative Neural Networks)
Revised: GNN-ES (Evolutionary Strategies with Generative Neural Networks)

Original: [28] is a combined method ofbijective neural networks and evolutions.
Revised: [28] is a combined method ofbijective neural networks and evolutions.

Original: GNN-ES assumes latent space z and bijective Generative
Revised: GNN-ES assumes latent space z and bijective Generative

Original: [Figure]
Revised: [Figure]

Original: ASB ASB
Revised: ASB ASB

Original: Linear …
Revised: Linear …

Original: [Figure]
Revised: [Figure]

Original: Linear
Revised: Linear

Original: ASB ASB
Revised: ASB ASB

Original: ASB ASB
Revised: ASB ASB

Original: Linear, Activation
Revised: Linear, Activation

Original: ASB
Revised: ASB

Original: Critic Network Branches
Revised: Critic Network Branches

Original: Linear
Revised: Linear

Original: ASB ASB
Revised: ASB ASB

Original: ASB ASB
Revised: ASB ASB

Original: Linear, Activation …
Revised: Linear, Activation …

Original: Concat.
Revised: Concat.

Original: Linear, Dropout, Act.
Revised: Linear, Dropout, Act.

Original: [Figure]
Revised: [Figure]

Original: Network (GNN) x
Revised: Network (GNN) x

Original: = g ( z ) , z = h ( x ) .
Revised: = g ( z ) , z = h ( x ) .

Original: It optimizes latent space and bijective network, while updatinglatent space is carried out by evolution strategies.
Revised: It optimizes latent space and bijective network, while updatinglatent space is carried out by evolution strategies.

Original: The study shows that GNN-ES can optimize testfunctions in dimension = 10 .
Revised: The study shows that GNN-ES can optimize testfunctions in dimension = 10 .

Original: GNN-ES is not a surrogate model-based optimizer and is restricted tobijective networks.
Revised: GNN-ES is not a surrogate model-based optimizer and is restricted tobijective networks.

Original: Conservative Objective Models (COMs) [29] is a surrogate model-based black-box optimizationmethod.
Revised: Conservative Objective Models (COMs) [29] is a surrogate model-based black-box optimizationmethod.

Original: The key idea of COMs is regularizing the loss function of a surrogate model in training.
Revised: The key idea of COMs is regularizing the loss function of a surrogate model in training.

Original: Along with a standard supervised regression, it adds COMs-regularizer to prevent erroneously largepredictions of the trained model.
Revised: Along with a standard supervised regression, it adds COMs-regularizer to prevent erroneously largepredictions of the trained model.

Original: Surrogate assisted evolution models is also related [58].
Revised: Surrogate assisted evolution models is also related [58].

Original: However, the algorithm does not guarantee
Revised: However, the algorithm does not guarantee

Original: O ( n ) computational complexity.
Revised: O ( n ) computational complexity.

Original: Because high-dimensional optimization requires a lot of functioncalls, slow speed can be a huge challenge.
Revised: Because high-dimensional optimization requires a lot of functioncalls, slow speed can be a huge challenge.

Original:  Methods
Revised:  Methods

Original: [Figure]
Revised: [Figure]

Original: GEO consists of two stages: the evolution and network training.
Revised: GEO consists of two stages: the evolution and network training.

Original: The evolution pool maintainsa certain number of generators on the basis of the fitness score.
Revised: The evolution pool maintainsa certain number of generators on the basis of the fitness score.

Original: The evaluation and sorting ofmulti-objective scores is determined by Pareto-efficiency.
Revised: The evaluation and sorting ofmulti-objective scores is determined by Pareto-efficiency.

Original: A generator training is also a mutation in evolution.
Revised: A generator training is also a mutation in evolution.

Original: A generator is randomly sampled from the pool.2.
Revised: A generator is randomly sampled from the pool.2.

Original: The critic network trains the selected generator (using backpropagation, to increase prediction ofcritic).
Revised: The critic network trains the selected generator (using backpropagation, to increase prediction ofcritic).

Original: The trained generator suggests a new variable x = G ( z ).
Revised: The trained generator suggests a new variable x = G ( z ).

Original: Check score =
Revised: Check score =

Original: F ( x ).
Revised: F ( x ).

Original: Sort anew ( G , score ) set in a pool.
Revised: Sort anew ( G , score ) set in a pool.

Original: ( x , score ) pairs are stored in buffer memory and used to train the critic network.
Revised: ( x , score ) pairs are stored in buffer memory and used to train the critic network.

Original: A critic network istrained to surrogate a corresponding black-box.
Revised: A critic network istrained to surrogate a corresponding black-box.

Original: 3.1 Generative model
Revised: 3.1 Generative model

Original: GEO consists of a pool of generator networks and critic networks that create backpropagations.
Revised: GEO consists of a pool of generator networks and critic networks that create backpropagations.

Original: For each iteration step, generators are randomly sampled to make mutated generators.
Revised: For each iteration step, generators are randomly sampled to make mutated generators.

Original: N -criticnetworks are prepared to make an N -objective surrogate model, each critic network corresponds toa single objective function.
Revised: N -criticnetworks are prepared to make an N -objective surrogate model, each critic network corresponds toa single objective function.

Original: The mutated generators create variables x = G ϕ ( z ) , and N -objectivescores s i
Revised: The mutated generators create variables x = G ϕ ( z ) , and N -objectivescores s i

Original: = f i ( x ) are measured.
Revised: = f i ( x ) are measured.

Original: The set of scores ( x, s ) , s =
Revised: The set of scores ( x, s ) , s =

Original: ( s 1 , s 2 , ..., s N ) is stored in the buffermemory.
Revised: ( s 1 , s 2 , ..., s N ) is stored in the buffermemory.

Original: Training of the critic network is carried out using the buffer memory.
Revised: Training of the critic network is carried out using the buffer memory.

Original: After training, criticnetworks mutate the generator networks in the next iteration step.
Revised: After training, criticnetworks mutate the generator networks in the next iteration step.

Original: The backpropagation servesoptimal mutations by increasing predictions of critic networks.
Revised: The backpropagation servesoptimal mutations by increasing predictions of critic networks.

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: performances Computational time in 8,192 dimension (real Optimization performances of and
Revised: single

Original: in single-objective functions. LSM a modification of L- Usually, traditional GAN generators feed random latent variables z through the input layer, whilesome GAN algorithms separate latent variables from the input feeds [33].
Revised: Usually, traditional GAN generators feed random latent variables z through the input layer, whilesome GAN algorithms separate latent variables from the input feeds [33].

Original: Because GEO does notneed inferences, we do not see z as a latent vector.
Revised: Because GEO does notneed inferences, we do not see z as a latent vector.

Original: We experimented with both random variables(Figure 5) and pseudo-random constants (Figure 4) as input feeds z .
Revised: We experimented with both random variables(Figure 5) and pseudo-random constants (Figure 4) as input feeds z .

Original: Since each critic network has a corresponding objective, it must be trained separately using itscorresponding objective function.
Revised: Since each critic network has a corresponding objective, it must be trained separately using itscorresponding objective function.

Original: We used L1 loss with a single objective function f j and a criticnetwork C j .
Revised: We used L1 loss with a single objective function f j and a criticnetwork C j .

Original: The loss function is defined as follows
Revised: The loss function is defined as follows

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The critic network learns variable x in a global region.
Revised: The critic network learns variable x in a global region.

Original: Global training is essential for multidimensional Pareto front searches.
Revised: Global training is essential for multidimensional Pareto front searches.

Original: (See 3.4)
Revised: (See 3.4)

Original: 3.2 Evolution strategy
Revised: 3.2 Evolution strategy

Original: Each generator is stored in the pool with a corresponding fitness score s j
Revised: Each generator is stored in the pool with a corresponding fitness score s j

Original: = ( s 1 j , ...s Nj )
Revised: = ( s 1 j , ...s Nj )

Original: =
Revised: =

Original: F ( x j ) .
Revised: F ( x j ) .

Original: Score data { ( x j , s j ) ... } are sorted and selected by Pareto efficiency at every iteration.
Revised: Score data { ( x j , s j ) ... } are sorted and selected by Pareto efficiency at every iteration.

Original: Optionally,age evolution can be added in the sorting part.
Revised: Optionally,age evolution can be added in the sorting part.

Original: The age evolution removes the oldest elements fromthe pool.
Revised: The age evolution removes the oldest elements fromthe pool.

Original: Thereby, it prevents "the high score due to stochasticity" from surviving in the pool.
Revised: Thereby, it prevents "the high score due to stochasticity" from surviving in the pool.

Original: Apool-refresh method is another option, but it make the calculation time almost doubled.
Revised: Apool-refresh method is another option, but it make the calculation time almost doubled.

Original: In GEO, evolution strategy is not just an auxiliary tool.
Revised: In GEO, evolution strategy is not just an auxiliary tool.

Original: Without an evolution strategy, the training ofnetworks can be unstable, which leads to the failure of optimization.
Revised: Without an evolution strategy, the training ofnetworks can be unstable, which leads to the failure of optimization.

Original: We discuss details in section 3.4.
Revised: We discuss details in section 3.4.

Original: 3.3 Neural networks
Revised: 3.3 Neural networks

Original: All kinds of neural networks, including Recurrent NeuralNetwork (RNN) [47][48][49], Convolutional
Revised: All kinds of neural networks, including Recurrent Neural

Original: Neural Network (CNN) [46], and Full Connected (FC) network can be used as generator networksand critic networks. We chose a multi-head-self-attention network [50] for operational convenience.
Revised: [Figure]

Original: Figure 2 shows the self-attention network we used.
Revised: Figure 2 shows the self-attention network we used.

Original: The overall structure is modified from the originaltransformer model.
Revised: The overall structure is modified from the originaltransformer model.

Original: [Figure]

Original: Because the attention network consumes gigantic memory size, it may cause GPU out-of-memory.
Revised: Because the attention network consumes gigantic memory size, it may cause GPU out-of-memory.

Original: Itwas a significant problem when we optimized high-dimensional functions.
Revised: Itwas a significant problem when we optimized high-dimensional functions.

Original: When using an NVIDIA
Revised: When using an NVIDIA

Original: Tesla V100 32G GPU in variable space of dimension d > 2 11 , the total required memory exceeds theavailable memory size.
Revised: Tesla V100 32G GPU in variable space of dimension d > 2 11 , the total required memory exceeds theavailable memory size.

Original: We devised an ad hoc trunk-branch network structure to solve this problem.
Revised: We devised an ad hoc trunk-branch network structure to solve this problem.

Original: This structure has one trunk network and several branch networks, and each branch network extendsfrom the trunk network.
Revised: This structure has one trunk network and several branch networks, and each branch network extendsfrom the trunk network.

Original: Branches have an identical structure with each other, and the length of theoutput tensor is defined by
Revised: Branches have an identical structure with each other, and the length of theoutput tensor is defined by

Original: [Equation]
Revised: [Equation]

Original: [Figure] [Equation] [Figure] The split branch trick serves a memory-efficient structure, but it could be detrimental to optimizationperformance.
Revised: The split branch trick serves a memory-efficient structure, but it could be detrimental to optimizationperformance.

Original: Therefore, we implement the trunk-branch structure only for the sake of memoryefficiency of GPU.
Revised: Therefore, we implement the trunk-branch structure only for the sake of memoryefficiency of GPU.

Original: The baseline attention network structure includes dropout layers [51], and the dropout layers’ randomness makes the generator stochastic.
Revised: The baseline attention network structure includes dropout layers [51], and the dropout layers’ randomness makes the generator stochastic.

Original: A random input feed z is also a source of stochastic behavior.
Revised: A random input feed z is also a source of stochastic behavior.

Original: We experimented with both stochastic generators (Figure 5) and non-stochastic generators (Figure 4).
Revised: We experimented with both stochastic generators (Figure 5) and non-stochastic generators (Figure 4).

Original: However, for critic networks, we maintained the dropout layers as non-zero.
Revised: However, for critic networks, we maintained the dropout layers as non-zero.

Original: 3.4 A complementary strategy of generative network and evolution
Revised: 3.4 A complementary strategy of generative network and evolution

Original: Figure 1 shows the full algorithm of GEO.
Revised: Figure 1 shows the full algorithm of GEO.

Original: Training the surrogate model (critic network) is theessential part of a surrogate model-based optimizer, but it is also the trickiest part.
Revised: Training the surrogate model (critic network) is theessential part of a surrogate model-based optimizer, but it is also the trickiest part.

Original: The point is thatthe training data (the true data in GAN concepts) is not prepared, and it can be only acquired throughan on-the-fly search.
Revised: The point is thatthe training data (the true data in GAN concepts) is not prepared, and it can be only acquired throughan on-the-fly search.

Original: Without prepared data, training can be unstable since outbreaks of new datamake the training region fluctuate.
Revised: Without prepared data, training can be unstable since outbreaks of new datamake the training region fluctuate.

Original: In this case, the algorithm diverges for the following reasons:
Revised: In this case, the algorithm diverges for the following reasons:

Original: The generator suggests an input variable x in a wrong direction.
Revised: The generator suggests an input variable x in a wrong direction.

Original: The critic network is trained
Revised: The critic network is trained

Original: with input variable x , but x has no information of a Pareto front.
Revised: with input variable x , but x has no information of a Pareto front.

Original: The critic network trains thegenerator, but it does not give meaningful information.
Revised: The critic network trains the generator, but it does not give meaningful information.

Original: In short, the divergence is a result of evil cycles of two networks.
Revised: In short, the divergence is a result of evil cycles of two networks.

Original: The local sampling method in L-GSO can be a solution for stabilizing a surrogate network in asingle-objective function.
Revised: The local sampling method in L-GSO can be a solution for stabilizing a surrogate network in asingle-objective function.

Original: It naturally creates a local trust region in the vicinity of an optimized point.
Revised: It naturally creates a local trust region in the vicinity of an optimized point.

Original: However, it can be challenging in N -objective functions.
Revised: However, it can be challenging in N -objective functions.

Original: Since the Pareto front is R n − 1 surface, itrequires R n − 1 dimensional sampling region, which cannot be local.
Revised: Since the Pareto front is R n − 1 surface, itrequires R n − 1 dimensional sampling region, which cannot be local.

Original: Therefore, we need to devise a stabilizing method for multi-objective functions.
Revised: Therefore, we need to devise a stabilizing method for multi-objective functions.

Original: We suggest that theevolution strategy can be a good solution.
Revised: We suggest that theevolution strategy can be a good solution.

Original: The role of evolution is to trap G and corresponding x nearthe Pareto front.
Revised: The role of evolution is to trap G and corresponding x nearthe Pareto front.

Original: At the same time, it slows down the fluctuation of the pool.
Revised: At the same time, it slows down the fluctuation of the pool.

Original: As a result, the trainingdata region becomes stabilized around the Pareto front.
Revised: As a result, the trainingdata region becomes stabilized around the Pareto front.

Original: We expect the stabilization method to serve as an anchor, which prevents the training data fromfloating.
Revised: We expect the stabilization method to serve as an anchor, which prevents the training data fromfloating.

Original: At the same time, a properly trained critic network provides better mutation strategies.
Revised: At the same time, a properly trained critic network provides better mutation strategies.

Original: Theinterdependent cooperation of evolution strategy and surrogate model is our main idea.
Revised: Theinterdependent cooperation of evolution strategy and surrogate model is our main idea.

Original:  Experimental results
Revised:  Experimental results

Original: In this section, we compare the black-box optimization results of GEO with baseline optimizers.
Revised: In this section, we compare the black-box optimization results of GEO with baseline optimizers.

Original: Wetested single and multi-objective functions, stochastic and non-stochastic functions.
Revised: Wetested single and multi-objective functions, stochastic and non-stochastic functions.

Original: The baseline optimizers are Bayesian optimization (Gaussian process), NSGA-II (NSGA2) [7], NSGA-III (NSGA3)[15], MOEA/D [18], and CMA-ES evolution algorithms.
Revised: The baseline optimizers are Bayesian optimization (Gaussian process), NSGA-II (NSGA2) [7], NSGA-III (NSGA3)[15], MOEA/D [18], and CMA-ES evolution algorithms.

Original: For the implementation of NSGA-II and
Revised: For the implementation of NSGA-II and

Original: CMA-ES, we used the Pymoo package [52].
Revised: CMA-ES, we used the Pymoo package [52].

Original: 4.1 Single objective functions
Revised: 4.1 Single objective functions

Original: [Figure] Figure 3a) shows performances of optimizers according to dimensions.
Revised: Figure 3a) shows performances of optimizers according to dimensions.

Original: For the single-objective testfunction, we used Styblinski-Tang function.
Revised: For the single-objective testfunction, we used Styblinski-Tang function.

Original: At dimension = 2 , Bayesian optimization shows thebest performance.
Revised: At dimension = 2 , Bayesian optimization shows thebest performance.

Original: However, as the dimension increases, GEO outperforms baseline optimizers.
Revised: However, as the dimension increases, GEO outperforms baseline optimizers.

Original: Atdimension = 8 , 192 , baseline optimizers rarely finds the global minimum, while the GEO showsbetter performances.
Revised: Atdimension = 8 , 192 , baseline optimizers rarely finds the global minimum, while the GEO showsbetter performances.

Original: To see how the depth of the generator network affects the performance, we also tested GEO with asingle-layer generator.
Revised: To see how the depth of the generator network affects the performance, we also tested GEO with asingle-layer generator.

Original: The single-layer generator appears to have little optimization capability.
Revised: The single-layer generator appears to have little optimization capability.

Original: Evenin the dimension = 2 problem, it shows a considerably slow optimization.
Revised: Evenin the dimension = 2 problem, it shows a considerably slow optimization.

Original: Bayesian optimization is a powerful method for high-cost black-box problems, but its performancecan be weakened when the problem requires numerous function calls.
Revised: Bayesian optimization is a powerful method for high-cost black-box problems, but its performancecan be weakened when the problem requires numerous function calls.

Original: The computational complexityof Bayesian optimization is known to be O ( n 3 ) [3].
Revised: The computational complexityof Bayesian optimization is known to be O ( n 3 ) [3].

Original: Figure 3b) shows the computational time of
Revised: Figure 3b) shows the computational time of

Original: GEO, NSGA-II, and Bayesian optimization.
Revised: GEO, NSGA-II, and Bayesian optimization.

Original: Like most evolution strategies, GEO is designed to havea computational complexity of O ( n ) .
Revised: Like most evolution strategies, GEO is designed to havea computational complexity of O ( n ) .

Original: Figure 3c) shows performance comparisons to a Local Surrogate Model (LSM), a modificationof the L-GSO algorithm.
Revised: Figure 3c) shows performance comparisons to a Local Surrogate Model (LSM), a modificationof the L-GSO algorithm.

Original: LSM follows the general outline of L-GSO GAN implementation butadopts the self-attention network used in GEO.
Revised: LSM follows the general outline of L-GSO GAN implementation butadopts the self-attention network used in GEO.

Original: Finding an optimal hyperparameter set for all testfunctions is nearly impossible since the behavior of LSM greatly varies according to hyperparameters.
Revised: Finding an optimal hyperparameter set for all testfunctions is nearly impossible since the behavior of LSM greatly varies according to hyperparameters.

Original: Therefore we investigate relative properties of LSM according to dimensions.
Revised: Therefore we investigate relative properties of LSM according to dimensions.

Original: In the Styblinski-Tangfunction, LSM shows worse performance than in other functions.
Revised: In the Styblinski-Tangfunction, LSM shows worse performance than in other functions.

Original: Also, the performance of LSMrapidly decreases as the dimension increases.
Revised: Also, the performance of LSM [Table] rapidly decreases as the dimension increases.

Original: On the other hand, GEO shows better optimizationperformance under various conditions.
Revised: On the other hand, GEO shows better optimizationperformance under various conditions.

Original: [Table]

Original: 4.2 Multi objective functions
Revised: 4.2 Multi objective functions

Original: In this section, we show optimization performance comparisons in multi-objective problems.
Revised: In this section, we show optimization performance comparisons in multi-objective problems.

Original: We onlycompare GEO and evolution strategies because the high-dimensional problems require immoderatefunction calls.
Revised: We onlycompare GEO and evolution strategies because the high-dimensional problems require immoderatefunction calls.

Original: Figure 4 shows optimization results in the non-stochastic-multi-objective functions.
Revised: Figure 4 shows optimization results in the non-stochastic-multi-objective functions.

Original: Each figure is a
Revised: Each figure is a

Original: Pareto front plot after 100,000 function calls.
Revised: Pareto front plot after 100,000 function calls.

Original: For the multi-objective test functions, ZDT functionsand combined functions F =
Revised: For the multi-objective test functions, ZDT functionsand combined functions F =

Original: [ f 1 , f 2 ] ( f i : Styblinski-Tang, Ackley and Rastrigin function) wereused.
Revised: [ f 1 , f 2 ] ( f i : Styblinski-Tang, Ackley and Rastrigin function) wereused.

Original: A Latin Hyper Cube (LHC) [53] method is a good guess for initial states.
Revised: A Latin Hyper Cube (LHC) [53] method is a good guess for initial states.

Original: However, GEOcannot implement LHC because it generates initial points x through the neural network.
Revised: However, GEOcannot implement LHC because it generates initial points x through the neural network.

Original: We gave twoinitial states in the baseline optimizer to control performance according to the initial state.
Revised: We gave twoinitial states in the baseline optimizer to control performance according to the initial state.

Original: NSGA 2 - Iuses GEO’s initial distribution x ∈ G ( z ) as its initial points, while NSGA 2 - LHC uses LHC-initialpoints.
Revised: NSGA 2 - Iuses GEO’s initial distribution x ∈ G ( z ) as its initial points, while NSGA 2 - LHC uses LHC-initialpoints.

Original: In most cases, GEO outperforms baseline optimizers.
Revised: In most cases, GEO outperforms baseline optimizers.

Original: The choice of initial points for NSGA-II rarelyaffects final results.
Revised: The choice of initial points for NSGA-II rarelyaffects final results.

Original: A slightly different result appears in ZDT2.
Revised: A slightly different result appears in ZDT2.

Original: In ZDT2, GEO finds an optimalpoint, but it fails to find a global shape of the Pareto front.
Revised: In ZDT2, GEO finds an optimalpoint, but it fails to find a global shape of the Pareto front.

Original: Figure 5 shows optimization results in the stochastic-multi-objective functions.
Revised: Figure 5 shows optimization results in the stochastic-multi-objective functions.

Original: The optimization ofstochastic functions is defined as follows
Revised: The optimization ofstochastic functions is defined as follows

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: f i ← f i + N i ( µ, σ )

Original: N is a random normal distribution, and we set µ = 0 . 0 and σ = 1 . 0 .
Revised: N is a random normal distribution, and we set µ = 0 . 0 and σ = 1 . 0 .

Original: Figure 5 shows optimizationresult of stochastic-multi-objective functions.
Revised: Figure 5 shows optimizationresult of stochastic-multi-objective functions.

Original: GEO outperforms baseline optimizers even in astochastic environment, but it still has a single-point-convergence problem in the ZDT2 function.
Revised: GEO outperforms baseline optimizers even in astochastic environment, but it still has a single-point-convergence problem in the ZDT2 function.

Original: 4.3 Cartpole-v1 
Revised: 4.3 Cartpole-v

Original: For the variety of test functions, we optimized the OpenAI [54] Cartpole-V1 by converting it into ablack-box problem.
Revised: For the variety of test functions, we optimized the OpenAI [54] Cartpole-V1 by converting it into ablack-box problem.

Original: It is also a simple toy model of time-sequential
Revised: It is also a simple toy model of time-sequential

Original: I/O problems.
Revised: I/O problems.

Original: Cartpole-V1 is a test package that is mainly used in reinforcement learning [34][35].
Revised: Cartpole-V1 is a test package that is mainly used in reinforcement learning [34][35].

Original: In the reinforcement learning perspective, its action space is regarded as two-dimensional and time-sequential.
Revised: In the reinforcement learning perspective, its action space is regarded as two-dimensional and time-sequential.

Original: Reinforcement learning is trained by continuously observing rewards and states for actions duringlearning.
Revised: Reinforcement learning is trained by continuously observing rewards and states for actions duringlearning.

Original: However, in the black-box problem, the entire input sequence is assumed as one large input,and only the final score is measured without observing the intermediate rewards and states.
Revised: However, in the black-box problem, the entire input sequence is assumed as one large input,and only the final score is measured without observing the intermediate rewards and states.

Original: The final score is measured in a relative score to the sequence length.
Revised: The final score is measured in a relative score to the sequence length.

Original: (If the Cartpole is alive form -length in n -sequences, the score is m/n .)
Revised: (If the Cartpole is alive form -length in n -sequences, the score is m/n .)

Original: Therefore, the maximum and minimum score set to 1 .
Revised: Therefore, the maximum and minimum score set to 1 .

Original: 0and 0 . 0 .
Revised: 0and 0 . 0 .

Original: GEO outperforms NSGA-II from 32 to 512 dimensions in the experiment (Table 1).
Revised: GEO outperforms NSGA-II from 32 to 512 dimensions in the experiment (Table 1).

Original: Since this experiment is a toy model of real-world problems which have stochastic environments, wekept the Cartpole stochastic.
Revised: Since this experiment is a toy model of real-world problems which have stochastic environments, wekept the Cartpole stochastic.

Original: 4.4 LeNet-5
Revised: [s = 0.999] 1 : [s = 0.959] 2 : [s = 0.999] 3 : [s = 0.999] 4 : [s = 0.999] [Figure]

Original: We also optimized LeNet-5, which is trained with the MNIST dataset.
Revised: We also optimized LeNet-5, which is trained with the MNIST dataset.

Original: The optimization goal is togenerate an image that makes the LeNet-5 predict a target number with a maximum score (maximum [s = 0.999] 1 : [s = 0.959] 2 : [s = 0.999] 3 : [s = 0.999] 4 : [s = 0.999] prediction score = 1 . 0 ).
Revised: The optimization goal is togenerate an image that makes the LeNet-5 predict a target number with a maximum score (maximumprediction score = 1 . 0 ).

Original: The LeNet-5 is regarded as a non-differentiable black-box.
Revised: The LeNet-5 is regarded as a non-differentiable black-box.

Original: After 50 , 000function calls, the final scores of generated images reach very close to the maximum score (Figure 6).
Revised: After 50 , 000function calls, the final scores of generated images reach very close to the maximum score (Figure 6).

Original: In the related experiment, L-GSO, generative models appear to be better at finding the local optimumin sub-manifolds.
Revised: In the related experiment, L-GSO, generative models appear to be better at finding the local optimumin sub-manifolds.

Original: We wanted to see the sub-manifold structure in the generated image, but thegenerated image does not seem intuitive.
Revised: We wanted to see the sub-manifold structure in the generated image, but thegenerated image does not seem intuitive.

Original:  Discussion
Revised:  Discussion

Original: Often, a neural network’s learning mechanism is likened to learning manifolds in a high-dimensionalspace.
Revised: Often, a neural network’s learning mechanism is likened to learning manifolds in a high-dimensionalspace.

Original: Similarly, we guess that the critic network in GEO learns low-dimensional manifolds in thehigh-dimensional space.
Revised: Similarly, we guess that the critic network in GEO learns low-dimensional manifolds in thehigh-dimensional space.

Original: Therefore, we expect that finding global or local optima would be easy ifthe optima are in low-dimensional manifolds.
Revised: Therefore, we expect that finding global or local optima would be easy ifthe optima are in low-dimensional manifolds.

Original: Also, we consider that it is the reason why GEO withsingle-layer generators optimizes a target very slowly.
Revised: Also, we consider that it is the reason why GEO withsingle-layer generators optimizes a target very slowly.

Original: Stabilization of critic network training is another key.
Revised: Stabilization of critic network training is another key.

Original: Although we implemented the evolutionstrategy in this study, any other methods which can stabilize the training process would be helpful.
Revised: Although we implemented the evolutionstrategy in this study, any other methods which can stabilize the training process would be helpful.

Original:  Conclusion
Revised:  Conclusion

Original: We have described a method for stochastic-multi-objective black-box optimization.
Revised: We have described a method for stochastic-multi-objective black-box optimization.

Original: GEO is aninterdependent cooperation method of generative neural networks and evolution strategy.
Revised: GEO is aninterdependent cooperation method of generative neural networks and evolution strategy.

Original: Theevolution strategy provides a stable training region for critic networks, and the critic networks provideefficient mutations to the evolution strategy.
Revised: Theevolution strategy provides a stable training region for critic networks, and the critic networks provideefficient mutations to the evolution strategy.

Original: As our design intent, GEO seems to work appropriatelyin stochastic and high-dimensional multi-objective test functions.
Revised: As our design intent, GEO seems to work appropriatelyin stochastic and high-dimensional multi-objective test functions.

Original: Meanwhile, the Pareto front collapsing problem, shown in ZDT2, is an important issue to be dealtwith.
Revised: Meanwhile, the Pareto front collapsing problem, shown in ZDT2, is an important issue to be dealtwith.

Original: Another limitation of GEO is the GPU memory consumption problem.
Revised: Another limitation of GEO is the GPU memory consumption problem.

Original: Excessive memoryconsumption of attention networks limits its search space to around 10 , 000 dimensions.
Revised: Excessive memoryconsumption of attention networks limits its search space to around 10 , 000 dimensions.

Original: In futureresearch, we can study other memory-efficient networks to solve this problem.
Revised: In futureresearch, we can study other memory-efficient networks to solve this problem.

Original: GEO is designed for optimization in a very high dimension.
Revised: GEO is designed for optimization in a very high dimension.

Original: Performance at lower dimensions is notguaranteed (see supplement).
Revised: Performance at lower dimensions is notguaranteed (see supplement).

Original: We think that d = [1,000 to 10,000] can be the practical range of useof GEO.
Revised: We think that d = [1,000 to 10,000] can be the practical range of useof GEO.

Original: In addition, GEO is designed to find local-optima, not a global-optimum.
Revised: In addition, GEO is designed to find local-optima, not a global-optimum.

Original: It is because themutation of generators only concetrates on an exploit.
Revised: It is because themutation of generators only concetrates on an exploit.

Original: In the next study, a strong exploit & explorestrategy (like what BO has) should be added to make better optimization capability. 
Revised: In the next study, a strong exploit & explorestrategy (like what BO has) should be added to make better optimization capability. Checklist For all authors... (a) Do

Revised: the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Revised: Yes

Revised: (b

Revised: Did

Revised: you describe the limitations of your work?

Revised: [Yes] See discussion and conclusion

Revised: (c)

Revised: Did you

Revised: discuss any potential negative societal impacts of your work

Revised: Yes

Revised: (d) Have

Revised: you read the ethics review guidelines and ensured that your paper conforms tothem

Revised: Yes

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [N/A]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] See supplementary for implementation details(b)

Revised: Did you specify all the training details (e

Revised: g., data splits, hyperparameters, how theywere chosen)?

Revised: [Yes] See supplementary for implementation details

Revised: (c) Did you report error bars (e

Revised: g., with respect to the random seed after running experi-

Revised: ments multiple times)?

Revised: [Yes] See supplementary for experimental details

Revised: (d) Did you include the total amount of compute and the type of resources used (e

Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [Yes] See supplementary for experimental

Revised: details

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] See references

Revised: (b) Did you mention the license of the assets?

Revised: [Yes] See supplementary(c)

Revised: Did you include any new assets either in the supplemental material or as a URL?

Revised: [No]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [Yes]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?

Revised: [Yes]

Revised: If you used crowdsourcing or conducted research with human subjects...

Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Revised: applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB