Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper presents a new black-box optimization method called GEO. It is designed for solving stochastic and multi-objective black-box large-scale optimization problem. 

GEO has two stages: evolution and surrogate/critic network training. The evolution pool stores a number of generators ranked by fitness scores. The backprogragation from the critic network is the mutations generators. 

When comparing with baseline methods including L-GSO, BO,  CMA-ES and NSGA-II, GEO shows competitive results in optimization functions, finding Pareto fronts, etc. 
strengths_and_weaknesses: The strengths of the paper include:
1) It proposes an interesting method that utilizes generative neural networks and evolution strategy. Evolution helps to stabilize the training of critic networks while the critic networks provide efficient mutations. 
2) Experiment results are better than baseline methods in many cases.

The weaknesses of the paper include:
1) For generative neural networks, stabilizing training often is very challenging, and training can be expensive. 
2) Due to the complexity of generative neural networks, it will be difficult for the proposed method to scale to handle high dimension problems. 
3) The organization of the paper can be better. The pictures and algorithm should be close to their description pages. The pictures' quality is not high. I failed to print the paper using two different printers. There are something wrong with the format to this paper. 
questions: I had a hard time to understand the Algorithm 1 on page 3. What is N and M? Should "while n = N " be "while n < N"?

In Figure 3, what is BO package used? 

In Figure 5, what does The noise is restored for visualization mean?

Why does GEO fail to find the Pareto front for ZDT2?

Why the generated images can not be recognized but still have high scores?

limitations: I think the authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors introduce a new black-box optimization technique that works for multi-objective, high-dimensional functions. They do this by learning a critic function and a population of "generator" functions. The outputs of the generator are used to train the critics. The generators backpropagate through the critics in order to improve their scores. The use of evolution strategies is key to stabilizing the training of the critics.
strengths_and_weaknesses: Originality:

The paper seems to be original, though I'm not extremely familiar with the SOTA on multi-objective high-dimensional blackbox optimization. 

Quality:

The paper has interesting experiments. However, there are many simple baselines that seem relevant for blackbox optimization of high-dimensional algorithms. In particular, some well-known works include:

Salimans, Tim, et al. "Evolution strategies as a scalable alternative to reinforcement learning." arXiv preprint arXiv:1703.03864 (2017).

Such, Felipe Petroski, et al. "Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning." arXiv preprint arXiv:1712.06567 (2017).

Sehnke, Frank, et al. "Policy gradients with parameter-based exploration for control." International conference on artificial neural networks. Springer, Berlin, Heidelberg, 2008.

In particular, OpenAI ES and PGPE are similar to this approach in that they estimate a gradient. These papers could be a more fair comparison since the baselines used in the paper are notoriously poor at high-dimensional optimization. Furthermore, the methods are fairly unsophisticated, making them easy to compare to.

Furthermore, I'm not sure what the purpose of the LeNet-5 experiments is, seeing as there is no baseline comparison. 

Clarity:

The paper is poorly structured and written to the point where it is quite difficult to understand. See the "Questions" section for the many points of uncertainty. 

There are numerous grammatical errors that make it difficult to read. In particular, the paper often either misuses or is missing articles (words like "the" and "a").

The pseudocode uses "while n=N", which is not at all clear. I'm assuming that they mean "for n $\in$ N".  

The authors often use imprecise language, which can cause confusion. For example, in Line 119 they write about "accidentally-high-scoring elements". I'm assuming that they are referring to points that are high due to stochasticity -- however, calling those "accidental" is imprecise/incorrect.

Significance:

The approach is interesting and seems to perform well -- however, with the missing baselines and lack of clarity on the details, it is very difficult to evaluate its significance.
questions: 1. Line 107 the authors write: "We experiment with both random variables and pseudo-random constants as input feeds z". Where are the results? If it is in a plot or figure, please reference it (I could not find which one it is referring to). If they are not, please at least mention what the results of the experiments were. What do you mean by "pseudo-random constants"? How is it different from a "random variable?" Are the random variables somehow not generated by a "pseudo-random" number generator? If $z$ is constant throughout training, then is it really a "generative" method? How would optimizing $G$ be any different than optimizing "x" directly?

2. Line 140 the authors write: "We experimented with both stochastic and non-stochastic generators". Again, I'm not sure where in the paper the results are. In particular, this is an interesting experiment: If non-stochastic generators worked, then this wouldn't really be a generative process. Can you get away with not using generators and instead just having a large population?

3. How do you measure pareto efficiency in the paper? In the pseudo-code "ParetoEfficiency" is written. In the paper, on Line 117 the authors write that data are "sorted and selected by Pareto efficiency". How exactly is this calculated?

4. Similar to question 3, there are simply many parts of the paper that are poorly explained, so much so that I cannot list out every question that I have. Here are a few:

A. Line 19: I understand that there are multiple objectives to optimize, but how are these objectives combined? The authors simply write "Optimize" without explanation. Do you take the min? Do you take the sum? If they're combined, how is it different from a single objective? Or instead, are you looking for multiple $X$ / trying to find the best $X$ for each of the $m$ functions?  I assume you take the min, but this is not made clear in the paper.

B. Line 95: "GEO uses only searched points without prepared data". I think I have a good idea of what this means, but it is not made explicitly clear at all. What do you mean by "prepared data?"

C. Line 113: "The critic network learns variable $x$ in a global region. Global training is essential". What does this mean? What would be the alternative approach? Again, while I think I have a decent guess of what the authors are trying to convey, it is not completely clear.

D. In Cartpole-V1 is the random seed fixed? My understanding is that cartpole's initialization is random, so observing the state is required.
limitations: The authors seem aware of the limitations of the work. They mention the "Pareto front collapsing problem" and the "GPU memory consumption problem".
review_title: null

final_score: rating: 2: Strong Reject: For instance, a paper with major technical flaws, and/or poor evaluation, limited impact, poor reproducibility and mostly unaddressed ethical considerations.
review_content: summary: This paper proposes a combination of deep generative model and evolution strategy, targeting high-dimensional multi-objective black-box optimization. The approach consists of N neural networks, called critic networks, where N is the number of objective, and p generative models, from which solutions to the original problem are generated by giving latent vectors z. The critic networks are trained to approximate the objective functions and the generative models are trained on the critic networks to generate a better solution.
strengths_and_weaknesses: Literature Review:

Though this paper targets high-dimensional black-box optimization, there is almost no literature review on this topic. For example, in evolution strategies, which is the basis of the proposed approach, different approaches have been proposed for high-dimensional problems, such as SEP-CMA [1], VD-CMA[2], LM-CMA[3], etc. 
[1] A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, PPSN 2008
[2] Comparison-Based Natural Gradient Optimization in High Dimension, GECCO 2014
[3] Lm-cma: An alternative to l-bfgs for large-scale black box optimization, Evolutionary Computation (2017)

Moreover, though one of the contributions of this paper is a surrogate-assisted evolution strategy, no literature review has been performed on this line. See for example [4, 5] and references therein.
[4] A global surrogate assisted CMA-ES, GECCO 2019.
[5] pysamoo: Surrogate-Assisted Multi-Objective Optimization in Python, arxiv (2022)

Multi-objective optimization has also not been reviewed. 

Experimental Evaluation:

The baseline approaches are very limited, as the literature review has not been performed well. NSGA-II is a traditional GA based approach for mainly bi-objective optimization. A lot of improvements are reported. Surprisingly, it has been applied to a single objective optimization in Figure 3, which is not understandable. CMA-ES is basically an approach for single objective optimization, and there are several extensions of CMA-ES to multi-objective optimization, but it has not been explained which algorithm is used. 

No surrogate-assisted evolution strategies or other surrogate-assisted approaches are compared in the experiments. No state-of-the-art evolutionary multi-objective optimization approaches such as MOEA/D or NSGA-III are compared. I suggest you to look at BBOB BiObjective Testbed and algorithms reported there (https://numbbo.github.io/data-archive/bbob-biobj/).

Because of the lack of the literature review and comarison with the state-of-the-art approaches, I can not judge the goodness of the proposed approach.

questions: The algorithm description looks strange to me. What do you mean by “while n = N do” and “while m = M do”?

limitations: See the comment above. The lack of literature review and the lack of experimental comparison.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This work proposed a new Generative Evolutionary Optimization method combined Evolutionary Strategies and Generative Neural Network, aiming to address stochastic multi-objective high-dimensional optimization problems. The proposed GEO is verified on a set of benchmark optimization functions (both single-objective and multi-objective), and a black-box optimization of neural networks in reinforcement learning and image recognition. 
strengths_and_weaknesses: The reviewer did not see similar work for addressing the stochastic multi-objective high-dimensional optimization problem. Though through related works, there are L-GSO, EGAN and other works, the authors clarify their differences inbetween.

The organization of the technique part of this manuscript should be considerably improved. For the reviewer, it is quite difficult to get how GEO exactly works and the introduction of sub-routine techques are quite vague, e.g., how generative model generates candidate solutions. Besides, the empirical studies section is also problematic. It is unclear why Cartpole-v1 and LeNet-5 are used for verification and how to fairly assess the quality of the GEO based on them. For example, why not compare GEO with RL methods on Cartpole-v1? What is Cartpole-v1? How representative it is? Why we have to regard it as a black-box optimization problem if some RL methods can solve it better with MDP structure. And it is also unclear what information has been delivered to the readers in section 4.4.

Though this work focuses on the  stochastic multi-objective high-dimensional optimization problem, it is not clear what real-world problem is with these features and there seems no empirical studies on the problem with all these features. Thus, it is difficult to judge how significant the proposed GEO is.
questions: 1. Please name a few real-world applications involving stochastic multi-objective high-dimensional optimization problems. The current experiment studies seem to consider 4 benchmarks each touching one or two aspects of the above problem features. It is interesting to see an empirical case study on the stochastic multi-objective high-dimensional optimization problem.

2. How Evolution Strategies contribute to GEO? Section 3.4 states that ES can help stablize the training, but why is this case? Please clarify this part. Besides, the ablation studies of verifying the importance of ES in GEO is missing. 

3. How generative model generates candidate solutions? The review suggests first introducing the general framework of GEO at the begining of section 3, so readers can easily get how GEO works. For the current manuscript, it is hard for those whoe do not farmiliar with L-GSO and EGAN to understand the GEO, even for the basic idea of it.

4. What is network structure and hyper-parameters of the generative models and critic models?

5. For Cartpole-v1, why not compare GEO with RL methods if RL methods perform better? Also, it would be much better to briefly introduce what is Carpole-v1 scenario.

6. It is not intuitive what does section 4.4 want to show about. 




limitations: No.
review_title: null

Meta-review: metareview: While the topic of the paper and the reported experimental results appeared to be of interest of the reviewing team, a number of limitations were put to the fore by the reviewers, who graded the paper with scores between 2 and 5, and often emphasized various issues such as a lacunary literature review (See in particular comments of Reviewer wE4h), insufficient comparison with state-of-the-art approaches and in particular on realistic test-casses (bPMJ), as well as on the lack of clarity (See for instance in Reviever uzBL’s review: „The paper is poorly structured and written to the point where it is quite difficult to understand“ ). For all these reasons I recommend rejection.     
confidence: Certain
Decision: Reject