Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors proposed an interesting approach based on what I can tell to be a novel connection between vision transformers and evolutionary algorithms. The paper then modifies the transformer architecture to implement evolutionary-specific operations like crossover, mutation and selection. The authors then demonstrate their method on six black-box functions and show superior performance to baselines. 
strength_and_weaknesses: Strengths: 
- Interesting approach with potential impact on black-box optimisation 
- Favourable results in application domains considered 

Weaknesses: 

- Paper writing: I have found the paper hard to follow and not clearly written. Of course, I feel with the authors who had to introduce many related concepts before proposing their method. Maybe the authors could consider moving their contributions before page 4 to make it easier to grasp the novelty of the paper. 

- There does not seem to be regard to sample efficiency in the proposed approach. Are the authors not worried about expensive of evaluating black-box functions? I ask since the domains considered in the experiments are relatively toy and running the experiments in a real-world problem beyond Ab initio docking. Looking at the literature on high-dimensional Bayesian optimisation (which needs to be cited) can help in designing such experiments.  

- I am also a bit confused about the experimental results, which seem to somewhat contradict the notion of black-box optimisation. To elaborate, it seems that the training set's similarity to the black box is a critical factor in the approach's success. If that is the case, how can we define this notion? If the target task (so to say) is a black box then what does it mean for us to be able to pick similar training data? What does similarity mean in this context? 

- Why haven't the authors compared to state-of-the-art bayesian optimisation solutions in their experiments? As far as I can tell, the dimensions vary from 10 to 100 d which SOTA BO should be able to handle. 

- What motivated the choices of the training and testing functions? After reading the paper, they seemed to be arbitrary. Maybe the authors can help me understand? 
clarity,_quality,_novelty_and_reproducibility: Please see above.
summary_of_the_review: In general, I think this is an interesting approach. In its current state, however, I can not recommend acceptance. I am of course willing to change my score if the authors convince me in the rebuttal phase.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new approach based on Transformer, namely Optformer, to optimize unconstrained black-box optimization problems. The key idea is based on an existing observation that Vision Transformer is similar to the evolutionary algorithm (EAs). Therefore, the paper aims to modify the components of the Transformer to implement the evolutionary algorithm's operations such as crossover, mutation, and selection. The proposed model Optformer then creates a mapping between a random population and the optimal solution of the black-box optimization problem. Experiments are conducted on 6 black-box functions and one real-world problem to show the efficacy of the proposed method.
strength_and_weaknesses: Strengths:

Overall, I think the key idea of the paper, which is to propose a transformer-based model to solve the unconstrained black-box optimization problems is very interesting and novel.
+ The key idea of using the similarity between the Vision Transformer and evolutionary algorithms (EAs), and then developing a new model by modifying the Transformer's components to implement the EA operations seems to be sound and novel to me.
+ The paper's writing is generally clear and easy to understand, although there are some parts that I feel I need to understand more (please see my questions below).
+ The experimental evaluation shows the strong performance of the proposed method. It also includes some ablation study and sensitivity analysis to further understand the behavior of the proposed method.

Weaknesses:
+ Using a very deep learning model to solve the black-box optimization problems will introduce a lot of extra hyper-parameters that may significantly affect the performance of the proposed method. I found it hard to believe that an untrained model could still yield good results. Perhaps the test problems are not complex enough?
+ The experimental evaluation only includes one real-world problem (protein docking). For these types of work, I expect more real-world problems to be included in the experimental evaluation.

Questions:
+ In Section 4.5, I feel very unclear about how the training dataset is generated. The paper states that the method to generate the training dataset is to "randomly produce a shifted objective function by adjusting the location of the optima", but we do not know the objective functions (they're black-box functions) so how can we generate these shifted objective functions? In the experimental evaluation, I notice that the paper uses 3 new functions to create the training datasets, so how are these functions chosen? Do they need to share the similarity with the objective functions that we need to optimize?
+ For the loss function in Eq. (12), a simple model that minimizes this loss function is the model where E_{\theta}(X0) = X(0), but if this happens, then we can never find the optimal solution set of the black-box optimization problem. Will this case ever happen for the proposed method?
+ The paper always states that X0 needs to be sorted in non-descending order of fitness (e.g., in the first sentence of page 6), but it never describes how we sort this set X0. Does the paper mean to say that the proposed Optformer will actually perform this ordering of fitness?
+ In the experiments, the number of generations of all the reference algorithms is set to 100. This seems to be quite a small value given that some objective functions have the dimensions of 100. Plots showing the optimal values found by all the algorithms w.r.t. the number of generations (and the number of generations is much larger than 100) might be helpful.
+ Is there any guideline on how to choose the hyper-parameters of the Optformer? E.g., learning rate, number of layers, etc.

clarity,_quality,_novelty_and_reproducibility: The paper writing is generally clear and easy to understand. There are only some parts that I don't feel clear enough and need to seek further information - please see my comments above.

The key idea of the paper is interesting and novel. The proposed method seems to be sound to me. The experiments show that the proposed method performs well on various problems (but note there is only one real-world problem) - please see my comments above.

The paper provides the code which can help the reproducibility.
summary_of_the_review: Overall, I think the key idea of the paper, which is to propose a transformer-based model to solve the unconstrained black-box optimization problems is very interesting and novel. The proposed method is sound to me. It is also shown to perform well on various problems. The paper's writing is generally clear. But I still have various concerns as listed in my questions. If these concerns are addressed, I can increase the score for the paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a novel method for continuous unconstrained black-box optimization problems, called Optformer. It is claimed that the Optformer is inspired by the similarity between Transformer and EA. Six simple benchmark functions and a real-world problem are used to verify the performance of Optformer. In general, this paper is poorly written. I could not find the scientific question therein and there are many errors. I think the contribution of this work is rare. 
strength_and_weaknesses: 1. The English writing is poor. Too many long sentence making it difficult for understanding. 
2. In the first paragraph, I do not think neural architecture search can be said that "we have no access to any other information about f". And it is not generally correct to say any hyperparameter optimization problems that "we have no access to any other information about f".
3. The second sentence, "evolutionary strategies" should be "evolution strategies". Also, Evolution strategies (ES) is a type of evolutionary algorithms(EAs), if the authors list EAs, it is no need to list ES.
4. What is "hand engineering"? And do not list 11 references at the same time.
5. "The rarely focus on black-box optimization problems", this is too strong and should be clearly justified. As they aim to solve general optimization problem, why cannot they solve black-box optimization problems.
6. "Although several efforts like Cao et al. (2019); Chen et al. (2017) have coped with these problems, they have limited performance due to poor representation ability of RNN, LSTM, and MLP." this is too strong and should be clearly justified.
7. The major motivation of this work is said as "Inspired by the similarity of EAs and Transformer", but there is no discussion about what is the similarity inbetween. And to the reviewer, EAs look not similar to Transformer.
8. For the experimental studies, the comparisons are too simple. First, the benchmark contains only six very simple functions. It is not enough to demonstrate the advantages of Optformer on these kinds of problems. Second, the real-world problem of Ab initio protein docking lacks of descriptions. It is unclear what the problem it is essentially. Third, the compared algorithm of DE look outdated as it is 12 years ago, and it is unclear what exact algorithms/versions of ES and CMA-ES are as there are references.


clarity,_quality,_novelty_and_reproducibility: Please see the comments above. I think this paper is not clearly written and its quality is poor. The scientific question is not clear and it is difficult to judge the novelty.
summary_of_the_review: In summary, I think this paper should be considerably re-organized and carefully improved in terms of at least motivation, related work, and experimental studies.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: # Summary 
At a high level, the paper in order can be summarized as:

1. Operations (e.g. crossover) in evolutionary algorithms over continuous search spaces, can be seen as specific matrix multiplications over a population of suggestions $X$.
2. This implies a form of learnability over these matrix multiplications, and a modified variant of the Transformer block can be interpreted as a large, learnable population mutator, mapping an initial population $X_{t}$ to an intermediate population $X_{t+1}$. These blocks can be stacked together to obtain population chains $X_{0} \rightarrow X_{1} \rightarrow X_{2} ...$. Note that then the output of this "modified Transformer" will be the final population.
3. The model is trained directly with a basic normalized performance loss, over **differentiable** functions. 
4. Experiments are conducted to check the end-to-end performance of the method, which seems to outperform standard evolutionary algorithm baselines. Ablations are also conducted to show:
    * Deeper models perform better.
    * A deep randomly initialized model "nws5-r" outperforms a trained shallow model "ws3"
strength_and_weaknesses: # Strengths
* The overall approach (i.e. a Transformer block has an analogy to an evolutionary algorithm population update) is interesting. If the weaknesses below are resolved, this could be a solid contribution to the field of evolutionary algorithms.
* The baselines used in the experiments (e.g. ES, CMA-ES, Dragonfly) are quite comprehensive. This at least currently establishes that the method has some promise in end-to-end performance.


# Weaknesses
* Experimental results do not present how the intermediate layers work, but only present the end result. This makes it hard to know if the method is doing what it's supposed to, especially given the large claims about the model's analogies to evolutionary algorithms. For example, some lingering questions exist, and can be resolved with more ablations:
    * What kinds of population updates / cross-over techniques is each block learning? Are they similar or different from baselines? 
    * "Perhaps using ES is a good training strategy": Is the untrained block performing some sort of local or global mutation?
* Currently, the paper only presents work on continuous search spaces. However, one big advantage of using regular evolutionary algorithms are their flexibility in dealing with combinatorial search spaces. Could the method be modified to approach combinatorial / categorical spaces? 
* Large amounts of unnecessary notation make it very difficult to comprehend what is going on. In general, it's not hard to convey my summary written above with fewer notation in general, and this could significantly improve the paper's quality and presentation. For example:
    * Section 3.2 can be first introduced by stating the intent (i.e., showing that crossover / mutations can be represented as matrix multiplications).
    * Section 3.3 can be moved to the Appendix or deleted altogether since they simply describe the original Transformer module. 
    * It would also make a huge difference in readability, if e.g. a diagram was made to represent Sections 4.1-4.3 (e.g. see Figure 1 in the original Transformer paper [1]). 
    * Too many abbreviations (e.g. "nws5") in the experimental section in Section 5. These could simply be written explicitly, e.g. "Trained"/"Untrained/"5 layers".
* (Minor): It seems that a previous paper [2] has already claimed the "OptFormer" name. I would suggest changing "OptFormer" in this paper to a different name.



[1] Attention is all you need. https://arxiv.org/abs/1706.03762

[2] Towards Learning Universal Hyperparameter Optimizers with Transformers. https://arxiv.org/abs/2205.13320
clarity,_quality,_novelty_and_reproducibility: ## Quality + Clarity:
Please see my above first point on "Weaknesses". Overall the paper can be made much more clear and polished following my suggestions for improvement.

## Originality: 
Moderate originality. As referenced already in this paper, there are multiple previous works which e.g. use LSTMs to perform blackbox optimization. One core contribution the current paper makes from previous works, is the interpretability of the Transformer block as an actual evolutionary population update. However, the paper does not experimentally ablate this contribution very well, which is unfortunate.
summary_of_the_review: Overall, while the approach and overall theme are solid, the execution of the paper is still currently flawed and needs much polishing (both in the writing/presentation, in addition to needing more experimental studies). Thus I currently recommend rejection.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new network architecture based on transformers to learn an evolutionary algorithm for black-box optimization. It modifies the structure of transformers and proposes new components for crossover, mutation and selection. By training on surrogate functions similar to the target test function, the paper shows better convergence than a few standard EA baselines.

Strengths:
- A novel architecture for learning EA operations.
- Strong performance than baselines on the problems evaluated

Weaknesses:
- English writing and presentation could be improved. The authors' feedback clarifies many questions from the authors. A significant amount of revision may be required to improve the quality of writing.
- Multiple reviewers have concerns on the set of experiments. Only two real-data experiments are conducted in addition to synthetic test functions.
- More visual ablations would help readers understand the behavior of the learned model.
justification_for_why_not_higher_score: The main concerns of clarification and lack of thorough empirical evaluation are not fully addressed after the review-author discussion.
Decision: Reject