Paper: Uniform as Glass: Gliding over the Pareto Front with Neural Adaptive Preferences (ICLR-2024)

Reviewer comment (Novelty):
The work related to MOEA/D with adaptive preference adjustment methods has not been adequately investigated. The most recent paper mentioned in this paper was published in 2014, which does not correspond to the extensive research MOEA/D has received over the years.

Author response:
**W1. The work related to MOEA/D with adaptive preference adjustment methods has not been adequately investigated. The most recent paper mentioned in this paper was published in 2014, which does not correspond to the extensive research MOEA/D has received over the years.**

Thanks for bringing it to us. We do notice some new preference adjustment methods in recent years, e.g., Dong et al 2020 ([2]), Xu et al 2019 ([3]), and Farias et al 2019 [4].

We have not compared the proposed method with them for the following reasons. 

- [2] only use linear segments to model the Pareto fronts which can only handle objectives with very few objectives. Such models with simple curves cannot represent the true front. 
- [3;4] are modifications of Qi et al.'s MOEA/D-AWA [4], employing some simple strategies. These heuristic methods lack the theoretical guarantees of our work, making their final performance less reliable compared to our proposed approach.

Reviewer comment (Experiments):
The effectiveness of the proposed method needs more evaluation by considering test problems with more complicated Pareto fronts, e.g., the WFG and UF test suite, and more state-of-the-art algorithms published within the last eight years.

Author response:
**W2. The effectiveness of the proposed method needs more evaluation by considering test problems with more complicated Pareto fronts, e.g., the WFG and UF test suite, and more state-of-the-art algorithms published within the last eight years.**

Thank you for highlighting this aspect. Our experiments indeed utilized the 'real-world' problem dataset introduced by Ryoji Tanabe and Hisao Ishibuchi in 2020 ([6]). As far as we are aware, this dataset is pretty 'new' in this field and has not yet been thoroughly examined. One of the key challenges presented by this dataset is the complexity and unpredictability of the Pareto front and set. Despite these complexities, we found that our proposed method performs effectively on these tasks, demonstrating its robustness and adaptability to intricate real-world problems.

The WFG and UF problem sets, while known for the complexity of their Pareto sets, exhibit regularity in the shape of their Pareto fronts. This observation forms a pivotal standpoint in our paper. Although modeling the Pareto front as an $m-1$ manifold within the $R^n$ space presents considerable challenges due to its complexity, the scenario changes when considering the Pareto front in the $R^m$ space. Here, it is represented as an $m-1$ manifold, which is significantly easier to model. So since the shape of the Pareto front of WFG and UF are still regular, the complexity of the landscape of these problems are not a bottleneck for the proposed method. 

[1]. Jiang Siwei, Cai Zhihua, Zhang Jie, and Ong Yew-Soon. Multiobjective optimization by decomposition with pareto-adaptive weight vectors. 

[2]. Dong, Zhiming, Xianpeng Wang, and Lixin Tang. "MOEA/D with a self-adaptive weight vector adjustment strategy based on chain segmentation." Information Sciences.

[3]. Xu, Siwen, et al. "A modified MOEAD with an adaptive weight adjustment strategy." 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS). 

[4] C. De Farias, Lucas RC, et al. "MOEA/D with uniformly randomly adaptive weights." Proceedings of the genetic and evolutionary computation conference. 2018. 

[5] D. Qi, Yutao, et al. "MOEA/D with adaptive weight adjustment." Evolutionary computation 22.2 (2014). 

[6] Ryoji Tanabe, Hisao Ishibuchi. An Easy-to-use Real-world Multi-objective Optimization Problem Suite.

Reviewer comment (Reproducibility):
More details of the proposed method need to be provided, e.g., when the method uses the real objective evaluation and the model-based estimation.

Author response:
**W3. More details of the proposed method need to be provided, e.g., when the method uses the real objective evaluation and the model-based estimation.**

Thanks for the reminder. We'll provide a detailed explanation of the experiment. Currently, we use real objective evaluations for the problems considered. We're planning to tackle more complex issues, like using simulations and models for multiobjective output evaluation.

Reviewer comment (Experiments):
The conclusion that MOEA/D fails to achieve uniform objectives shown in Section 4 is not rigorous, given that many MOEA/D variants have been proposed. Specific descriptions or references that hold for the conclusion should be provided.

Author response:
**W4. The conclusion that MOEA/D fails to achieve uniform objectives shown in Section 4 is not rigorous, given that many MOEA/D variants have been proposed. Specific descriptions or references that hold for the conclusion should be provided.**

Thank you for noting this. Section 4.1 details how MOEA/D achieves an asymptotically uniform distribution only when 'h' is an affine mapping. Otherwise, the objective distribution density aligns with $\frac{\partial h}{\partial \lambda}$, influenced by 'h's' fluctuations. This new insight in Section 4.1 enhances our understanding of MOEA/D in multi-objective optimization (MOO).

This theoretical observation leads us to understand that adjusting the preference distribution to non-uniformity is key to generating uniform Pareto objectives. Hence, by applying the continuous mapping theorem, the final objective distribution $h \circ p'$, with $p'$ as the adjusted preference, achieves asymptotic uniformity.

This paper appears to be the first rigorous study of MOEA/D uniformity theory. While many MOEA/D variants exist, such as preference adjustment techniques, previous studies mostly relied on heuristic strategies, like modifying preferences in sparse or dense regions, or used very simple models for the Pareto front. Our experiments confirm that heuristic methods and simple models alone are insufficient for uniform solution generation.

Reviewer comment (Experiments):
What is the scope of the proposed adaptive weight adjustment method? Is it suitable only for decomposition-based multi-objective evolutionary algorithms? If not, how would it be used in other frameworks, e.g., dominance relation-based, indicator-based frameworks?

Author response:
**Q1. Is it suitable only for decomposition-based multi-objective evolutionary algorithms?**

In our current research, we have effectively demonstrated the efficiency of our novel uniform MOEA/D approach, integrating preference adjustment through a neural model, within two methodologies: decomposition-based and indicator-based. However, we are yet to ascertain its compatibility with dominance-based MOEAs, an aspect we aim to investigate in future studies.

We have tested the proposed method within the MOEA/D framework, referring to the decomposition-based method as you mentioned. In fact, we have also innovatively improved the PaLam [1] method, originally designed to maximize hypervolume (referring to the indicator-based method as you mentioned) through adaptive reference vectors, by applying a neural adjustment strategy (firstly proposed in this paper). Our results, particularly observable in Tables 3 and 5, reveal that the hypervolume-based method, when enhanced with our neural weight adjustment technique, outperforms other methods of achieving a higher hypervolume.

Reviewer comment (Experiments):
How were the test problems in the experiments chosen? For example, for the DTLZ test suite, why were only DTLZ1-2 used, but not the more complex other problems?

Author response:
**Q2. Why were only DTLZ1-2 used, but not the more complex other problems?**

Our codebase, developed mainly in Python using pymoo and pytorch, currently faces limitations with the DTLZ4 problem due to pymoo's performance issues. Efforts are underway to rectify these limitations and we plan to release an updated, open-source version of pymoo soon. Notably, our method models the Pareto front directly. Since DTLZ2 and DTLZ4 share identical Pareto fronts, our successful results on DTLZ2 indicate similar performance on DTLZ4.

Reviewer comment (Theory):
The motivation of this paper assumes that the optimization problem is a black box and thus proposes a neural model as a differential surrogate. The main concern is that this motivation is improper for neural network optimization, in which you can use a gradient descent optimizer. Equation (5) can also be optimized without the proposed neural surrogate model.

Author response:
This **'differential surrogate'** is actually pretty **essential**, and up to our latest research, it **can not be replaced**. 

The primary challenge with the 'gradient descent optimizer', as you have mentioned, is the **time-intensive** task of estimating the gradient of the uniformity function (defined in Equation (5)). The Pareto objective, denoted as $y(\lambda)$, under a given preference $\lambda$, is **not directly ascertainable** and **requires solving a MOEA/D** (Multi-Objective Evolutionary Algorithm based on Decomposition) subproblem for its estimation. To compute the numerical gradient $\frac{y(\lambda + \delta \lambda) - y(\lambda)}{\delta \lambda}$, it is necessary to solve two MOEA/D subproblems for preferences $\lambda + \delta \lambda$ and $\lambda$ respectively. Given that solving a single MOEA/D problem takes approximately **one minute**, and considering the **frequent need** for this gradient, the use of a numerical gradient becomes **excessively time-consuming**.

Reviewer comment (Experiments):
The baselines used in the experiment are weak and old. A lot of work in the field of evolutionary multi-objective optimization has discussed adaptive reference/preference vectors.

Author response:
Thank you for your feedback. We agree with you that our paper's problems and benchmarks might not fully capture the latest developments in the field. Nonetheless, our work's key contributions are significant: (1) theoretical asymptotical uniform Pareto objectives, and (2) the successful integration of a neural network for Pareto front modeling.

In light of your suggestions, we're now comparing our method with more recent works to strengthen our findings. It's important to note, however, that some recent studies (e.g., [1-3]) are primarily minor strategy adaptations of the foundational MOEA/D-AWA work [4]. Their heuristic approach or reliance on simple linear models (like [1]) remains unchanged. Given our theoretical contributions and strong performance, we believe our work are valuable. 

[1] Dong, Zhiming, Xianpeng Wang, and Lixin Tang. "MOEA/D with a self-adaptive weight vector adjustment strategy based on chain segmentation." Information Sciences. 

[2] Xu, Siwen, et al. "A modified MOEAD with an adaptive weight adjustment strategy." 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS). 

[3] De Farias, Lucas RC, et al. "MOEA/D with uniformly randomly adaptive weights." Proceedings of the genetic and evolutionary computation conference. 2018. 

[4] D. Qi, Yutao, et al. "MOEA/D with adaptive weight adjustment." Evolutionary computation 22.2 (2014).

Reviewer comment (Presentation):
Figure 1 is unclear and needs improvement.

Author response:
Thanks for mentioning that, we have made this figure more clear in the revised version.

Reviewer comment (Theory):
The claim in section 3.3, "whereas the proposed method aims to achieve global optimal MOO solutions," seems to be improper.

Author response:
Thank you for pointing that out. We acknowledge that our initial statement may have been vague and we intend to clarify it in the revised version of our paper. To express this more accurately, unlike gradient-based methods, which focus on finding Pareto stationary points characterized by a zero gradient, evolutionary algorithms employ improvement and repair mechanisms, such as crossover and mutation operators. These techniques enable evolutionary methods to effectively bypass local optima and often yield superior performance, especially when computational resources are abundant. In contrast, gradient-based methods can become trapped in local optima under similar conditions.

Reviewer comment (Experiments):
The main body of this paper is missing an ablation study section.

Author response:
Thank you for your valuable suggestion. Actually, we have done a lot of ablation studies in the current paper, though they are not marked in an explicit manner. For your convenience, we summarized the ablation studies in the current paper.  

- 1. Initially, we examine the effects of integrating a neural-based adjustment method as opposed to the conventional MOEA/D approach. 
- 2. Subsequently, we compare our neural-based adjustment strategy with the heuristic methods that have been utilized in earlier studies.

It's noteworthy that our proposed method significantly outperforms these existing approaches. We will make the ablations more clear in the revised version.

Reviewer comment (Novelty):
The relevance to previous works is not clear enough. It seems that the technique of replacing the preference-to-objective mapping by a neural network as the surrogate model is developed from (Borodachov et al., 2019), and the generalization error analysis is adapted from prior works; hence, it would be helpful to clarify the technical difficulty or novelty compared to these works.

Author response:
**W1. The relevance to previous works is not clear enough. It seems that the technique of replacing the preference-to-objective mapping by a neural network as the surrogate model is developed from (Borodachov et al., 2019), and the generalization error analysis is adapted from prior works; hence, it would be helpful to clarify the technical difficulty or novelty compared to these works.**

Firstly, we shall note that the citation (Borodachov et al., 2019) ([1]) is a statistics monograph, which does not mention the preference-to-objective neural model. The preference-to-objective neural model was **originally** developed by our paper and is a crucial mechanism to search uniform Pareto objectives, which is the main focus of our article.

Secondly, since we have developed this new model in the uniform setting, we provide a theoretical analysis of this model. To our knowledge, the analysis for the uniform setting we employ is also **novel**.

Lastly, the proposed **generalization bound**, as far as we know, **has not been previously proposed** in other works. The papers [1-3] cited in our work are actually pure mathematical papers. Paper [3] is used to prove the 3rd line of Theorem 2, while papers [1-2] are referenced as Lemma 6 to prove Theorem 1. These mathematical papers serve as tools to prove intermediate steps of our conclusions, and we have cited them appropriately. 

[1]. S. Borodachov, Douglas P Hardin, and Edward B Saff. Discrete energy on rectifiable sets.
Springer, 2019.

[2]. S. Borodachov, D Hardin, and E Saff. Asymptotics of best-packing on rectifiable sets. Proceedings
of the American Mathematical Society. 

[3]. Minwoo Chae and Stephen G Walker. Wasserstein upper bounds of the total variation for smooth densities. Statistics \& Probability Letters.