Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper has two contributions. First it argues for the need to benchmark multi-objective problems  using analytic functions for convex and non-convex conditions. Second, it presents a two-stage method for finding the Pareto front. 
main_review: Pros:
- I resonate with the call for analytical benchmarks. Multi-objective optimization as a field may be growing within ML, but I feel many work are not evaluated in a way that is conducive to a holistic understanding of the methods capabilities and limitations. Using these benchmarks, the paper establishes clear cases where recent method fails. 
- The paper helps bridge some of the gap between multi-objective work in ML and Operations Research.  I think an ML audience will learn much from the paper, just like I did. 
- The proposed two-stage method is reasonable, and shows good performance on the proposed benchmarks.

Cons: 
- The proposed two-stage method is a somewhat orthogonal contribution, and I am not sure if it really connects well with the benchmark contribution. 
- I wonder if the neural net for weak Pareto front will be practical in applications where it is difficult to compute the value in objective space. For example, if we were to optimize the hyperparameters of deep neural nets like [1] and [2], it may be expensive to obtain samples for this neural net. Some discussion would be good.
- Pareto density (Table 2) is not the only way to evaluate, and it does penalize methods like MTL. There are other evaluation metrics, like hyper volume, which may result in a different ranking of methods. Some discussion would be desired. 

[1] Zhang & Duh. Reproducible and Efficient Benchmarks for Hyperparameter Optimization of NMT Systems
. TACL2020 https://aclanthology.org/2020.tacl-1.26/
[2] Klein & Hutter. Tabular benchmarks for joint architecture and hyper-parameter optimization. https://arxiv.org/abs/1905.04970

Additional comments: 
- It will be great if the authors release the data or generator code (and evaluation code) for the benchmarks in an easy-to-use fashion. I understand it is relatively easy to implement, but it can help encourage more people to work on the benchmark.
summary_of_the_review: I think this paper presents a strong argument for benchmarks in multi-objective problems. More discussion that ties the benchmark and the evaluation of methods back to various practical considerations (e.g. feasibility of sampling in the objective space, desirability of Pareto density vs. hypervolume) would make this paper even stronger.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work proposes a Hybrid Neural Pareto Front (HNPF) framework to solve multi-objective optimization problems. The proposed method needs to first generate a huge number of feasible solutions to cover the whole decision space (e.g., random sampling in this work), and then use a two-stage approach to select the Pareto optimal solutions. In the first stage, it builds a Fritz-John Condition (FJC) based neural network to verify the weak Pareto optimality for given solutions. In the second stage, it further filters the truly strong Pareto optimal subset from the obtained solutions. Experiments have been conducted on several low-dimensional problems to validate the proposed method's performance.

main_review: **Strengths:**

+ It studies how to find the whole Pareto set for multi-objective optimization problems, which could be useful for different applications.    

+ The call for new benchmarks for multi-objective optimization other than multi-task learning is valuable, also see a related work [1] (I am not involved in this paper). However, this discussion is in the appendix and not the main point of this work. 

**Weakness:**

*Method:*

**1. Verification and Optimization:** 

The proposed HNPF method is for verification (e.g., check whether a given solution x is Pareto optimal), but not for optimization (e.g., find an (approximate) Pareto solution x). It needs an extra search method, such as random sampling in this work, to first generate a large number of feasible solutions to cover the whole search space. Therefore, the underlying optimization is indeed random sampling (independent from HNPF), which could be extremely inefficient for a non-trivial search space. It is not suitable to put and compare the proposed HNPF method with other optimization methods that can directly find the (approximate) Pareto solution. 

Since HNPF depends on random sampling, it is not surprising that it can only work for small scale problems.

**2. The Reason to Build the Model:**

HNPF needs to first build a neural network to check whether a given solution x satisfies the Fritz-John Condition (FJC), which requires a large number of training samples (e.g., 11K for a two-dimensional problem). The learned model is mainly used to classify whether the extra randomly sampled solutions (e.g., 9K) are weak Pareto optimal or not. The reason for model building, such as the advantage over the simple FJC rule-based classification, is not well motivated and justified in this work.    

The proposed Pareto filter in stage 2 is not discussed and compared with other related nondominated sorting algorithms (e.g., [2]).

**3. Necessary Condition for Pareto Optimality:**

The KKT[3] and FJC[4] are two types of first order necessary conditions for (local) Pareto efficiency (Pareto optimality). In my understanding, the multi-objective optimization based MTL algorithms mentioned in this work (Sener & Koltun, 2018; Lin et al., 2019a; Mahapatra & Rajan, 2020; Ma et al., 2020; Navon et al., 2021) mainly use the gradient-based multi-objective optimization methods (e.g., MGDA) [5-7], which is based on the KKT condition. For these methods, in each update step, the gradient can be written as a linear combination of the gradient for each objective with adaptive weights derived from the KKT condition. Therefore, they are all different from the simple linear scalarization with fixed weights. All the claims and analyses in this work that the previous works use simple linear scalarization is not correct.

The FJ condition is also for local Pareto convergence, similar to the KKT condition. The global convergence property is solely due to random sampling that only works for extremely low-dimensional problems. It is unfair to say the proposed algorithm can overcome the local convergence of other gradient-based methods. In addition, the proposed algorithm heavily depends on the Fritz-John condition, but the original work [4] is not cited.

**4. Linear Scalarization and Convex Pareto Front:**

It is well-known that the simple linear scalarization cannot find the non-convex part of the Pareto front [8]. This finding leads to the seminal work on NBI scalarization (Das & Dennis.,1998), which is indeed one fundamental work that inspires the proposed method in this work (section 4, first sentence). The claim "it is incorrect to state that LS itself fails if the Pareto front is non-convex" (appendix, page 15) is questionable. 

Since the proposed HNPF can only verify whether a given solution is weak Pareto optimal, its ability to find the whole Pareto front totally depends on the extra sampling method (e.g., random sampling) to generate all the Pareto solutions (might be infinite). It is misleading to indicate the proposed HNPF method itself can find the whole Pareto front. In addition, since the Pareto set has measure zero and infinite cardinality, the random sampling + HNPF method can at most find a dense approximation to the Pareto set.     

*Experiment:*

**5. Algorithms for Comparison:**

All the multi-objective optimization based MTL algorithms are designed for optimizing a deep neural network with millions of parameters. They implicitly depend on the assumption that the deep neural network has good properties (e.g., no bad local optimum [9][10]) on its loss functions, which is consistent with other gradient-based single-objective optimization methods. They are not designed to find the global Pareto front for low-dimensional problems. 

For low-dimensional problems, it is more suitable to compare with the model-free multi-objective optimization methods such as the multi-objective evolutionary algorithm [11,12] and multi-objective CMA-ES [13]. If the model building is allowed, Multi-Objective Bayesian Optimization (MOBO) algorithms can have a very good sampling efficiency for the low-dimensional problems [14,15]. It is also very common to conduct non-dominated filtering at the end of those model-free algorithms or MOBOs (e.g., only keeping the current non-dominated solutions). 

**6. Training + Sampling:**

The proposed method needs to first sample 11k solutions to train the neural network model, then randomly generate extra 9K solutions for filtering. Is there any advantage over simply using FJC to filter 9K (or 11k + 9K) randomly sampling solutions?

**7. Figure from Other Works:**

Many figures in the main paper and the appendix are directly borrowed from other works. I think this is not appropriate even the credits are given to the original works. 

**Reference:**

[1] Ruchte, Michael, and Josif Grabocka. Multi-task problems are not multi-objective. arXiv preprint arXiv:2110.07301, 2021.

[2] Roy, Proteek Chandan, Kalyanmoy Deb, and Md Monirul Islam. An efficient nondominated sorting algorithm for large number of fronts. IEEE transactions on cybernetics 49, no. 3: 859-869, 2018.

[3] Kuhn, H. W., and A. W. Tucker. Nonlinear Programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, pp. 481-492. University of California Press, 1951.

[4] Da Cunha, N. O., and E. Polak. Constrained minimization under vectorvalued criteria in finite dimensional spaces. Journal of Mathematical Analysis and Applications, 19(1), 103–124 ,1967.

[5] Fliege, Jorg, and Benar Fux Svaiter. Steepest descent methods for multicriteria optimization. Mathematical methods of operations research 51, no. 3: 479-494, 2000.

[6] Fliege, Jorg, and A. Ismael F. Vaz. A method for constrained multiobjective optimization based on SQP techniques. SIAM Journal on Optimization 26, no. 4: 2091-2119, 2016.

[7] Desideri, Jean-Antoine. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. Comptes Rendus Mathematique 350, no. 5-6: 313-318, 2012.

[8] Das, Indraneel, and John E. Dennis. A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems. Structural optimization 14, no. 1: 63-69, 1997.

[9] Kawaguchi, Kenji. Deep learning without poor local minima. NeurIPS 2016.

[10] Kawaguchi, Kenji, and Leslie Kaelbling. Elimination of all bad local minima in deep learning. AISTATS 2020.

[11] Deb, Kalyanmoy, Amrit Pratap, Sameer Agarwal, and T. A. M. T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, no. 2: 182-197, 2002.

[12] Zhang, Qingfu, and Hui Li. "MOEA/D: A multiobjective evolutionary algorithm based on decomposition." IEEE Transactions on evolutionary computation 11, no. 6: 712-731, 2007.

[13] Igel, Christian, Nikolaus Hansen, and Stefan Roth. Covariance matrix adaptation for multi-objective optimization. Evolutionary computation 15, no. 1: 1-28, 2007.

[14] Daulton, Samuel, Maximilian Balandat, and Eytan Bakshy. Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization. NeurIPS 2020.

[15] Konakovic Lukovic, Mina, Yunsheng Tian, and Wojciech Matusik. Diversity-Guided Multi-Objective Bayesian Optimization With Batch Evaluations. NeurIPS 2020.


summary_of_the_review: This work aims at finding the Pareto front for a multi-objective optimization problem, which is important for many applications. However, there are many major concerns on both the method (verification rather than optimization, the reason for model building, questionable claims) and experiments (suitable comparison, figures from other works) that make this work unacceptable in the current form.

============================= post rebuttal response ============================= 

I have read the other reviewers' comments with the authors' feedback, but still think the proposed verification method (FJC discriminator) should not be directly compared with other optimization methods (not limited to MTL). Therefore, I keep my initial score (3) and lean toward rejection.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper intends to find points on the Pareto frontier of a multiobjective optimization problem. Despite other baseline methods for this problem, they claim that their proposed algorithm is suitable for non-convex optimization, and their solutions are evenly spread across the frontier. Also, their algorithm can handle constraints for the MOO problems similar to solutions proposed in existing operation research methods. Empirical evaluations show that the proposed algorithm can find Pareto points and omit non-dominated points in the final stage.
main_review: Although they are claiming that their proposed algorithm solves the issues mentioned in the paper, I think the literature is not represented fairly to show novelty here. For instance, they mentioned that "the theorems presented in MTL works rely upon convexity assumption that are never justified through numerical experiments." This claim is not true. Just for one case of [A], they explicitly discuss that they have no convexity assumptions. There are other proposals, mostly with multiple gradients descent algorithm (MGDA) variations that have convergence guarantees for both convex and non-convex objectives such as [B]. Besides, there is confusion regarding the convexity, whether they are discussing the convexity of objective functions or the Pareto frontier. Since even with convex objective functions, the Pareto frontier can be still non-convex.

In addition to the lack of fair representation of the literature to justify the claims, I have several other concerns regarding the approaches in this paper as follows:
1. In most parts, they are using algorithms and proposals from other papers, hence it is not clear what is their own novelty added in this paper. For instance, the FJC for the optimality and Kd-tree algorithm seems to be borrowed from other papers. Hence, it is not clear, what is the novelty in this paper since these are the main parts of this proposal.  They use the optimality condition derived from FJC to use as a loss function for an MLP in order to decide whether a point is a Pareto or not.

2. In addition, as they mentioned themselves, their proposal has some similarities with PHN [C]. In the case of PHN, the algorithm and training procedure is an end-to-end solution and very clear. However, in this paper, the training procedure is not clear at all. The main concern is that how they are getting the $X$ parameters in the first place to feed to their MLP model for prediction of Pareto and not Pareto. Are they using some existing methods like EPO to find $X$? The training procedure should be stated clearly. 

3. The filtering of Pareto points is not novel in my opinion. This is the procedure most of the proposed algorithms in this domain are doing one way or another. Since this is a search space over candidates for dominated points on the weak Pareto set is not that big, the computational complexity is not that big a concern to be claimed as the novelty here. See this [link](https://stackoverflow.com/a/40239615) for several solutions for this problem in the community.

In addition to the aforementioned concerns, one of the main concerns, I reflected upon several times, is regarding the clarity of the manuscript. I believe many parts of this paper require more clarification and in-depth discussion while some parts can be omitted or moved to the appendix. The distinction between the contribution of this paper and other proposals should be clear, and the contribution of the literature should addressed fairly and clearly.



[A] Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In International Conference on Machine Learning (ICML), pp. 6597–6607, 2020.

[B] Kamani, Mohammad Mahdi, et al. "Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing." arXiv preprint arXiv:2104.01634 (2021).

[C] Aviv Navon, Aviv Shamsian, Gal Chechik, and Ethan Fetaya. Learning the pareto front with hypernetworks. In International Conference on Learning Representations (ICLR), 2021. URL https://openreview.net/forum?id=NjF772F4ZZR.
summary_of_the_review: Overall, I believe the novelty of the proposed algorithms is marginal and the clarity of the paper is very low at this stage.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors present key shortcomings of MTL solvers in addition to Hybrid Neural Pareto Front (HNPF) that aims to handle non-convex functions and constraints.

The authors claim the following contributions:
- New strategy for weak Pareto front identification based on Fritz-John conditions.
- New Pareto filter to remove dominated points from the weak Pareto front.

main_review: My main concern about this work is its scalability. At first phase of finding the weak pareto front HNPF needs to sample the solution space in uniform manner. Covering the solution space for real life solution is not feasible (in terms of memory and runtime) as such models contain hundreds of millions of parameters. Next, the authors propose a Neural Net (NN) to approximate the weak pareto front, since the NN optimization process is depend on labels there is additional labeling process. Although, this labeling process is automatically (i.e. without human intervention) it requires the calculation of $det(L^TL)$ where $L\in\mathbb{R}^{\(n+m\)\times\(k+m\)}$ per sampled solution. The NN network is working on the solution/model parameter space which is not scalable as NNs contain much more parameters than their input space to ensure degrees of freedom.

For the second step HNPF utilizes variant of Kd trees to filter our dominated solutions. In my perspective the novelty of this section is limited, it can be applied to any MTL/MOO approach. It will be interesting to see how the compared methods' results change after applying this filter.

The experiment part is a bit weak. Due to the scalability problem the authors did not test HNPF on more challenging datasets (not even MNIST).



summary_of_the_review: Some questions were raised during the review process.
review_title: null

Meta-review: The paper makes two contributions: (1) Multi-task benchmarks where the pareto solution is known analytically; and (2) a verification method for testing if solutions are on the pareto front. The authors make the point that MTL methods are applied to large-scale problems, but fail to find the pareto front in problems where it is known. 

Reviewers appreciated the discussion and insights by the approach, and the idea that correctness of scalable methods should be evaluated with problems that have analytic solutions, but they also had grave concerns. The primary concern is that without an efficient search, a verification method that builds on filters randomly generated solutions cannot scale to high dimensional problems. There were also disagreement about the role of LS and comparison with previous literature. 

As a result, the contribution of the submission is not sufficient for acceptance to ICLR
Decision: Reject