Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed a gradient-based MOO algorithm with adaptive reference vectors. The core idea is to view it as a bilevel optimization problem. Then the authors proposed an interactive solution for this bi-level optimization.
strengths_and_weaknesses: Strength:
- A new way to reformulate the task to bi-level optimization 
- Better performance than state-of-the-art methods.


Weakness:
- It is interesting to view this task as a bi-level optimization problem, but the solver seems not novel. The bi-level solving is similar with existing well-known Expectation–maximization algorithm or the way used in neural architecture search (NAS). So I have concerns about the technical novelty of this paper.

- There are two assumptions proposed in this paper. In which case we should the uniform one (GMOOAR-U) rather than HV-optimal one (GMOOAR-HV)?

- Another concern is the scalability of the proposed method. It can be seem from Table 1 and Table 7 that the performance gain is getting worse and worse when extending to large models. 


questions: - It is interesting to view this task as a bi-level optimization problem, but the solver seems not novel. The bi-level solving is similar with existing well-known Expectation–maximization algorithm or the way used in neural architecture search (NAS). So I have concerns about the technical novelty of this paper.

- There are two assumptions proposed in this paper. In which case we should the uniform one (GMOOAR-U) rather than HV-optimal one (GMOOAR-HV)?

- Another concern is the scalability of the proposed method. It can be seem from Table 1 and Table 7 that the performance gain is getting worse and worse when extending to large models. 


limitations: The authors did not talk about limitations of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Paper tackles multi objective optimization (MOO), and proposes a gradient based MOO using reference vectors(RVs). Previous RV based MOO algorithms use a fixed set of RVs, and they might not be able to cover some parts of the Pareto Front (PF). Authors propose to jointly learn RVs along with the solving the MOO problem by formulating it as a bilevel optimization problem. Authors present an algorithm for solving the bilevel optimization problem, and theoretical convergence analysis of the algorithm is also provided. 
strengths_and_weaknesses: The paper is novel and interesting. It seems to be more reasonable to learn the RVs rather than using a fixed set of RVs.

1. Some more details about how the FilM conditioning is done using RVs should be added for completeness.

2. Datasets in Section 4.1 and even the Multi-MNIST, Multi-Fashion and Multi-Fashion+MNIST are synthetically created. It would have been interesting to see results on more real world tasks such as the multi-objective tasks considered in YOTO [14] (image compression).
questions: See strengths and weaknesses.
limitations: Authors should add some discussion about whether their algorithm can scale to large scale datasets (for e.g. large scale MTL tasks like image classification + object detection + semantic segmentation ).
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a novel multi-objective optimization algorithm based on reference vectors (RV).
In contrast to prior works, the paper proposes learning these RVs using gradient-based optimization instead of using pre-specified and fixed RVs. 
This is formulated as a bilevel optimization problem, where the lower level obtains the optimal solution given the RVs, and the upper level optimizes a quality measure w.r.t. the learned RVs (uniformity and the hypervolume indicator are considered as quality measures).  
An efficient solver for the bilevel optimization problem is proposed, requiring just one gradient descent step for lower and upper-level problems. A convergence analysis is also provided.
The algorithm is evaluated on both synthetic and real-world problems and achieves state-of-the-art performance. 
strengths_and_weaknesses: Strengths:
- The approach to extend existing RV-based approaches with a mechanism to adaptively learn said RVs is sensible and appears to improve upon the baselines in all cases
- The formulation as a bilevel optimization problem is elegant, and the provided solver is efficient, quite simple, and appears to work well in practice.  
- The overall presentation is excellent. The technical presentation of the method and background material is sound and clear, and the method is also well-motivated.
- The experiments cover a good number of benchmarks from more artificial/synthetic ones to more real-world applications. 

Weaknesses:
- The experiments are most convincing on the somewhat artificial benchmarks around MNIST and FashionMNIST. The experiments might be more compelling with more real-world applications, e.g., multi-label image classification with more than two objectives. 
- Adding a comparison in terms of computational efficiency (e.g., in terms of runtime) to Table 1, for example, could be helpful for the readers. 
- A discussion of the limitations of the method appears to be missing. 
questions: I would appreciate it if the authors could address the weaknesses mentioned above, i.e., 
- how does the method compare in terms of efficiency with prior approaches
- a discussion of the limitations of the method
limitations: A section with limitations is unfortunately missing and would definitely benefit the paper.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper aims to obtain a set of Pareto-optimal solutions that can better uniformly cover the Pareto front by learning a set of appropriate reference vectors (RVs). The proposed method is formulated as a bilevel optimization problem where we can obtain the Pareto-optimal solution in the inner loop and optimize the RVs by maximizing the quality function in the outer loop. The convergence analysis of the proposed method is provided and empirical studies show the effectiveness of the proposed method.
strengths_and_weaknesses: Strengths：

1. This paper aims to obtain a set of Pareto-optimal solutions that can better uniformly cover the Pareto front by learning a set of appropriate reference vectors (RVs), which is meaningful for multi-objective optimization.

2. This paper is well-written and easy to follow.

Weaknesses：

1. The novelty of this paper is limited and it is just an ensemble of the existing works. The RV-conditioned network is adapted from [14] (as mentioned in Section 3.1). The solver (Section 3.3) of the bilevel problem in Eqs. (8)-(9) is more trivial, which is similar to MAML.  

2. The description of Pareto domination in Lines 38-39 is problematic.

3. How to compute the gradient w.r.t. $R$ in Eq. (7)?

4. The Eq. (9) is equivalent to minimize $\sum_{j=1}^m (\sum_{i=1}^n r_i(j))f_j(\phi)$ and can only obtain a Pareto-optimal solution based on a new reference vector $(\sum_{i=1}^n r_i(1), \cdots, \sum_{i=1}^n r_i(m))$ rather than a set of Pareto-optimal solutions (as mentioned in Line 120). Thus, how the proposed method can obtain multiple Pareto-optimal solutions and how to guarantee those solutions can uniformly cover the PF.

5. In Section 3.3 and Algorithm 1, it seems the hypergradient (i.e., the gradient of $\phi_{k+1}$ w.r.t. $R$) is ignored, which may cause a large approximation error.
questions: Refer to above.
limitations: Refer to above.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper propose a novel gradient-based multi-objective optimization (MOO) algorithm with adaptive reference vectors (RVs). They formulate reference vector adaption as a bilevel optimization problem, and solve it with an efficient solver. Experiments on an extensive set of learning scenarios demonstrate the superiority of the proposed algorithm over the state-of-the-art.
strengths_and_weaknesses: Strengths:
- The motivation is clear and straightforward.  
- This paper is well-written.
- The authors provide theoretical results on the convergence of the proposed Algorithm.
- Experiments on several learning scenarios, including synthetic problems, multi-task learning, accuracy-fairness trade-off, and usage on larger networks demonstrate the effectiveness of the method.

Weaknesses:
- The novelty of this paper is limited. They simply formulate how to generate a proper RV and the model's parameters as a bilevel optimization problem and solve it with an solver, which is limited novelty. Since the idea of RV adaption is commonly used in some evolutionary algorithms [40][26] and the bilevel optimization problem also is a common problem, the authors simply combine these two together to generate their method.
- The solver proposed in this work (GMOOAR) to solve the Bilevel Optimization Problem seems exists some problems. I am not sure can solve the bilevel problem in this simple way, the algorithm needs to be explained more detail.
- How the authors can guarantee the generated RVs are proper RVs?  There needs more experiments to demonstrate this point.
- There miss some SOTA methods to compare, such as *Multi-Objective Meta Learning* (Ye et. al. , NeurIPS 2021).
questions: I have some questions about this work:
- As the authors claim in the Line 5 " they may still produce undesirable solutions due to mismatch between the pre-specified reference vectors and the problem’s underlying Pareto front." Then why do we need to get the PF by utilizing the RVs ? I don't understand this point and hope the authors can make some explanations, since this is the major motivation of this work.
- The authors propose to learn the set of RVs simultaneously with the model parameters and formulate the lower-level optimization problem obtains the Pareto-optimal solutions with the given set of RVs, while the upper-level problem optimizes the RVs based on some quality measure. Do the authors try the different level orders? That is the lower-level optimization problem optimizes the RVs while the upper-level optimization problem obtains the Pareto-optimal solutions. I'm just curious about that.
limitations: For limitations see Weaknesses and Questions. No social impact discussed. Given the effort NeurIPS has put into this effort, I believe something should be added to discuss this.
review_title: null

Meta-review: metareview: The paper tackles the multi-objective optimization (MOO) problem using adaptive reference vectors(RVs). The authors claim two main contributions:  (1) the use of adaptive reference vectors (RVs), instead of fixed ones as in existing gradient-based MOO algorithms； (2) RV adaptation is based on the gradient of the quality function.  It is reasonable to learn the RVs rather than using a fixed set of RVs. The authors also provide theoretical results about the proposed method. However, the concerns raised by reviewers should be carefully considered and addressed. Also, the authors also consider only 2 or 3 objectives. It would be interesting to see results with more objectives  (see >10) on classical numerical multi-objective optimization problems.  Last, Figure 1 is confusing and difficult to understand.  
confidence: Less certain
Decision: Accept