Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: In this work the authors propose a gradient-based method for identifying the Pareto Front of a Multi-Objective Optimization problem.   More specifically, they propose two methods, one based on Stein Variational Gradient Descent, and another based on overdamped Langevin dynamics to evolve forward a set of particles to the (local) pareto front.

The approach is based on the multiple gradient descent (MGD) formulation where the next particle descent direction is chosen to maximise the slowest decreasing rate amongst the objectives.  Assuming particles are advected along an unknown vector field $\phi^*$ the authors introduce a variational formulation which selects the $\phi$ according to MGD objective while encouraging diversity through the usual entropy regularisation.   This gives rise to SVGD-like dynamics.   An analogous formulation for Langevin Dynamics is also proposed.

The authors provide some theoretical guarantees for both in terms of concentration of the dynamics to the Pareto front.  Results are demonstrated for a number of numerical experiments.

main_review: The proposed method is an interesting contribution provides an interesting and novel application of SVGD dynamics.   The fact that the resulting dynamics are not driven by a gradient adds interesting challenges, which the authors recognize.  Both the paper and the supplementary information was clearly written, and the proofs relatively straightforward to follow.  However,  I do feel that there are some issues which needed to be addressed more carefully.

* For example: in the conclusion the authors state: "Theoretically, analyzing our methods are challenging due to the lack of explicit form of the invariant distributions".   This presupposes the assumption that the dynamics do concentrate onto an invariant measure -- and if so -- is it unique.    Theorems 3.1, 3.2 and 3.3 pre-suppose the existence of $\rho^*$ and implicitly it is assumed that it is unique.  The authors should address why they believe this is true a bit more carefully.   In the case of Theorem 3.3 this should be discharged using relatively standard results, and I would imagine it to be the case in the other results.

* Theorem 3.2 doesn't quite characterise the concentration to the Pareto front as is claimed in the text, but rather it provides a bound for the weighted kernel mean embedding of $g^*$.   This is understandable from the form of the proof, however, it does complicate affairs as there is strong parameter dependence $(m_d, \sigma^2, etc)$ on the LHS of both inequalities.   Would it be possible to extract bounds, asymptotics, etc specifically for $g^*$?   If not, then the authors should write the LHS explicitly, using the kernel trick etc, to demonstrate the explicit dependence on the constants.  

* The first part of Theorem 3.3 is essentially the Helmholtz-decomposition for weighted measures.  Ideally, the authors should cite this well-known result if possible, looking in the references of [8] for example which makes use of this result.

* The authors should explicitly define the notation $f_{[\rho^*]}$ somewhere a bit more clearly, as it was not immediately obvious on a first read of the text.

* I find the justification of the second factor on page 3 to be a bit unclear.  As written the argument suggests we want to maximise $-\mathbb{E}_{x\sim \rho}[\nabla \cdot \phi(x)]$, not as stated. 

Minor comments:

l17: power systems

l31: have been

l80: gradient

l125: it's a divergence operator not a diversity operator

l143: $\rho^*(x) \propto \exp(-f^*(x)/\alpha)$.

l158 The notation is completely unclear, and needs to be written far more carefully.
limitations_and_societal_impact: The authors have adequately addressed these
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies how to acquire diverse and representative Pareto solutions by gradient-based methods. Two algorithms -- MOO-SVGD and MOO-LD are proposed by using stein variational gradient descent and Langevin dynamics respectively, with convergence (to local Pareto optimal) guarantees. The experimental results demonstrate that MOO-SVGD can obtain visually uniform solution sets of ZDT and MaF1 problems, and get better Pareto fronts on fair ML and multi-task learning.

main_review: Strength:

The motivation is clear to the readers that the diversity of the Pareto solution set is crucial to MOO.
The intuitive review of the algorithm design is clear for both two algorithms.
Theoretical convergence of strongly convex functions is provided.
The experimental results are convincing and attractive, especially the ZDT and MaF1 problems.

Weakness:

Although the intuition behind using SVGD is clear, it is unclear that why MOO-SVGD is superior to find diverse Pareto solutions from a theoretical review (the authors show this limitation in Section 6).
Assumption 1 is too strong for the deep learning model, where the ReLu layer is not strongly convex, which is not sufficient to claim “we mainly focus on the finding locally optimal solutions” (line 68).
I think MOO-LD is not important to the paper logic, and the experiments show its impractical performance. What’s the purpose that the authors care about this? In the experiments, the authors also do not give an intuitive interpretation on why MOO-LD fails.
In the fair ML and multi-task learning experiments, the solutions are not that uniform as the previous 2 experiments. Could the authors provide more explanations?

Typos:
Line 168, “j,j,k” under the partial differential symbol.
Line 291, Fig 5 → Fig 2

limitations_and_societal_impact: No
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a method to learn multiple Pareto solutions for multi-objective optimization (MOO).
Specifically, the method iteratively updates a set of points by Stein variational gradient descent (SVGD) to encourage the diversity between the solutions. Another counterpart method based on Langevin dynamics is also provided.
The methods are theoretically guaranteed to converge to the Pareto front.
The experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.
main_review: Strengths:
1. The idea of encouraging the diversity between the solutions by SVGD is interesting.
2. The proposed methods are theoretically guaranteed to converge to the Pareto front.
3. The experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.

Weaknesses:
1. As a method to learn multiple Pareto solutions for MOO, the proposed method discussed no related works that learn multiple Pareto solutions, e.g., [1,2,3,4,5].
2. The proposed method cannot control the specific preference vector for each solution, while [1,2] can, suggesting a type of weakness of the proposed method.
3. The proposed method requires maintaining multiple solutions, which may have computation and storage problems, whereas [5] maintains one solution only, suggesting another type of weakness of the proposed method.
4. Since MOO-Langevin Dynamics seems inferior for every aspect, the necessity to present this method is not clear.

[1] Navon, A., Shamsian, A., Chechik, G., & Fetaya, E. (2020). Learning the Pareto front with hypernetworks. ICLR 2021.

[2] Lin, X., Yang, Z., Zhang, Q., & Kwong, S. (2020). Controllable Pareto multi-task learning. arXiv preprint arXiv:2010.06313.

[3] Shah, A., & Ghahramani, Z. (2016, June). Pareto frontier learning with expensive correlated objectives. In International Conference on Machine Learning (pp. 1919-1927). PMLR.

[4] Deist, T. M., Grewal, M., Dankers, F. J., Alderliesten, T., & Bosman, P. A. (2021). Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximization. arXiv preprint arXiv:2102.04523.

[5] Ruchte, M., & Grabocka, J. (2021). Efficient Multi-Objective Optimization for Deep Learning. arXiv preprint arXiv:2103.13392.

Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited? 

1. The idea of encouraging the diversity between the solutions by SVGD is interesting.
2. No related work that learn multiple Pareto solutions is discussed.

Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work? 

-Yes.

Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.) 

-Fair.

Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach? 

-The experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes novel gradient-based algorithms for finding solutions across the Pareto front by using Stein variational gradient descent (SVGD) (and Langevin dynamics), in multi-objective optimization (MOO) problems.

main_review: Strengths:
- The paper is well structured and well written.
- The paper focuses on the main challenge of MOO problems: find representative solutions on the Pareto front, and propose a novel and scalable algorithm for this problem.
- The general idea is well explained, and the motivation is clear and straightforward. 
- The authors provide theoretical guarantees and analysis for the proposed method.

Questions/suggestions/Weaknesses:

- Overall it is a good paper with clear motivation and good theoretical analysis. I have two main concerns:
   - Previous literature in the same direction as discussed in the paper is omitted from discussion and empirical evaluation. Specifically, recent methods propose profiling the entire Pareto front in a scalable manner by conditioning the model on the preference direction (e.g. [1, 2]).
   - Second, scalability of the approach: the method essentially trains multiple models. The number of models that are needed for achieving good cover of the Pareto front grows exponentially with the number of objectives. This may be a major drawback with large-scale problems and models.

- Experiments:
   - Missing comparisons to recent state-of-the-art MOO approaches such as [1, 2].
   - The Linear scalarization baseline is missing from most comparisons.
   - The authors only experiment with small-scale and/or synthetic examples. I encourage the authors to provide an empirical evaluation on large-scale problems, like common MTL baselines (e.g., NUYv2/CityScapes dataset).
   - In addition, experiments with a larger number of objectives ($>3$) would be beneficial.
   - How did you select the hyper-parameters (HPs) for your method and baseline methods?
      - For example, in Appendix C.1, you state that $c$ (or the bandwidth) parameter is selected to optimize the hyper-volume. However, you did not use a validation split, so does this mean $c$ is optimized according to the test set? How did you select other HPs like $\alpha$?
      - All methods use the same learning rate (lr) in a given experimental setup if I understand correctly. How did you select this lr? Is it possible baseline methods can be improved by selecting an appropriate learning rate for each method?


[1] Learning the Pareto Front with Hypernetworks, Navon et al., ICLR 2021.

[2] Efficient Multi-Objective Optimization for Deep Learning, Ruchte et al., 2021.





limitations_and_societal_impact: Yes.
review_title: null

Meta-review: This paper proposes and studies a novel algorithm, inspired by Stein variational gradient descent, whose aim is to find distinct elements on the Pareto front of a multi-objective optimisation problem.  Theoretical and empirical results support the proposed method.  All reviewers agreed that the paper would make an excellent contribution to NeurIPS, but I suggest the author(s) carefully consider the comments of Reviewer 6uVc, who explains that there are elements of the presentation of theoretical results that ought to be clarified in the manuscript.
Decision: Accept (Spotlight)