Profiling Pareto Front With Multi-Objective Stein Variational  Gradient Descent
Authors: Xingchao Liu
Xin Tong
qiang liu
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: Pareto Front
Sampling
Multi-objective Learning

Abstract: Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO). In this work, we propose a novel gradient-based algorithm for profiling Pareto front by using Stein variational gradient descent (SVGD). We also provide a counterpart of our method based on Langevin dynamics. Our methods iteratively update a set of points in a parallel fashion to push them towards the Pareto front using multiple gradient descent, while encouraging the diversity between the particles by using the repulsive force mechanism in SVGD, or diffusion noise in Langevin dynamics. Compared with existing gradient-based methods that require predefined preference functions, our method can work efficiently in high dimensional problems, and can obtain more diverse solutions evenly distributed in the Pareto front. Moreover, our methods are theoretically guaranteed to converge to the Pareto front. We demonstrate the effectiveness of our method, especially the SVGD algorithm, through extensive experiments, showing its superiority over existing gradient-based algorithms.

Profiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Xingchao Liu
Xin T. Tong
Department of Computer Science University of Texas at Austin xcliu@utexas.c edu
Department of Mathematics
National University of Singapore mattxin@nus, .ww.uuuuueeudu.... sg
Qiang Liu
Department of Computer Science University of Texas at Austin lqiangBcs. texas. edu
Abstract
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO). In this work, propose novel gradient-base based algorithm for profiling Pareto front by using Stein variational gradient descent (SVGD). We also provide counterpart of our method based Langevin dynamics. Our methods iteratively update a set of points in a parallel fashion to push them towards the Pareto front using multiple gradient descent, while encouraging the diversity between the particles by using the repulsive force mechanism in SVGD, or diffusion noise in Langevin dynamics. Compared with existing gradient-based methods that require predefined preference functions, our method can work efficiently in high dimensional problems, and can obtain more diverse solutions evenly distributed in the Pareto front. Moreover, our methods are theoretically guaranteed converge the Pareto front. We demonstrate the effectiveness of our method, especially the SVGD algorithm, through extensive experiments, showing superiority over existing gradient-based algorithms.
Introduction
Many scientific and engineering problems involve optimizing multiple conflicting objectives [5, 28, 4, 24), including, for example, designing wireless sensors [12], building electric power systems [30]. and training neural networks with multiple tasks [35]. With multiple conflicting objectives, it is
impossible to find a single solution that optimizes all the objectives simultaneously. Instead, it is
essential to find set of diverse solutions in the Pareto front that represent different preferences on
the different objective functions, so that the users can have a global view of how the optimal trade-off' of the different objectives look like and select the solution according their own preference.
Unfortunately, profiling the Pareto fronts casts a key computational challenge, especially for high dimensional problems [5. 12, 28). Traditionally, large literature has been devoted to developing black-box, derivative-free algorithms that are suitable for black-box optimization, such as these based on evolutionary algorithms [8] and Bayesian optimization [25, 1, 39]. However, the black-box algorithms tend to be expensive and can only be applied to small scale problems due to the lack of gradient information. Gradient-based MOO algorithms have been catching attention only recently, which include mainly multiple gradient descent (MGD) based methods [10, 22, 27J. However,
"Code is available at httpp://ggiee com/ http://wt..........a/aaaaaa//taa
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

the results of these methods are still dissatisfying in finding diverse and evenly distributed Pareto solutions in complex problems.
We introduce Stein variational gradient descent (SVGD) [20, 19, 21] and Langevin dynamics [37] as efficient approaches for profiling Pareto fronts. These methods iteratively evolve group of particles to represent the target distribution. The main difference of the two algorithms is how they distribute points. The Langevin dynamics uses stochastic noises to perturb the partiele trajectories. so they can visit different areas of the Pareto front. On contrast, SVGD is deterministic sampling algorithm that pushes the particles to high probability regions using gradient information, while enforcing diversity between the particles using a repulsive force.
In this work, propose to a simple approach to integrate SVGD and Langevin Dynamies with MGD to draw samples from the Pareto front. Theoretical analyses are provided for both algorithms to understand their limiting distributions and their convergence speed. One challenge with MGD based sampling is that the limiting distributions do not admit explicit formulations. This is mainly because the forcing from MGD is in general not the gradient of any function. However, we can show its non-gradient component is orthogonal to large class of functions. Assuming each objective function is strongly convex and regular, we can also show the limiting distributions concentrate on the Pareto front. Moreover, we can show the two algorithms converge to good solutions of MOO with (1/t) and linear rate.
We test our methods on a variety of tasks, ranging from low-dimensional optimization to multi-task neural network optimization. On all the tasks tested, our method can obtain diverse and high quality Pareto solutions that distribute evenly on the Pareto front, without predefined preference vectors. Quantitatively, we substantially outperform PF-SMG and EPO with respect to the hypervolume metric.
2 Background
In Multi-objective optimization (MOO), we are interested in minimizing a vector-valued loss function F(x) = (Ji (x), [2())))
E Rm, where x E IRd and ʃi Rd + R i [m]] is the i-th scalar-valued objective function. Here we use notation [m]] = (1, ... m Obviously, we can not fully optimize all the objective functions simultancously because they may be conflicting with each other. Instead, are interested in finding the points which can not be improved simultancously for the objective functions, yielding the notion of Pareto optimality.
Definition ( Pareto Optimality). For 21, F2 E TRd, We say that xı is dominated by D2 iff fi f1(22) S f((a))) Vi [m), and F(x1) + F(x2), point x" called globally Pareto optimal on Rdd iff is not dominated by any other x' E Rdd A point x² is called locally Pareto optimal iff there exists an open neighborhood (xx) xf, such that x" is not dominated by any x E NN(x).. ) The collection of globally (resp. locally) Pareto optimal points are called the global (resp. local) Pareto set. The collection of function values F(x") of all the Pareto points x is called the Pareto front.
Our goal is to find diverse and representative solutions {m:)²_ from the Pareto set, so that their function values ( (F (F(x)) covers different preferences on different objectives. This would allow the end-users to have a global view on the optimal trade-off between the different objective functions and decide which solution based on their own preference.
Linear Scalarization One standard approach to solve MOO is using preference vector À
[Ai...., Am_] from the probability simplex on [m]] i.e., S = (> n > = 1, 2 0, i E [m]]. Each X E S leads to a weighted objective function (x(x = E (x) and its minimizer ox = arg min, fx (zr). Then as we take À in grid of S, we hope that the corresponding 2i gives rough sketching of the Pareto front.
Although this strategy is simple and easy to implemented, it suffers from the a number of weaknesses. key problem is that F(=t) can only lie on the convex envelope of the Pareto front, and hence only works in cases when the Pareto front is convex. In addition, uniform grid of X on S does not necessarily yield uniformly distributed points on the Pareto front.
Multiple Gradient Descent (MGD) MGD a natural extension of the single-objective gradient descent to finding Pareto point [10], which, unlike linear scalarization, works for non-convex Pareto

---

fronts. The idea is to iteratively update the variable along direction that ensures that all the objectives are decreased simultaneously (which is called Pareto improvement).
Let g;(x) = V((()) be the gradient of the i-th objective. Suppose we update the variable by o' - eg" (1:), where g" 9a(xx) is vector field he determined and is small step size. By Taylor approximation, we have (gt, 9²) 2 - (fe(x') fi(x)))/e, which represents the decreasing rate of fi when we update x along direction g² (z). In MGD, g" chosen to maximize the slowest decreasing rate among all the objectives, that is,
g" (x) X arg max min (g, 9;())), s.t. Illll S (:E(m)
(1)
gERe
Therefore, (x) is encouraged to have positive inner products with all (x). If this is impossible to achieve. (x)) 1 will contain conflicting directions, and we would have (x() = which terminates the algorithm. Using Lagrangian duality, can show that the optimal solution to (1) is
(x) x 1= X; (x)gi(z), where {x)
m is the solution of
im1
m
min Σ (z)
s.t.
A = A 2 0, Vi.
(2)
1=1
This optimization has a simple closed-form solution when m 2. and fast algorithm offered by [35] for m > 2. By construction, when the step size e is small, MGD monotonically decreases all the objectives simultancously and will terminate when it arrives local Pareto point; in this case we have g*(x)= = Em 1 X7 (x) = 0, suggesting that the zero vector 0 is inside the convex hull of (g)
The MGD direction g" provides a natural notion of gradient for multiple objectives. point x is said Similar to the case of differentiable single-objective optimization, every Pareto local optimal is Pareto stationary point. See [10, 35, 22] for more discussion.
be Pareto stationary if g (xx)
However, the vanilla MGD suffers from several key weaknesses. Although it promises to return a point on the (local) Pareto set, and it is difficult to explicitly control which Pareto point will converge to. The convergence point of MGD is implicitly determined by the initialization and the other pareeerrrsmeterss of the algorithm (c.g., step size) in a complicated way. can be harder for MGD to converge to some Pareto points than the others. In fact, assume the Pareto front is open set in Rdd, then the vanilla MGD, when initialized from outside of Pareto front, will terminate when reaches the boundary of the Pareto front, and hence never reach the interior points. Below is simple example to demonstrate this.
Example 1. Consider fi(z) x² and = - 1)2 for x E R as shown in Figure 1. Then the Pareto set is the interval [0, 1]. However, when initialized outside of [0, 1), MGD will converge to either = 0 or = and can not reach the interior points of the Pareto set unless we add snsersity-promotinng mechanism, such as random noise or deterministic repulsive force. See figure I.
1.0
1.0
10
Tocie Frea
1.0
Dercs Fosel Serplc
1.0
Paraa Fost
Sample
Sungl:
0.0.5
0.00
< 0.5
<2 0.5
< 0.5
0.0
-1.0
0.0
0.0
0.0
0.5
Objectives
(b) MGD grddient
(0) MGD
(d) MMOO-VVD
(e) MOO-LD
Figure I: (a) The plot of fi and fa in Example 1. (b) The MGD direction g (x). The Pareto set is [0, 1), within
which we have 9" (=)==0 = 0. (c) MGD can only converge to either of the two end points (z = 0 or 1) of the Pareto set [0, 1] when initialized outside of [0. 1). (d) Solutions from MOO-SVGD;
30 particles with 3000 iterations. (e) Solutions from MOO-LD; we run the algorithm 10 steps, and uniformly sample 30 points from the last 1000 points.
3 Profiling Pareto front with Particle Dynamics
We improve MGD by integrating with sampling algorithms for approximating distributions to obtain diverse solutions that cover the Pareto front more evenly. In this section, we first introduce our
3

---

main algorithm, which integrates MGD with Stein variational gradient descent (SVGD), yielding an interacting particle system that evolves diverse Pareto solutions with deterministic repulsive forces (Section 3.1). We also introduce a related idea of combining MGD with Langevin dynamics to obtain diverse solutions by injecting diffusion noises, which is found less favorable in practice than SVGD-based method but is of independent interest (Section 3.2).
3.1 Multi-Objective Stein Variational Gradient Descent
Stein Variational Gradient Descent (SVGD) [20] is deterministic sampling method for finding diverse and representative points (a. particles) for approximating an un-normalized distribution, by iteratively evolving the particles with both gradient information and special repulsive force. In this section, we integrate SVGD with MGD and adapt it to profile the Pareto front.
We derive our algorithm following how SVGD was derived in [20]. We initialize set of points { (a.k.a. particles) and iteratively updates them to 1) mimic the MGD trajectory and 2) maintain diversity between the points. For the derivation, assume the particles {z;} follow distribution p and we update them with 2; < I; cΦ"(x) (z), , Vi E [n), where ø is an update direction, selected from a predefined function space H. to strike the balance of the following two factors:
1) We want to choose & to make the dynamics mimic the MGD trajectory as close as possible; this can be framed as maximizing the average inner product Ez~pl(9" (1), (xxr))), to make the direction of Φ and g" as close as possible.
2) We also want to choose o to encourage the diversity between the particles. Assume is a continuous distribution; the diversity of p can be measured by the entropy H1 (p), and hence we want to choose e such that the entropy of the distribution of the updated particles, denoted by p',
maximized. [20] showed that H(p')= = Hp)) cEzupl V x(x)] + O(e)), where V. denotes the
divergence operator in vector calculus. Therefore, we should choose & to maximize - -Ez~pl (V (xx)
to encourage the diversity of the updated particles.
Overall, we optimize & to maximize a linear combination of the two terms above: %f arg max (Bz=o[(g"( d(x) a Mxx]] where is a positive coefficient that controls the importance of the divergence term. Following [20], we take F to be the unit ball of a reproducing kernel Hilbert space (RKHS) with a positive definite kernel k(x, x²). With the same derivation as [20], we get a closed form solution: (z(z) x Ex'~o (2'))(( x) a V )=]
By approximating P with the empirical distribution of the particles o E, / and iteratively updating the particles with of above, we obtain the following MOO-SVGD:
- eδ(x;), where g(z.) 1 Σ g" g0,z) a M,)
We can see that MOO-SVGD simply replaces the gradient in SVGD with the multiple gradient g".
The intuition of the update is clear: the first term in å makes the dynamic follow the MGD direction so that the particles are pushed towards the Pareto set when {g (x:)} have large magnitudes; the second term pushes the different particles away from each other, so that more areas of the Pareto set is approximated. In particular, when all the particles are in the Pareto set with (²(x;) R O, then only the pairwise repulsive force will push the particles to be away from each other and form uniform distribution inside the Pareto set.
If g" is the gradient vector of some scalar function, i.e., (ee..gg(()) (x) then our method reduces
to the standard SVGD for sampling from distribution p² (xx x exp(-f"(x)/o In general, g" is not a gradient vector field of any cealar-valuce objective function, and hence our method yields a general "non -oaadind"nnn" variant SVGD. Similar to vanilla SVGD, we can characterize the evolution of the density of the particles with a nonlinear different equation. In the large particle (n - oc) and continuous time limit, let Pt be the limit density of the particles at time i (c -> 0). Following [19]. the evolution of Pt is governed by.
dpr(x) - d (o)
dt Unlike the case of the vanilla SVGD, the behavior of (4) is less clear due to the non-gradient nature of g" (r). In the following, we show that the stationary distribution of (4) is connected to Helmholtz

---

like decomposition of g" (z) into gradient component and non non-gradient components. To facilitate our discussion, we define the kernel embedding of any function f with density p as
f(1(z)= = ((le,))(())p(yddd
Theorem 3.1. Assume p² is a fixed point of for which log p" (a) is continously differentiable and satisfies the Stein's identity Ex~g- v log p² (z)k(z, x') + Vk(2, x') = 0 for every E R. Then we have the following decomposition of the vector field g":
(5) where V log' D" and can be viewed as the gradient and non- gradiend components of and )(x) satisfies the following property: its kernel embedding orthogonal the kernel embedding of any gradient field in RKHS H. that is, let h be any C function with a compact support then
p (I) + f()),
(Mee), 1/p 00
The existence of p" is discussed in the Appendix after the proof of Theorem 3.1. Next, we show that p" concentrates on the Pareto set. In order do so, recall that (x)]] indicates that x is on the Pareto set. Therefore we will show the norm of the MGD field g" is small over p² and SVGD updates can find a solution with small ||
To simplify our discussion, we limit our discussion on the case where a truncated (a, mau--sassiaan kernel is used: k(x, 2) (det(2ro²1)) exp(- शेत || yll²) f|= | Note that its gradient may be singular at the truncation threshold. Formally we can ignore such singularity and simply define formally: V Vzk(z, y) == - k(., 1 vil
We also assume the objectives functions
are regular and strongly convex:
Assumption 1. Each objective function Ji is C iee, third-order continuously differentiable, and 1 3 V 3 c2l, 10,kk S for i, k,1 E [d], where O < 61 S C2 S co, 0 S 03 < oc are constants. Here f is the Hessian matrix of fo and A 1 indicates that B is positive semi-definite matrix.
is well known that, with strongly convex single objective function, gradient based algorithm can converge to the global minimum. We show that this also true for MOO-SVGD method.
Theorem 3.2. Suppose Ig= C g" is L-Lipschitz and Assumption holds. If we apply MOO- SVGD using the (o,ma) trruunatdde Gaussian kernel k. the equilibrium distribution p" of MOO-SVGD satisfies
100 - (x))(((,,)))) v)0" (y)g" (x) g" (y)dxdy
< f (x) w)o " (v) ll (x)) dody S
11
22
If the MOO-SVGD density Ps following (4) considered, for any there is an s S such that
( 2c
(L a/o3) (1. 11 Eenpolll (2) (z)
Claim of Theorem 3.2 says that, at the equilibrium p², the kernel embedding of g" is O((1 + a/o²)²a²) which can be arbitrarily small a and a are chosen to be small numbers. Claim 2 of Theorem 3.2 indicates the SVGD density flow Pt find solution of similar property in finite time.
3.2 Multi-Objective Langevin Dynamics
Besides SVGD, MGD can also be integrated with Langevin dynamics (LD), which yields diverse solutions by injecting random noise into the updates. Similar to SVGD, we can achieve this by simply replacing the gradient field in LD with g" yielding MOO-LD:
レ (x) + V20cg
(6)
5

---

where Ç is a standard Gaussian noise and a is a positive coefficient that determines the magnitude of the noise. The intuition again simple: if æ is far away from the Pareto set with large multiple gradient Ilo" (a:)l ||. the dynamics will mainly drive towards the Pareto set; when æ is close to Pareto set with a small [[g"()) (x))]l, the noise term will dominate and allow randomly diffuse on Pareto set.
If 9" is the gradient vector of some sclar-valued function f", that is, g"(x) V f (x), then its stationary distribution equals p² (x) x exp(-f((z)/a) If g" is not gradient field, the Langevin dynamics is known to be irreversible [e.g., 32, 11].
As shown in our empirical results, the SVGD based method is more practically attractive than the Langevin method. We find that SVGD tends to provide more diverse and uniformly distributed points than Langevin dynamics. And for some of the challenge MOO benchmarks, Langevin dynamics can be stuck in a small local region and slow to mix. In comparison, thanks to the deterministic updates, SVGD is found generate mores uniformly distributed Pareto solutions in faster speed. Another practical advantage of SVGD is that we can define the kernel to be of form k( y) = ka (f(),, F(V)). so that the diversity force is w.r.t. the function values {F(z)) }. rather than the variables {
In the following, we consider the asymptotic property of the dynamics in (6). Consider the continuous time limit of the dynamics in (6) (e + 0), which yields stochastic differential equation
dx. = -9" (x+)dt + V2ddwe
where wa denotes standard Wiener process and t denotes time. Assume we initialize zo from distribution with continuous differential density function po. then the evolution the density of ar(t), denoted by Pt. is governed by a Fokker-Planck equation:
d di (+(x)
V (x)) +VV(()))))))))))))))))))))))))))
(8)
where V. denotes the divergence operator in vector calculus.
The stationary distribution of (7) and (8), denoted as p", uniquely exists as long as g" (xx) is coercive (e.g., if Assumption holds). See e.g., [29]. It yields decomposition of g" (xx analogous to that of (5). Moreover, we can show the KL. divergence between Pt and p" decays:
Theorem 3.3. Assume p² a fixed point of for which log (x) is in Ca (B2).. Then we have the following Helmholtz decomposition of the vector field g": g²(x) aa lo (x) + (()), where satisfies that, for any C'l function h with a compact support, J is orthogonal with Vh under p²: {a)] Vh(z)pp (x)dx= = O. Further, if p((x))(a)) - when ||ж||| + oc, the KL divergence between Pr and p" decreases monotonically with
KL(p: I1 = (pt p'),
(9)
dt
where F(pe p²) En-p, III (x) (I3 is the Fisher divergence.
Theorem 3.3 similar to Theorem 3.1. The differences are: 1) there is no kernel, and the decomposi-
tion is more well -ell-undesstodd, see for example Section of [11]: 2) Interestingly, SVGD does not seem to have a similar monotonie decreasing property similar to (9). In particular, if p is strongly log-concave, it is well known that (9) indicates that Pt converges to p" linearly in KI. divergence.
Analogues to Theorem 3.2, we also show the MOO-LD yields distribution that concentrates on the Pareto (stationary) set. In fact, we can further show ||g* (I)]l is sub-Gaussian under p"
Theorem 3.4. Under Assumption I, let M = / + cg. For any constant b S point p" to (8) satisfies
Ct the fixed aueg.
Ex-pr I1g" (2) < 4dMd//1, Ex-p exp(bllg" (x)11) < <3exp(16abMd//o
The solution Pt to (8) satisfies the following
Ex 110" (x)]1 exp(- (lg 4aMd/e1,
Ex-p, (0oblll((x)111+
(xp(:)))))ep(((dddd exp(-abbdt) exp(bllg' (z) + (tll))))xx((lbblll)) 3esp(16abMd/cc).

---

4
Related Works
We discuss related works MOO and also introduce the hypervolume (HV) indicator which we will use as the evaluation metric in our experiments.
Gradient-free MOO Algorithms A large volume of literature focuses black box and derivative- free MOO, where f(()), E
m are observed through the outputs [e.g., 8, 25, 15, 39, 25, 13]. These methods are typically based evolutionary algorithm or Bayesian optimization. However, due to the lack of gradient information, they can perform well only in low dimensional problems and can not be applied to large-scale optimization problems in modern machine learning, such as learning neural networks.
Gradient-based MOO Algorithms Another important line of work is gradient-based methods. The foundation of these methods is MGD. Representative methods include Pareto Front Stochastic Multiple Gradient (PF-SMG) [10, 22], Pareto Multi-task Learning (PMTL) [18], and Exact Pareto Optimization (EPO) [27].
PF-SMG leverages the randomness in stochastic multiple gradient descent to profile the whole Pareto front. The method keeps a set of solutions. every optimization step, the algorithm removes dominated solutions from the set, randomly sample new solutions from the neighborhood of the survived solutions, and run MGD for another multiple steps. PF-SMG is similar to the Langevin method we discuss in Section 3.2 in that both of them leverage random noise, but PF-SMG leverages the noise in stochastic gradient, which will vanish to zero when the step size decreases to zero. Therefore, PF-SMG would reduce to MGD except the selection step. In constrast, the diffusion noise converges to Brownian motion with small step size.
PMTL and EPO are designed to find local Pareto optimal solutions that satisfy specific user preference. They can also be used trace the Pareto front by varying the user preference vector uniformly. However, sampling the preference vector uniformly will not necessarily result in uniform solution set on the Pareto front, which is what we observe in the experiments. Compared with PTML and EPO, our method does not require pre-defined user preference and can obtain more diverse and uniformly distributed points as we show in experiments.
Pareto Hypernet Methods Several methods, such as [31, 17, 33), proposed to learn a neural network, called Pareto hypernet, which can generate pareto-optimal solutions given a preference vector as input. In comparison, our methods directly generates a set of solutions (or particles) that covers the Pareto front. The usage of neural network can be helpful in some cases, but also makes the algorithm more complicated and less transparent, and the choice of neural network structure has heavy influence on the result. In comparison, our algorithm is much simpler, transparent and directly outputs the solutions that we want. More empirical evidence and discussion can be found in the Appendix.
Hypervolume
The hypervolume (HV) indicator is a standard metric for evaluating the quality of sets of solutions in MOO. Assume [ri,..., F'm] is an reference point that is an upper bound of the objectives, such that sup,, fa(x) I ri, Vi E [m]] For given set of solutions X = {1;))_ its hypervolume indicator HV(X) is the measure of the region between FXX) and
x a II(A)) 1) (f E Rd 1
(
12
HV(x)=1 =
(10)
where A(-) denotes the Lebesgue measure. Directly optimizing the hypervolume metric can be problematic, because is piece-wise constant. Hypervolume indicator gradient ascent (HIGA) [36] relaxes the hypervolume indicator to get usable gradient for optimization, and recent work [9] is mathematically similar to HIGA. However, [36] and [9] have two main drawbacks compared with sampling- based method: (1) its results depend on the choice of reference point; (2) it may get stuck in bad local minima. See the Appendix for more empirical comparison.
Experiments
We show our method, Multi-Objectio Stein Variational Gradient Descent (MOO-SVGD) can effi- ciently profile the Pareto front on a variety of problems, including the ZDT problem set, tri-objective problem MaFl, trade-off between accuracy and fairness metrics in fair ML, as well as multi-task
7

---

Naive Hopae-Vlumee
PF-SMG
COSMOS
MOO-ID
MOD-5VG
10
1.2
9.5
0.5
0.0
1.0
000
0,5
1.0
100
055
177
0.5
0.5
1.0
1.0
1.0
1.0
0.5
0.5
0.5-
0.5
6.5
0.0
0.0-
0.0
6.0
Wali-bak Tine
2.0
0.5
3.5
0.0
2.5
1.0
0.0
835
1.0
0.3
0.5
2.5
075
0.5
175
50
855
175
5.0
1.0
03
05
10
10
Runtime Compaisson
(b) PF-SMG
(c) COSMUS
(4) MOO-LD (Ouns)
EPO
MOO-SVGD (Ours)
Figure 2: (a) The hypervolume of results of different methods w.r.t. the wall clock time. (b) The solution set obtained by PF- SM- All the points converge to the same solution because they do not interact with each other. (c) The solution set obtained by COSMOS. In ZDTI, uniform preference vector causes non-uniform distribution of the final solution; in ZDT3, the solutions not on the real Pareto front because they are limited The solution
by the imperfect optimization of the function approximator. the random noise, it has slightly larger coverage than PF-SMG, but is still not widespread even with 200,000 samples. (e) The solution set obtained by EPO. The orange lines refer to the preference vectors. Note that in ZDTI, uniform preference vector causes non-uniform distribution of the final solution; in ZDT3, EPO cannot deal with negative objectives. (f) Benefited from the explicit repulsive force, MOO-SVGD obtains uniform Pareto front. All the results are averaged over runs.
ohtained hy MOO-LD. Thanks
learning on MultiMNIST, MultFashhion and MultFFsshonnMMNIST. Our method outperforms EPO and PF-SMG, two previous baselines for profiling the Pareto front with gradients. We also include Multi-Objective Langevin Dynamics (MO0-LD) for comparison, and find that MOO-SVGD yields much better empirical performance against MOO-LD. MOO-LD in general needs longer iterations to cover the Pareto Front. Please refer to the Appendix for more discussion. We leave the detailed algorithm configuration in the Appendix due to space limitation.
ZDT Bi-objective Problems We compare our methods with other baseline methods on the ZDT problem set [40]. The ZDT problems have a 30-dimensional variable and two objective functions. The details of ZDT problems can be found in the Appendix. In ZDT problems, both objective functions are differentiable. We choose ZDT-1, ZDT-2, and ZDT-3 demonstrate the performance of our algorithm when the Pareto front is convex, concave, and discontinuous. For baselines, we compare with naive optimization on hypervolume, PF-SMG [22]) and Pareto hypernet algorithms (COSMOS [33]). All the results of the baselines are reproduced using their official implementation. For fair comparison between black-box and white-box optimization algorithms, we report the increase of the hypervolume indicator (Eq. (10)) over the wall-clock time. For fairness of comparison, we use the same reference point when computing hypervolume indicator for every algorithm. Moreover, to make the comparison of wall-clock time fair, we run all the algorithms with CPU on the same computational platform with 192GB memory and 48 cores. The results are shown in Fig. 2. We constrain the maximal wall-clock time in the figure to be 60 seconds. We use 50 particles for MOO-SVGD, Because EPO and COSMOS need user preference vectors, we uniformly sample 50 unit vectors from a semi-circle to trace Pareto front, following the strategy provided by the authors [27, 33]. We use a two-layer ReLU network with 1,000 hidden units for the hypernet in COSMOS.
soate-of-the-ree gradient-based MOO algorithms (EPO [27]
As the figures shows, our method, MOO-SVGD, obtains the largest hyper-volume in all three problems. In the limit of 60 seconds, EPO generally has the best performance among all the three baselines. Naive optimization of hyper-volume get similar performance as the stochastic method, PF-SMG, though the latter is much slower than the former. COSMOS cannot cover the entire Pareto
8

---

front. In ZDTI and ZDT3, MOO-SVGD converged to better solution. In comparison, another sampling based method, MOO-LD, can only cover limited subset of the whole Pareto front.
MaF1 Tri-objective Problem We test our algorithm on MaF1 [3], a problem with three objectives to showcase its ability when there more than two objectives. The objective functions are defined on [0, 1j]. The specific expression of f1, f2 and f3 can be found in Appendix due to limited space. The Pareto front of the problem can be analytically solved, which is {x::
fa = 2]. The reference point for calculating hypervolume is (1.0, 1.0, 2.0). The results are shown in Fig 3 and Table. All the three methods, EPO, MOO-LD, and MOO-SVGD can find Pareto optimal solutions. However, MOO-LD can only cover a very small subset of the Pareto front. EPO can find diverse Pareto optimal solutions, but they are not distributed uniformly. MOO-SVGD obtains the most visually uniform solution set, and hence reaches the highest performance w.r.t. hypervolume. MOO-LD is not tested the following parts due to its poor performance on toy experiments.
Method EPO
Hypervolume
A+
0.5714
MOO-LD (Ours)
0.2613
EPO
(Oun)
(Uang
MOO-SVGD (Ours)
0.5867
Figure 3: The solution obtained by EPO, MOO- LD and MOO-SVGD, The red plane the Pareto front. The red dots are the final solutions.
Table 1: The Hyper-volume of the solution sets ob- tained by EPO, MOO-LD and MOO-SVGD on MaF1.
PF-SMG (50 poine)
PF-SM (838 poizhs)
EPO
MOO-SYG (Oun)
Pomceees
Parisles
Particos
0.38
Faluess Rouly
Dyper Warme
Pypes Volums
CF Sore
0.00
0.00
ClosiBatiiu Low
(n) Training
0.37
271 625
Ciniiiatiion ACcurRcY
(h) TToetingg
(a) Ablazion Study un Bandwidth
6
Ingh
8
10
112
(b) AMlatiu Study on Temperature
2
lug.r
0
Figure 4: (a) The Pareto front of classification loss and fairness penalty in the training set. MOO-SVGD obtains a uniform solution set that substantially domi- nates the basclines. (b) Tradc-off between classifica- tion accuracy and CV score in the test set. PF-SMG (k point) denotes PF-SMG with & solution.
Figure 5: Ablation study of MOO-SVGD on ZDTI. (a) The final hypervolume w.r the bandwidth h when a The optimal h is larger for smaller number of particles. (b) The final hypervolume w.r.t. the temperature a when h = le - 5. All the results are averaged over 3 runs.
Trade-off Between Accuracy and Fairness When a MI. system applied to a real-world scenario, it may be biased against people with sensitive attributes, e.g., their genders or races. This leads to unfairness in ML, which is a key issue in real applications of ML. A common practice to avoid unfaimess is to add fairness regularization during training. Unfortunately, introducing fairness regularization may hurt the prediction accuracy of the model. The user may have various requirements for the trade-off between accuracy and faimnsss in different environments. This can be framed into MOO problem, which simultancously optimize the typical training loss and the fairness penalties.
For this experiment, please refer to the appendix to see how we construct our dataset. We choose gender to be the sensitive attribute, and we use disparate impact as the fairness criterion. Disparate impact can be measured by the CV score [2], CV(J)= II(Ú = 1| 0) p(ğ = 0) = 0)], where ff is our classifier, ÿ is the prediction, and A is the sensitive attribute. ÿ = if f(x) > O; otherwise, y 0. CV score implies the gap between the probabilities of getting positive outcomes in different sensitive groups. As in [23], we use a linear classifier with 87 parameters. The classification loss L is the typical binary cross eross-entropy loss. For faimess penalty, instead of directly optimizing the CV score, we use convex surrogate of CV score provided in [23], f2(s) [T a.E. (a, a)f(x;;) where 2ú is the feature of instance i; a; is the sensitive attribute of instance i; 'D is the dataset, and is the averaged value of all a, in the dataset. Basically, it is minimizing the empirical covariance between the the prediction and the sensitive attribute.
We compare with PF-SMG [23] and EPO [27]. The number of particles is 10 for MOO-SVGD and EPO. For PF-SMG, we use two different versions. The first version has a maximal particle number of 600; the second is 50. The two versions denoted by PF-AMG-50 and PF-SMG-60X The results are shown in Fig. 4. Generally, MOO-SVGD yields the best solution set at convergence. Among all

---

Linear
EPO
PMTT
MOO-SVGD (Ours)
0.75
0.75
Taok Ace
Task 2-Nce
Took: bane
0.50
0.50
1.0
0:25
0.75
0.25
0.2
1.0 Task loos
Talk
Ase
1.7
Trsk 1oo
Talk boss
1.5
Task: Toae
3.5
2.0-
2.0
3.0-
0.3
0.5-
0.3
2.0
1.0
3.5
0.2
1.0
1.3
Talk Jos4
Task I Joss
Task loss
(d) Top: 10 partioles on MutiMNIST
(2) MulIMNIST
(b) Mulii ashion
(c) MultiFashiom IMNIST
Bottom: Optimization unjectiory
Figure 6: The solution set of the neural networks obtained by different methods two tasks, w.r.t. the training loss (lower panel) and testing accuracy (upper panel). MOO-SVGD yields the best Pareto front on training loss and a widespread Pareto front on test accuracy.
the methods, only MOO-SVGD and PF-SMG-600 can get a solution set that is uniformly distributed on different preferences. PF-SMG-50, due poor exploration ability of the random strategy, shrinks to the two end points of PF-SMG-600. EPO totally fails in this task. Uniformly sampling preference vector from the semi-circle is problematic, since the Pareto front lies in very small region of the semi-circle, and most of the preference vectors do not ends up with an 'exact' solution at all.
Multi-task Learning with Neural Networks MultiMNIST, MultiFashion and MultiFash- ion+MNIST, introduced in [34], are three standard benchmarks for multi-task deep learning. The datasets are created in a similar way: each image in the datasets is a combination of two different images. One of them is on the top-left, and the other is on the bottom-right. In MultiMNIST, both images are from the MNIST dataset [16]. Similarly, MultiFashion uses two images from the MNSSIonMNIST dataset [38]. For MultiFashion-MNNIST the top-left image is from MNIST, while the bottom-right one is from FashionMNIST. All three datasets have 120,000 samples in the training set, and 20,000 samples for test set. These datasets are used in [18, 27, 26].
The baseline methods are linear scalarization, EPO [27] and Pareto Multi Task Learning (PMTL) [18]. EPO and PMTL both require preference vectors to work. We follow the setting in [27], where the authors use LeNet [16] and 5 preference vectors uniformly sampled from semi-circle. We also use 5 particles for MOO-SYGD. We find that training with MOO-SVGD can be problematic in the early stage, where networks with different initialization cause gradient explosion. Therefore, we initialize the networks with same parameters, then warm-up them by training with naive linear scalarization objective for epoch, where the weights sampled uniformly from [0. 1]. We switch back to MOO-SVGD in the remaining epochs. For all the experiments, all the methods are trained for 100 epochs with batch size of 256. Moreover, they optimized with SGD and the learning rate is 0.001.
be too close to each other and
6 Conclusion, Limits, Impacts, and Open Questions
In this paper, we propose Multi-objective Stein Variational Gradient Descent (MOO-SVGD). Em- pirically, it can trace the Pareto front evenly without user preference. There are a number of limits that deserve future efforts. On the practical side, since our methods leverage SVGD and Langevin dynamics, they necessarily inherent their limits, such as the sensitivity on the choice of kernel and step sizes; studies on automatizing the selection of these significantly. Theoretically, analyzing our methods are challenging due to the lack of explicit form of the invariant distributions. For example, although we show that the limiting distributions concentrate on the Pareto front, it is an open question to theoretically characterize in what sense they are evenly distributed in the Pareto front. Moreover, it would be interesting to draw theoretical understandings on why the MOO-SVGD performs better than the MOO-Langevin dynamics.
can improve the method
In terms of social impact, as a new computational tool for MOO, it allows us to better trade-off different objectives in machine learning and hence improve ML systems in both accuracy and various trustworthy metrics such as faimess, safety, as we demonstrate with our experiments. We will release code online to promote the applications of our approach, although it does open the possibility of malicious use of our techniques for adversarial purposes.
10

---

Acknowledgements
The work is conducted in the statistical learning and AI group computer science at UT Austin, which is supported in part by FAREER-1846422 SenSE-2037267, EAGER-204132 and Office of Navy Research, and NSF AT Institute for Foundations of Machine Learning (IFML), Xingchao Liu is supported in part by a funding from BP. X. T. Tong's s research is supported by Singapore MOE Academic Research Funds R- 4-146-000-292-111 The authors thank the reviewers for all the suggestions made in the reviewing process.
References
[1] Syrine Belakaria, Aryan Deshwal, Nitthilan Kannappan Jayakodi, and Janardhan Rao Doppa. Uncertainty-awarg search framework for multi-objective bayesian optimization. In AAAI, pages 10044-10052, 2020.
[2] Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination-free classifi- cation. Data Mining and Knowledge Discovery, 21(2):277-292, 2010.
[3] Ran Cheng, Miging Li, Ye Tian, Xingyi Zhang, Shengxiang Yang. Yaochu Jin, and Xin Yao. A benchmark test suite for evolutionary majy-objective optimization. Complex & Intelligent Systems, 3(1):67-81, 2017.
[4] Altannar Chinchuluun and Panos M Pardalos. A survey of recent developments in multiobjective optimization. Annals of Operations Research, 154(1):29-50, 2007.
[5] Carlos A Coello Coello. comprehensive survey of evolutionary-base multiobjective opti- mization techniques. Knowledge and Information systems, 1(3):269-308, 1999.
[6] Giuseppe Da Prato and Jerzy Zabezyk. Stochastic equations in infinite dimensions. Cambridge university press, 2014.
[7] Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Differentiable expected hyper- volume improvement for parallel multi-objective bayesian optimization. arXiv preprint arXiv:2006.05078, 2020.
[8] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions evolutionary computation, 6(2):182-19, 2002.
[9] Timo M Deist, Monika Grewal, Frank JWM Dankers, Tanja Alderliesten, and Peter AN Bosman. Multi-objective learning to predict pareto fronts using hypervolume maximization. arXiv preprint arXiv:2102.04552. 2021.
[10] Jean-Antoine Désidéri. Multiple graddentgr descent algorithm (mgda) for multiobjective opti- mization. Comptes Rendus Mathematique, 350(5-6):31-3115 2012.
[11] Andrew B Duncan, Tony Lelievre, and GA Pavliotis. Variance reduction using nonreversible langevin samplers. Journal of statistical physics, 163(3):457-491, 2016.
[12) Zesong Fei, Bin Li, Shaoshi Yang, Chengwen Xing, Hongbin Chen, and Lajos Hanzo. A survey of multi-objective optimization in wireless sensor networks: Metries, algorithms, and open problems. IEEE Communications Surveys & Tutorials, 19(1):550-555 2016.
[13] Daniel Golovin al. Random hypervolume scalarizations for provable multi-objective black box optimization. arXiv preprint arXiv:2006.04655 2020.
[14] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2015.
[15] Joshua Knowles. Parego: hybrid algorithm with on-line landscape approximation for expen- sive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10(1):50-66, 2006.

---

[16] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))27782222 1998.
[17] Xi Lin, Zhiyuan Yang, Qingfu Zhang, and Sam Kwong. Controllable pareto multi-task learning. arXiv preprint urXiv:2010.06313, 2020.
[18] Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qingfu Zhang, and Sam Kwong. Pareto multi-task learning. arXiv preprint arXiv: 12854,
[19] Qiang Liu. Stein variational gradient descent as gradient flow, In Advances in neural information processing systems, pages 3115-3123, 2017.
[20] Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Advances In Neural Information Processing Systems, pages 2370-2378, 2016.
[21] Qiang Liu and Dilin Wang, Stein variational gradient descent as moment matching. In Neural networks for signal processing, pages 8854-8863, 2018.
[22] Suyun Liu and Luis Nunes Vicente. The stochastic multi-gradient algorithm for multi- objective optimization and its application supervised machine learning. arXiv preprint arXiv:1907.04472 2019.
[23] Suyun Liu and Luis Nunes Vicente. Accuracy and faimess trade-offs in machine learning: A stochastic multi-objective approach. arXiv preprint arXiv:2008.013322 2020.
[24] Xingchao Liu, Xing Han, Na Zhang, and Qiang Liu. Certified monotonic neural networks. arXiv preprint arXiv:2011.1022 2020.
[25] Mina Konakovic Lukovic, Yunsheng Tian, and Wojciech Matusik. Diversity-guide multi- objective bayesian optimization with batch evaluations. Advances in Neural Information Processing Systems, 2020.
[26] Pingchuan Ma. Tao Du, and Wojeiech Matusik. Efficient continuous pareto exploration in multi-task learning. In International Conference on Machine Learning, pages 6522-6531. PMLR, 2020.
[27] Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In International Conference on Machine Learning, pages 6597-6607. PMLR, 2020.
(28) R Timothy Marler and Jasbir S Arora. Survey of multi-objective optimization methods for engineering. Structural and multidisciplinary optimization, 26(6):369-39) 2004.
[29] Jonathan C Mattingly, Andrew M Stuart, and Desmond J Higham. Ergodicity for sdes and approximations: locally lipschitz vector fields and degenerate noise. Stochastic processes and their applications, 101(2):185-232. 2002.
[30] JE Mendoza, ME Lopez, CA Coello Coello, and EA Lopez. Microgenetic multiobjective reconfiguration algorithm considering power losses and reliability indices for medium voltage distribution network. IET Generation, Transmission Distribution, 3(9):825-840, 2009.
[31] Aviv Navon, Aviv Shamsian, Gal Chechik, and Ethan Fetaya. Learning the pareto front with hypernetworks. arXiv preprint arXiv:2010.044 04.044011444, 2020.
[32] Hong Qian. A decomposition of irreversible diffusion processes without detailed balance. Journal of Mathematical Physics, 54(5):053302, 2013.
[33] Michael Ruchte and Josif Grabocka. Efficient multi-objective optimization for deep learning. arXiv preprint arXiv:2 103.
[34] Sara Sabour, Nicholas Frosst, and Geoffrey Hinton. Dynamic routing between capsules. arXiv preprint 1tXiv:1710.098299 2017. 2017.
12

---

[35] Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. arXiv preprint arXiv:1810.046500 2018.
[36] Hao Wang, André Deutz, Thomas Bäck, and Michael Emmerich. Hypervolume indicator gradient ascent multi-objective optimization. In International conference on evolutionary multi-criterion optimization, pages 354-669. Springer, 2017.
[37] Max Welling and Yee W Teh. Bayesian learning via stochastic gradient Langevin dynamics. In International Conference Machine Learning. 2011.
[38] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist. novel image dataset for benchmarking machine learning algorithms. arXiv preprint
2017.
[39] Qingfu Zhang, Wudong Liu, Edward Tsang, and Botond Virginas. Expensive multiobjective optimization by moea/d with gaussian process model. IEEE Transactions on Evolutionary Computation, 14(3):456-473) 2009.
[40] Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, 8(2):133-192 2000.
13


Reviews:
[{"review_id": "Ub8FqC2BT94", "reviewer": "Reviewer_6uVc", "summary": "In this work the authors propose a gradient-based method for identifying the Pareto Front of a Multi-Objective Optimization problem.   More specifically, they propose two methods, one based on Stein Variational Gradient Descent, and another based on overdamped Langevin dynamics to evolve forward a set of particles to the (local) pareto front.\n\nThe approach is based on the multiple gradient descent (MGD) formulation where the next particle descent direction is chosen to maximise the slowest decreasing rate amongst the objectives.  Assuming particles are advected along an unknown vector field $\\phi^*$ the authors introduce a variational formulation which selects the $\\phi$ according to MGD objective while encouraging diversity through the usual entropy regularisation.   This gives rise to SVGD-like dynamics.   An analogous formulation for Langevin Dynamics is also proposed.\n\nThe authors provide some theoretical guarantees for both in terms of concentration of the dynamics to the Pareto front.  Results are demonstrated for a number of numerical experiments.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method is an interesting contribution provides an interesting and novel application of SVGD dynamics.   The fact that the resulting dynamics are not driven by a gradient adds interesting challenges, which the authors recognize.  Both the paper and the supplementary information was clearly written, and the proofs relatively straightforward to follow.  However,  I do feel that there are some issues which needed to be addressed more carefully.\n\n* For example: in the conclusion the authors state: \"Theoretically, analyzing our methods are challenging due to the lack of explicit form of the invariant distributions\".   This presupposes the assumption that the dynamics do concentrate onto an invariant measure -- and if so -- is it unique.    Theorems 3.1, 3.2 and 3.3 pre-suppose the existence of $\\rho^*$ and implicitly it is assumed that it is unique.  The authors should address why they believe this is true a bit more carefully.   In the case of Theorem 3.3 this should be discharged using relatively standard results, and I would imagine it to be the case in the other results.\n\n* Theorem 3.2 doesn't quite characterise the concentration to the Pareto front as is claimed in the text, but rather it provides a bound for the weighted kernel mean embedding of $g^*$.   This is understandable from the form of the proof, however, it does complicate affairs as there is strong parameter dependence $(m_d, \\sigma^2, etc)$ on the LHS of both inequalities.   Would it be possible to extract bounds, asymptotics, etc specifically for $g^*$?   If not, then the authors should write the LHS explicitly, using the kernel trick etc, to demonstrate the explicit dependence on the constants.  \n\n* The first part of Theorem 3.3 is essentially the Helmholtz-decomposition for weighted measures.  Ideally, the authors should cite this well-known result if possible, looking in the references of [8] for example which makes use of this result.\n\n* The authors should explicitly define the notation $f_{[\\rho^*]}$ somewhere a bit more clearly, as it was not immediately obvious on a first read of the text.\n\n* I find the justification of the second factor on page 3 to be a bit unclear.  As written the argument suggests we want to maximise $-\\mathbb{E}_{x\\sim \\rho}[\\nabla \\cdot \\phi(x)]$, not as stated. \n\nMinor comments:\n\nl17: power systems\n\nl31: have been\n\nl80: gradient\n\nl125: it's a divergence operator not a diversity operator\n\nl143: $\\rho^*(x) \\propto \\exp(-f^*(x)/\\alpha)$.\n\nl158 The notation is completely unclear, and needs to be written far more carefully.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed these", "review_text": "", "extra_scores": {"ethical_concerns": "I can see no ethical concerns", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Gyi28WQtzPH", "reviewer": "Reviewer_1ANJ", "summary": "The paper proposes novel gradient-based algorithms for finding solutions across the Pareto front by using Stein variational gradient descent (SVGD) (and Langevin dynamics), in multi-objective optimization (MOO) problems.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper is well structured and well written.\n- The paper focuses on the main challenge of MOO problems: find representative solutions on the Pareto front, and propose a novel and scalable algorithm for this problem.\n- The general idea is well explained, and the motivation is clear and straightforward. \n- The authors provide theoretical guarantees and analysis for the proposed method.\n\nQuestions/suggestions/Weaknesses:\n\n- Overall it is a good paper with clear motivation and good theoretical analysis. I have two main concerns:\n   - Previous literature in the same direction as discussed in the paper is omitted from discussion and empirical evaluation. Specifically, recent methods propose profiling the entire Pareto front in a scalable manner by conditioning the model on the preference direction (e.g. [1, 2]).\n   - Second, scalability of the approach: the method essentially trains multiple models. The number of models that are needed for achieving good cover of the Pareto front grows exponentially with the number of objectives. This may be a major drawback with large-scale problems and models.\n\n- Experiments:\n   - Missing comparisons to recent state-of-the-art MOO approaches such as [1, 2].\n   - The Linear scalarization baseline is missing from most comparisons.\n   - The authors only experiment with small-scale and/or synthetic examples. I encourage the authors to provide an empirical evaluation on large-scale problems, like common MTL baselines (e.g., NUYv2/CityScapes dataset).\n   - In addition, experiments with a larger number of objectives ($>3$) would be beneficial.\n   - How did you select the hyper-parameters (HPs) for your method and baseline methods?\n      - For example, in Appendix C.1, you state that $c$ (or the bandwidth) parameter is selected to optimize the hyper-volume. However, you did not use a validation split, so does this mean $c$ is optimized according to the test set? How did you select other HPs like $\\alpha$?\n      - All methods use the same learning rate (lr) in a given experimental setup if I understand correctly. How did you select this lr? Is it possible baseline methods can be improved by selecting an appropriate learning rate for each method?\n\n\n[1] Learning the Pareto Front with Hypernetworks, Navon et al., ICLR 2021.\n\n[2] Efficient Multi-Objective Optimization for Deep Learning, Ruchte et al., 2021.\n\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4-5 hours"}, "extra_text": {}}, {"review_id": "74hOvSzei9w", "reviewer": "Reviewer_WBcz", "summary": "This paper proposes a method to learn multiple Pareto solutions for multi-objective optimization (MOO).\nSpecifically, the method iteratively updates a set of points by Stein variational gradient descent (SVGD) to encourage the diversity between the solutions. Another counterpart method based on Langevin dynamics is also provided.\nThe methods are theoretically guaranteed to converge to the Pareto front.\nThe experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The idea of encouraging the diversity between the solutions by SVGD is interesting.\n2. The proposed methods are theoretically guaranteed to converge to the Pareto front.\n3. The experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.\n\nWeaknesses:\n1. As a method to learn multiple Pareto solutions for MOO, the proposed method discussed no related works that learn multiple Pareto solutions, e.g., [1,2,3,4,5].\n2. The proposed method cannot control the specific preference vector for each solution, while [1,2] can, suggesting a type of weakness of the proposed method.\n3. The proposed method requires maintaining multiple solutions, which may have computation and storage problems, whereas [5] maintains one solution only, suggesting another type of weakness of the proposed method.\n4. Since MOO-Langevin Dynamics seems inferior for every aspect, the necessity to present this method is not clear.\n\n[1] Navon, A., Shamsian, A., Chechik, G., & Fetaya, E. (2020). Learning the Pareto front with hypernetworks. ICLR 2021.\n\n[2] Lin, X., Yang, Z., Zhang, Q., & Kwong, S. (2020). Controllable Pareto multi-task learning. arXiv preprint arXiv:2010.06313.\n\n[3] Shah, A., & Ghahramani, Z. (2016, June). Pareto frontier learning with expensive correlated objectives. In International Conference on Machine Learning (pp. 1919-1927). PMLR.\n\n[4] Deist, T. M., Grewal, M., Dankers, F. J., Alderliesten, T., & Bosman, P. A. (2021). Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximization. arXiv preprint arXiv:2102.04523.\n\n[5] Ruchte, M., & Grabocka, J. (2021). Efficient Multi-Objective Optimization for Deep Learning. arXiv preprint arXiv:2103.13392.\n\nOriginality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited? \n\n1. The idea of encouraging the diversity between the solutions by SVGD is interesting.\n2. No related work that learn multiple Pareto solutions is discussed.\n\nQuality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work? \n\n-Yes.\n\nClarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.) \n\n-Fair.\n\nSignificance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach? \n\n-The experimental results demonstrate the superiority of the proposed SVGD type of method compared with EPO. The superiority is shown visually and by the hypervolume of the solutions.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "6LTBjuCGMgC", "reviewer": "Reviewer_ag1C", "summary": "This paper studies how to acquire diverse and representative Pareto solutions by gradient-based methods. Two algorithms -- MOO-SVGD and MOO-LD are proposed by using stein variational gradient descent and Langevin dynamics respectively, with convergence (to local Pareto optimal) guarantees. The experimental results demonstrate that MOO-SVGD can obtain visually uniform solution sets of ZDT and MaF1 problems, and get better Pareto fronts on fair ML and multi-task learning.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\nThe motivation is clear to the readers that the diversity of the Pareto solution set is crucial to MOO.\nThe intuitive review of the algorithm design is clear for both two algorithms.\nTheoretical convergence of strongly convex functions is provided.\nThe experimental results are convincing and attractive, especially the ZDT and MaF1 problems.\n\nWeakness:\n\nAlthough the intuition behind using SVGD is clear, it is unclear that why MOO-SVGD is superior to find diverse Pareto solutions from a theoretical review (the authors show this limitation in Section 6).\nAssumption 1 is too strong for the deep learning model, where the ReLu layer is not strongly convex, which is not sufficient to claim “we mainly focus on the finding locally optimal solutions” (line 68).\nI think MOO-LD is not important to the paper logic, and the experiments show its impractical performance. What’s the purpose that the authors care about this? In the experiments, the authors also do not give an intuitive interpretation on why MOO-LD fails.\nIn the fair ML and multi-task learning experiments, the solutions are not that uniform as the previous 2 experiments. Could the authors provide more explanations?\n\nTypos:\nLine 168, “j,j,k” under the partial differential symbol.\nLine 291, Fig 5 → Fig 2\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "48"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes and studies a novel algorithm, inspired by Stein variational gradient descent, whose aim is to find distinct elements on the Pareto front of a multi-objective optimisation problem.  Theoretical and empirical results support the proposed method.  All reviewers agreed that the paper would make an excellent contribution to NeurIPS, but I suggest the author(s) carefully consider the comments of Reviewer 6uVc, who explains that there are elements of the presentation of theoretical results that ought to be clarified in the manuscript.
Decision: Accept (Spotlight)