Profiling Pareto Front With Multi-Objective Stein Variational  Gradient Descent

1 Introduction . Many scientific and engineering problems involve optimizing multiple conflicting objectives [ 5 , 28 , 4 , 24 ] , including , for example , designing wireless sensors [ 12 ] , building electric power systems [ 30 ] , and training neural networks with multiple tasks [ 35 ] . With multiple conflicting objectives , it is impossible to find a single solution that optimizes all the objectives simultaneously . Instead , it is essential to find a set of diverse solutions in the Pareto front that represent different preferences on the different objective functions , so that the users can have a global view of how the optimal trade-off of the different objectives look like and select the solution according to their own preference . Unfortunately , profiling the Pareto fronts casts a key computational challenge , especially for high dimensional problems [ 5 , 12 , 28 ] . Traditionally , a large literature has been devoted to developing black-box , derivative-free algorithms that are suitable for black-box optimization , such as these based on evolutionary algorithms [ 8 ] and Bayesian optimization [ 25 , 1 , 39 ] . However , the black-box algorithms tend to be expensive and can only be applied to small scale problems due to the lack of gradient information . Gradient-based MOO algorithms have been catching attention only recently , which include mainly multiple gradient descent ( MGD ) based methods [ 10 , 22 , 27 ] . However , ∗Code is available at https : //github.com/gnobitab/MultiObjectiveSampling . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the results of these methods are still dissatisfying in finding diverse and evenly distributed Pareto solutions in complex problems . We introduce Stein variational gradient descent ( SVGD ) [ 20 , 19 , 21 ] and Langevin dynamics [ 37 ] as efficient approaches for profiling Pareto fronts . These methods iteratively evolve a group of particles to represent the target distribution . The main difference of the two algorithms is how they distribute points . The Langevin dynamics uses stochastic noises to perturb the particle trajectories , so they can visit different areas of the Pareto front . On contrast , SVGD is a deterministic sampling algorithm that pushes the particles to high probability regions using gradient information , while enforcing diversity between the particles using a repulsive force . In this work , we propose to a simple approach to integrate SVGD and Langevin Dynamics with MGD to draw samples from the Pareto front . Theoretical analyses are provided for both algorithms to understand their limiting distributions and their convergence speed . One challenge with MGD based sampling is that the limiting distributions do not admit explicit formulations . This is mainly because the forcing from MGD is in general not the gradient of any function . However , we can show its non-gradient component is orthogonal to a large class of functions . Assuming each objective function is strongly convex and regular , we can also show the limiting distributions concentrate on the Pareto front . Moreover , we can show the two algorithms converge to good solutions of MOO with O ( 1/t ) and linear rate . We test our methods on a variety of tasks , ranging from low-dimensional optimization to multi-task neural network optimization . On all the tasks tested , our method can obtain diverse and high quality Pareto solutions that distribute evenly on the Pareto front , without predefined preference vectors . Quantitatively , we substantially outperform PF-SMG and EPO with respect to the hypervolume metric . 2 Background . In Multi-objective optimization ( MOO ) , we are interested in minimizing a vector-valued loss function F ( x ) = ( f1 ( x ) , f2 ( x ) , . . . , fm ( x ) ) ∈ Rm , where x ∈ Rd and fi : Rd → R , i ∈ [ m ] is the i-th scalar-valued objective function . Here we use notation [ m ] = { 1 , 2 , . . . , m } . Obviously , we can not fully optimize all the objective functions simultaneously because they may be conflicting with each other . Instead , we are interested in finding the points which can not be improved simultaneously for the objective functions , yielding the notion of Pareto optimality . Definition 1 ( Pareto Optimality ) . For x1 , x2 ∈ Rd , We say that x1 is dominated by x2 iff fi ( x2 ) ≤ fi ( x1 ) , ∀i ∈ [ m ] , and F ( x1 ) 6= F ( x2 ) . A point x∗ is called globally Pareto optimal on Rd iff it is not dominated by any other x′ ∈ Rd . A point x∗ is called locally Pareto optimal iff there exists an open neighborhood N ( x∗ ) of x∗ , such that x∗ is not dominated by any x ∈ N ( x∗ ) . The collection of globally ( resp . locally ) Pareto optimal points are called the global ( resp . local ) Pareto set . The collection of function values F ( x∗ ) of all the Pareto points x∗ is called the Pareto front . Our goal is to find diverse and representative solutions { xi } ni=1 from the Pareto set , so that their function values { F ( xi ) } ni=1 covers different preferences on different objectives . This would allow the end-users to have a global view on the optimal trade-off between the different objective functions and decide which solution based on their own preference . Linear Scalarization One standard approach to solve MOO is using a preference vector λ = [ λ1 , . . . , λm ] from the probability simplex on [ m ] , i.e. , S = { λ : ∑m i=1 λi = 1 , λi ≥ 0 , i ∈ [ m ] } . Each λ ∈ S leads to a weighted objective function fλ ( x ) = ∑m i=1 λifi ( x ) and its minimizer x∗λ = arg minx fλ ( x ) . Then as we take λ in a grid of S , we hope that the corresponding x∗λ gives a rough sketching of the Pareto front . Although this strategy is simple and easy to implemented , it suffers from the a number of weaknesses . A key problem is that F ( x∗λ ) can only lie on the convex envelope of the Pareto front , and hence it only works in cases when the Pareto front is convex . In addition , a uniform grid of λ on S does not necessarily yield uniformly distributed points on the Pareto front . Multiple Gradient Descent ( MGD ) MGD is a natural extension of the single-objective gradient descent to finding a Pareto point [ 10 ] , which , unlike linear scalarization , works for non-convex Pareto fronts . The idea is to iteratively update the variable x along a direction that ensures that all the objectives are decreased simultaneously ( which is called Pareto improvement ) . Let gi ( x ) = ∇fi ( x ) be the gradient of the i-th objective . Suppose we update the variable by x′ ← x− g∗ ( x ) , where g∗ ( x ) is a vector field to be determined and is a small step size . By Taylor approximation , we have 〈gi , g∗〉 ≈ − ( fi ( x′ ) − fi ( x ) ) / , which represents the decreasing rate of fi when we update x along direction g∗ ( x ) . In MGD , g∗ is chosen to maximize the slowest decreasing rate among all the objectives , that is , g∗ ( x ) ∝ arg max g∈Rd { min i∈ [ m ] 〈g , gi ( x ) 〉 , s.t . ‖g‖ ≤ 1 } . ( 1 ) Therefore , g∗ ( x ) is encouraged to have positive inner products with all gi ( x ) . If this is impossible to achieve , { gi ( x ) } mi=1 will contain conflicting directions , and we would have g∗ ( x ) = 0 which terminates the algorithm . Using Lagrangian duality , we can show that the optimal solution to ( 1 ) is g∗ ( x ) ∝ ∑m i=1 λ ∗ i ( x ) gi ( x ) , where { λ∗i ( x ) } mi=1 is the solution of min { λi } ∥∥∥∥∥ m∑ i=1 λigi ( x ) ∥∥∥∥∥ s.t . m∑ i=1 λi = 1 , λi ≥ 0 , ∀i . ( 2 ) This optimization has a simple closed-form solution when m = 2 , and a fast algorithm is offered by [ 35 ] for m > 2 . By construction , when the step size is small , MGD monotonically decreases all the objectives simultaneously and will terminate when it arrives a local Pareto point ; in this case we have g∗ ( x ) = ∑m i=1 λ ∗ i ( x ) gi ( x ) = 0 , suggesting that the zero vector 0 is inside the convex hull of { gi ( x ) } mi=1 . The MGD direction g∗ provides a natural notion of gradient for multiple objectives . A point x is said to be Pareto stationary if g∗ ( x ) = 0 . Similar to the case of differentiable single-objective optimization , every Pareto local optimal is a Pareto stationary point . See [ 10 , 35 , 22 ] for more discussion . However , the vanilla MGD suffers from several key weaknesses . Although it promises to return a point on the ( local ) Pareto set , and it is difficult to explicitly control which Pareto point it will converge to . The convergence point of MGD is implicitly determined by the initialization and the other hyper-parameters of the algorithm ( e.g. , step size ) in a complicated way . It can be harder for MGD to converge to some Pareto points than the others . In fact , assume the Pareto front is an open set in Rd , then the vanilla MGD , when initialized from outside of Pareto front , will terminate when it reaches the boundary of the Pareto front , and hence never reach the interior points . Below is a simple example to demonstrate this . Example 1 . Consider f1 ( x ) = x2 and f2 ( x ) = ( x − 1 ) 2 for x ∈ R as shown in Figure 1 . Then the Pareto set is the interval [ 0 , 1 ] . However , when initialized outside of [ 0 , 1 ] , MGD will converge to either x = 0 or x = 1 and can not reach the interior points of the Pareto set unless we add a diversity-promoting mechanism , such as random noise or deterministic repulsive force . See figure 1 . 3 Profiling Pareto front with Particle Dynamics . We improve MGD by integrating it with sampling algorithms for approximating distributions to obtain diverse solutions that cover the Pareto front more evenly . In this section , we first introduce our main algorithm , which integrates MGD with Stein variational gradient descent ( SVGD ) , yielding an interacting particle system that evolves diverse Pareto solutions with deterministic repulsive forces ( Section 3.1 ) . We also introduce a related idea of combining MGD with Langevin dynamics to obtain diverse solutions by injecting diffusion noises , which is found less favorable in practice than SVGD-based method but is of independent interest ( Section 3.2 ) .

Summary: This paper studies how to acquire diverse and representative Pareto solutions by gradient-based methods. Two algorithms -- MOO-SVGD and MOO-LD are proposed by using stein variational gradient descent and Langevin dynamics respectively, with convergence (to local Pareto optimal) guarantees. The experimental results demonstrate that MOO-SVGD can obtain visually uniform solution sets of ZDT and MaF1 problems, and get better Pareto fronts on fair ML and multi-task learning.