P ARETO N AVIGATION G RADIENT D ESCENT : A F IRSTO RDER A LGORITHM FOR O PTIMIZATION IN P ARETO S ET
Anonymous authors
Paper under double-blind review
A BSTRACT
Many modern machine learning applications, such as multi-task learning, require ﬁndingoptimal model parameters to trade-off multiple objective functions that may conﬂict witheach other.
The notion of the Pareto set allows us to focus on the set of (often inﬁnite numberof) models that cannot be strictly improved.
But it does not provide an actionable procedurefor picking one or a few special models to return to practical users.
In this paper, weconsider optimization in Pareto set (OPT-in-Pareto) , the problem of ﬁnding Pareto modelsthat optimize an extra reference criterion function within the Pareto set.
This function caneither encode a speciﬁc preference from the users, or represent a generic diversity measurefor obtaining a set of diversiﬁed Pareto models that are representative of the whole Paretoset.
Unfortunately, despite being a highly useful framework, efﬁcient algorithms for OPTin-Pareto have been largely missing, especially for large-scale, non-convex, and non-linearobjectives in deep learning.
A naive approach is to apply Riemannian manifold gradientdescent on the Pareto set, which yields a high computational cost due to the need for eigencalculation of Hessian matrices.
We propose a ﬁrst-order algorithm that approximatelysolves OPT-in-Pareto using only gradient information, with both high practical efﬁciencyand theoretically guaranteed convergence property.
Empirically, we demonstrate that ourmethod works efﬁciently for a variety of challenging multi-task-related problems.
I NTRODUCTION
Although machine learning tasks are traditionally framed as optimizing a single objective.
Many modernapplications, especially in areas like multitask learning, require ﬁnding optimal model parameters to minimizemultiple objectives (or tasks) simultaneously.
As the different objective functions may inevitably conﬂictwith each other, the notion of optimality in multi-objective optimization (MOO) needs to be characterized bythe Pareto set: the set of model parameters whose performance of all tasks cannot be jointly improved.
Focusing on the Pareto set allows us to ﬁlter out models that can be strictly improved.
However, the Paretoset typically contains an inﬁnite number of parameters that represent different trade-offs of the objectives.
For m objectives (cid:96) 1 , . . .
, (cid:96) m , the Pareto set is often an ( m − 1) dimensional manifold.
It is both intractableand unnecessary to give practical users the whole exact Pareto set.
A more practical demand is to ﬁnd someuser-speciﬁed special parameters in the Pareto set, which can be framed into the following optimization in
Pareto set (OPT-in-Pareto) problem:
Finding one or a set of parameters inside the Pareto set of (cid:96) 1 , . .
, (cid:96) m that minimize a reference criterion F .
Here the criterion function F can be used to encode an informative user-speciﬁc preference on the objectives(cid:96) 1 , . . .
, (cid:96) m , which allows us to provide the best models customized for different users.
F can also be annon-informative measure that encourages, for example, the diversity of a set of model parameters.
In this case, optimizing F in Pareto set gives a set of diversiﬁed Pareto models that are representative of the whole
Pareto set, from which different users can pick their favorite models during the testing time.
OPT-in-Pareto provides a highly generic and actionable framework for multi-objective learning and optimization.
However, efﬁcient algorithms for solving OPT-in-Pareto have been largely lagging behind in deeplearning where the objective functions are non-convex and non-linear.
Although has not been formally studied,a straightforward approach is to apply manifold gradient descent on F in the Riemannian manifold formed bythe Pareto set (Hillermeier, 2001; Bonnabel, 2013).
However, this casts prohibitive computational cost dueto the need for eigen-computation of Hessian matrices of { (cid:96) i } .
In the optimization and operation researchliterature, there has been a body of work on OPT-in-Pareto viewing it as a special bi-level optimizationproblem (Dempe, 2018).
However, these works often heavily rely on the linearity and convexity assumptionsand are not applicable to the non-linear and non-convex problems in deep learning; see for examples in Ecker& Song (1994); Jorge (2005); Thach & Thang (2014); Liu & Ehrgott (2018); Sadeghi & Mohebi (2021) (justto name a few).
In comparison, the OPT-in-Pareto problem seems to be much less known and under-exploredin the deep learning literature.
In this work, we provide a practically efﬁcient ﬁrst-order algorithm for OPT-in-Pareto, using only gradientinformation of the criterion F and objectives { (cid:96) i } .
Our method, named Pareto navigation gradient descent(PNG), iteratively updates the parameters following a direction that carefully balances the descent on F and{ (cid:96) i } , such that it guarantees to move towards the Pareto set of { (cid:96) i } when it is far away, and optimize F in aneighborhood of the Pareto set.
Our method is simple, practically efﬁcient and has theoretical guarantees.
In empirical studies, we demonstrate that our method works efﬁciently for both optimizing user-speciﬁccriteria and diversity measures.
In particular, for ﬁnding representative Pareto solutions, we propose anenergy distance criterion whose minimizers distribute uniformly on the Pareto set asymptotically (Hardin& Saff, 2004), yielding a principled and efﬁcient Pareto set approximation method that compares favorablywith recent works such as Lin et al.
; Mahapatra & Rajan (2020).
We also apply PNG to improve theperformance of JiGen (Carlucci et al., 2019b), a multi-task learning approach for domain generalization, byusing the adversarial feature discrepancy as the criterion objective.
Related Work There has been a rising interest in MOO in deep learning, mostly in the context of multi-tasklearning.
But most existing methods can not be applied to the general OPT-in-Pareto problem.
A large bodyof recent works focus on improving non-convex optimization for ﬁnding some model in the Pareto set, butcannot search for a special model satisfying a speciﬁc criterion (Chen et al., 2018;
Kendall et al., 2018; Sener& Koltun, 2018; Yu et al., 2020; Chen et al., 2020; Wu et al., 2020; Fifty et al., 2020; Javaloy & Valera, 2021).
One exception is Mahapatra & Rajan (2020);
Kamani et al.
(2021), which searches for models in the Paretoset that satisfy a constraint on the ratio between the different objectives.
The problem they study can beviewed as a special instance of OPT-in-Pareto.
However, their approaches are tied with special properties ofthe ratio constraint and do not apply to the general OPT-in-Pareto problem.
There has also been increasing interest in ﬁnding a compact approximation of the Pareto set.
Navon et al.(2020)
; Lin et al.
(2020) use hypernetworks to approximate the map from linear scalarization weights tothe corresponding Pareto solutions; these methods could not fully proﬁle non-convex Pareto fronts dueto the limitation of linear scalarization (Boyd et al., 2004), and the use of hypernetwork introduces extraoptimization difﬁculty.
Another line of works (Lin et al., 2019; Mahapatra & Rajan, 2020) approximatethe Pareto set by training models with different user preference vectors that rank the relative importanceof different tasks; these methods need a good heuristic design of preference vectors, which requires priorknowledge of the Pareto front.
Ma et al.
(2020) leverages manifold gradient to conduct a local random walkon the Pareto set but suffers from the high computational cost.
Deist et al.
(2021) approximates the Pareto setby maximizing hypervolume, which requires prior knowledge for choosing a good reference vector.
Multi-task learning can also be applied to improve the learning in many other domains including domaingeneralization (Dou et al., 2019; Carlucci et al., 2019a; Albuquerque et al., 2020), domain adaption (Sun et al., 2019; Luo et al., 2021), model uncertainty (Hendrycks et al., 2019; Zhang et al., 2020; Xie et al.
, 2021),adversarial robustness (Yang & Vondrick, 2020) and semi-supervised learning (Sohn et al., 2020).
All ofthose applications utilize a linear scalarization to combine the multiple objectives and it is thus interesting toapply the proposed OPT-in-Pareto framework, which we leave for future work.
B ACKGROUND ON M ULTI - OBJECTIVE O PTIMIZATION
We introduce the background on multi-objective optimization (MOO) and Pareto optimality.
For notation,we denote by [ m ] the integer set { 1 , 2 , ...., m } , and R + the set of non-negative real numbers.
Let C m = 87 (cid:8)
ω ∈ R m + , (cid:80) mi =1 ω i =
1 (cid:9) be the probability simplex.
We denote by (cid:107)·(cid:107) the Euclidean norm.
Let θ ∈ R n be a parameter of interest (e.g., the weights in a deep neural network).
Let (cid:96) ( θ ) =[
(cid:96) 1 ( θ ) , . . .
, (cid:96) m ( θ )] be a set of objective functions that we want to minimize.
For two parameters θ, θ (cid:48) ∈ R n ,we write (cid:96) ( θ ) (cid:23) (cid:96) ( θ (cid:48) )
if (cid:96) i ( θ ) ≥ (cid:96) i ( θ (cid:48) ) for all i ∈ [ m ] ; and write (cid:96) ( θ ) (cid:31) (cid:96) ( θ (cid:48) ) if (cid:96) ( θ ) (cid:23) (cid:96) ( θ (cid:48) )
and(cid:96)
( θ ) (cid:54) =
(cid:96) ( θ (cid:48) ) .
We say that θ is Pareto dominated (or Pareto improved) by θ (cid:48)
if (cid:96) ( θ ) (cid:31) (cid:96) ( θ (cid:48) ) .
We say that θ is
Pareto optimal on a set Θ ⊆ R n , denoted as θ ∈ Pareto(Θ) , if there exists no θ (cid:
) ∈ Θ such that (cid:96) ( θ ) (cid:31) (cid:96) ( θ (cid:48) ) .
The Pareto global optimal set P ∗∗ :=
Pareto( R n ) is the set of points (i.e., θ ) which are Pareto optimal onthe whole domain R n .
The Pareto local optimal set of (cid:96) , denoted by P ∗ , is the set of points which are Paretooptimal on a neighborhood of itself:
P
[Equation]
: there exists a neighborhood N θ of θ , such that θ ∈ Pareto( N θ ) } .
The (local or global)
Pareto front is the set of objective vectors achieved by the Pareto optimal points, e.g.,the local Pareto front is F ∗ = { (cid:96) ( θ ) : θ ∈ P ∗ } .
Because ﬁnding global Pareto optimum is intractable fornon-convex objectives in deep learning, we focus on Pareto local optimal sets in this work; in the rest of thepaper, terms like “Pareto set” and “Pareto optimum” refer to Pareto local optimum by default.
Pareto Stationary Points Similar to the case of single-objective optimization, Pareto local optimum implies a notion of Pareto stationarity deﬁned as follows.
Assume (cid:96) is differentiable on R n .
A point θ is called Pareto stationary if there must exists a set of non-negative weights ω 1 , . . .
, ω m with (cid:80) mi =1 ω i = 1 , such that θ is a stationary point of the ω -weighted linear combination of the objectives: (cid:96) ω
( θ ) :
= (cid:80) mi =1 ω
i (cid:96) i ( θ ) .
Therefore, the set of Pareto stationary points, denoted by P , can be characterized by
[Equation]
where g ( θ ) is the minimum squared gradient norm of (cid:96) ω among all ω in the probability simplex C m on [ m ] .
Because g ( θ ) can be calculated in practice, it provides an essential way to access Pareto local optimality.
Finding Pareto Optimal Points
A main focus of the MOO literature is to ﬁnd a (set of) Pareto optimalpoints.
The simplest approach is linear scalarization , which minimizes (cid:96) ω for some weight ω (decided, e.g.,by the users) in C m .
However, linear scalarization can only ﬁnd Pareto points that lie on the convex envelopof the Pareto front (see e.g., Boyd et al., 2004), and hence does not give a complete proﬁling of the Paretofront when the objective functions (and hence their Pareto front) are non-convex.
Multiple gradient descent (MGD) (Désidéri, 2012) is an gradient-based algorithm that can converge to a
Pareto local optimum that lies on either the convex or non-convex parts of the Pareto front, depending on the initialization.
MGD starts from some initialization θ 0 and updates θ at the t -th iteration by
[Equation]
where ξ is the step size and v t is an update direction that maximizes the worst descent rate among allobjectives, since ∇ (cid:96) i ( θ t ) (cid:62) v ≈ ( (cid:96)
i ( θ t ) − (cid:96) i
( θ t − ξv )) /ξ approximates the descent rate of objective (cid:96) i whenfollowing direction v .
When using a sufﬁciently small step size ξ , MGD ensures to yield a Pareto improvement(i.e, decreasing all the objectives) on θ t unless θ t is Pareto (local) optimal; this is because the optimization in
Equation (2) always yields min i ∈
[ m ] ∇ (cid:96) i ( θ t ) (cid:62) v t ≤ 0 (otherwise we can simply ﬂip the sign of v t ).
Using Lagrange strong duality, the solution of Equation (2) can be framed into
[Equation]
where
[Equation]
It is easy to see from Equation (3) that the set of ﬁxed points of MDG (which satisfy v t = 0 ) coincides withthe Pareto stationary set P ∗ .
A key disadvantage of MGD, however, is that the Pareto point that it converges to depends on the initializationand other algorithm conﬁgurations in a rather implicated and complicated way.
It is difﬁcult to explicitlycontrol MGD to make it converge to points with speciﬁc properties.
O PTIMIZATION I N P ARETO S ET
The Pareto set typically contains an inﬁnite number of points.
In the optimization in Pareto set (OPT-inPareto) problem, we are given an extra criterion function F ( θ ) in addition to the objectives (cid:96) , and we want tominimize F in the Pareto set of (cid:96) , that is,
[Equation]
For example, one can ﬁnd the Pareto point whose loss vector (cid:96) ( θ ) is the closest to a given reference pointr ∈ R m by choosing F ( θ )
= (cid:107) (cid:96) ( θ ) − r (cid:107) 2 .
We can also design F to encourages (cid:96) ( θ ) to be proportional to r ,i.e., (cid:96) ( θ ) ∝ r ; a constraint variant of this problem was considered in Mahapatra & Rajan (2020).
We can further generalize OPT-in-Pareto to allow the criterion F to depend on an ensemble of Pareto points{ θ 1 , ..., θ N } jointly, that is,
[Equation]
For example, if F ( θ 1 , . . .
, θ N ) measures the diversity among { θ i } Ni =1 , then optimizing it provides a set of diversiﬁed points inside the Pareto set P ∗ .
An example of diversity measure is
F
[Equation]
, with
[Equation]
where E is known as an energy distance in computational geometry, whose minimizer can be shown to givean uniform distribution asymptotically when N → ∞ (Hardin & Saff, 2004).
This formulation is particularlyuseful when the users’ preference is unknown during the training time, and we want to return an ensemble ofmodels that well cover the different areas of the Pareto set to allow the users to pick up a model that ﬁts theirneeds regardless of their preference.
The problem of proﬁling Pareto set has attracted a line of recent works(e.g.
, Lin et al., 2019; Mahapatra & Rajan, 2020; Ma et al., 2020; Deist et al., 2021), but they rely on speciﬁccriterion or heuristics and do not address the general optimization of form Equation (5).
Manifold Gradient Descent One straightforward approach to OPT-in-Pareto is to deploy manifold gradientdescent (Hillermeier, 2001; Bonnabel, 2013), which conducts steepest descent of F ( θ ) in the Riemannianmanifold formed by the Pareto set P ∗ .
Initialized at θ 0 ∈ P ∗ , manifold gradient descent updates θ t at the t -thiteration along the direction of the projection of ∇ F ( θ t ) on the tangent space T ( θ t ) at θ t in P ∗ ,θ t +
=
θ t − ξ Proj T ( θ t ) ( ∇ F ( θ t )) .
By using the stationarity characterization in Equation (1), under proper regularity conditions, one canshow that the tangent space T ( θ t ) equals the null space of the Hessian matrix ∇ 2 θ (cid:96) ω
t ( θ t ) , where ω t =arg min ω ∈C m (cid:107)∇ θ (cid:96) ω ( θ t ) (cid:107) .
However, the key issue of manifold gradient descent is the high cost for calculatingthis null space of Hessian matrix.
Although numerical techniques such as Krylov subspace iteration (Maet al., 2020) or conjugate gradient descent (Koh & Liang, 2017) can be applied, the high computational cost(and the complicated implementation) still impedes its application in large scale deep learning problems.
See
Section 1 for discussions on other related works.
P ARETO N AVIGATION G RADIENT D ESCENT FOR OPT- IN -P ARETO
We now introduce our main algorithm, Pareto Navigating Gradient Descent (PNG), which provides a practicalapproach to OPT-in-Pareto.
For convenience, we focus on the single point problem in Equation (4) in thepresentation.
The generalization to the multi-point problem in Equation (5) is straightforward.
We ﬁrstintroduce the main idea and then present theoretical analysis in Section 4.1.
Main Idea We consider the general incremental updating rule of form
[Equation]
where ξ is the step size and v t is an update direction that we shall choose to achieve the following desideratain balancing the decent of { (cid:96) i } and F :
i)
When θ t is far away from the Pareto set, we want to choose v t to give Pareto improvement to θ t , moving it
towards the Pareto set.
The amount of Pareto improvement might depend on how far θ t is to the Pareto set.
ii)
If the directions that yield Pareto improvement are not unique, we want to choose the Pareto improvement
direction that decreases F ( θ ) most.
iii)
When θ t is very close to the Pareto set, e
.g.
, having a small g ( θ ) , we want to fully optimize F ( θ ) .
We achieve the desiderata above by using the v t that solves the following optimization:
[Equation]
where we want v t to be as close to ∇ F ( θ t ) as possible (hence decrease F most), conditional on that thedecreasing rate ∇ θ (cid:96) i ( θ t ) (cid:62) v t of all losses (cid:96) i are lower bounded by a control parameter φ t .
A positive φ tenforces that ∇ θ t (cid:96) i ( θ ) (cid:62) v t is positive for all (cid:96) i , hence ensuring a Pareto improvement when the step size issufﬁciently small.
The magnitude of φ t controls how much Pareto improvement we want to enforce, so wemay want to gradually decrease φ t when we move closer to the Pareto set.
In fact, varying φ t provides anintermediate updating direction between the vanilla gradient descent on F and MGD on { (cid:96) i } :i)
If φ t =
−∞ , we have v t = ∇ F ( θ t ) and it conducts a pure gradient descent on F without considering { (cid:96) i } .ii)
If φ t → + ∞ , then v t approaches to the MGD direction of { (cid:96) i } in Equation (2) without considering F .
In this work, we propose to choose φ t based on the minimum gradient norm g ( θ t ) in Equation (1) as a surrogate indication of Pareto local optimality.
In particular, we consider the following simple design:
[Equation]
where e is a small tolerance parameter and α t is a positive hyper-parameter.
When g ( θ t ) >
e , we set φ t to beproportional to g ( θ t ) , to ensure Pareto improvement based on how far θ t is to Pareto set.
When g ( θ t ) ≤ e ,we set φ t = −∞ which “turns off” the control and hence fully optimizes F ( θ ) .
In practice, the optimization in Equation (7) can be solved efﬁciently by its dual form as follows.
Theorem 1.
The solution v t of Equation (7), if it exists, has a form of
[Equation]
with { λ i,t } mt =1 the solution of the following dual problem
[Equation]
The optimization in Equation (10) can be solved efﬁciently for a small m (e..g, m ≤ 10 ), which is the casefor typical applications.
We include the details of the practical implementation in Appendix B.
4. T HEORETICAL P ROPERTIES
We provide a theoretical quantiﬁcation on how PNG guarantees to i) move the solution towards the Paretoset (Theorem 2); and ii) optimize F in a neighborhood of Pareto set (Theorem 3).
To simplify the result andhighlight the intuition, we focus on the continuous time limit of PNG, which yields a differentiation equationd θ t = − v t d t with v t deﬁned in Equation (7), where t ∈ R + is a continuous integration time.
Assumption 1.
Let { θ t : t ∈ R + } be a solution of d θ t = − v t d t with v t in Equation (7); φ k in Equation (8);e > 0 ; and α t ≥ 0 , ∀ t ∈ R + .
Assume F and (cid:96) are continuously differentiable on R n , and lower boundedwith F ∗ :=
inf θ ∈ R n F ( θ ) >
−∞ and (cid:96) ∗ i := inf θ ∈ R n (cid:96) i ( θ ) >
−∞ .
Assume sup θ ∈ R n (cid:107)∇ F ( θ ) (cid:107) ≤ c .
Technically, d θ t = − v t d t is a piecewise smooth dynamical system whose solution should be taken in the
Filippov sense using the notion of differential inclusion (Bernardo et al., 2008).
The solution always existsunder mild regularity conditions although it may not be unique.
Our results below apply to all solutions.
Pareto Optimization on (cid:96) We now show that the algorithm converges to the vicinity of Pareto set quantiﬁed by a notion of Pareto closure.
For (cid:15) ≥ 0 , let P (cid:15) be the set of Pareto (cid:15) -stationary points: P (cid:15)
= { θ ∈
R n : g ( θ ) ≤ (cid:15) } .
The Pareto closure of a set P (cid:15) , denoted by P (cid:15) is the set of points that perform no worse than at least one point in P (cid:15) , that is,
[Equation]
Therefore, P (cid:15) is better than or at least as good as P (cid:15) in terms of Pareto efﬁciency.
Theorem 2 (Pareto Improvement on (cid:96) ) .
Under Assumption 1, assume θ 0 (cid:54)∈ P e , and t e is the ﬁrst time when θ t e ∈ P e , then for any time t < t e , d
[Equation]
Therefore, the update yields Pareto improvement on (cid:96) when θ t (cid:54)∈ P e and α t g ( θ t ) > 0 .
Further, if (cid:82) t
α s d s = + ∞ , then for any (cid:15) > e , there exists a ﬁnite time t (cid:15) ∈ R + on which the solution enters
P (cid:15) and stays within P (cid:15) afterwards, that is, we have θ t (cid:15) ∈ P (cid:15) and θ t ∈ P (cid:15) for any t ≥ t (cid:15) .
Here we guarantee that θ t must enter P (cid:15) for some time (in fact inﬁnitely often), but it is not conﬁned in P (cid:15) .
On the other hand, θ t does not leave P (cid:15) after it ﬁrst enters P (cid:15) thanks to the Pareto improvement property.
Optimization on F We now show that PNG ﬁnds a local optimum of F inside the Pareto closure P (cid:15) in anapproximate sense.
We ﬁrst show that a ﬁxed point θ of the algorithm that is locally convex on F and (cid:96) mustbe a local optimum of F in the Pareto closure of { θ } , and then quantify the convergence of the algorithm.
Lemma 1.
Under Assumption 1, assume θ t (cid:54)∈ P e is a ﬁxed point of the algorithm, that is, d θ t d t = − v t = 0 ,and F , (cid:96) are convex in a neighborhood θ t , then θ t is a local minimum of F in the Pareto closure { θ t } , that is,there exists a neighborhood of θ t in which there exists no point θ (cid:48)
such that F ( θ (cid:48) )
<
F ( θ t ) and (cid:96) ( θ (cid:48) )
(cid:22) (cid:96) ( θ t ) .
On the other hand, if θ t ∈ P e , we have v t = ∇ F ( θ t ) , and hence a ﬁxed point with d θ t d t = − v t = 0 is anunconstrained local minimum of F when F is locally convex on θ t .
Theorem 3.
Let (cid:15) >
e and assume g (cid:15) :
= sup θ { g ( θ ): θ ∈ P (cid:15) } < + ∞ and sup t ≥
α t < ∞ .
Under
Assumption 1, when we initialize from θ 0 ∈ P (cid:15) , we have
[Equation]
In particular, if we have α t = α = const , then min s ∈ [0 ,t ] (cid:107) d θ
s / d s (cid:107)
= O (cid:0) 1 /t + α √ g (cid:15) (cid:1) .
If (cid:82) ∞
α γt d t < + ∞ for some γ ≥ 1 , we have min s ∈ [0 ,t ] (cid:107) d θ s / d s (cid:107)
= O (1 /t + √ g (cid:15) /t 1 /γ ) .
Combining the results in Theorem 2 and 3, we can see that the choice of sequence { α t : t ∈ R + } controls howfast we want to decrease (cid:96) vs. F .
Large α t yields faster descent on (cid:96) , but slower descent on F .
Theoretically,using a sequence that satisﬁes (cid:82) α t d t = + ∞ and (cid:82) α γt d t < + ∞ for some γ > 1 allows us to ensure thatboth min s ∈ [0 ,t ] g ( θ s ) and min s ∈ [0 ,t ] (cid:107) d θ/ d s (cid:107) 2 converge to zero.
If we use a constant sequence α t = α , itintroduces an O ( α √ g (cid:15) ) term that does not vanish as t → + ∞ .
However, we can expect that g (cid:15) is small when(cid:15) is small for well-behaved functions.
In practice, we ﬁnd that constant α t works sufﬁciently well.
E MPIRICAL R ESULTS
We introduce three applications of OPT-in-Pareto with PNG:
Singleton Preference, Pareto approximation andimproving multi-task based domain generalization method.
We also conduct additional study on how thelearning dynamics of PNG changes with different initialization and hyper-parameters ( α t and e ), which areincluded in Appendix C.3.
Other additional results that are related to the experiments in Section 5.1 and 5.2and are included in the Appendix will be introduced later in their corresponding sections.
5. F INDING P REFERRED P ARETO M ODELS
We consider the synthetic example used in Lin et al.
; Mahapatra & Rajan (2020), which consists oftwo losses: (cid:96) 1 ( θ ) = 1 − exp( − (cid:107) θ − η (cid:107) 2 ) and (cid:96) 2 ( θ ) = 1 − exp( − (cid:107) θ + η (cid:107) 2 ) , where η = n − 1 / 2 and n = 10is dimension of the parameter θ .
Ratio-based Criterion
We ﬁrst show that PNG can solve the search problem under the ratio constraint ofobjectives in Mahapatra & Rajan (2020), i.e., ﬁnding a point θ ∈ P ∗ ∩ Ω with Ω
= { θ : r 1 (cid:96) 1 ( θ ) =
r 2 (cid:96) 2 ( θ ) =...
=
r m (cid:96) m ( θ ) } , given some preference vector r =
[ r 1 , ..., r m ] .
We apply PNG with the non-uniformityscore deﬁned in Mahapatra & Rajan (2020) as the criterion, and compare with their algorithm called exact
Pareto optimization (EPO).
We show in Figure 1(a)-(b) the trajectory of PNG and EPO for searching modelswith different preference vector r , starting from the same randomly initialized point.
Both PNG and EPOconverge to the correct solutions but with different trajectories.
This suggests that PNG is able to achievethe same functionality of ﬁnding ratio-constraint Pareto models as Mahapatra & Rajan (2020);
Kamani et al.(2021) do but being versatile to handle general criteria.
We refer readers to Appendix C.1.1 for more resultswith different choices of hyper-parameters and the experiment details.
Other Criteria We demonstrate that PNG is able to ﬁnd solutions for general choices of F .
We considerthe following designs of F : 1) weighted (cid:96) 2 distance w.r.t.
a reference vector r ∈ R m + , that is, F wd ( θ ) =
l 1 0 .0 .0 .
.0 .1 .l
[Figure]
Pareto
[Figure]
{+EPO 0 .0 .0 .
+}
l 1 0 .0 .0 .
.0 .1 .l
[Figure]
Pareto
[Figure]
l 1 0 .0 .0 {+.
+}
0 lweighted distance
[Figure]
l 1 0 {+.0 .0 .
+} l
[Figure]
(cid:80) mi
=1 ( (cid:96) i ( θ ) − r i ) 2 /r i ; and 2) complex cosine: in which F is a complicated function related to the cosineof task objectives, i.e., F cs = − cos ( π ( (cid:96) 1 ( θ ) − r 1 ) / 2) + ( cos ( π ( (cid:96) ( θ 2 ) − r 2 ))
+ 1) 2 . Here the weighted (cid:96) 2distance can be viewed as ﬁnding a Pareto model that has the losses close to some target value r , which can beviewed as an alternative approach to partition the Pareto set. The design of complex cosine aims to test whether PNG is able to handle a very non-linear criterion function.
In both cases, we take r 1 = [0 . 2 , 0 . 4 , 0 . 6 , 0 . 8] andr 2 = 1 − r 1 .
We show in Fig 1(c)-(d)
the trajectory of PNG.
As we can see, PNG is able to correctly ﬁnd theoptimal solutions of OPT-in-Pareto.
We also test PNG on a more challenging ZDT2-variant used in Ma et al.(2020) and a larger scale MTL problem (Liu et al., 2019).
We refer readers to Appendix C.1.2 and C.1.3 forthe setting and results.
5. F INDING D IVERSE P ARETO M ODELS
Setup We consider the problem of ﬁnding diversiﬁed points from the Pareto set by minimizing the energydistance criterion in Equation (6).
We use the same setting as Lin et al.
; Mahapatra & Rajan (2020).
We consider three benchmark datasets: (1) MultiMNIST, (2) MultiFashion, and (3) MultiFashion+MNIST.
For each dataset, there are two tasks (classifying the top-left and bottom-right images).
We consider LeNetwith multihead and train N = 5 models to approximate the Pareto set.
For baselines, we compare with linearscalarization, MGD (Sener & Koltun, 2018), and EPO (Mahapatra & Rajan, 2020).
For the MGD baseline,we ﬁnd that naively running it leads to poor performance as the learned models are not diversiﬁed and thus weinitialize the MGD with 60-epoch runs of linear scalarization with equally distributed preference weights andruns MGD for the later 40 epoch.
We refer the reader to Appendix C.2.1 for more details of the experiments.
Metric and Result We measure the quality of how well the found models { θ 1 , . . .
, θ N } approximate the
Pareto set using two standard metrics: Inverted Generational Distance Plus (IGD+) (Ishibuchi et al.
, 2015)and hypervolume (HV) (Zitzler & Thiele, 1999); see Appendix C.2.2 for their deﬁnitions.
We run all themethods with 5 independent trials and report the averaged value and its standard deviation in Table 1.
Wereport the scores calculated based on loss (cross-entropy) and accuracy on the test set.
The bolded valuesindicate the best result with p-value less than 0.05 (using matched pair t-test).
In most cases, PNG improvesthe baselines by a large margin.
We include ablation studies in Appendix C.2.3 and additional comparisonswith the second-order approach proposed by Ma et al.
(2020) in Appendix C.2.4.
5. A PPLICATION TO M ULTI - TASK BASED D OMAIN G ENERALIZATION A LGORITHM
JiGen (Carlucci et al., 2019b) learns a domain generalizable model by learning two tasks based on linearscalarization, which essentially searches for a model in the Pareto set and requires choosing the weight of [Table] [Table] linear scalarization carefully.
It is thus natural to study whether there is a better mechanism that dynamicallyadjusts the weights of the two losses so that we eventually learn a better model.
Motivated by the adversarialfeature learning (Ganin et al., 2016), we propose to improve JiGen such that the latent feature representationsof the two tasks are well aligned.
This can be framed into an OPT-in-Pareto problem where the criterion isthe discrepancy of the latent representations (implemented using an adversarial discrepancy module in thenetwork) of the two tasks.
PNG is applied to solve the optimization.
We evaluate the methods on PACS (Liet al., 2017), which covers 7 object categories and 4 domains (Photo, Art Paintings, Cartoon, and Sketches).
The model is trained on three domains and tested on the rest of them.
Our approach is denoted as JiGen+PNGand we also include JiGen + adv, which simply adds the adversarial loss as regularization and two otherbaseline methods (D-SAM (D’Innocente & Caputo, 2018) and DeepAll (Carlucci et al., 2019b)).
For the three
JiGen based approaches, we run 3 independent trials and for the other two baselines, we report the results intheir original papers.
Table 2 shows the result using ResNet-18, which demonstrates the improvement by theapplication of the OPT-in-Pareto framework.
We also include the results using AlexNet in the Appendix.
Werefer readers to Appendix C.4 for the additional results and more experiment details.
C ONCLUSION
This paper studies the OPT-in-Pareto, a problem that has been studied in operation research with restrictivelinear or convexity assumption but largely under-explored in deep learning literature, in which the objectivesare non-linear and non-convex.
Applying algorithms such as manifold gradient descent requires eigencomputation of the Hessian matrix at each iteration and thus can be expensive.
We propose a ﬁrst-orderapproximation algorithm called Pareto Navigation Gradient Descent (PNG) with theoretically guaranteeddescent and convergence property to solve OPT-in-Pareto.
A T HEORETICAL A NALYSIS
Theorem
[Dual of Equation (7)]
The solution v t of Equation (7), if it exists, has a form of
[Equation]
with { λ i,t } mi =1 the solution of the following dual problem
[Equation]
where R m + is the set of nonnegative m -dimensional vectors, that is, R m + = { λ ∈ R m : λ i ≥ 0 ,
∀ i ∈ [ m ] } .
Proof.
By introducing Lagrange multipliers, the optimization in Equation (7) is equivalent to the followingminimax problem:
[Equation]
With strong duality of convex quadratic programming (assuming the primal problem is feasible), we can exchange the order of min and max, yielding
[Equation]
It is easy to see that the minimization w.r.t.
v is achieved when v = ∇ F ( θ t ) + (cid:80) mi
=1 λ i ∇ (cid:96) i ( θ t ) .
Correspondingly, the Φ( λ ) has the following dual form:
[Equation]
This concludes the proof.
Theorem 2 [Pareto Improvement on (cid:96) ]
Under Assumption 1, assume θ 0 (cid:54)∈ P e , and t e is the ﬁrst time when θ t e ∈ P e , then for any time t < t e , [Equation] min s [Equation]
[Equation]
t
[Equation]
t
[Equation] min i ∈ [Equation] Therefore, the update yields Pareto improvement on (cid:96) when θ t (cid:54)∈ P e and α t g ( θ t ) > 0 .
Further, if (cid:82) t
α s d s = + ∞ , then for any (cid:15) > e , there exists a ﬁnite time t (cid:15) ∈ R + on which the solution enters
P (cid:15) and stays within P (cid:15) afterwards, that is, we have θ t (cid:15) ∈ P (cid:15) and θ t ∈ P (cid:15) for any t ≥ t (cid:15) .
Proof.
i)
When t < t e , we have g ( θ t ) >
e and hence
[Equation]
where we used the constraint of ∇ (cid:96) i ( θ t ) (cid:62) v t ≥ φ t in Equation (7).
Therefore, we yield strict decent on all thelosses { (cid:96) i } when α t g ( θ t ) > 0 . ii) Integrating both sides of Equation (11):
[Equation]
This yields the result since it holds for every i ∈ [ m ] .
If (cid:82) ∞
α t d t = + ∞ , then we have min s ∈ [0 ,t ] g ( θ s ) → 0 when t → + ∞ .
Assume there exists an (cid:15) >
e ,such that θ t never enters P (cid:15) at ﬁnite t .
Then we have g ( θ t ) ≥ (cid:15) for t ∈ R + , which contradicts withmin s ∈ [0 ,t ] g ( θ s ) → 0 .iii)
Assume there exists a ﬁnite time t (
cid:48) ∈ ( t (cid:15) , + ∞ ) such that θ t (cid:48) (cid:54)∈ P (cid:15) .
Because (cid:15) >
e and g is continuous, P eis in the interior of P (cid:15) ⊆
P (cid:15) .
Therefore, the trajectory leading to θ t (cid:48) (cid:54)∈ P (cid:15) must pass through P (cid:15) \ P e at somepoint, that is, there exists a point t (cid:48)(cid:48) ∈
[ t (cid:15) , t (cid:48) ) , such that { θ t : t ∈ [ t (cid:48)(cid:48) , t (cid:48) ] } (cid:54)∈ P e .
But because the algorithm cannot increase any objective (cid:96) i outside of P e , we must have (cid:96) ( θ t (cid:48) )
(cid:22) (cid:96) ( θ t (cid:48)(cid:48) ) , yielding that θ t (cid:48) ∈ { θ t (cid:48)(cid:48) } ⊆ P (cid:15) ,where { θ t (cid:48)(cid:48) } is the Pareto closure of { θ t (cid:48)(cid:48) } ; this contradicts with the assumption.
Lemma
Under Assumption 1, assume θ t (cid:54)∈ P e is a ﬁxed point of the algorithm, that is, d θ t d t = − v t = 0 ,and F , (cid:96) are convex in a neighborhood θ t , then θ t is a local minimum of F in the Pareto closure { θ t } ,that is, there exists a neighborhood of θ t in which there exists no point θ (cid:48)
such that F ( θ (cid:48) )
<
F ( θ t ) and(cid:96)
( θ (cid:48) )
(cid:22) (cid:96) ( θ t ) .
Proof.
Note that minimizing F in { θ t } can be framed into a constrained optimization problem:
[Equation]
In addition, by assumption, θ = θ t satisﬁes v t = ∇ F ( θ
[Equation]
+ (cid:80) mi
=
λ i,t ∇ (cid:96) i ( θ t ) = 0 , which is the KKTstationarity condition of the constrained optimization.
It is also obvious to check that θ
= θ t satisﬁes thefeasibility and slack condition trivially.
Combining this with the local convexity assumption yields theresult.
Theorem 3 [Optimization of F ] Let (cid:15) >
e and assume g (cid:15) :
= sup θ { g ( θ ): θ ∈ P (cid:15) } < + ∞ and sup t ≥
α t < ∞ .
Under Assumption 1, when we initialize from θ 0 ∈ P (cid:15) , we have min s ∈ [0 ,t ] (cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
In particular, if we have α t = α = const , then min s ∈ [0 ,t ] (cid:107) d θ
s / d s (cid:107)
= O (cid:0) 1 /t + α √ g (cid:15) (cid:1) .
If (cid:82) ∞
α γt d t < + ∞ for some γ ≥ 1 , we have min s ∈ [0 ,t ] (cid:107) d θ s / d s (cid:107)
= O (1 /t + √ g (cid:15) /t 1 /γ ) .
Proof.
i)
The slack condition of the constrained optimization in Equation (7) says that
[Equation]
This gives that
[Equation]
plugging
[Equation] [Equation]
If θ t (cid:54)∈ P e , we have φ t = α t g ( θ t ) and this gives
[Equation]
If θ t is in the interior of P e , then we run typical gradient descent of F and hence has
[Equation]
If θ t is on the boundary of P e , then by the deﬁnition of differential inclusion, d θ/ d t belongs to the convex hull of the velocities that it receives from either side of the boundary, yielding that
[Equation]
where β ∈ [0 , 1] .
Combining all the cases gives
[Equation]
(cid:13) (cid:13) (cid:13)(cid:13)
d θ
[Equation]
d t (cid:13) (cid:13) (cid:13)(cid:13) + m (cid:88) i
= λ
i,t [Equation] t g ( θ t ) .
min
[Equation]
where the last step used Lemma 2 with φ t
= α t g ( θ t ) : m (cid:88) i =
[Equation]
θ
[Equation]
+
[Equation]
, and here we used g ( θ t ) ≤ g (cid:15) because the trajectory is contained in P (cid:15) following Theorem 2.
The remaining results follow Lemma 4.
A.0. T ECHNICAL L EMMAS
Lemma 2.
Assume Assumption 1 holds.
Deﬁne g ( θ ) =
min ω ∈C m (cid:107) (cid:80) mi
=1 ω i ∇ (cid:96) i
( θ ) (cid:107) 2 , where C m is the probability simplex on [ m ] .
Then for the v t and λ i,t deﬁned in Equation (7) and Equation (10), we have
[Equation]
Proof.
The slack condition of the constrained optimization in Equation (7) says that
[Equation]
Sum the equation over i ∈ [ m ] and note that v t = ∇ F ( θ t ) + (cid:80) mi =1 λ i,t ∇ (cid:96) i ( θ t ) .
We get
[Equation]
Deﬁne
[Equation]
Then it is easy to see that x t ≥ ¯ λ 2 t g t .
Using Cauchy-Schwarz inequality,
[Equation]
where we used (cid:107)∇ F ( θ t ) (cid:107) ≤ c by Assumption 1.
Combining this with Equation (14), we have
[Equation]
Applying Lemma 3 yields the result.
Lemma 3.
Assume φ ∈ R , and x, λ, c, g ∈ R + are non-negative real numbers and they satisfy
[Equation]
≥ λ
[Equation]
Then we have λg ≤ max(0 , φ + c √ g ) .
Proof.
Square the ﬁrst equation, we get
[Equation]
where f is a quadratic function.
To ensure that f ( x ) ≤ 0 has a solution that satisﬁes x ≥ λ 2 g , we need to have f ( λ 2 g ) ≤ 0 , that is,
[Equation]
This can hold under two cases:
Case 1: λ = 0;
Case 2: | λg − φ | ≤ c √ g , and hence φ − c √ g ≤ λg ≤ φ + c √ g .
Under both case, we have
[Equation]
Lemma 4.
Let { α t : t ∈ R + } ⊆ R + be a non-negative sequence with A := (cid:0)(cid:82) ∞
α γt d t (cid:1) 1 /γ < ∞ , where γ ≥ 1 , and B = sup t
α t < ∞ .
Then we have
[Equation]
Proof.
Let η = γγ − 1 , so that 1 /η + 1 /γ = 1 .
We have by Holder’s inequality, (cid:90) t
[Equation]
(cid:18)(cid:90) {+t 
+}
α γs d s (cid:19) 1 /γ (cid:18)(cid:90) t
1 η d s (cid:19) 1 /η ≤
At 1 /η
=
At 1 − 1 /γ .
and hence
[Equation] [Equation]
[Equation]
) where λ i,t = 0 , ∀ i ∈ [ m ] if g ( θ t ) ≤ e , and { λ i,t } mt =1 is the solution of Equation (10) with φ ( θ t ) =
α t g ( θ t ) when g ( θ t ) > e .
B P RACTICAL I MPLEMENTATION
Hyper-parameters
Our algorithm introduces two hyperparameters { α t } and e over vanilla gradient descent.
We use constant sequence α t
= α
and we take α = 0 .
5 unless otherwise speciﬁed.
We choose e bye = γe 0 , where e 0 is an exponentially discounted average of 1 m (cid:80) mi =1 (cid:107)∇ (cid:96) i ( θ t ) (cid:107) 2 over the trajectory so thatit automatically scales with the magnitude of the gradients of the problem at hand.
In the experiments of thispaper, we simply ﬁx γ = 0 . 1 unless speciﬁed.
Solving the Dual Problem Our method requires to calculate { λ i,t } mt =1 with the dual optimization problemin Equation (10), which can be solved with any off-the-shelf convex quadratic programming tool.
In thiswork, we use a very simple projected gradient descent to approximately solve Equation (10).
We initialize{ λ i,t } mt =1 with a zero vector and terminate when the difference between the last two iterations is smaller thana threshold or the algorithm reaches the maximum number of iterations (we use 100 in all experiments).
The whole algorithm procedure is summarized in Algorithm 1.
C E XPERIMENTSC. F INDING P REFERRED P ARETO M ODELSC.1. R ATIO - BASED C RITERION
The non-uniformity score from (Mahapatra & Rajan, 2020) that we use in Figure 1 is deﬁned as
[Equation] [Equation]
We ﬁx the other experiment settings the same as Mahapatra & Rajan (2020) and use γ = 0 .
and α = 0 .
25for this experiment reported in the main text.
We defer the ablation studies on the hyper-parameter α and γ to
Section C.3.
C.1. ZDT2-V ARIANT
We consider the ZDT2-Variant example used in Ma et al.
(2020) with the same experiment setting, inwhich the Pareto set is a cylindrical surface, making the problem more challenging.
We consider thesame criteria, e.g. weighted distance and complex cosine used in the main context with different choicesof r
=
We use the default hyper-parameter set up, choosing α = 0 .
5 and r = 0 .
[Figure]
For complex cosine, we use MGD updating for the ﬁrst 150 iterations.
Figure 2 shows the trajectories,demonstrating that PNG works pretty well for the more challenging ZDT2-Variant tasks.
C.1. G ENERAL C RITERIA : T HREE - TASK LEARNING ON THE NYU V 2 D ATASET
We show that PNG is able to handle large-scale multitask learning problems by deploying it on a threetask learning problem (segmentation, depth estimation, and surface normal prediction) on NYUv2 dataset(Silberman et al., 2012).
The main goal of this experiment is to show that: 1.
PNG is able to handle
OPT-in-Pareto in a large-scale neural network; 2.
With a proper design of criteria, PNG enables to dotargeted ﬁne-tuning that pushes the model to move towards a certain direction.
We consider the sametraining protocol as Liu et al.
and use the MTAN network architecture.
Start with a model trainedwith equally weighted linear scalarization and our goal is to further improve the model’s performanceon segmentation and surface normal estimation while allowing some sacriﬁce on depth estimation.
Thiscan be achieved by many different choices of criterion and in this experiment, we consider the followingdesign:
[Equation]
001 + (cid:96) depth ( θ )) .
Here (cid:96) seg , (cid:96) surface and (cid:96) depth are the loss functionsfor segmentation, surface normal prediction and depth estimation, respectively.
The constant 0.001 in thedenominator is for numeric stability.
We point out that our design of criterion is a simple heuristic and mightnot be an optimal choice and the key question we study here is to verify the functionality of the proposed
PNG.
As suggested by the open-source repository of Liu et al.
(2019), we reproduce the result based on theprovided conﬁguration.
To show that PNG is able to move the model along the Pareto front, we show theevolution of the criterion function and the norm of the MGD gradient during the training in Figure 3.
As wecan see, PNG effectively decreases the value of criterion function while the norm of MGD gradient remainsthe same.
This demonstrates that PNG is able to minimize the criterion by searching the model in the Paretoset.
Table 3 compares the performances on the three tasks using standard training and PNG, showing that
PNG is able to improve the model’s performance on segmentation and surface normal prediction tasks whilesatisfying a bit of the performance in depth estimation based on the criterion.
C. F INDING D IVERSE P ARETO M ODELS
C.2. E XPERIMENT D ETAILS
[Table] [Table]
We train the model for 100 epochs using Adam optimizer with batch size 256 and 0.001 learning rate.
To encourage diversity of the models, following thesetting in Mahapatra & Rajan (2020), we use equallydistributed preference vectors for linear scalarizationand EPO.
Note that the stochasticity of using minibatches is able to improve the performance of Paretoapproximation for free by also using the intermediate checkpoints to approximate P .
To fully exploitthis advantage, for all the methods, we collect checkpoints every epoch to approximate P , starting fromepoch 60.
E VALUATION M ETRIC D ETAILS
We introduce the deﬁnition of the used metric forevaluation.
Given a set ˆ P = { θ 1 , . . .
, θ N } that weuse to approximate P , its IGD+ score is deﬁned as:
[Equation]
where µ is some base measure that measures the importance of θ ∈ P and ( t ) + := max( t, 0) , applied oneach element of a vector.
Intuitively, for each θ , we ﬁnd a nearest ˆ θ ∈ ˆ P that approximates θ best.
Herethe ( · ) + is applied as we only care the tasks that ˆ θ is worse than θ .
In practice, a common choice of µ canbe a uniform counting measure with uniformly sampled (or selected) models from P .
In our experiments,since we can not sample models from P , we approximate P by combining ˆ P from all the methods, i.e.,
P ≈ ∪ m ∈ {Linear,MGD,EPO,PNG} ˆ P m , where ˆ P m is the approximation set produced by algorithm m .
This approximation might not be accurate but is sufﬁcient to compare the different methods,
The Hypervolume score of ˆ P , w.r.t.
a reference point (cid:96) r ∈ R m + , is deﬁned as
[Equation]
, where µ is again some measure.
We use (cid:96) r = [0 . 6 , 0 . 6] for calculating the Hypervolume based on loss andset µ to be the common Lebesgue measure.
Here we choose 0.6 as we observe that the losses of the two tasksare higher than 0.6 and 0.6 is roughly the worst case.
When calculating Hypervolume based on accuracy, we simply ﬂip the sign.
[Table]
A BLATION S TUDY
We conduct ablation study to understand the effect of α and γ using the Pareto approximation task on
Multi-Mnist.
We compare PNG with α = 0 .
75 and γ = 0 .
Figure 4 summarizes theresult.
Overall, we observe that PNG is not sensitive to the choice of hyper-parameter.
C.2. C OMPARING WITH THE S ECOND O RDER A PPROACH
We give a discussion on comparing our approach with the second order approaches proposed by Ma et al.(2020).
In terms of algorithm, Ma et al.
(2020) is a local expansion approach.
To apply Ma et al.
(2020),in the ﬁrst stage, we need to start with several well distributed models (i.e., the ones obtained by linearscalarization with different preference weights) and Ma et al.
(2020) is only applied in the second stage toﬁnd the neighborhood of each model.
The performance gain comes from the local neighbor search of eachmodel (i.e. the second stage).
In comparison, PNG with energy distance is a global search approach.
It improves the well-distributednessof models in the ﬁrst stage (i.e. it’s a better approach than simply using linear scalarization with differentweights).
And thus the performance gain comes from the ﬁrst stage.
Notice that we can also apply Ma et al.(2020) to PNG with energy distance to add extra local search to further improve the approximation.
In terms of run time comparison.
We compare the wall clock run time of each step of updating the 5 modelsusing PNG and the second order approach in Ma et al.
We calculate the run time based on themulti-MNIST dataset using the average of 100 steps.
PNG uses 0.3s for each step while Ma et al.
2020 uses16.8s.
PNG is 56x faster than the second order approach.
And we further argue that, based on time complexitytheory, the gap will be even larger when the size of the network increases.
C. U NDERSTANDING PNG D YNAMICS
We draw more analysis to understand the training dynamics of PNG.
Different Staring Points We give analysis on PNG with different initializations showing that PNG ismore robust to the initialization than other approaches such as Lin et al.
We consider the Pareto setapproximation tasks and reuse synthetic example introduced in Section 5.1.
We consider learning 5 models toapproximate the Pareto front staring from two different bad starting points.
Speciﬁcally, in the upper row of
Figure 4, we consider initializing the models using linear scalarization.
Due to the concavity of the Paretofront, linear scalarization can only learns models at the two extreme end of the Pareto front.
The second rowuses MGD for initialization and the models is scattered at an small region of the Pareto front.
Different fromthe algorithm proposed by Lin et al.
(2019) which relies on a good initialization, using the proposed energy
l 1 0 .0 .0 .
l
Pareto Front
Models 0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
0 l distance function, PNG pushes the models to be equally distributed on the Pareto Front without the need ofany prior information of the Pareto front even with extremely bad starting point.
Trajectory Visualization with Different Hyper-parameters We also give more visualization on the PNGtrajectory when using different hyper-parameters.
We reuse synthetic example introduced in Section 5.1for studying the hyper-parameters α and γ .
We ﬁx α = 0 .
25 and vary γ = 0 .
01 , 0 . 1 ; and ﬁxγ = 0 .
and vary α = 0 .
Figure 5 plots the trajectories.
As we can see, when γ is properlychosen, with different α , PNG ﬁnds the correct models with different trajectories.
Different α determines thealgorithm’s behavior of balancing the descent of task losses or criterion objectives.
On the other hand, withtoo large γ , the algorithm fails to ﬁnd a model that is close to P ∗ , which is expected.
C. I MPROVING M ULTITASK B ASED D OMAIN G ENERALIZATION
We argue that many other deep learning problems also have the structure of multitask learning when multiplelosses presents and thus optimization techniques in multitask learning can also be applied to those domains.
In this paper we consider the JiGen (Carlucci et al., 2019b).
JiGen learns a model that can be generalized tounseen domain by minimizing a standard cross-entropy loss (cid:96) class for classiﬁcation and an unsupervised loss(cid:96) jig based on Jigsaw Puzzles:
[Equation]
The ratio between two losses, i.e. ω , is important to the ﬁnal performance of the model and requires acareful grid search.
Notice that JiGen is essentially searching for a model on the Pareto front using the linearscalarization.
Instead of using a ﬁxed linear scalarization to learn a model, one natural questions is that
l 1 0 .0 .0 .
.ltask preference
Pareto Front PNG 0 .0 .0 .0 .0 .1 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference 0 .0 .0 .
l 1 0 .0 .0 .
.ltask preference whether it is possible to design a mechanism that dynamically adjusts the ratio of the losses so that we canachieve to learn a better model.
We give a case study here.
Motivated by the adversarial feature learning (Ganin et al., 2016), we proposeto improve JiGen such that the latent feature representations of the two tasks are well aligned.
Speciﬁcally,suppose that Φ class ( θ ) = { φ class ( x i , θ ) } ni =1 and Φ jig ( θ ) = { φ jig x i , θ ) } ni =1 is the distribution of latent featurerepresentation of the two tasks, where x i is the i -th training data.
We consider F PD as some probability metricthat measures the distance between two distributions, we consider the following problem:
[Equation]
With PD as the criterion function, our algorithm automatically reweights the ratio of the two tasks such thattheir latent space is well aligned.
Setup We ﬁx all the experiment setting the same as Carlucci et al.
(2019b).
We use the Alexnet and Resnet-18with multihead pretrained on ImageNet as the multitask network.
We evaluate the methods on PACS (Li et al.,2017), which covers 7 object categories and 4 domains (Photo, Art Paintings, Cartoon and Sketches).
Same to
Carlucci et al.
(2019b), we trained our model considering three domains as source datasets and the remainingone as target.
We implement F PD that measures the discrepancy of the feature space of the two tasks usingthe idea of Domain Adversarial Neural Networks (Ganin & Lempitsky, 2015) by adding an extra predictionhead on the shared feature space to predict the whether the input is for the classiﬁcation task or Jigsaw task.
Speciﬁcally, we add an extra linear layer on the shared latent feature representations that is trained to predictthe task that the latent space belongs to, i.e., [Equation] Notice that the optimal weight and bias for the linear layer depends on the model parameter θ , during thetraining, both w, b and θ are jointly updated using stochastic gradient descent. We follow the default trainingprotocol provided by the source code of Carlucci et al.
(2019b). Baselines Our main baselines are JiGen (Carlucci et al., 2019b); JiGen + adv, which adds an extra domainadversarial loss on JiGen; and our PNG with domain adversarial loss as criterion function.
In order to runstatistical test for comparing the methods, we run all the main baselines using 3 random trials.
We use thereleased source code by Carlucci et al.
(2019b) to obtained the performance of JiGen.
For JiGen+adv, we usean extra run to tune the weight for the domain adversarial loss.
Besides the main baselines, we also includes
TF (Li et al., 2017), CIDDG (Li et al., 2018b), MLDG (Li et al., 2018a) , D-SAM (D’Innocente & Caputo,2018) and DeepAll (Carlucci et al., 2019b) as baselines with the author reported performance for reference.
Result
The result is summarized in Table 5 with bolded value indicating the statistical signiﬁcant best methodswith p-value based on matched-pair t-test less than 0.1.
Combining Jigen and PNG to dynamically reweightthe task weights is able to implicitly regularizes the latent space without adding an actual regularizer whichmight hurt the performance on the tasks and thus improves the overall result.
[Table]