Title: Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspective
Venue: NeurIPS 2023
Accepted: yes
Keywords: multi-task learning

scalarization

Pareto front
Abstract: Linear scalarization, i.e., combining all loss functions by a weighted sum, has been the default choice in the literature of multi-task learning (MTL) since its inception. In recent years, there is a surge of interest in developing Specialized Multi-Task Optimizers (SMTOs) that treat MTL as a multi-objective optimization problem. However, it remains open whether there is a fundamental advantage of SMTOs over scalarization. In fact, heated debates exist in the community comparing these two types of algorithms, mostly from an empirical perspective. To approach the above question, in this paper, we revisit scalarization from a theoretical perspective. We focus on linear MTL models and study whether scalarization is capable of fully exploring the Pareto front. Our findings reveal that, in contrast to recent works that claimed empirical advantages of scalarization, scalarization is inherently incapable of full exploration, especially for those Pareto optimal solutions that strike the balanced trade-offs between multiple tasks. More concretely, when the model is under-parametrized, we reveal a multi-surface structure of the feasible region and identify necessary and sufficient conditions for full exploration. This leads to the conclusion that scalarization is in general incapable of tracing out the Pareto front. Our theoretical results partially answer the open questions in Xin et al. (2021), and provide a more intuitive explanation on why scalarization fails beyond non-convexity. We additionally perform experiments on a real-world dataset using both scalarization and state-of-the-art SMTOs. The experimental results not only corroborate our theoretical findings, but also unveil the potential of SMTOs in finding balanced solutions, which cannot be achieved by scalarization.

Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspective

Abstract

Linear scalarization, i.e., combining all loss functions by a weighted sum, has been the default choice in the literature of multi-task learning (MTL) since its inception. In recent years, there is a surge of interest in developing Specialized Multi-Task Optimizers (SMTOs) that treat MTL as a multi-objective optimization problem. However, it remains open whether there is a fundamental advantage of SMTOs over scalarization. In fact, heated debates exist in the community comparing these two types of algorithms, mostly from an empirical perspective. To approach the above question, in this paper, we revisit scalarization from a theoretical perspective. We focus on linear MTL models and study whether scalarization is capable of fully exploring the Pareto front. Our findings reveal that, in contrast to recent works that claimed empirical advantages of scalarization, scalarization is inherently incapable of full exploration, especially for those Pareto optimal solutions that strike the balanced trade-offs between multiple tasks. More concretely, when the model is under-parametrized, we reveal a multi-surface structure of the feasible region and identify necessary and sufficient conditions for full exploration. This leads to the conclusion that scalarization is in general incapable of tracing out the Pareto front. Our theoretical results partially answer the open questions in Xin et al. (2021), and provide a more intuitive explanation on why scalarization fails beyond non-convexity. We additionally perform experiments on a real-world dataset using both scalarization and state-of-the-art SMTOs. The experimental results not only corroborate our theoretical findings, but also unveil the potential of SMTOs in finding balanced solutions, which cannot be achieved by scalarization.

1 Introduction

When labeled data from multiple tasks are available, multi-task learning (MTL) aims to learn a joint model for these tasks simultaneously in order to improve generalization (Caruana, 1997; Zhang and Yang, 2021; Ruder, 2017). One common approach towards this goal is learning a joint feature representation that is shared among multiple tasks, where each task then uses a separate task-specific head for its own prediction. The hope is that by sharing a joint feature representation, the effective sample size for feature learning will be larger than that for single-task learning, hence leading to better generalization across related tasks. This simple and natural idea has proven to be successful in many applications, including object segmentation (Misra et al., 2016), natural language processing (Collobert and Weston, 2008), autonomous driving (Yu et al., 2020a), to name a few.

One typical approach to learning the shared feature model from multiple tasks is called linear scalarization , where the learner provides a set of convex coefficients (in the form of a probability vector) to

1

†

linearly combine the loss from each task and reduces the learning problem into a scalar optimization problem. Since its inception, scalarization has been the default approach for MTL (Caruana, 1997; Ruder, 2017; Zhang and Yang, 2018; Crawshaw, 2020; Zhang and Yang, 2021), mainly due to its simplicity, scalability, as well as empirical success. However, in recent years, variants of the multiple gradient descent algorithm (MGDA) (Fliege and Svaiter, 2000) have been proposed and used in MTL (Sener and Koltun, 2018; Yu et al., 2020b; Chen et al., 2020; Liu et al., 2021a,b; Zhou et al., 2022). Referred to as specialized multi-task optimizers (SMTOs), the main idea behind them is to cast the learning problem explicitly as a multi-objective (multi-criteria) optimization problem, wherein the goal is to enforce convergence to a Pareto optimal solution across tasks. In the literature, a main driving force behind these studies has been the belief that they are better than linear scalarization when conflicting gradients among tasks are present (Yu et al., 2020b; Liu et al., 2021a). Nevertheless, recent empirical results tell an opposite story (Kurin et al., 2022; Xin et al., 2022; Lin et al., 2022)with proper choices of hyperparameters and regularization techniques, scalarization matches or even surpasses SMTOs. Hence, it largely remains an open problem regarding whether there is an inherent advantage of using SMTOs for MTL over scalarization, at least from a theoretical perspective.

In this paper, we revisit scalarization from a theoretical perspective. We study whether scalarization is capable of fully exploring the Pareto front in linear MTL. Technically, we aim to understand the following question on full exploration :

For every Pareto optimum v on the Pareto front induced by multiple tasks, does there exist a weighted combination of task losses such that the optimal solution of the linearly scalarized objective corresponds to v ?

Our fundamental thesis is that, if the answer to the above question is affirmative, then there is no inherent advantage of SMTOs over scalarization on the representation level; otherwise, there might be cases where SMTOs can converge to Pareto optimal solutions that cannot be obtained via any scalarization objective. Our findings suggest that, in contrast to recent works that claimed empirical advantages of scalarization, it is inherently incapable of full exploration. Specifically, our contributions are as follows:

We reveal a multi-surface structure of the feasible region, and identify a common failure mode of scalarization, which we refer to as 'gradient disagreement' (Section 3.1). This serves as an important first step towards understanding when and why scalarization fails to achieve a specific Pareto optimum.

We provide necessary and sufficient conditions for scalarization to fully explore the Pareto front, which lead to the conclusion that scalarization is incapable of tracing out the Pareto front in general (Section 3.2). To the best of our knowledge, we are the first to establish both necessary and sufficient conditions for full exploration in the non-convex setting.

We proceed to discuss implications of our theoretical results (Section 4.1), demonstrating how they answer the open questions in Xin et al. (2022), and provide a remedy solution via randomization to counteract the fundamental limitation of scalarization (Section 4.2).

We additionally perform experiments on a real-world dataset using both scalarization and state-ofthe-art SMTOs (Section 5). The experimental results not only corroborate our theoretical findings, but also demonstrate the potential of SMTOs in finding balanced solutions.

2 Preliminaries

We introduce the setup of multi-task learning (MTL) and relevant notations, along with essential background on the optimal value of the scalarization objective and the concept of Pareto optimality.

Problem setup. Following prior works, we consider multi-task learning with k regression tasks (Wu et al., 2020; Du et al., 2021). The training examples { ( x j , y j 1 , · · · , y j k ) } n j =1 are drawn i.i.d. from some µ ∈ P ( X ×Y 1 ×···×Y k ) , where X ⊆ R p is the shared input space and Y i ⊆ R is the output space for task i . We denote the label of task i as y i = ( y 1 i , · · · , y n i ) ⊤ ∈ R n , and concatenate the inputs and labels in matrix forms: X = [ x 1 , · · · , x n ] ⊤ ∈ R n × p and Y = [ y 1 , · · · , y k ] ∈ R n × k .

We focus on linear MTL, in which a linear neural network serves as the backbone for prediction. Despite its simplicity, this setting has attracted extensive research interest in the literature (Maurer, 2006; Maurer et al., 2016; Wu et al., 2020; Du et al., 2021; Tripuraneni et al., 2021). Without loss of

generality, the model that will be used for our study is a two-layer linear network 3 , which consists of a shared layer W ∈ R p × q and a task-specific head a i ∈ R q for each task. The prediction for task i on input x is given by f i ( x, W, a i ) := x ⊤ Wa i . We adopt Mean Squared Error (MSE) as the loss function, and the training loss for task i is given by L i ( W,a i ) := ∥ XWa i -y i ∥ 2 2 .

Notation. We use [ k ] to denote the index set { 1 , · · · , k } . For two square matrices M and N with the same dimension, we write M ⪰ N if M -N is positive semi-definite, and M ≻ N if M -N is positive definite. We denote diag( D ) as the diagonal matrix whose diagonal entries are given by D = { d i } i ∈ [ k ] , a set of real numbers; and span( V ) as the linear space spanned by V = { v i } i ∈ [ k ] , a set of vectors. For two vectors with the same size, we use ⊙ and ⊘ to denote entry-wise multiplication and division. When there is no ambiguity, operations on vectors (e.g., absolute value, square, square root) are performed entry-wise. For any matrix M , M † stands for its Moore-Penrose inverse. We use range( M ) and rank( M ) to denote its column space and rank, respectively. The expression ∥ · ∥ denotes the ℓ 2 norm of a vector, ⟨· , ·⟩ the inner product of two vectors, ∥ · ∥ F the Frobenius norm of a matrix, and ρ ( · ) the spectral radius of a square matrix.

Optimal value of scalarization (restated from Wu et al. (2020)). The objective function of scalarization is a weighted sum of losses: ∑ k i =1 λ i L i ( W,a i ) . Here, the convex coefficients { λ i } i ∈ [ k ] satisfy λ i ≥ 0 and ∑ i λ i = 1 , and are typically obtained via grid search (Vandenhende et al., 2021). For each task, we can decompose its loss based on the orthogonality principle (Kay, 1993)

where ˆ y i = X ( X ⊤ X ) † X ⊤ y i is the optimal predictor for task i . Denote Λ = diag ( { √ λ i } k i =1 ) , A = [ a 1 , · · · , a k ] ∈ R q × k , ˆ Y = [ˆ y 1 , · · · , ˆ y k ] ∈ R n × k . The scalarization objective can be compactly expressed as

Throughout this paper, we assume the optimal linear predictions { ˆ y i } i ∈ [ k ] are linearly independent. 4 This leads to two distinct cases, depending on the width of the hidden layer q :

When q ≥ k , the minimum value of the first term is zero, i.e., every task can simultaneously achieve its optimal loss ∥ ˆ y i -y i ∥ 2 .

When q < k , the minimum value of the first term is given by the rankq approximation. In particular, let ˆ Y Λ = ∑ k i =1 σ i u i v ⊤ i be the SVD of ˆ Y Λ . According to the Eckart-Young-Mirsky Theorem (Eckart and Young, 1936), the best rankq approximation of ˆ Y Λ under the Frobenius norm is given by the truncated SVD: ˆ Y q, Λ Λ := ∑ q i =1 σ i u i v ⊤ i . Hence, the minimum training risk is ∥ ( ˆ Y q, Λ -ˆ Y )Λ ∥ 2 F + ∥ ( ˆ Y -Y )Λ ∥ 2 F . This regime, referred to as the under-parameterized regime, constitutes the main focus of our study.

Pareto optimality. In the context of MTL, a point θ ∗ (shared model parameters) is a Pareto optimum (PO) if there does not exist a θ ′ that achieves strictly better performance on at least one task and no worse performance on all other tasks compared to θ ∗ . Formally, for a set of objectives { L i } i ∈ [ k ] that need to be minimized, if L i ( θ ′ ) < L i ( θ ∗ ) for some point θ ′ , then there must exist another task j such that L j ( θ ′ ) > L j ( θ ∗ ) . The Pareto front (a.k.a. Pareto frontier) is the collection of objective values ( L 1 ( θ ) , · · · , L k ( θ )) corresponding to all PO.

3 Linear scalarization through the lens of exploring the Pareto front

In this section, we provide a fine-grained analysis of the feasible region (formally defined in Section 3.1), revealing its multi-surface structure. We proceed to analyze whether scalarization is capable of tracing out the Pareto front, and identify necessary and sufficient conditions for full exploration.

Reformulation. Recall that minimizing Equation (1) is equivalent to minimizing the squared loss ∥ XWa i -ˆ y i ∥ 2 , where ˆ y i = X ( X ⊤ X ) † X ⊤ y i is the optimal linear predictor for task i . Additionally, the task-specific heads can be optimized independently for each task. Thus, if we let Z = XW and

Feasible MSEs restricted

to each surface

union

All feasible MSEs

1

0

0

1

0

1

MSE 1

MSE 3

MSE 2

E

E

E

intersection (balanced PO)

E

1

2

3

∅

MSE 3

MSE 1

0

1

0

1

Gradient disagreement

g

1

g

3

a i = ( Z ⊤ Z ) † Z ⊤ ˆ y i (the optimal task-specific head), then minimizing Equation (1) is equivalent to

which can be further transformed into

Therefore, the original MTL problem can be reformulated into the following multi-objective problem

where P Z is a projection matrix with rank at most q (i.e., hidden layer width) and maps to the column space of X . Each quadratic term can be further expressed as ∥ P Z ˆ y i ∥ 2 . From here it is clear (also see Section 2) that when q ≥ k (the over-parametrized regime), the k quadratic terms can be maximized simultaneously, meaning that the network has sufficient capacity to fit the target tasks. In this case, the Pareto front reduces to a singleton , and can be achieved by scalarization with any choice of convex coefficients. In fact, this holds true even for general non-linear networks as long as it is wide enough, i.e., over-parametrized, and we defer more discussions to Appendix A.2.

The main focus of our study, in contrast, lies in the under-parametrized regime q < k , where various tasks compete for the network's capacity for better prediction. We comment that this regime has been the focus of previous work and demonstrated to be beneficial in MTL (Wu et al., 2020; Wang et al., 2022). We study two extreme cases within the under-parameterized regime, q = 1 and q = k -1 . By doing so, we anticipate that our results will hold for other k 's in between.

3.1 The multi-surface structure of the feasible region

The objective function of task i is ∥ P Z ˆ y i ∥ 2 , i.e., the square of the projection of ˆ y i on P Z . Since we are interested in the Pareto front, we can assume w.l.o.g. that rank( P Z ) = q and range( P Z ) ⊂ span( { ˆ y i } i ∈ [ k ] ) (otherwise there is a loss of model capacity, and the corresponding point will no longer be a PO). Consequently, we define the feasible region as

We now provide a fine-grained characterization of F 1 (corresponding to q = 1 ) in the following theorem, which also serves as the basis for q = k -1 .

where D i 1 ··· i l is a diagonal matrix with all diagonal entries equal to 1 , except for the entries at positions i 1 , · · · , i l which are equal to -1 . Consider the following surface parameterized by v :

We have:

We have:

i.e., F 1 is the union of 2 k -1 surfaces in the non-negative orthant (note E i 1 ··· i l = E i 1 ··· i l by symmetry).

To help the readers gain some intuitions on the multi-surface structure demonstrated in Theorem 3.1, we visualize the feasible region of a three-task (i.e., k = 3 ) example in Figure 1 (more details at the end of the subsection). We also provide a proof sketch below (detailed proof in Appendix A.3).

Proof sketch of Theorem 3.1. A rank1 projection matrix can be expressed as ss ⊤ with ∥ s ∥ = 1 . The feasible region can therefore be equivalently characterized as

Define v i = ⟨ ˆ y i , s ⟩ and v = ( v 1 , · · · , v k ) ⊤ . We will show the set of v forms the boundary of an ellipsoid determined by Q . As a consequence, the set of | v | is the union of multiple surfaces, each corresponding to the boundary of an ellipsoid (reflection of Q ) in the non-negative orthant. The reflection is represented by the diagonal matrix D i 1 ··· i l in the theorem statement. Finally, F 1 , which is the set of v 2 , can be obtained by squaring the coordinates in the previous set. This finishes the proof of Theorem 3.1.

Since the orthogonal complement of a 1 -dimensional subspace of span( { ˆ y i } i ∈ [ k ] ) is a ( k -1) -dimensional subspace, we immediately have the following result (proof deferred to Appendix A.4).

Theorem 3.2. Let t := ( ∥ ˆ y 1 ∥ 2 , · · · , ∥ ˆ y k ∥ 2 ) ⊤ . Consider the following surface parametrized by v :

i.e., F k -1 is the union of 2 k -1 surfaces in the non-negative orthant ( I i 1 ··· i l = I i 1 ··· i l by symmetry).

Theorems 3.1 and 3.2 not only offer us a glimpse into the shape of the feasible region but also provide valuable insights into the fundamental limitation of linear scalarization. The key observation is that, scalarization fails to reach the intersection points of two (or more) surfaces when the gradients with respect to the two (or more) surfaces disagree (i.e., they lie in different directions). We refer to this phenomenon as gradient disagreement , and provide an illustrating example in Figure 2. Essentially, the multi-surface structure that we have revealed implies the abundance of intersection points on the feasible region F q . Moreover, if any of these points were to be an PO, scalarization would not be able to trace out the Pareto front (see Appendix A.5 for a detailed explanation).

We illustrate this observation using a concrete example with three tasks in Figure 1. The feasible region consists of four surfaces, among which E ∅ is dominated by other surfaces. The intersection of E 1 , E 2 , E 3 (the red point) is a PO that achieves balanced performance across tasks, yet scalarization cannot reach this point since it lies in a valley.

3.2 Necessary and sufficient conditions for full exploration

In this section, we provide an in-depth analysis of the Pareto front, which is a subset of the feasible region. Concretely, we develop necessary and sufficient conditions for scalarization to fully explore the Pareto front.

We first review some important concepts that will be frequently used in this section. Denote Q := { Q i 1 ··· i l | 0 ≤ l ≤ k, 1 ≤ i 1 < · · · < i l ≤ k } as the collection of matrices which define the surfaces in the previous section. Denote G := { G i 1 ··· i l = Q -1 i 1 ··· i l | 0 ≤ l ≤ k, 1 ≤ i 1 < · · · < i l ≤ k } . Note G i 1 ··· i l = D i 1 ··· i l ˆ Y ⊤ ˆ Y D i 1 ··· i l = ( ˆ Y D i 1 ··· i l ) ⊤ ( ˆ Y D i 1 ··· i l ) . Therefore, G is a collection of gram matrices which measure the task similarity , and each element is obtained by flipping the signs of some optimal predictors. We use E and I to denote the collection of surfaces defined in Equations (7) and (10), respectively. We also formally define a class of matrix as follows.

Definition 3.3 (Doubly non-negative matrix) . A doubly non-negative matrix M is a real positive semi-definite (PSD) matrix with non-negative entries.

Our analysis begins with an observation that the distribution of PO across different surfaces plays a vital role in the success of scalarization. Specifically, we want to avoid the 'bad' case like the one demonstrated in Figure 1, where a PO lies on the intersection of multiple surfaces. A natural solution to circumvent this issue is to force the Pareto front to lie on a single surface. We therefore draw our attention to this particular scenario, and provide a necessary and sufficient condition for such configuration in the following lemma.

Lemma 3.4. When q = 1 , the Pareto front of the feasible region belongs to a single surface E ∗ ∈ E , if and only if the corresponding G ∗ = ( Q ∗ ) -1 ∈ G is doubly non-negative. We refer to the existence of such a doubly non-negative G ∗ in G as C1 .

We outline the proof sketch below. The detailed proof can be found in Appendix B.1.

Proof sketch of Lemma 3.4. For the if part, we employ a geometric argument. Specifically, denote the acute cone formed by the optimal predictors as A . We will show that, if s (a unit vector in Equation (9)) does not belong to the dual cone A ∗ , then its reflection w.r.t. the dual cone s ′ is a better point than s . As a consequence, we can remove the absolute value from the proof sketch of Theorem 3.1. This further implies that the Pareto front belongs to a single surface.

For the only if part, we consider the points which achieve maximum value along an axis. We will show these points are PO. Finally, we will show that if C1 does not hold, then these points must belong to different surfaces in E .

We have a similar result for q = k -1 . The details of the proof are deferred to Appendix B.2.

Lemma 3.5. When q = k -1 , the Pareto front of the feasible region belongs to a single surface I ∗ ∈ I , if and only if the corresponding Q ∗ ∈ Q is doubly non-negative. We refer to the existence of such a doubly non-negative Q ∗ in Q as C2 .

Lemmas 3.4 and 3.5 provide a refined characterization of how the PO is distributed within the feasible region: unless C1 (resp. C2 ) holds, the Pareto front will belong to multiple surfaces in E (resp. I ). This easily leads to the existence of a PO on the intersection of different surfaces (see Figure 1), which will render the failure of scalarization as a consequence. It is therefore natural for us to conjecture that C1 (resp. C2 ) is necessary for full exploration. We make the following assumption regarding the topology of the Pareto front.

Assumption 3.6. For q = 1 and q = k -1 , we assume the Pareto front is path-connected, i.e., for every pair of PO x and y , there exists a continuous curve on the Pareto front with x and y being the two endpoints.

Assumption 3.6, combined with Lemma 3.4 (resp. Lemma 3.5), will help locate a PO P on the intersection of two surfaces when C1 (resp. C2 ) does not hold. We make an additional technical assumption to circumvent degenerated cases.

Assumption 3.7. We assume P (the intersection point) is a relative interior point (Rockafellar, 1997, Section 6) of the Pareto front.

We are now ready to present the main results in this section (full proofs in Appendices B.3 and B.5).

Theorem 3.8 (Necessity of C1 (resp. C2 )) . For q = 1 (resp. q = k -1 ), under Assumptions 3.6 and 3.7, C1 (resp. C2 ) is a necessary condition for scalarization to fully explore the Pareto front.

Proof sketch of Theorem 3.8. Assumption 3.7 allows us to perform dimensionality analysis in a small neighborhood of the intersection point P , from which we can demonstrate the existence of a PO lying on the intersection of two surfaces with gradient disagreement. This point cannot be achieved by linear scalarization.

Remark 3.9. Assumptions 3.6 and 3.7 are mainly used to simplify analysis, and they are not necessary for Theorem 3.8 to hold. As a matter of fact, we can show the necessity of C1 (resp. C2 ) when k = 3 without these assumptions. We refer the readers to Appendix B.4 for more details.

Finally, without relying on the above assumptions, we can show that C1 and C2 are sufficient.

Theorem 3.10 (Sufficiency of C1 (resp. C2 )) . For q = 1 (resp. q = k -1 ), C1 (resp. C2 ) is a sufficient condition for scalarization to fully explore the Pareto front.

Proof sketch of Theorem 3.10. For q = 1 , we consider the tangent plane at a PO and attempt to show it lies above the feasible region. This is transformed into demonstrating the positive semidefiniteness of a matrix, which can be conquered by applying the Perron-Frobenius theorem (ShakedMonderer and Berman, 2021, Section 1.3) on non-negative matrices. For q = k -1 , we show the boundary of the feasible region is concave, and apply the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) to conclude the proof.

4 Discussion

We discuss the implications of our theoretical results, and provide a remedy solution for linear scalarization to address its inability of full exploration.

4.1 Implications of the theoretical results

Fundamental limitation of linear scalarization. From Theorem 3.8, it is clear that linear scalarization faces significant challenges in fully exploring the Pareto front in the case of q = 1 and q = k -1 , since both C1 and C2 require the existence of a doubly non-negative matrix, which is a very small subset of the PSD matrix. To illustrate this point, we consider a simplified probabilistic model: when the optimal predictors { ˆ y i } i ∈ [ k ] are drawn independently from a symmetric distribution centered at the origin, the probability that C1 holds is 2 -O ( k 2 ) , which decays exponentially with the number of tasks. Thus, we comment that C1 and C2 in general does not hold in practical cases.

Towards understanding when and why scalarization fails. Among the vast literature that studies Pareto optimality in MTL, there is a noticeable absence of a systematic and rigorous exploration of when and why scalarization fails to reach a Pareto optimum . Prior work typically resorts to hypothetical examples (e.g., Figure 4.9 in (Boyd et al., 2004)) to demonstrate the failure of scalarization. Our study, which identifies the multi-surface structure of the feasible region and reveals the phenomenon of gradient disagreement in a concrete setting, serves as a pioneering step in addressing this gap in the literature.

A closer examination of the failure mode. For the extreme under-parametrized case ( q = 1 ), on the one hand, we can prove that scalarization is able to achieve points which attain maximum value along one direction (i.e., best performance on one task). On the other hand, by examining Figure 1, we shall see that scalarization is incapable of exploring the interior points (i.e., balanced performance across all tasks). This suggests that scalarization has the tendency to overfit to a small fraction of tasks, and fails to strike the right balance between different tasks. We provide more empirical evidence to support this argument in Section 5.

Criteria for practitioners. Theorem 3.10 demonstrates that C1 and C2 are sufficient for scalarization to fully explore the Pareto front. As such, these conditions can serve as valuable criteria for practitioners when deciding which tasks to combine in linear MTL. Note a naive enumeration of G or Q will lead to a time complexity of O (2 k ) . We instead provide an O ( k 2 ) algorithm checking whether C1 holds (Algorithm 1 in Appendix C). In essence, the algorithm aims to find a set of flipping signs { s i } i ∈ [ k ] ∈ { + , -} that ensure positive or zero correlation between all pairs in { s i ˆ y i } i ∈ [ k ] . It scans through the predictors in ascending order and either determines the signs s j for each optimal predictor ˆ y j if not determined yet, or checks for conflicts between signs that are already determined. A sign can be determined when the predictors ˆ y i and ˆ y j are correlated. A conflict indicates the impossibility of finding such a set of flipping signs, thus falsifying C1 . A similar algorithm can be devised for C2 with an additional step of Cholesky decomposition beforehand, leading to an O ( k 3 ) time complexity.

Open questions in Xin et al. (2022). We (partially) answer the open questions in Xin et al. (2022).

Q1 . 'Is it the case that neural networks trained via a combination of scalarization and standard first-order optimization methods are not able to reach the Pareto Frontier?'

A1 . For Linear MTL, our answer is a definitive 'Yes'. We demonstrate that linear scalarization with arbitrary convex coefficients cannot trace out all the Pareto optima in general. Our results are strong in the sense that they hold on a representation level, meaning that they are independent of the specific

optimization algorithm employed. This suggests that the weakness we have revealed in scalarization is fundamental in nature.

Q2 . 'Is non-convexity adding additional complexity which makes scalarization insufficient for tracing out the Pareto front?'

A2 . Yes, even some mild non-convexity introduced by two-layer linear networks will render scalarization insufficient. On the other hand, we do believe the multi-surface structure that we have unveiled in Section 3.1 is both universal and fundamental, which also contributes to the observed non-convexity in practical scenarios. In essence, what we have done partly is to penetrate through the surface and gain a holistic view of the feasible region. Notably, the key obstacle in linear scalarization, as revealed by our analysis, lies in the phenomenon of gradient disagreement . This cannot be readily discerned by solely examining the surface.

We make two additional comments to further clarify our standing point in comparison to this work.

We do not aim to refute the claims in Xin et al. (2022); as a matter of fact, we consider their empirical findings highly valuable. Instead, we would like to emphasize a discrepancy between their theory and experiments. Given that we have shown the conclusion-that linear scalarization can fully explore the PF-no long holds in the non-convex setting, their theory does not account for their experiments on modern neural networks. Therefore, we urge the research community to develop a new theory elucidating the empirical efficacy of linear scalarization.

While the differing settings render the results from these two papers non-comparable, they collectively offer a broader perspective on the pros and cons of scalarization and SMTOs. We believe this is of great significance to both researchers and practitioners.

4.2 A remedy solution

Given the fundamental weakness of linear scalarization in fully exploring the Pareto front, one might wonder whether there exists any model class where linear scalarization is sufficient to fully explore the Pareto front. The answer is affirmative if we allow ourselves to use randomized multi-task neural networks . More specifically, given two MTL networks f (0) and f (1) over the same input and output spaces and a fixed constant t ∈ [0 , 1] , we can construct a randomized MTL network as follows. Let S ∼ U (0 , 1) be a uniform random variable over (0 , 1) that is independent of all the other random variables, such as the inputs. Then, consider the following randomized neural network:

Essentially, the randomized network constructed by Equation (12) will output f (0) ( x ) with probability t and f (1) ( x ) with probability 1 -t . Now if we consider the loss of f ( · ) , we have

which means that the loss of the randomized multi-task network is a convex combination of the losses from f (0) ( · ) and f (1) ( · ) . Geometrically, the argument above shows that randomization allows us to 'convexify' the feasible region and thus the Pareto front will be convex. Now, by a standard application of the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) we can see that every point on the Pareto front can be realized via linear scalarization. We empirically verify the effectiveness of randomization in Appendix D.3.

We comment that randomization is a powerful tool that has wide applicability in machine learning. For instance, it has achieved great success in online learning, specifically in the Follow-the-PerturbedLeader algorithm (Kalai and Vempala, 2005). It also appears in the literature of fairness to achieve the optimal regressor or classifier (Zhao and Gordon, 2022; Xian et al., 2023). Here we further demonstrate that it can be applied to MTL to facilitate the exploration of linear scalarization. In essence, randomization increases the representation power of the original neural network.

5 Experiments

In this section, we corroborate our theoretical findings with a linear MTL experiment, where we show that linear scalarization fails to fully explore the Pareto front on a three-task problem that does not satisfy C1 with q = 1 . In comparison, SMTOs can achieve balanced Pareto optimum solutions that are not achievable by scalarization.

Dataset and setup. We use the SARCOS dataset for our experiment (Vijayakumar and Schaal, 2000), where the problem is to predict the torque of seven robot arms given inputs that consist of the position, velocity, and acceleration of the respective arms. For ease of visualization, we restrict the tasks to arms 3, 4, and 5; in particular, they do not satisfy C1 .

Our regression model is a two-layer linear network with hidden size q = 1 (no bias). To explore the portion of the Pareto front achievable by linear scalarization, we fit 100,000 linear regressors with randomly sampled convex coefficients and record their performance. These are compared to the solutions found by SMTOs, for which we use MGDA (Désidéri, 2012) and MGDA-UB (Sener and Koltun, 2018). Specifically, to guarantee their Pareto optimality, we run the SMTOs with fullbatch gradient descent till convergence (hence each SMTO method has one deterministic solution). Additional details, including hyperparameter settings, can be found in Appendix D.1.

Results. In Figure 3, we plot the MSEs achieved by scalarization under randomly sampled weights. We observe that the solutions achieved by scalarization are concentrated near the vertices and partly along the edges, indicating their tendency to overfit to 12 of the tasks. Scalarization also fails to explore a large portion of the Pareto front in the center-which have more balanced performance across task; the inability of full exploration corroborates the necessary condition (Theorem 3.8) discussed in Section 3. In contrast, SMTOs are capable of finding solutions not achievable by scalarization that are located in the center of the Pareto front. This showcases the relative advantage and practical utility of SMTOs in MTL over scalarization, especially in scenarios where equitable performance on all tasks are desired. To strengthen our results, we experiment with multiple random initializations and observe consistent results (see Appendix D.2).

MSE1

1.0

0.8

0.6

0.4

0.2

0.0

MSE2

1.0

0.8

0.6

0.4

0.2

0.0

MSE3

1.0

0.8

0.6

0.4

0.2

0.0

(.55, .81, .69)

(.57, .60, .77)

MGDA

MGDA-UB

Scalarization

Discussion. We would like to note that we are not endorsing any specific SMTO algorithm, such as MGDAor MGDA-UB. As a matter of fact, we hypothesize that no single algorithm is universally effective (also see (Chen et al., 2023)), notably in overcoming the limitation of scalarization in terms of full exploration. Instead, by revealing the limitation of scalarization and the potential benefits of some SMTO algorithms, we aim to strengthen SMTO research, challenge the notion that linear scalarization is sufficient, and advocate for a more balanced progression in the field of MTL.

6 Limitations and future directions

Limitations. There are two major limitations of our work. First, while the techniques we develop in this paper are highly non-trivial, we only cover the two extreme cases q = 1 and q = k -1 in the under-parametrized regime, and the analysis for general q < k is missing. Second, the setting of our study is relatively restricted. Specifically, we focus on linear MTL where different tasks share the same input. A more realistic setting should factor in different inputs for different tasks (Wu et al., 2020), as well as some non-linearity in the model architecture (Kurin et al., 2022).

Future directions. We expect the geometry of the feasible region to transition smoothly as we increase q , meaning that it will not abruptly collapse into a single surface. Providing a rigorous basis for this assertion would facilitate the analysis for general q 's, thereby complementing our study. Regarding the assumption of shared input, while our analysis persists under the 'proportional covariance assumption' adopted by Wu et al. (2020), we find this assumption excessively restrictive,

and hope future work could relax it further. The generalization to neural networks with ReLU activation is interesting but also challenging, and it is unclear whether the techniques and results developed in this paper can be extended to the non-linear setting. Another interesting avenue to explore is interpreting the multi-surface structure we have uncovered, which we hypothesize to have connections with game theory. Finally, our paper dedicates to demonstrating the weaknesses of linear scalarization, while we consider it an important future direction to theoretically analyze the advantage of SMTOs, as partially revealed in our experiments. In all, we hope our study can initiate a line of works on MTL, which provide theoretical justifications for the usage of specific algorithms beyond empirical evaluations across various settings.

7 Related work

SMTOs. There is a surge of interest in developing specialized multi-task optimizers (SMTOs) in the literature. One of the earliest and most notable SMTOs is the multiple-gradient descent algorithm (MGDA) (Désidéri, 2012). For each task, MGDA solves an optimization problem and performs updates using the gradient of the objective function with both shared and task-specific parameters. However, MGDA cannot scale to high dimension (e.g., neural networks), and computing the gradients for every task is prohibitively expensive. To counter these drawbacks, Sener and Koltun (2018) introduce a scalable Frank-Wolfe-based optimizer which is more efficient. Another prominent issue in MTL is that the gradients from multiple tasks could conflict with each other and the performance of some tasks could be significantly affected. To this end, Yu et al. (2020b) propose gradient surgery to mitigate the negative interference among tasks; Chen et al. (2020) and Liu et al. (2021a) propose to manipulate the gradients or the gradient selection procedures; Fernando et al. (2022) devise a stochastic gradient correction method and prove its convergence. More recently, Navon et al. (2022) cast the gradient update as a Nash bargaining game, yielding well-balanced solutions across the PF.

Empirical comparison of SMTOs and scalarization. Arecent line of works empirically compare SMTOs and linear scalarization. Their central claim is that the performance of linear scalarization is comparable with the state-of-the-art SMTOs despite the added complexity of SMTOs. Xin et al. (2022) argue that multi-task optimization methods do not yield any improvement beyond what can be achieved by linear scalarization with carefully-tuned hyperparameters; Kurin et al. (2022) show that the performance of linear scalarization matches or even surpasses the more complex SMTOs when combined with standard regularization techniques. In addition to classification tasks, Vandenhende et al. (2021) conduct extensive experiments on dense prediction tasks, i.e., tasks that produce pixellevel predictions. They conclude that avoiding gradient competition among tasks can actually lead to performance degradation, and linear scalarization with fixed weights outperforms some SMTOs.

Exploration of the Pareto front. It is well-known that minimizing scalarization objectives with positive coefficients yields Pareto optimal solutions (Boyd et al., 2004), while using non-negative coefficients yields weak Pareto optimal ones (Zadeh, 1963). On the other hand, exploration of the entire Pareto front has been a long-standing and difficult open problem in MTL. For the simplest case, if all loss functions are convex or the achievable region is a convex set, then scalarization can fully explore the PF (Boyd et al., 2004; Xin et al., 2022). A slightly weaker condition for full exploration is directional convexity (Holtzman and Halkin, 1966), established by Lin (1976). Several recent works in MTL develop algorithms to generate PO subject to user preferences, in particular on trade-offs among certain tasks (Lin et al., 2019; Mahapatra and Rajan, 2020; Momma et al., 2022). There is also a line of empirical works that focuses on designing algorithms to efficiently explore the PF (Ma et al., 2020; Lin et al., 2020; Liu et al., 2021c; Navon et al., 2021; Ruchte and Grabocka, 2021; Ye and Liu, 2022). As far as we know, we are the first to establish both necessary and sufficient conditions for full exploration in the non-convex setting.

8 Conclusion

In this paper, we revisit linear scalarization from a theoretical perspective. In contrast to recent works that claimed consistent empirical advantages of scalarization, we show its inherent limitation in fully exploring the Pareto front. Specifically, we reveal a multi-surface structure of the feasible region, and identify necessary and sufficient conditions for full exploration in the under-parametrized regime. We also discuss several implications of our theoretical results. Experiments on a real-world dataset corroborate our theoretical findings, and suggest the benefit of SMTOs in finding balanced solutions.

References

Bhatia. Matrix analysis , volume 169. Springer Science & Business Media, 2013.

Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization . Cambridge university press, 2004.

Caruana. Multitask learning. Machine learning , 28:41-75, 1997.

Chen, H. Fernando, Y. Ying, and T. Chen. Three-way trade-off in multi-objective learning: Optimization, generalization and conflict-avoidance. arXiv preprint arXiv:2305.20057 , 2023.

Chen, J. Ngiam, Y. Huang, T. Luong, H. Kretzschmar, Y. Chai, and D. Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. Advances in Neural Information Processing Systems , 33:2039-2050, 2020.

G. Ciarlet. Linear and nonlinear functional analysis with applications , volume 130. Siam, 2013.

Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning , pages 160-167, 2008.

Crawshaw. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 , 2020.

J.-A. Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique , 350(5-6):313-318, 2012.

S. Du, W. Hu, S. M. Kakade, J. D. Lee, and Q. Lei. Few-shot learning via learning the representation, provably. In International Conference on Learning Representations , 2021. URL https:// openreview.net/forum?id=pW2Q2xLwIMD .

Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika , 1(3):211-218, 1936.

D. Fernando, H. Shen, M. Liu, S. Chaudhury, K. Murugesan, and T. Chen. Mitigating gradient bias in multi-objective learning: A provably convergent approach. In The Eleventh International Conference on Learning Representations , 2022.

Fliege and B. F. Svaiter. Steepest descent methods for multicriteria optimization. Mathematical methods of operations research , 51(3):479-494, 2000.

Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman. The elements of statistical learning: data mining, inference, and prediction , volume 2. Springer, 2009.

M. Holtzman and H. Halkin. Directional convexity and the maximum principle for discrete systems. SIAM Journal on Control , 4(2):263-275, 1966.

Kalai and S. Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences , 71(3):291-307, 2005.

M. Kay. Fundamentals of statistical signal processing: estimation theory . Prentice-Hall, Inc., 1993.

Kurin, A. De Palma, I. Kostrikov, S. Whiteson, and P. K. Mudigonda. In defense of the unitary scalarization for deep multi-task learning. Advances in Neural Information Processing Systems , 35:12169-12183, 2022.

Lin, Y. Feiyang, Y. Zhang, and I. Tsang. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. Transactions on Machine Learning Research , 2022.

G. Lin. Three methods for determining pareto-optimal solutions of multiple-objective problems. In Directions in Large-Scale Systems: Many-Person Optimization and Decentralized Control , pages 117-138. Springer, 1976.

Lin, H.-L. Zhen, Z. Li, Q.-F. Zhang, and S. Kwong. Pareto multi-task learning. Advances in neural information processing systems , 32, 2019.

Lin, Z. Yang, Q. Zhang, and S. Kwong. Controllable pareto multi-task learning. arXiv preprint arXiv:2010.06313 , 2020.

Liu, X. Liu, X. Jin, P. Stone, and Q. Liu. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems , 34:18878-18890, 2021a.

Liu, Y. Li, Z. Kuang, J.-H. Xue, Y. Chen, W. Yang, Q. Liao, and W. Zhang. Towards impartial multi-task learning. In International Conference on Learning Representations , 2021b. URL https://openreview.net/forum?id=IMPnRXEWpvr .

Liu, X. Tong, and Q. Liu. Profiling pareto front with multi-objective stein variational gradient descent. Advances in Neural Information Processing Systems , 34:14721-14733, 2021c.

Ma, T. Du, and W. Matusik. Efficient continuous pareto exploration in multi-task learning. In International Conference on Machine Learning , pages 6522-6531. PMLR, 2020.

Mahapatra and V. Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In International Conference on Machine Learning , pages 6597-6607. PMLR, 2020.

Maurer. Bounds for linear multi-task learning. The Journal of Machine Learning Research , 7: 117-139, 2006.

Maurer, M. Pontil, and B. Romera-Paredes. The benefit of multitask representation learning. Journal of Machine Learning Research , 17(81):1-32, 2016.

Misra, A. Shrivastava, A. Gupta, and M. Hebert. Cross-stitch networks for multi-task learning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3994-4003, 2016.

Momma, C. Dong, and J. Liu. A multi-objective/multi-task learning framework induced by pareto stationarity. In International Conference on Machine Learning , pages 15895-15907. PMLR, 2022.

Navon, A. Shamsian, E. Fetaya, and G. Chechik. Learning the pareto front with hypernetworks. In International Conference on Learning Representations , 2021. URL https://openreview. net/forum?id=NjF772F4ZZR .

Navon, A. Shamsian, I. Achituve, H. Maron, K. Kawaguchi, G. Chechik, and E. Fetaya. Multitask learning as a bargaining game. In International Conference on Machine Learning , pages 16428-16446. PMLR, 2022.

T. Rockafellar. Convex analysis , volume 11. Princeton university press, 1997.

Ruchte and J. Grabocka. Scalable pareto front approximation for deep multi-objective learning. In 2021 IEEE international conference on data mining (ICDM) , pages 1306-1311. IEEE, 2021.

Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 , 2017.

Sener and V. Koltun. Multi-task learning as multi-objective optimization. Advances in neural information processing systems , 31, 2018.

Shaked-Monderer and A. Berman. Copositive and completely positive matrices . World Scientific, 2021.

Tripuraneni, C. Jin, and M. Jordan. Provable meta-learning of linear representations. In International Conference on Machine Learning , pages 10434-10443. PMLR, 2021.

Vandenhende, S. Georgoulis, W. Van Gansbeke, M. Proesmans, D. Dai, and L. Van Gool. Multitask learning for dense prediction tasks: A survey. IEEE transactions on pattern analysis and machine intelligence , 2021.

S. Varga. Matrix iterative analysis , volume 27. Springer Science & Business Media, 1999.

Vijayakumar and S. Schaal. Locally weighted projection regression: An o (n) algorithm for incremental real time learning in high dimensional space. In Proceedings of the seventeenth international conference on machine learning (ICML 2000) , volume 1, pages 288-293. Morgan Kaufmann, 2000.

Wang, Z. Zhao, B. Dai, C. Fifty, D. Lin, L. Hong, L. Wei, and E. H. Chi. Can small heads help? understanding and improving multi-task generalization. In Proceedings of the ACM Web Conference 2022 , pages 3009-3019, 2022.

Wu, H. R. Zhang, and C. Ré. Understanding and improving information transfer in multitask learning. In International Conference on Learning Representations , 2020. URL https: //openreview.net/forum?id=SylzhkBtDB .

Xian, L. Yin, and H. Zhao. Fair and optimal classification via post-processing. In Proceedings of the International Conference on Machine Learning (ICML) , 2023.

Xin, B. Ghorbani, J. Gilmer, A. Garg, and O. Firat. Do current multi-task optimization methods in deep learning even help? Advances in Neural Information Processing Systems , 35:13597-13609, 2022.

Ye and Q. Liu. Pareto navigation gradient descent: a first-order algorithm for optimization in pareto set. In Uncertainty in Artificial Intelligence , pages 2246-2255. PMLR, 2022.

Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2636-2645, 2020a.

Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems , 33:5824-5836, 2020b.

Zadeh. Optimality and non-scalar-valued performance criteria. IEEE transactions on Automatic Control , 8(1):59-60, 1963.

Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM , 64(3):107-115, 2021.

Zhang and Q. Yang. An overview of multi-task learning. National Science Review , 5(1):30-43, 2018.

Zhang and Q. Yang. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering , 34(12):5586-5609, 2021.

Zhao and G. J. Gordon. Inherent tradeoffs in learning fair representations. The Journal of Machine Learning Research , 23(1):2527-2552, 2022.

Zhou, W. Zhang, J. Jiang, W. Zhong, J. Gu, and W. Zhu. On the convergence of stochastic multi-objective gradient manipulation and beyond. Advances in Neural Information Processing Systems , 35:38103-38115, 2022.

A Omitted details from Section 3.1

A.1 Implementation of Figure 1

Figure 1 is generated from a simple three-task linear MTL problem that we constructed, using Equation (4). Specifically, we set ˆ y 1 ≈ (0 . 98 , 0 , 0 . 2) , ˆ y 2 ≈ ( -0 . 49 , -0 . 85 , 0 . 2) , ˆ y 3 ≈ ( -0 . 49 , 0 . 85 , 0 . 2) (the number of data points is n = 3 ; this is a rotated version of the equiangular tight frame), set q = 1 (the width of the network is one, i.e., under-parameterized), and plotted the achievable points of Equation (4) by sweeping P Z (the set of rank-1 projection matrices). The software we used is Mathematica.

A.2 Over-parametrization reduces the Pareto front to a singleton

For general non-linear multi-task neural networks, we will show that increasing the width reduces the Pareto front to a singleton, where all tasks simultaneously achieve zero training loss. As a consequence, linear scalarization with any convex coefficients will be able to achieve this point.

To illustrate this point, we follow the same setting as in Section 2, except changing the model to be a two-layer ReLU multi-task network with bias terms. Concretely, for task i , the prediction on input x is given by f i ( x, W, b, a i ) = a ⊤ i max( Wx + b, 0) . We have the following result.

Theorem A.1. There exists a two-layer ReLU multi-task network with hidden width q = nk and parameters ( W,b,a 1 , · · · , a k ) , such that

This implies that the network achieves zero training loss on each task.

Proof. For every i ∈ [ k ] , we can apply Theorem 1 in (Zhang et al., 2021) and find ( W i , b i , ˜ a i ) such that

Here, W i ∈ R n × p and b i , ˜ a i ∈ R n . Now consider

where e i denotes the i -th unit vector in R n and ⊗ stands for the Kronecker product. It is straightforward to see that

This finishes the proof as desired.

A.3 Proof of Theorem 3.1

Proof of Theorem 3.1. Define v i = ⟨ ˆ y i , s ⟩ and v = ( v 1 , · · · , v k ) ⊤ . We are interested in the set

We will show S is equivalent to the following set

which is essentially the boundary of an ellipsoid.

Step 1: S ⊂ B . Note v Y ⊤ s

Denote P ˆ Y = ˆ Y ( ˆ Y ⊤ ˆ Y ) -1 ˆ Y ⊤ , which is a projection matrix that maps to the column space of ˆ Y . Since s ∈ span( { ˆ y i } i ∈ [ k ] ) , we have

Step 2: B ⊂ S . Since ˆ Y ⊤ has full row rank, for every v ∈ R k , we can always find s ∈ R n such that ˆ Y ⊤ s = v . We can further assume s ∈ span( { ˆ y i } i ∈ [ k ] ) by removing the component in the null space of ˆ Y ⊤ . Assuming v ∈ B , we have

Since P ˆ Y is a projection matrix that maps to the column space of ˆ Y , we have s ⊤ s = 1 , which implies ∥ s ∥ = 1 .

Now we have S = B . The set

can be obtained by reflecting E to the non-negative orthant. Formally, a reflection is determined by a collection of axes { i 1 , · · · , i l } ( 0 ≤ l ≤ k ). The image of such reflection in the non-negative orthant is given by

where Q i 1 , ··· i l = D i 1 ··· i l QD i 1 ··· i l . As a consequence, we have

Finally, the feasible region can be equivalently characterized as

which can be obtained by squaring the coordinates of S ′ . Therefore, if we denote

then (note the square root naturally implies v ≥ 0 )

This finishes the proof as desired.

A.4 Proof of Theorem 3.2

Proof of Theorem 3.2. Note the orthogonal complement of a 1 -dimensional subspace of span( { ˆ y i } i ∈ [ k ] ) is a ( k -1) -dimensional subspace, and that the objective function for each task is the square of the projection of ˆ y i . Denote t := ( ∥ ˆ y 1 ∥ 2 , · · · , ∥ ˆ y k ∥ 2 ) ⊤ . By the Pythagorean theorem (Bhatia, 2013, Section I.6), if v ∈ F 1 , then we must have t -v ∈ F q -1 , and vice versa. As a consequence, denote and we have

by Theorem 3.1.

A.5 Why scalarization fails in the presence of gradient disagreement

We explain why linear scalarization is incapable of exploring an intersection point with gradient disagreement.

Wefirst recall the geometric interpretation (see Figure 4.9 in (Boyd et al., 2004)) of linear scalarization: a point P lying at the boundary of the feasible region can be achieved by scalarization, if and only if there exists a hyperplane at P and the feasible region lies above the hyperplane. The normal vector of the hyperplane is proportional to the scalarization coefficients. By definition, if a hyperplane at P lies below the feasible region, its normal vector must be a subgradient of the surface. When the boundary of the feasible region is differentiable and the subdifferential set is non-empty, the normal vector must be the gradient of the surface, and the hyperplane becomes the tangent plane at P .

Now suppose P lies at the intersection of two differentiable surfaces S 1 and S 2 , and that P can be achieved by scalarization. Applying the above argument to S 1 and P , we know that the scalarization coefficients are proportional to the gradient w.r.t. S 1 . Similarly, applying the above argument to S 2 and P yields that the scalarization coefficients are proportional to the gradient w.r.t. S 2 . This will result in a contradiction if the two gradients w.r.t. S 1 and S 2 at P lie in different directions, a phenomenon we refer to as gradient disagreement . In this case, scalarization cannot reach P .

B Omitted details from Section 3.2

B.1 Proof of Lemma 3.4

We begin by reviewing some basics of convex analysis. We refer the readers to Rockafellar (1997) for more background knowledge.

Definition B.1 (Convex hull) . Let X = { x i } i ∈ [ m ] be a collection of points in R n . The convex hull of X , denoted as conv( X ) , is the set of all convex combinations of points in X , i.e.,

Definition B.2 (Cone and convex cone) . A set K ⊂ R n is called a cone if x ∈ K implies αx ∈ K for all α > 0 . A convex cone is a cone that is closed under addition. The convex cone generated by X = { x i } i ∈ [ m ] ⊂ R n , denoted as cone( X ) , is given by

where ray( X ) = { λx i | λ ≥ 0 , i ∈ [ m ] } .

Definition B.3 (Dual cone) . The dual cone of C ⊂ R n , denoted as C ∗ , is given by

Definition B.4 (Projection onto a convex set) . Let C be a closed convex set in R n . The orthogonal projection of y ∈ R n onto the set C , denoted as P C y , is the solution to the optimization problem

Theorem B.5 (Bourbaki-Cheney-Goldstein inequality (Ciarlet, 2013)) . Let C be a closed convex set in R n . For any x ∈ R n , we have

Theorem Theorem B.5 is also known as the variational characterization of convex projection. We will use it to prove the following lemma, which serves as a crucial ingredient in the proof of Lemma 3.4.

Lemma B.6. Let C be a closed convex set in R n . For any z ∈ R n , we have

where z ′ = 2 P C z -z is the reflection of z w.r.t. C .

Proof. Plugging in z ′ = 2 P C z -z , it suffices to show

We are now ready to prove Lemma 3.4.

This is true because by Theorem B.5, and that

Proof of Lemma 3.4. For the if part, we assume w.l.o.g. that G = ˆ Y ⊤ ˆ Y is doubly non-negative. This implies that the angle between each pair of optimal predictors is non-obtuse. Denote A = cone( { ˆ y i } i ∈ [ k ] ) and A ∗ as its dual cone. We have A ⊂ A ∗ .

Our goal is to show that, for every ∥ s ∥ = 1 , we can always find s ′ ∈ A ∗ , such that 1) ∥ s ′ ∥ = 1 ; 2) ⟨ s ′ , ˆ y i ⟩ ≥ |⟨ s, ˆ y i ⟩| , ∀ i ∈ [ k ] . This implies that when restricting our discussion to the Pareto front, we can safely ignore the absolute value which appears in the proof of Theorem 3.1. As a consequence, the Pareto front belongs to the surface determined by Q = G -1 .

Consider s ′ = 2 P A ∗ s -s . It is straightforward to see that ∥ s ′ ∥ = 1 since it is the reflection of s w.r.t. A ∗ . To show 2), we will prove:

We break the discussion into two cases.

Case 1: ⟨ s, x ⟩ ≥ 0 . Since x ∈ A ⊂ A ∗ , we have

Case 2: ⟨ s, x ⟩ < 0 . Consider u = -s and denote C := { a | ⟨ a, P A ∗ s ⟩ ≥ 0 } . It is straightforward to see that A ⊂ C .

We will show P C u = P A ∗ s -s . In fact, for any z ∈ C , we have

Therefore, we have

With another application of Lemma B.6, we have

This finishes the proof of the if part.

For the only if part, we consider the point p i which achieves maximum value along the i -axis. For a given i , it is straightforward to see that such p i is unique, and therefore is a PO of the feasible region.

Now, p i belongs to a surface determined by Q ′ , if and only if there exists a non-negative vector v i , such that Q ′ v i = e i (the normal vector at p i aligns with the i -th axis). In other words, G ′ e i ≥ 0 where G ′ = ( Q ′ ) -1 . When C1 does not hold, there does not exist a G ∗ ∈ G , such that G ∗ e i ≥ 0 for all i ∈ [ k ] . This implies that these p i must belong to different surfaces.

B.2 Proof of Lemma 3.5

Proof of Lemma 3.5. For the if part, we assume w.l.o.g. that Q ∅ = G -1 ∅ is doubly non-negative. This essentially implies that E ∅ is dominated by all other surfaces in E , which further implies that I ∅ dominates all other surfaces in I . Therefore, the Pareto front must belong to I ∅ .

For the only if part, we study the Pareto front of F q -1 through the lens of F 1 . Specifically, a point z is a PO of F q -1 if and only if F q -1 and the non-negative orthant of z intersects only at z . Denote the corresponding point of z on F 1 as z ′ = t -z . An equivalent characterization is that F 1 and the non-positive orthant of z ′ intersects only at z ′ . We can therefore consider a special type of z ′ whose coordinates are zero except for i, j -th entry. If z ′ further lies on a surface Q with q ij > 0 , then the corresponding z will be a PO of F q -1 .

When C2 does not hold, there exists a row of Q ∅ which contains both positive and negative entries. We assume w.l.o.g. the first row of Q ∅ satisfies this condition. By the above observation, we can find two PO of F q -1 that correspond to Q ∅ and Q 1 , respectively. This finishes the proof as desired.

B.3 Proof of Theorem 3.8

Proof of Theorem 3.8. We only prove for the case of q = 1 , and the proof for q = k -1 can be done similarly. Suppose P (whose coordinate is v ) lies at the intersection of two surfaces defined by Q and Q ′ . We denote the intersection of Q and Q ′ as S , which is a non-linear manifold with dimension ( k -2) . Since P is a relative interior point of the PF, there exists some ε > 0 , such that any point in S ∩ B ε ( P ) is a PO. We will show there exists some P ′ ∈ S ∩ B ε ( P ) (whose coordinate is v ′ ), such that the gradients at P ′ w.r.t. the two surfaces disagree, i.e.,

̸

To see why this is true, note that Q ′ can be expressed as Q ′ = DQD , where D is a diagonal matrix whose diagonal entries are either 1 or -1 . As a consequence, the set

is a subspace of R k whose dimension is at most ( k -2) , so it cannot fully contain a non-linear manifold whose dimension is ( k -2) . This finishes the proof as desired.

B.4 Proof of Theorem 3.8 without assumptions ( k = 3 )

Here we provide a proof for Theorem 3.8 under k = 3 , without relying on Assumptions 3.6 and 3.7.

Proof. We prove the necessity of C1 and C2 separately.

C1 is necessary. Suppose C1 is not true. We assume w.l.o.g. that ⟨ ˆ y 1 , ˆ y 2 ⟩ < 0 , ⟨ ˆ y 1 , ˆ y 3 ⟩ > 0 , ⟨ ˆ y 2 , ˆ y 3 ⟩ > 0 . Now we can write

where q ij > 0 for all pairs of ( i, j ) . We also have

The boundary of the feasible region is formed by E ∅ , E 1 , E 2 . Our goal is to find a point I ∈ E ∅ ∩ E 1 , such that 1) I is a PO of the feasible region; 2) the gradients at I w.r.t. E ∅ and E 1 disagree. We write the coordinate of I as ( x, y = q 2 13 /s, z = q 2 12 /s ) ⊤ , where

and s < q 11 q 2 23 + q 22 q 2 13 + q 33 q 2 12 -2 q 12 q 23 q 31 .

It is straightforward to see that 2) can be satisfied as long as x = 0 . Therefore, we will focus on the realization of 1). This further requires two conditions: i) I is a PO w.r.t. E ∅ and E 1 , meaning that the normal vectors are non-negative; ii) I is a PO w.r.t. E 2 , meaning that the non-negative orthant at I does not intersect with E 2 .

Some simple calculations yield that i) is equivalent to

Setting x = 0 , we have

̸

by Equation (47). Therefore, it suffices to show the non-negative orthant at (0 , q 2 13 /s, q 2 12 /s ) ⊤ ( s = q 22 q 2 13 + q 33 q 2 12 -2 q 12 q 23 q 31 ) does not intersect with E 2 , and by continuity we can find some x 0 > 0 such that i) and ii) hold simultaneously.

To prove the above claim, suppose a ≥ 0 , b ≥ q 2 13 /s, c ≥ q 12 2 /s . We will show ( a, b, c ) ⊤ must lie above the surface defined by E 2 , i.e.,

As a consequence, by choosing a small x 0 , we can guarantee both 1) and 2). I is therefore a PO that cannot be achieved via scalarization.

C2 is necessary. Suppose C2 is not true. We can write

where q ij > 0 for all pairs of ( i, j ) . The inner boundary of F 1 is formed by E 1 , E 2 , E 3 , and the coordinates of their intersection I is given by v = ( q 2 23 /r, q 13 2 /r, q 2 12 /r ) ⊤ , where r = q 11 q 2 23 + q 22 q 2 13 + q 33 q 2 12 +2 q 12 q 23 q 31 . Our goal is to show the corresponding I ′ on F 2 (i.e., t -v ) is a PO which cannot be achieved by scalarization.

To see why this is true, first note that the gradients at I w.r.t. the three surfaces disagree, since q ij > 0 for all pairs of ( i, j ) . Now, to demonstrate I ′ is a PO of F 2 , it suffices to show the non-positive orthant of I intersects with F 1 only at I . By symmetry, it suffices to show that I does not dominate any point on E 1 .

To prove the above claim, suppose a ≤ q 2 23 /r, b ≤ q 2 13 /r, c ≤ q 2 12 /r . We will show, J = ( a, b, c ) ⊤ must lie below the surface defined by E 1 unless J equals I . In fact, let

We have

where the equalities hold iff I = J . This finishes the proof as desired.

B.5 Proof of Theorem 3.10

We first present several useful lemmas regarding non-negative matrices.

Lemma B.7. Let A be a non-negative, irreducible and real symmetric matrix, then there only exists one eigenvector v of A (up to scaling), such that v > 0 .

Proof. Since A is non-negative and irreducible, by Perron-Frobenius theorem (Shaked-Monderer and Berman, 2021), the eigenvector associated with the largest eigenvalue ρ ( A ) (a.k.a. Perron root ) can be made positive, and is unique up to scaling. We denote it as v .

We will show the eigenvectors associated with other eigenvalues cannot be positive. In fact, assume u > 0 is an eigenvector associated with λ < ρ ( A ) . Since A is symmetric, we have

This is a contradiction since v ⊤ u > 0 and ρ ( A ) > λ .

Lemma B.8. Let A be a non-negative and real symmetric matrix. Suppose v is an eigenvector of A such that v > 0 , then v corresponds to the largest eigenvalue of A .

Proof. Consider the normal form (Varga, 1999) of A :

̸

where P is a permutation matrix and B ii ( i ∈ [ h ] ) are irreducible. It is straightforward to see that B is non-negative. Furthermore, since a permutation matrix is an orthogonal matrix, B is also symmetric (implying that B ij = 0 for i = j ) and has the same spectrum as A .

Now suppose Av = λv for some v > 0 . Since B = PAP ⊤ , we have Bψ = λψ with ψ = Pv > 0 . Denote

where each ψ i has the same dimension as B ii . Now we have

Since B ii is non-negative, irreducible and real symmetric, by Lemma B.7, ψ i corresponds to the largest eigenvalue of B ii . Note B is a block diagonal matrix, so the spectrum of B is essentially the union of the spectrum of B ii . As a consequence,

Proof of Theorem 3.10. We prove the sufficiency of C1 and C2 separately.

C1 is sufficient. We have demonstrated in Lemma 3.4 that the Pareto front belongs to a single surface E , whose gram matrix is doubly non-negative. Let p = ( a 1 , · · · , a k ) ⊤ be a PO of E , and denote t = √ p . We will show that the tangent plane at this point lies above the feasible region, implying that it can be achieved by scalarization.

Concretely, the tangent plane is given by

To show it lies above the feasible region, it suffices to prove that it lies above every surface in E , i.e.,

where q i 1 ··· i l ij represents the ( i, j ) -th entry of Q i 1 ··· i l . Note Equation (60) can be treated as a quadratic form w.r.t. √ v , so it suffices to prove the corresponding matrix is PSD. Writing compactly, it suffices to show

is PSD. As a consequence, we only need to show the positive semi-definiteness of the following matrix

Since p is a PO of E , we have Qt ≥ 0 . Let w = Qt , we have

Assume w.l.o.g. that w > 0 (otherwise we simply remove the corresponding rows and columns in G -1 ). Denote √

We have

By Lemma B.8, we have ρ ( RGR ) = 1 . Since RGR is positive definite, we have

This further implies that R -2 ⪰ G , and

C2 is sufficient. We have demonstrated in Lemma 3.5 that the Pareto front belongs to a single surface I ∅ . Here we will further show I ∅ is concave, and apply the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) to conclude the proof. In fact, E ∅ is parametrized by

Since the first term is a summation of linear functions, f ( x, y ) = √ xy is a concave function and that q ij ≥ 0 , we conclude that E ∅ is convex, and I ∅ is concave. This finishes the proof as desired.

C A polynomial algorithm for checking C1

We explain how Algorithm 1 works.

Essentially, the goal is to find a set of flipping signs { s i } i ∈ [ k ] ∈ { + , -} that ensure positive correlation between all pairs in { s i ˆ y i } i ∈ [ k ] (except for those pairs that are not correlated). We state an observation which is critical for our algorithm-given a pair ˆ y i and ˆ y j , whether their signs are the same or opposite can be determined by whether ˆ y i and ˆ y j is positively or negatively correlated. Thus, given a determined s i , the signs of its subsequent predictors s j , j > i can be uniquely determined by evaluating ⟨ ˆ y i , ˆ y j ⟩ . This allows us to determine the signs for the predictors s i in ascending order, and check for potential conflicts in the mean time.

We state one loop invariant in our algorithm-when entering iteration i in the outer loop, s i has been determined by its preceding predictors and no conflict has occurred, or it has remained undetermined (i.e., None ) all the way because it is not correlated with any preceding predictors. At this point, what is left to be done is to examine the subsequent predictors. If s i is undetermined (lines 2-7), we attempt to use its subsequent predictors to determine it (line 5), and break out of the loop once it gets determined. Now that we make sure that s i is determined after line 9, we can proceed to check the remaining subsequent predictors, to either determine their signs (lines 12-13) or check for potential conflicts (lines 14-15).

We comment that caution needs to be taken for the pairs that are not correlated (i.e., ⟨ ˆ y i , ˆ y j ⟩ = 0 ). In this case, one cannot determine the sign of s j by the relationship between these two predictors; basically, the sign has no influence on their correlation. Thus, the predictor ˆ y j retains the flexibility to be determined by other predictors other than ˆ y i . Additionally, it is possible that at the end of line 9 s j is still undetermined; this only occurs when ˆ y i is not correlated with any other optimal predictor. In this case, t will take the value of k +1 and the following loop (lines 10-15) will not be executed.

D Additional experimental details and results

D.1 Additional experimental details

Dataset. The SARCOS dataset (Vijayakumar and Schaal, 2000) consists of the configurations (position, velocity, acceleration) of the robotic arms, and the problem is to predict the torque of the

̸

̸

respective arm given its triplet configuration. Since our study concentrates on the training procedure and does not concern generalization, we conduct the experiments on the training set alone, where the size of the training set of the SARCOS dataset (Vijayakumar and Schaal, 2000) is 40 , 000 . Regression tasks normally benefit from standardization as a pro-processing step (Hastie et al., 2009), so we standardize the dataset to zero mean and unit variance.

Tasks. Following our theoretical analysis in Section 3, we study the scenario where q = 1 . In this case, it is straightforward to see that the minimal k that may lead to a violation of C1 is 3 . To show the violation of C1 leading to the failure of scalarization exploring the Pareto Front, we constrain our study to k ≥ 3 ; furthermore, for ease of visualization, we focus on k = 3 . In our experiments, we take the arms 3 , 4 , and 5 for which do not satisfy C1 .

Network. We use a two-layer linear network for the reason explained in Section 2. We merge the bias term b into the weight matrix W by adding an additional all-one column to the input X . The input dimension of the network is 22 (seven (position, velocity, acceleration) triplets plus the dummy feature induced by b ), the hidden size is q = 1 , and the output size is 1 (i.e., the predicted torque). The same network architecture is used for both experiments in scalarization and that in SMTOs.

Implementation of linear scalarization. For linear scalarization, we uniformly sample 100 , 000 sets of convex coefficients from a three-dimensional simplex (i.e., a tetrahedron). Concretely, we first sample m 1 , m 2 i.i.d. ∼ U (0 , 1) , and then craft λ = ( λ 1 , λ 2 , λ 3 ) = (min( m 1 , m 2 ) , max( m 1 , m 2 ) -min( m 1 , m 2 ) , 1 -max( m 1 , m 2 ) + min( m 1 , m 2 )) as the weights. For a given set of convex coefficients, we calculate the optimal value directly based on the analysis in Section 2 instead of performing actual training. More specifically, the optimum of the scalarization objective is given by ˆ Y q, Λ Λ := ∑ q i =1 σ i u i v ⊤ i , from which we can compute the corresponding MSE for each task.

̸

SMTOs. We consider two state-of-the-art SMTOs, MGDA (Désidéri, 2012) and MGDA-UB (Sener and Koltun, 2018). MGDA is an optimization algorithm specifically devised for handling multiple objectives concurrently. It utilizes a composite gradient direction to facilitate simultaneous progress across all objectives. MGDA-UB is an efficient variant of MGDA that focuses on maximizing the minimum improvement among all objectives. It is a gradient-based multi-objective optimization algorithm aimed at achieving balanced optimization outcomes across all objectives.

Implementation of SMTOs. Our code is based on the released implementation 5 of MGDA-UB, which also includes the code for MGDA. We apply their code on the SARCOS dataset. For both methods, we use vanilla gradient descent with a learning rate of 0 . 5 for 100 epochs, following the default choice in the released implementation. We comment that early stopping can also be adopted, i.e., terminate once the minimum norm of the convex hull of gradients is smaller than a threshold, for which we set to be 10 -3 .

D.2 Additional experiments on random initialization

To eliminate the influence of random initialization, we perform 300 trials for each algorithm using different random seeds. We filter out solutions whose maximum MSE is larger than 1 for clearer presentation, which leads to 198 solutions for MGDA and 240 solutions for MGDA-UB. We plot all these solutions in Figure 4. We see that MGDA and MGDA-UB are consistently capable of finding balanced solutions regardless of the random seed.

MSE1

1.0

0.8

0.6

0.4

0.2

0.0

MSE2

1.0

0.8

0.6

0.4

0.2

0.0

MSE3

1.0

0.8

0.6

0.4

0.2

0.0

MGDA

Scalarization

(a) MGDA with multiple initializations

MSE1

1.0

0.8

0.6

0.4

0.2

0.0

MSE2

1.0

0.8

0.6

0.4

0.2

0.0

MSE3

1.0

0.8

0.6

0.4

0.2

0.0

MGDA-UB

Scalarization

(b) MGDA-UB with multiple initializations

D.3 Additional experiments on randomization

To verify the effectiveness of randomization as discussed in Section 4.2, we plot the region achievable by randomized linear scalarization. Specifically, according to Equation (13), the region can be expressed as the collection of the convex combination of two points from the feasible region. To this end, we randomly sample 100,000 weight pairs (the same number as in Figure 3). For each weight pair ( w 1 , w 2 ) , we uniformly draw t ∼ U (0 , 1) and get two corresponding optimal networks f 1 and f 2 by SVD. For each sample, with probability t , the model uses f 1 , otherwise f 2 , to calculate the MSE. The final result is demonstrated in Figure 5. It is straightforward to see that randomization allows scalarization to trace out the PF since there is no hole within the blue region, thus validating our theoretical analysis. We additionally comment that randomization convexifies the feasible region, as such, the solutions found by MGDA and MGDA-UB are dominated.

MSE1

1.0

0.8

0.6

0.4

0.2

0.0

MSE2

1.0

0.8

0.6

0.4

0.2

MSE3

1.0

0.8

0.6

0.4

0.2

0.0

MGDA

MGDA-UB

Randomized Scalarization

0.0

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn recent years, there has been a surge in papers suggesting Specialized Multi-Task Optimizers (SMTOs). These papers show the empirical advantage of using SMTOs compared to linear scalarization (LS). However, recently, there have been several papers that suggested that LS with proper tunning can match SMTOs performance. The paper analyzes scalarization from a theoretical perspective, and studies whether scalarization is capable of fully exploring the Pareto front in linear MTL. It reveals some inherent limitations of LS for MTL optimization.\n\nSTRENGTHS:\n1. The paper addresses an important open question in MTL optimization.\n1. The paper is (mostly) well-written and well-structured.\n1. The paper provides an in-depth theoretical analysis with implications for both MTL researchers and practitioners.\n\nWEAKNESSES:\n1. A primary argument is that LS cannot reach a point in the intersection of surfaces. However, it is not clear from Section 3.2 why this is the case. Please elaborate on this point and revise the manuscript accordingly.\n1. Please discuss the main results and novelty w.r.t the known result ([1]), which states that LS cannot reach non-convex parts of the Pareto front.\n1. It would be beneficial to revise Section 3, by either breaking it into two sections or summarizing the main results at the beginning of the Section.\n1. Please add an experiment with the proposed randomization approach to empirically verify that it can explore the entire Pareto front.\n1. Missing citations in related work:\n    1. SMTOs: \n        - Multi-Task Learning as a Bargaining Game, ICML 2022. \n        - Towards Impartial Multi-task Learning, ICLR 2021.\n        - Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models, ICLR 2021.\n    1. Exploration of the Pareto front: There’s a line of work for methods trying to learn the entire Pareto front, so it is important to mention it. The two pioneering papers for Pareto front learning are:\n        - Learning the Pareto Front with Hypernetworks, ICLR 2021.\n        - Controllable Pareto Multi-Task Learning, 2021. \n\n[1] S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization. 2004.\n\nQUESTIONS:\n1. How realistic is the case $q<k$ for real-world applications?\n\nLIMITATIONS:\nAddressed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-26"}

comments: SUMMARY:
This paper is related to multi-objective optimization area.
They post a research question: if linearly weighting multiple objectives can fully explore the Pareto front?
Through theoretical analysis and a simple experiment, the answer is negative.
Hence, this might prove that multiple gradient descent algorithm is inherently better than linear objective weighting.

STRENGTHS:
Recently, many report that linear weighting is no worse and even better than gradient balancing algorithms like multiple gradient descent algorithm (MGDA). 
However, these findings are empirical,
This paper targets on an important question: if multiple gradient descent algorithm is inherently better than simple linear weighting when conflict gradients are presenting.
Through theoretical analysis and a simple experiment, the answer is yes, which lay a good foundation for this area.

WEAKNESSES:
The MTL model is this paper is very simple. I am not sure if this could reflect the real problem in the industry system.
This model only have one layer of shared layer and one layer of task-head. And this is a linear neural network. (I am not sure if pure linear can be called neural network).
So, seems this MTL model is a convex optimization.
However, in real industry systems, there are many layers (maybe millions of parameters) with non-linear transformation and hence a non-convex optimization.

QUESTIONS:
1. if MTL has different objectives like one regression task and one classification task, will your conclusion hold?

2. If MTL has non-linear transformation like Sigmoid, will your conclusion hold?

3. Do we have practical usage of randomization for linear scalarization?

LIMITATIONS:
No negative social impact
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper revisits linear scalarization from a theoretical perspective. The authors study multi-task learning with a two-layer linear network and reveal a multi-surface structure of the feasible region. They show the necessary and sufficient conditions for full exploration in the under-parameterized regime. Their theoretical results imply that linear scalarization has fundamental limitations and that scalarization tends to overfit a small fraction of tasks. They answer some open questions proposed in Xin et at.(2022) and also provide experiments on a three-task learning problem to verify the theoretical results.\n\nSTRENGTHS:\n- It is a novel and interesting problem to study whether linear scalarization can fully explore the Pareto front. The authors provide necessary and sufficient conditions for this in the under-parametrized regime, which is the main contribution of this paper.\n- The notations and presentations in this paper are clear and the proofs seem to be correct.\n- They empirically study the validity of the theoretical results, and the experiments are reasonable, and the results are convincing to me.\n\nWEAKNESSES:\n- The authors study only two cases (q = 1 and q = k - 1) in the under-parametrized regime, but are these two extreme cases representative for other cases in the under-parametrized regime?\n- This paper discusses linear scalarization in the under-parametrized regime. However, over-parameterization is common in deep learning, where the network has sufficient capacity to adapt to the target tasks. Does this mean that linear scalarization in the overparameterized regime does not have the fundamental limitations discussed in this paper?\n- I commend the authors for attempting to answer some open questions in Xin et al. (2022), but it lacks an explanation for why SMTOs have no improvement over linear scalarization in deep multitask learning, which is found in Xin et al. (2022).\n\nQUESTIONS:\nPlease see the weaknesses.\n\nLIMITATIONS:\nThe theoretical analysis in this paper is only applied to the linear network and under-parametrized regime, which is a limitation of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the linear scalarization approach in multi-task learning (MTL). It shows theoretically that the linear scalarization is not able to fully capture the Pareto optimal (PO) solutions. It also identifies necessary and sufficient conditions of full exploration of the PO solutions for under-parameterized two-layer linear MTL models.\nThis explains why scalarization fails in certain cases. Experiments are performed to further verify the theoretical findings.\n\nSTRENGTHS:\n1. The paper studies an important and timely problem, focusing on whether the popular approach -- linear scalarization for MTL are able to fully capture the Pareto front.\n\n2. The perspective of this paper and techniques to develop the theory under the specific two-layer linear models are unique.\n\n3. Paper is well written and easy to follow.\n\nWEAKNESSES:\n1. There is a lack of discussion of related works that also identifies the failure cases of linear scalarization fully capturing the Pareto front. See [1,2,3,4].  \nWhat is the unique contribution of this paper compared to these prior works? Some discussions need to be provided to distinguish with the prior works.\n\n---\n[1] Lin, Jiguan G. “Three Methods for Determining Pareto-Optimal Solutions of Multiple-Objective Problems”\n\n[2] Zadeh LA, \"Optimality and non-scalar-valued performance criteria\"\n\n[3] Goicoechea A, Hansen DR, Duckstein L, \"Multiobjective decision analysis with engineering and business applications\".\n\n[4] R. Timothy Marler, Jasbir S. Arora, \"The weighted sum method for multi-objective optimization: new insights\"\n\n[5] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, Qiang Liu, \"Conflict-Averse Gradient Descent for Multi-task Learning\"\n\n---\n\n2. In line 56-57, it is too restrictive to claim that \"if scalarization cannot fully explore the Pareto front, there is no inherent advantage of SMTOs over scalarization.\" In fact, there are other benefits of SMTOs beyond this. For SMTO methods like MGDA, the motivation is to find the steepest common descent direction for all objectives at each iteration. And the benefit sometimes lies in the optimization procedure besides the solutions they find. See the toy example in [5]\n\n3. Even though the paper shows that linear scalarization approach is not able to fully explore the Pareto set in some settings, there is no discussion on whether SMTOs are able to fully explore the Pareto set in the same settings.\n\n4. Some questions need to be clarified. See **Questions**\n\n\n\n-----------------------------update post rebuttal-------------------------------\n\nI have read the rebuttal and participated in the discussion. \n\nThe rebuttal partially addressed my concerns so I remained relatively positive about this work.\n\nIt could further improve with a more thorough discussion of the classical works and more precise positioning of their contributions.\n\n--------------------------------------------------------------------------------------\n\nQUESTIONS:\n1. I do not agree with the claimed contribution 2, line 63-65, where the authors claimed to give the \"first guarantee for full exploration in the presence of non-convexity.\"\n\n- Please see reference [1], where necessary conditions of Pareto optimality are given for the linear scalarization method with \"p-directionally convex\" requirement of the criterion space, which is not convex.\n\n- Please see reference [2,3], where sufficient conditions of (weak) Pareto optimality are given for the linear scalarization method.\n\n- Please discuss the difference and relations with the listed prior works. Perhaps you need to add the specific settings, such as \"for under-parameterized two-layer linear networks\".\n\n2. Since this paper is focused on the underparameterized model with $q < k$, how does the theory in this paper helps understanding the question posed in Xin et al. (2022), which is for deep models usually overparameterized?\n\n3. What are the implementation details for visualization in Figure 1?\n\n4. In line 170, what does \"gradients disagree\" mean? Does it mean they are not in the exactly same direction or the angle between the two gradients is larger than a threshold? There should be a formal definition.\n\n\n\n---\n[1] Lin, Jiguan G. “Three Methods for Determining Pareto-Optimal Solutions of Multiple-Objective Problems”\n\n[2] Zadeh LA, \"Optimality and non-scalar-valued performance criteria\"\n\n[3] Goicoechea A, Hansen DR, Duckstein L, \"Multiobjective decision analysis with engineering and business applications\".\n\n\n\n---\n\n### Minor\n\nTypos and grammar errors:\n\nLine 2: \"since its inception\" -> \"because of its inception\"\nLine 96: \"vecotrs\" -> \"vectors\"\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations in Section 5.\n\nI expect the authors to conduct a thorough review with the most relevant works, see Weaknesses-1.\n\nOverall, this paper studies an important problem with a unique perspective.\nI am willing to increase my score if the authors can address these questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

comments: SUMMARY:
This paper revisits linear scalarization in multi-task learning from a theoretical perspective.
The authors reveal that scalarization is, in general, incapable of tracing out the Pareto front.
Specifically, when the model is under-parametrized, a multi-surface structure of the feasible region is revealed by the authors and it can be used to identify necessary and sufficient conditions for full exploration.
Experimental results further verify the theoretical findings.

STRENGTHS:
1. This paper provides rigorous analysis to answer an important question in multi-task learning: whether linear scalarization is capable of
fully exploring the Pareto front?
The authors provide insightful remarks for the theories, and the figures provide intuitive explanations for the multi-surface structure of the feasible region.

2. The paper is well-written and easy to follow. Sufficient preliminaries and explanations are provided.

WEAKNESSES:
1. The paper only studies the under-parametrized regime. It remains unclear whether the conclusion still holds for over-parametrized regimes where the model capacity is large enough and task competence does not exist.

2. The paper mainly focuses on a two-layer neural network. Therefore, it remains unclear whether the conclusion still holds for non-convex functions.

QUESTIONS:
Please check the weakness.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce a novel mathematical framework to answer the problem “Under what conditions can (or cannot) linear scalarization recover the Pareto front of a given multi-objective optimization problem?”. To this end, the paper considers the multi-task learning setting  using a simple linear multi-task learning model in the under-parameterized regime, and introduce new concepts like “feasible regions” and \"gradient disagreement\", and state that the intersection points of aforementioned feasible regions are unattainable by optimizing any linear scalarization of the multiple objectives. Accordingly, the paper derives necessary and sufficient conditions as to when optimizing any linear scalarization of the multiple objectives can recover the Pareto front by checking for situations where the Pareto front of the problem lies entirely in one so called feasible region. The paper then provides some empirical validation of the theory that suggest linear scalarization indeed fail to cover the Pareto front, especially the “balanced” Pareto optimal points. The empirical results further suggest that Specialized Multi-Task Optimizers (SMTOs) like MGDA and MGDA-UB can achieve a more “balanced” Pareto optimal points compared to linear scalarization.\n\nSTRENGTHS:\n* The paper is aimed at finding necessary and sufficient conditions for when scalarization can not recover the Pareto front, which is an important and pertinent problem in the field of multi-objective optimization.\n\n* This paper provides a mathematical framework for analyzing the aforementioned problem, which may provide insights into how to select a method of solution for a multi-objective optimization problem based on the nature of the problem.\n\nWEAKNESSES:\n* The necessary and sufficient conditions derived for linear scalarization to recover the Pareto front in this paper requires Pareto optimal objective vectors to be “on the same feasible surface”, rather than being at an “intersection point of feasible surfaces”, as described in  Lemmas 3.4, 3.5. Yet, it is unclear why these intersection points cannot be attained by some scalarization of the objectives. Specifically, the “gradient disagreement” concept is unclear; for example how to mathematically characterize this gradient in the objective space, and why this gradient is related to the optimization of the multiple objectives.\n\n* It is unclear what is the origin ((0, 0, 0) point ) of the objective spaces used in the Figures 1 and 3. Thus, it is a bit hard to understand the provided illustrations of feasible regions in the objective space.\n\n* While the paper provide some weakness (i.e. the inability to recover the whole Pareto front of a problem) of scalarization in the under-parameterized regime, it does not give theoretical validation whether SMTOs can overcome this weakness. The empirical validation for showing that SMTOs can overcome the aforementioned weakness have some limitations, as described in following points. Thus, the conclusion of the paper seems to be not justified from the theoretical and empirical results.\n\n* The experiments done to verify theory seems inconsistent for SMTOs and linear scalarization methods. Specifically, SMTOs are optimized using iterative methods for 100 epochs, while scalarization solutions are obtained from the corresponding closed form expression for the optimum solution. A fair comparison of methods would be either running SMTO methods for a very large number of iterations, or using an iterative updating scheme for scalarizations for the same number of epochs. \n\n* By nature of the MGDA algorithm, if deterministic gradients are used, the algorithm should terminate at the first encounter of a Pareto stationary point (a notion of stationarity for multiple objectives similar to stationarity in single objective optimization), which is often a point either at least one objective have achieved stationarity, or the point where the objectives begin to conflict with one another (see some discussion on this here [1]). Thus, this kind of point is usually a point that will favor one objective over the others (unless the objectives are perfectly aligned). In this sense, it is unclear how MGDA have achieved an interior point of the Pareto front, as implied by Figure 3.\n\n* At the beginning of the paper (such as in the abstract) the authors pose the weakness of scalarization over SMTOs as the inability to “fully recover” the Pareto front, while the empirical evaluations suggests that SMTOs can also only recover a single Pareto optimal point, which seems to not align with the initial message.\n\nMinor comments\n\n* Definition of Pareto front introduced in the paper seems not standard (line 119). Specifically, in this paper it is referred to as the collection of  Pareto optimal points (PO) $\\theta^*$, while usually Pareto front is the set of objective values $\\\\{L_i(\\theta^*)\\\\}_{i\\in[k]}$  corresponding to Pareto optimal points.\n\n[1] Liu, Xingchao, Xin Tong, and Qiang Liu. \"Profiling pareto front with multi-objective stein variational gradient descent.\" Advances in Neural Information Processing Systems 34 (2021): 14721-14733.\n\nEdit: Added missing reference.\n\nQUESTIONS:\n* What is the mathematical definition of the gradients depicted in Figure 3, and what is the precise relationship between the “gradient disagreement” and the inability to recover this point by optimizing a linear scalarization of the objectives?\n\n* Could the authors provide some intuition or some physical meaning of the feasible regions described in the paper, and how they relate to the multiple objectives?\n\n* Would an implementation change of the experiments as described in the previous section (point 4) change the solution distribution of each algorithm compared to Figure 3?\n\n* Given the nature of MGDA algorithm as described in previous section (point 5), could the authors provide the reason for the convergence of MGDA like algorithms for the interior point of the Pareto front of the problem?\n\nLIMITATIONS:\nLimitations of the proposed framework are discussed discussed in the paper. In addition to these limitations, the paper does have limited validation as to SMTOs can overome the weaknesses of linear scalarization, as described in previous sections.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-18"}