SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For the only if part, we consider the points which achieve maximum value along an axis. We will show these points are PO. Finally, we will show that if C1 does not hold, then these points must belong to different surfaces in  $\mathcal{E}$ .
We have a similar result for  $q = k - 1$ . The details of the proof are deferred to Appendix B.2.
Lemma 3.5. When  $q = k - 1$ , the Pareto front of the feasible region belongs to a single surface  $I^{*} \in \mathcal{I}$ , if and only if the corresponding  $Q^{*} \in \mathcal{Q}$  is doubly non-negative. We refer to the existence of such a doubly non-negative  $Q^{*}$  in  $\mathcal{Q}$  as C2.
Lemmas 3.4 and 3.5 provide a refined characterization of how the PO is distributed within the feasible region: unless C1 (resp. C2) holds, the Pareto front will belong to multiple surfaces in  $\mathcal{E}$  (resp.  $\mathcal{I}$ ). This easily leads to the existence of a PO on the intersection of different surfaces (see Figure 1), which will render the failure of scalarization as a consequence. It is therefore natural for us to conjecture that C1 (resp. C2) is necessary for full exploration. We make the following assumption regarding the topology of the Pareto front.
Assumption 3.6. For  $q = 1$  and  $q = k - 1$ , we assume the Pareto front is path-connected, i.e., for every pair of  $PO$ $x$  and  $y$ , there exists a continuous curve on the Pareto front with  $x$  and  $y$  being the two endpoints.
Assumption 3.6, combined with Lemma 3.4 (resp. Lemma 3.5), will help locate a PO  $P$  on the intersection of two surfaces when C1 (resp. C2) does not hold. We make an additional technical assumption to circumvent degenerated cases.
Assumption 3.7. We assume  $P$  (the intersection point) is a relative interior point (Rockafellar, 1997, Section 6) of the Pareto front.
We are now ready to present the main results in this section (full proofs in Appendices B.3 and B.5).
Theorem 3.8 (Necessity of C1 (resp. C2)). For  $q = 1$  (resp.  $q = k - 1$ ), under Assumptions 3.6 and 3.7, C1 (resp. C2) is a necessary condition for scalarization to fully explore the Pareto front.
Proof sketch of Theorem 3.8. Assumption 3.7 allows us to perform dimensionality analysis in a small neighborhood of the intersection point  $P$ , from which we can demonstrate the existence of a PO lying on the intersection of two surfaces with gradient disagreement. This point cannot be achieved by linear scalarization.
Remark 3.9. Assumptions 3.6 and 3.7 are mainly used to simplify analysis, and they are not necessary for Theorem 3.8 to hold. As a matter of fact, we can show the necessity of C1 (resp. C2) when  $k = 3$  without these assumptions. We refer the readers to Appendix B.4 for more details.
Finally, without relying on the above assumptions, we can show that C1 and C2 are sufficient.
Theorem 3.10 (Sufficiency of C1 (resp. C2)). For  $q = 1$  (resp.  $q = k - 1$ ), C1 (resp. C2) is a sufficient condition for scalarization to fully explore the Pareto front.
Proof sketch of Theorem 3.10. For  $q = 1$ , we consider the tangent plane at a PO and attempt to show it lies above the feasible region. This is transformed into demonstrating the positive semi-definiteness of a matrix, which can be conquered by applying the Perron-Frobenius theorem (Shaked-Monderer and Berman, 2021, Section 1.3) on non-negative matrices. For  $q = k - 1$ , we show the boundary of the feasible region is concave, and apply the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) to conclude the proof.
We discuss the implications of our theoretical results, and provide a remedy solution for linear scalarization to address its inability of full exploration.
4.1 Implications of the theoretical results
Fundamental limitation of linear scalarization. From Theorem 3.8, it is clear that linear scalarization faces significant challenges in fully exploring the Pareto front in the case of  $q = 1$  and  $q = k - 1$ , since both C1 and C2 require the existence of a doubly non-negative matrix, which is a very small subset of the PSD matrix. To illustrate this point, we consider a simplified probabilistic model: when the optimal predictors  $\{\hat{g}_i\}_{i \in [k]}$  are drawn independently from a symmetric distribution centered at the origin, the probability that C1 holds is  $2^{-O(k^2)}$ , which decays exponentially with the number of tasks. Thus, we comment that C1 and C2 in general does not hold in practical cases.
Towards understanding when and why scalarization fails. Among the vast literature that studies Pareto optimality in MTL, there is a noticeable absence of a systematic and rigorous exploration of when and why scalarization fails to reach a Pareto optimum. Prior work typically resorts to hypothetical examples (e.g., Figure 4.9 in (Boyd et al., 2004)) to demonstrate the failure of scalarization. Our study, which identifies the multi-surface structure of the feasible region and reveals the phenomenon of gradient disagreement in a concrete setting, serves as a pioneering step in addressing this gap in the literature.
A closer examination of the failure mode. For the extreme under-parametrized case  $(q = 1)$ , on the one hand, we can prove that scalarization is able to achieve points which attain maximum value along one direction (i.e., best performance on one task). On the other hand, by examining Figure 1, we shall see that scalarization is incapable of exploring the interior points (i.e., balanced performance across all tasks). This suggests that scalarization has the tendency to overfit to a small fraction of tasks, and fails to strike the right balance between different tasks. We provide more empirical evidence to support this argument in Section 5.
Criteria for practitioners. Theorem 3.10 demonstrates that C1 and C2 are sufficient for scalarization to fully explore the Pareto front. As such, these conditions can serve as valuable criteria for practitioners when deciding which tasks to combine in linear MTL. Note a naive enumeration of  $\mathcal{G}$  or  $\mathcal{Q}$  will lead to a time complexity of  $O(2^k)$ . We instead provide an  $O(k^2)$  algorithm checking whether C1 holds (Algorithm 1 in Appendix C). In essence, the algorithm aims to find a set of flipping signs  $\{s_i\}_{i\in [k]}\in \{+, - \}$  that ensure positive or zero correlation between all pairs in  $\{s_i\hat{y}_i\}_{i\in [k]}$ . It scans through the predictors in ascending order and either determines the signs  $s_j$  for each optimal predictor  $\hat{y}_j$  if not determined yet, or checks for conflicts between signs that are already determined. A sign can be determined when the predictors  $\hat{y}_i$  and  $\hat{y}_j$  are correlated. A conflict indicates the impossibility of finding such a set of flipping signs, thus falsifying C1. A similar algorithm can be devised for C2 with an additional step of Cholesky decomposition beforehand, leading to an  $O(k^{3})$  time complexity.
Open questions in Xin et al. (2022). We (partially) answer the open questions in Xin et al. (2022).
Q1. "Is it the case that neural networks trained via a combination of scalarization and standard first-order optimization methods are not able to reach the Pareto Frontier?"
A1. For Linear MTL, our answer is a definitive "Yes". We demonstrate that linear scalarization with arbitrary convex coefficients cannot trace out all the Pareto optima in general. Our results are strong in the sense that they hold on a representation level, meaning that they are independent of the specific
optimization algorithm employed. This suggests that the weakness we have revealed in scalarization is fundamental in nature.
Q2. "Is non-convexity adding additional complexity which makes scalarization insufficient for tracing out the Pareto front?"
A2. Yes, even some mild non-convexity introduced by two-layer linear networks will render scalarization insufficient. On the other hand, we do believe the multi-surface structure that we have unveiled in Section 3.1 is both universal and fundamental, which also contributes to the observed non-convexity in practical scenarios. In essence, what we have done partly is to penetrate through the surface and gain a holistic view of the feasible region. Notably, the key obstacle in linear scalarization, as revealed by our analysis, lies in the phenomenon of gradient disagreement. This cannot be readily discerned by solely examining the surface.
We make two additional comments to further clarify our standing point in comparison to this work.
1. We do not aim to refute the claims in Xin et al. (2022); as a matter of fact, we consider their empirical findings highly valuable. Instead, we would like to emphasize a discrepancy between their theory and experiments. Given that we have shown the conclusion—that linear scalarization can fully explore the PF—no long holds in the non-convex setting, their theory does not account for their experiments on modern neural networks. Therefore, we urge the research community to develop a new theory elucidating the empirical efficacy of linear scalarization.
2. While the differing settings render the results from these two papers non-comparable, they collectively offer a broader perspective on the pros and cons of scalarization and SMTOs. We believe this is of great significance to both researchers and practitioners.
4.2 A remedy solution
Given the fundamental weakness of linear scalarization in fully exploring the Pareto front, one might wonder whether there exists any model class where linear scalarization is sufficient to fully explore the Pareto front. The answer is affirmative if we allow ourselves to use randomized multi-task neural networks. More specifically, given two MTL networks  $f^{(0)}$  and  $f^{(1)}$  over the same input and output spaces and a fixed constant  $t \in [0,1]$ , we can construct a randomized MTL network as follows. Let  $S \sim U(0,1)$  be a uniform random variable over  $(0,1)$  that is independent of all the other random variables, such as the inputs. Then, consider the following randomized neural network:
$$
f (x) := \left\{ \begin{array}{l l} f ^ {(0)} (x) & \text {i f} S \leq t \\ f ^ {(1)} (x) & \text {o t h e r w i s e} \end{array} \right. \tag {12}
$$
Essentially, the randomized network constructed by Equation (12) will output  $f^{(0)}(x)$  with probability  $t$  and  $f^{(1)}(x)$  with probability  $1 - t$ . Now if we consider the loss of  $f(\cdot)$ , we have
$$
\begin{array}{l} \mathbb {E} _ {\mu} \mathbb {E} _ {S} [ \| f (X) - Y \| _ {2} ^ {2} ] = \mathbb {E} _ {S} \mathbb {E} _ {\mu} [ \| f (X) - Y \| _ {2} ^ {2} ] \\ = t \mathbb {E} _ {\mu} [ \| f (X) - Y \| _ {2} ^ {2} \mid S \leq t ] + (1 - t) \mathbb {E} _ {\mu} [ \| f (X) - Y \| _ {2} ^ {2} \mid S > t ] \\ = t \mathbb {E} _ {\mu} [ \| f ^ {(0)} (X) - Y \| _ {2} ^ {2} ] + (1 - t) \mathbb {E} _ {\mu} [ \| f ^ {(1)} (X) - Y \| _ {2} ^ {2} ], \tag {13} \\ \end{array}
$$
which means that the loss of the randomized multi-task network is a convex combination of the losses from  $f^{(0)}(\cdot)$  and  $f^{(1)}(\cdot)$ . Geometrically, the argument above shows that randomization allows us to "convexify" the feasible region and thus the Pareto front will be convex. Now, by a standard application of the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) we can see that every point on the Pareto front can be realized via linear scalarization. We empirically verify the effectiveness of randomization in Appendix D.3.
We comment that randomization is a powerful tool that has wide applicability in machine learning. For instance, it has achieved great success in online learning, specifically in the Follow-the-Perturbed-Leader algorithm (Kalai and Vempala, 2005). It also appears in the literature of fairness to achieve the optimal regressor or classifier (Zhao and Gordon, 2022; Xian et al., 2023). Here we further demonstrate that it can be applied to MTL to facilitate the exploration of linear scalarization. In essence, randomization increases the representation power of the original neural network.
In this section, we corroborate our theoretical findings with a linear MTL experiment, where we show that linear scalarization fails to fully explore the Pareto front on a three-task problem that does not satisfy C1 with  $q = 1$ . In comparison, SMTOs can achieve balanced Pareto optimum solutions that are not achievable by scalarization.
Dataset and setup. We use the SARCOS dataset for our experiment (Vijayakumar and Schaal, 2000), where the problem is to predict the torque of seven robot arms given inputs that consist of the position, velocity, and acceleration of the respective arms. For ease of visualization, we restrict the tasks to arms 3, 4, and 5; in particular, they do not satisfy C1.
Our regression model is a two-layer linear network with hidden size  $q = 1$  (no bias). To explore the portion of the Pareto front achievable by linear scalarization, we fit 100,000 linear regressors with randomly sampled convex coefficients and record their performance. These are compared to the solutions found by SMTOs, for which we use MGDA (Désideri, 2012) and MGDA-UB (Sener and Koltun, 2018). Specifically, to guarantee their Pareto optimality, we run the SMTOs with full-batch gradient descent till convergence (hence each SMTO method has one deterministic solution). Additional details, including hyperparameter settings, can be found in Appendix D.1.
Results. In Figure 3, we plot the MSEs achieved by scalarization under randomly sampled weights. We observe that the solutions achieved by scalarization are concentrated near the vertices and partly along the edges, indicating their tendency to overfit to 1-2 of the tasks. Scalarization also fails to explore a large portion of the Pareto front in the center—which have more balanced performance across task; the inability of full exploration corroborates the necessary condition (Theorem 3.8) discussed in Section 3. In contrast, SMTOs are capable of finding solutions not achievable by scalarization that are located in the center of the Pareto front. This showcases the relative advantage and practical utility of SMTOs in MTL over scalarization, especially in scenarios where equitable performance on all tasks are desired. To strengthen our results, we experiment with multiple random initializations and observe consistent results (see Appendix D.2).
Figure 3: Linear scalarization has the tendency to overfit to a subset of tasks by finding peripheral blue points of the Pareto front. MGDA (magenta) and MGDA-UB (red) converge to more balanced PO in the interior.
Discussion. We would like to note that we are not endorsing any specific SMTO algorithm, such as MGDA or MGDA-UB. As a matter of fact, we hypothesize that no single algorithm is universally effective (also see (Chen et al., 2023)), notably in overcoming the limitation of scalarization in terms of full exploration. Instead, by revealing the limitation of scalarization and the potential benefits of some SMTO algorithms, we aim to strengthen SMTO research, challenge the notion that linear scalarization is sufficient, and advocate for a more balanced progression in the field of MTL.
6 Limitations and future directions
Limitations. There are two major limitations of our work. First, while the techniques we develop in this paper are highly non-trivial, we only cover the two extreme cases  $q = 1$  and  $q = k - 1$  in the under-parametrized regime, and the analysis for general  $q < k$  is missing. Second, the setting of our study is relatively restricted. Specifically, we focus on linear MTL where different tasks share the same input. A more realistic setting should factor in different inputs for different tasks (Wu et al., 2020), as well as some non-linearity in the model architecture (Kurin et al., 2022).
Future directions. We expect the geometry of the feasible region to transition smoothly as we increase  $q$ , meaning that it will not abruptly collapse into a single surface. Providing a rigorous basis for this assertion would facilitate the analysis for general  $q$ 's, thereby complementing our study. Regarding the assumption of shared input, while our analysis persists under the 'proportional covariance assumption' adopted by Wu et al. (2020), we find this assumption excessively restrictive,
and hope future work could relax it further. The generalization to neural networks with ReLU activation is interesting but also challenging, and it is unclear whether the techniques and results developed in this paper can be extended to the non-linear setting. Another interesting avenue to explore is interpreting the multi-surface structure we have uncovered, which we hypothesize to have connections with game theory. Finally, our paper dedicates to demonstrating the weaknesses of linear scalarization, while we consider it an important future direction to theoretically analyze the advantage of SMTOs, as partially revealed in our experiments. In all, we hope our study can initiate a line of works on MTL, which provide theoretical justifications for the usage of specific algorithms beyond empirical evaluations across various settings.
SMTOs. There is a surge of interest in developing specialized multi-task optimizers (SMTOs) in the literature. One of the earliest and most notable SMTOs is the multiple-gradient descent algorithm (MGDA) (Désidéri, 2012). For each task, MGDA solves an optimization problem and performs updates using the gradient of the objective function with both shared and task-specific parameters. However, MGDA cannot scale to high dimension (e.g., neural networks), and computing the gradients for every task is prohibitively expensive. To counter these drawbacks, Sener and Koltun (2018) introduce a scalable Frank-Wolfe-based optimizer which is more efficient. Another prominent issue in MTL is that the gradients from multiple tasks could conflict with each other and the performance of some tasks could be significantly affected. To this end, Yu et al. (2020b) propose gradient surgery to mitigate the negative interference among tasks; Chen et al. (2020) and Liu et al. (2021a) propose to manipulate the gradients or the gradient selection procedures; Fernando et al. (2022) devise a stochastic gradient correction method and prove its convergence. More recently, Navon et al. (2022) cast the gradient update as a Nash bargaining game, yielding well-balanced solutions across the PF.
Empirical comparison of SMTOs and scalarization. A recent line of works empirically compare SMTOs and linear scalarization. Their central claim is that the performance of linear scalarization is comparable with the state-of-the-art SMTOs despite the added complexity of SMTOs. Xin et al. (2022) argue that multi-task optimization methods do not yield any improvement beyond what can be achieved by linear scalarization with carefully-tuned hyperparameters; Kurin et al. (2022) show that the performance of linear scalarization matches or even surpasses the more complex SMTOs when combined with standard regularization techniques. In addition to classification tasks, Vandenhende et al. (2021) conduct extensive experiments on dense prediction tasks, i.e., tasks that produce pixel-level predictions. They conclude that avoiding gradient competition among tasks can actually lead to performance degradation, and linear scalarization with fixed weights outperforms some SMTOs.
Exploration of the Pareto front. It is well-known that minimizing scalarization objectives with positive coefficients yields Pareto optimal solutions (Boyd et al., 2004), while using non-negative coefficients yields weak Pareto optimal ones (Zadeh, 1963). On the other hand, exploration of the entire Pareto front has been a long-standing and difficult open problem in MTL. For the simplest case, if all loss functions are convex or the achievable region is a convex set, then scalarization can fully explore the PF (Boyd et al., 2004; Xin et al., 2022). A slightly weaker condition for full exploration is directional convexity (Holtzman and Halkin, 1966), established by Lin (1976). Several recent works in MTL develop algorithms to generate PO subject to user preferences, in particular on trade-offs among certain tasks (Lin et al., 2019; Mahapatra and Rajan, 2020; Momma et al., 2022). There is also a line of empirical works that focuses on designing algorithms to efficiently explore the PF (Ma et al., 2020; Lin et al., 2020; Liu et al., 2021c; Navon et al., 2021; Ruchte and Grabocka, 2021; Ye and Liu, 2022). As far as we know, we are the first to establish both necessary and sufficient conditions for full exploration in the non-convex setting.
In this paper, we revisit linear scalarization from a theoretical perspective. In contrast to recent works that claimed consistent empirical advantages of scalarization, we show its inherent limitation in fully exploring the Pareto front. Specifically, we reveal a multi-surface structure of the feasible region, and identify necessary and sufficient conditions for full exploration in the under-parametrized regime. We also discuss several implications of our theoretical results. Experiments on a real-world dataset corroborate our theoretical findings, and suggest the benefit of SMTOs in finding balanced solutions.
We thank the anonymous reviewers for their constructive feedback. Yuzheng Hu would like to thank Fan Wu for useful suggestions in writing. Han Zhao's work was partly supported by the Defense Advanced Research Projects Agency (DARPA) under Cooperative Agreement Number: HR00112320012, an IBM-IL Discovery Accelerator Institute research award, a Facebook Research Award, a research grant from the Amazon-Illinois Center on AI for Interactive Conversational Experiences, and Amazon AWS Cloud Credit.