SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that in the strongly convex case, the gradient norm  $\| \nabla F_z(x)\|_{\mathrm{F}}$  can be unbounded in  $\mathbb{R}^d$ . Therefore, one cannot assume Lipschitz continuity of  $f_{z,m}(x)$  w.r.t.  $x\in \mathbb{R}^d$ . We address this challenge by showing that  $\{x_t\}$  generated by the MoDo algorithm is bounded as stated in Lemma 1. Notably, combining with Assumption 1, we can derive that the gradient norm  $\| \nabla F_z(x_t)\|_{\mathrm{F}}$  is also bounded, which serves as a stepping stone to derive the MOL stability bound.
Lemma 1 (Boundedness of  $x_{t}$  for strongly convex and smooth objectives). Suppose Assumptions 1, 2 hold. For  $\{x_{t}\} , t\in [T]$  generated by MoDo algorithm or other dynamic weighting algorithm with weight  $\lambda \in \Delta^{M}$ , step size  $\alpha_{t} = \alpha$ , and  $0\leq \alpha \leq \ell_{f,1}^{-1}$ , there exists a finite positive constant  $c_{x}$  such that  $\| x_{t}\| \leq c_{x}$ . And there exists finite positive constants  $\ell_f$ ,  $\ell_F = \sqrt{M}\ell_f$ , such that for all  $\lambda \in \Delta^M$ , we have  $\| \nabla F(x_t)\lambda \| \leq \ell_f$ , and  $\| \nabla F(x_t)\|_F\leq \ell_F$ .
With Lemma 1, the stability bound and PS generalization is provided below.
Theorem 2 (PS generalization error of MoDo in strongly convex case). Suppose Assumptions 1, 2 hold. Let  $A$  be the MoDo algorithm (Algorithm 1). For the MOL uniform stability  $\epsilon_F$  of algorithm  $A$  in Definition 2, if the step sizes satisfy  $0 < \alpha_t \leq \alpha \leq 1 / (2\ell_{f,1})$ , and  $0 < \gamma_t \leq \gamma \leq \min \left\{\frac{\mu^2}{120\ell_f^2\ell_{g,1}}, \frac{1}{8(3\ell_f^2 + 2\ell_{g,1})}\right\} / T$ , then it holds that
$$
\epsilon_ {\mathrm {F}} ^ {2} \leq \frac {4 8}{\mu n} \ell_ {f} ^ {2} \ell_ {F, 1} ^ {2} \left(\alpha + \frac {1 2 + 4 M \ell_ {f} ^ {2}}{\mu n} + \frac {1 0 M \ell_ {f} ^ {4} \gamma}{\mu}\right) \quad a n d \quad \mathbb {E} _ {A, S} [ R _ {\text {g e n}} (A (S)) ] = \mathcal {O} \left(n ^ {- \frac {1}{2}}\right). \tag {3.4}
$$
And there exist functions  $F_{z}(x)$  that satisfy Assumptions 1, 2, neighboring datasets  $S$ ,  $S'$  that differ in at most one sample, and MoDo algorithm with step sizes  $0 < \alpha_{t} \leq \alpha \leq 1 / (2\ell_{f,1})$ , and  $0 < \gamma_{t} \leq \gamma \leq \min \left\{\frac{\mu^{2}}{120\ell_{f}^{2}\ell_{g,1}}, \frac{1}{8(3\ell_{f}^{2} + 2\ell_{g,1})}\right\} / T$  such that
$$
\mathbb {E} _ {A} \left[ \| \nabla F _ {z} (A (S)) - \nabla F _ {z} (A \left(S ^ {\prime}\right)) \| _ {\mathrm {F}} ^ {2} \right] \geq \frac {M \mu^ {2}}{2 5 6 n ^ {2}}. \tag {3.5}
$$
Theorem 2 provides both upper and lower bounds for the MOL uniform stability. In this case, we choose  $\alpha = \Theta (T^{-\frac{1}{2}})$ ,  $\gamma = o(T^{-1})$ , and  $T = \Theta (n^{2})$  to minimize the PS population risk upper bound, as detailed in Section 3.3. With this choice, the MOL uniform stability upper bound matches the lower bound in an order of  $n^{-2}$ , suggesting that our bound is tight. The generalization error bound in (3.4) is a direct implication from the MOL uniform stability bound in (3.4), Propositions 2, and 3.
It states that the PS generalization error of MoDo is  $\mathcal{O}(n^{-\frac{1}{2}})$ , which matches the generalization error of static weighting up to a constant coefficient [22]. Our result also indicates that when all the objectives are strongly convex, choosing small step sizes  $\alpha$  and  $\gamma$  can benefit the generalization error.
3.2 Multi-objective optimization error
In this section, we bound the multi-objective PS optimization error  $\min_{\lambda \in \Delta^M} \| \nabla F_S(x) \lambda \|$  [9, 30, 10]. As discussed in Section 2.2, this measure being zero implies the model  $x$  achieves a Pareto stationarity for the empirical problem.
Below we list an additional standard assumption used to derive the optimization error.
Assumption 3 (Lipschitz continuity of  $F_{z}(x)$ ). For all  $m \in [M]$ ,  $f_{z,m}(x)$  are  $\ell_{f}$ -Lipschitz continuous for all  $z$ . Then  $F_{z}(x)$  are  $\ell_{F}$ -Lipschitz continuous in Frobenius norm for all  $z$  with  $\ell_{F} = \sqrt{M}\ell_{f}$ .
Lemma 2 (Distance to CA direction). Suppose either: 1) Assumptions 1, 3 hold; or 2) Assumptions 1, 2 hold, with  $\ell_f$  and  $\ell_F$  defined in Lemma 1. For  $\{x_t\}$ ,  $\{\lambda_t\}$  generated by MoDo, it holds that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} _ {A} [ \| d _ {\lambda_ {t}} (x _ {t}) - d (x _ {t}) \| ^ {2} ] \leq \frac {4}{\gamma T} + 6 \sqrt {M \ell_ {f , 1} \ell_ {f} ^ {2} \frac {\alpha}{\gamma}} + \gamma M \ell_ {f} ^ {4}. \tag {3.6}
$$
Lemma 2 analyzes convergence to the CA direction using the measure introduced in Section 2.2. By, e.g., choosing  $\alpha = \Theta (T^{-\frac{3}{4}})$ , and  $\gamma = \Theta (T^{-\frac{1}{4}})$ , the RHS of (3.6) converges in a rate of  $\mathcal{O}(T^{-\frac{1}{4}})$ .
Theorem 3 (PS optimization error of MoDo). Suppose either: 1) Assumptions 1, 3 hold; or, 2) Assumptions 1, 2 hold, with  $\ell_f$  defined in Lemma 1. Define  $c_{F}$  such that  $\mathbb{E}_A[F_S(x_0)\lambda_0] - \min_{x\in \mathbb{R}^d}\mathbb{E}_A[F_S(x)\lambda_0]\leq c_F$ . Considering  $\{x_t\}$  generated by MoDo (Algorithm 1), with  $\alpha_{t} = \alpha \leq 1 / (2\ell_{f,1})$ ,  $\gamma_t = \gamma$ , then under either condition 1) or 2), it holds that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} _ {A} \left[ \min  _ {\lambda \in \Delta^ {M}} \| \nabla F _ {S} (x _ {t}) \lambda \| \right] \leq \sqrt {\frac {c _ {F}}{\alpha T}} + \sqrt {\frac {3}{2} \gamma M \ell_ {f} ^ {4}} + \sqrt {\frac {1}{2} \alpha \ell_ {f , 1} \ell_ {f} ^ {2}}. \tag {3.7}
$$
The choice of step sizes  $\alpha = \Theta (T^{-\frac{3}{4}})$ , and  $\gamma = \Theta (T^{-\frac{1}{4}})$  to ensure convergence to CA direction is suboptimal for the convergence to Pareto stationarity (see Theorem 3), exhibiting a trade-off between convergence to the CA direction and convergence to Pareto stationarity; see discussion in Section 3.3.
3.3 Optimization, generalization and conflict avoidance trade-off
Combining the results in Sections 3.1 and 3.2, we are ready to analyze and summarize the three-way trade-off of MoDo in MOL. With  $A_{t}(S) = x_{t}$  denoting the output of algorithm  $A$  at the  $t$ -th iteration, we can decompose the PS population risk  $R_{\mathrm{pop}}(A_t(S))$  as (cf. (2.1) and (3.1))
$$
\mathbb {E} _ {A, S} \left[ R _ {\text {p o p}} (A _ {t} (S)) \right] \leq \mathbb {E} _ {A, S} \left[ \min  _ {\lambda \in \Delta^ {M}} \| \nabla F _ {S} (A _ {t} (S)) \lambda \| \right] + \mathbb {E} _ {A, S} \left[ \| \nabla F (A _ {t} (S)) - \nabla F _ {S} (A _ {t} (S)) \| _ {\mathrm {F}} \right].
$$
The general nonconvex case. Suppose Assumptions 1, 3 hold. By the generalization error in Theorem 1, and the optimization error bound in Theorem 3, the PS population risk of the output of MoDo can be bounded by
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} _ {A, S} \left[ R _ {\text {p o p}} \left(A _ {t} (S)\right) \right] = \mathcal {O} \left(\alpha^ {- \frac {1}{2}} T ^ {- \frac {1}{2}} + \alpha^ {\frac {1}{2}} + \gamma^ {\frac {1}{2}} + T ^ {\frac {1}{2}} n ^ {- \frac {1}{2}}\right). \tag {3.8}
$$
Discussion of trade-off. Choosing step sizes  $\alpha = \Theta (T^{-\frac{1}{2}})$ ,  $\gamma = \Theta (T^{-\frac{1}{2}})$ , and number of steps  $T = \Theta (n^{\frac{2}{3}})$ , then the expected PS population risk is  $\mathcal{O}(n^{-\frac{1}{6}})$ , which matches the PS population risk upper bound of a general nonconvex single objective in [22]. A clear trade-off in this case is between the optimization error and generalization error, controlled by  $T$ . Indeed, increasing  $T$  leads to smaller optimization errors but larger generalization errors, and vice versa. To satisfy convergence to CA
direction, it requires  $\gamma = \omega(\alpha)$  based on Lemma 2, and the optimization error in turn becomes worse, so does the PS population risk. Specifically, choosing  $\alpha = \Theta(T^{-\frac{1}{2}})$ ,  $\gamma = \Theta(T^{-\frac{1}{4}})$ , and  $T = \Theta(n^{\frac{4}{5}})$  leads to the expected PS population risk in  $\mathcal{O}(n^{-\frac{1}{10}})$ , and the distance to CA direction in  $\mathcal{O}(n^{-\frac{1}{10}})$ . This shows another trade-off between conflict avoidance and optimization error.
The strongly convex case. Suppose Assumptions 1, 2 hold. By the generalization error and the optimization error given in Theorems 2 and 3, MoDo's PS population risk can be bounded by
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} _ {A, S} \left[ R _ {\text {p o p}} \left(A _ {t} (S)\right) \right] = \mathcal {O} \left(\alpha^ {- \frac {1}{2}} T ^ {- \frac {1}{2}} + \alpha^ {\frac {1}{2}} + \gamma^ {\frac {1}{2}} + n ^ {- \frac {1}{2}}\right). \tag {3.9}
$$
Discussion of trade-off. Choosing step sizes  $\alpha = \Theta (T^{-\frac{1}{2}})$ ,  $\gamma = o(T^{-1})$ , and number of steps  $T = \Theta (n^{2})$ , we have the expected PS population risk in gradients is  $\mathcal{O}(n^{-\frac{1}{2}})$ . However, choosing  $\gamma = o(T^{-1})$  leads to large distance to the CA direction according to Lemma 2 because the term  $\frac{4}{\gamma T}$  in (3.6) increases with  $T$ . To ensure convergence to the CA direction, it requires  $\gamma = \omega (T^{-1})$ , under which the tighter bound in Theorem 2 does not hold but the bound in Theorem 1 still holds. In this case, the PS population risk under proper choices of  $\alpha, \gamma, T$  is  $\mathcal{O}(n^{-\frac{1}{6}})$  as discussed in the previous paragraph. Therefore, to avoid conflict of gradients, one needs to sacrifice the sample complexity of PS population risk, demonstrating a trade-off between conflict avoidance and PS population risk.
4 Related Works and Our Technical Contributions
Multi-task learning (MTL). MTL, as one application of MOL, leverages shared information among different tasks to train a model that can perform multiple tasks. MTL has been widely applied to natural language processing, computer vision, and robotics [15, 38, 50, 43]. From the optimization perspective, a simple method for MTL is to take the weighted average of the per-task losses as the objective. However, as studied in [16], the static weighting method is not able to find all the Pareto optimal models in general. Alternatively, the weights can be updated dynamically during optimization, and the weights for different tasks can be chosen based on different criteria such as uncertainty [17], gradient norms [6], or task difficulty [13]. These methods are often heuristic and designed for specific applications. Another line of work tackles MTL through MOL [39, 48, 29, 12]. A foundational algorithm in this regard is MGDA [9], which takes dynamic weighting of gradients to obtain a CA direction for all objectives. Stochastic versions of MGDA have been proposed in [30, 52, 10]. Besides finding one single model, algorithms for finding a set of Pareto optimal models rather than one have been proposed in [28, 35, 31, 20, 46, 51, 32, 27, 34].
Theory of MOL. Optimization convergence analysis for the deterministic MGDA algorithm has been provided in [11]. Later on, stochastic variants of MGDA were introduced [30, 52, 10]. However, the vanilla stochastic MGDA introduces biased estimates of the dynamic weight, resulting in biased estimates of the CA directions during optimization. To address this, Liu et al. [30] proposed to increase the batch size during optimization, Zhou et al. [52] and Fernando et al. [10] proposed to use momentum-based bias reduction techniques. Compared to these works on the optimization analysis, we have improved the assumptions and/or the final convergence rate of the PS optimization error. A detailed comparison is summarized in Table 2. While the community has a rich history of investigating the optimization of MOL algorithms, their generalization guarantee remains unexplored until recently. In [7], a min-max formulation to solve the MOL problem is analyzed, where the weights are chosen based on the maximum function values, rather than the CA direction. More recently, [41] provides generalization guarantees for MOL for a more general class of weighting. These two works analyze generalization based on the Rademacher complexity of the hypothesis class, with generalization bound independent of the training process. Different from these works, we use algorithm stability to derive the first algorithm-dependent generalization error bounds, highlighting the effect of the training dynamics. In contrast to previous MOL theoretical works that focus solely on either optimization [10, 52] or generalization [7, 41], we propose a holistic framework to analyze three types of errors, namely, optimization, generalization, and CA distance in MOL with an instantiation of the MoDo algorithm. This allows us to study the impact of the hyperparameters on the theoretical testing performance, and their optimal values to achieve the best trade-off among the errors. Our theory and algorithm design can also be applied to algorithms such as PCGrad [48], CAGrad [29], and GradNorm [6]. Specifically, for example, the implementation of CAGrad takes iterative updates of the dynamic weight using a single stochastic estimate, resulting in a biased estimate of the update
Table 2: Comparison with prior stochastic MOL algorithms in terms of assumptions and the guarantees of the three errors, where logarithmic dependence is omitted, and Opt., CA dist., and Gen. are short for optimization error, CA distance, and generalization error, respectively.
Algorithm	Batch size	NC	Lipschitz λ*(x)	Bounded function	Opt.	CA dist.	Gen.
SMG [30, Thm 5.3]	O(t)	X	✓	X	T-1/8	-	-
CR-MOGM [52, Thm 3]	O(1)	✓	X	✓	T-1/4	-	-
MoCo [10, Thm 2]	O(1)	✓	X	X	T-1/20	T-1/5	-
MoCo [10, Thm 4]	O(1)	✓	X	✓	T-1/4	O(1)	-
MoDo (Ours, Thms 1,2,3)	O(1)	✓	X	X	T-1/4	O(1)	T1/2n-1/2
MoDo (Ours, Thms 1,2,3)	O(1)	✓	X	X	T-1/8	T-1/4	T1/2n-1/2
direction, thus no guarantee of convergence for the stochastic algorithm. This issue can be addressed by the double sampling technique introduced in this paper. In addition, our analysis techniques have been applied to improve the analysis of convergence rates for other algorithms [44, 5].
Algorithm stability and generalization. Stability analysis dates back to the work [8] in 1970s. Uniform stability and its relationship with generalization were studied in [3] for the exact minimizer of the ERM problem with strongly convex objectives. The work [14] pioneered the stability analysis for stochastic gradient descent (SGD) algorithms with convex and smooth objectives. The results were extended and refined in [19] with data-dependent bounds, in [4, 37, 23] for non-convex objectives, and in [1, 24] for SGD with non-smooth and convex losses. However, all these studies mainly focus on single-objective learning problems. To our best knowledge, there is no existing work on the stability and generalization analysis for multi-objective learning problems and our results on its stability and generalization are the first-ever-known ones.
Challenges and contributions. Our contributions are highly non-trivial as summarized below.
- The definition of PS testing risk in gradient (2.1) is unique in MOL, and overcomes the unnecessarily small step sizes usually brought by the classical function value-based risk analysis. Specifically, prior stability analysis in function values for single objective learning [14] requires  $1 / t$  step size decay in the nonconvex case, otherwise, the generalization error bound will depend exponentially on the number of iterations. However such step sizes lead to very slow convergence of the optimization error. This is addressed by the definitions of gradient-based measures and sampling-determined MOL algorithms, which yield stability bounds in  $\mathcal{O}(T / n)$  without any step size decay. See Theorem 1.
- The stability of the dynamic weighting algorithm in the strongly convex (SC) case is non-trivial compared to single objective learning [14] because it involves two coupled sequences during the update. As a result, the classical contraction property for the update function of the model parameters that are often used to derive stability does not hold. This is addressed by controlling the change of  $\lambda_{t}$  by the step size  $\gamma$ , and using mathematical induction to derive a tighter bound. See Appendix B.4.
- In the SC case with an unbounded domain, the function is not Lipschitz or the gradients are unbounded, which violates the commonly used bounded gradient assumption for proving the stability and optimization error. We relax this assumption by proving that the iterates generated by dynamic weighting algorithms in the SC case are bounded on the optimization trajectory in Lemma 1.
In this section, we conduct experiments to further demonstrate the three-way trade-off among the optimization, generalization, and conflict avoidance of the MoDo algorithm. An average of 10 random seeds with 0.5 standard deviation is reported if not otherwise specified.
Our theory in the SC case is first verified through a synthetic experiment; see the details in Appendix D.1. Figure 3a shows the PS optimization error and PS population risk, as well as the distance to CA direction, decreases as  $T$  increases, which corroborates Lemma 2, and Theorem 3. In addition, the generalization error, in this case, does not vary much with  $T$ , verifying Theorems 2. In Figure 3b, the optimization error first decreases and then increases as  $\alpha$  increases, which is consistent with Theorem 3. Notably, we observe a threshold for  $\alpha$  below which the distance to the CA direction converges even when the optimization error does not converge, while beyond which the distance to the CA direction becomes larger, verifying Lemma 2. Additionally, Figure 3c demonstrates that
(a) Number of iterations  $T$
(b) Different step size  $\alpha$
(c) Different step size  $\gamma$
Figure 3: Optimization, generalization, and CA direction errors of MoDo in the strongly convex case under different  $T, \alpha, \gamma$ . The default parameters are  $T = 100$ ,  $\alpha = 0.01$ ,  $\gamma = 0.001$ .
(a) Number of iterations  $T$
Figure 4: Optimization, generalization, and CA direction errors of MoDo for MNIST image classification under different  $T$ ,  $\alpha$ , and  $\gamma$ . The default parameters are  $T = 1000$ ,  $\alpha = 0.1$ , and  $\gamma = 0.01$ .
increasing  $\gamma$  enlarges the PS optimization error, PS generalization error, and thus the PS population risk, but decreases the distance to CA direction, which supports Lemma 2.
5.2 Image classification experiments
We further verify our theory in the NC case on MNIST image classification [21] using a multi-layer perceptron and three objectives: cross-entropy, mean squared error (MSE), and Huber loss. Following Section 2.2, we evaluate the performance in terms of  $R_{\mathrm{pop}}(x)$ ,  $R_{\mathrm{opt}}(x)$ ,  $R_{\mathrm{gen}}(x)$ , and  $\mathcal{E}_{\mathrm{ca}}(x,\lambda)$ . The exact PS population risk  $R_{\mathrm{pop}}(x)$  is not accessible without the true data distribution. To estimate the PS population risk, we evaluate  $\min_{\lambda \in \Delta^M} \| \nabla F_{S_{\mathrm{te}}}(\boldsymbol{x})\lambda \|$  on the testing data set  $S_{\mathrm{te}}$  that is independent of training data set  $S$ . The PS optimization error  $R_{\mathrm{opt}}(x)$  is obtained by  $\min_{\lambda \in \Delta^M} \| \nabla F_S(x)\lambda \|$ , and the PS generalization error  $R_{\mathrm{gen}}(x)$  is estimated by  $\min_{\lambda \in \Delta^M} \| \nabla F_{S_{\mathrm{te}}}(\boldsymbol{x})\lambda \| - R_{\mathrm{opt}}(x)$ .
We examine the impact of different  $T$ ,  $\alpha$ ,  $\gamma$  on the errors in Figure 4. Figure 4a shows that increasing  $T$  reduces optimization error and CA direction distance but increases generalization error, aligning with Theorems 1, 2, and 3. Figure 4b shows that increasing  $\alpha$  leads to an initial decrease and subsequent increase
Table 3: Classification results on Office-31 dataset.
Method	Amazon	Webcam
Test Acc ↑	Test Acc ↑	Test Acc ↑	ΔA% ↓
Static	84.62 ± 0.71	94.43 ± 0.96	97.44 ± 1.20	2.56 ± 0.37
MGDA	79.45 ± 0.11	96.56 ± 1.20	97.89 ± 0.74	3.65 ± 0.64
MoDo	85.13 ± 0.58	95.41 ± 0.98	96.78 ± 0.65	2.26 ± 0.31
in PS optimization error and population risk. which aligns with Theorem 3 and (3.8). On the other hand, there is an overall increase in CA direction distance with  $\alpha$ , which aligns with Theorem 2. Figure 4c shows that increasing  $\gamma$  increases both the PS population and optimization errors but decreases CA direction distance. This matches our bounds for PS optimization error in Theorem 3, PS population risk in (3.8), and CA direction distance in Theorem 2.
This work studies the three-way trade-off in MOL – among optimization, generalization, and conflict avoidance. Our results show that, in the general nonconvex setting, the traditional trade-off between optimization and generalization depending on the number of iterations also exists in MOL. Moreover, dynamic weighting algorithms like MoDo introduce a new dimension of trade-off in terms of conflict avoidance compared to static weighting. We demonstrate that this three-way trade-off can be controlled by the step size for updating the dynamic weighting parameter and the number of iterations. Proper choice of these parameters can lead to decent performance on all three metrics.
Broader impacts and limitations
This work has a potential impact on designing dynamic weighting algorithms and choosing hyperparameters such as step sizes and number of iterations based on the trade-off for MTL applications such as multi-language translation, and multi-agent reinforcement learning. No ethical concerns arise from this work. A limitation of this study is that the theory focuses on a specific algorithm, MoDo, for smooth objectives in unconstrained learning. Future research could explore the theory of other algorithms for non-smooth objectives or constrained learning, which would be interesting directions to pursue.