SlowGuess/NeurIPS_2023_OCR huggingface dataset:
MGDA-variant methods find a direction  $d(\theta)$  (e.g.,  $d^{*} = g_{w_{\lambda}^{*}} + \lambda g_{0}$  with  $w_{\lambda}^{*}$  given by eq. (8)) that tends to optimize all objective functions simultaneously, which is called a conflict-avoidant (CA)
direction [14]. Thus, it is important to measure the distance between a stochastic direction estimate  $\widehat{d}(\theta)$  and the CA direction, which we call as CA distance, as defined below.
Definition 2.  $\| \mathbb{E}_{\widehat{d} (\theta)}[\widehat{d} (\theta)] - d(\theta)\|$  denotes the CA distance.
The following assumption imposes the Lipschitz continuity on the objectives and their gradients.
Assumption 1. For every task  $i \in [K]$ ,  $L_i(\theta)$  is  $l_i$ -Lipschitz continuous and  $\nabla L_i(\theta)$  is  $l_{i,1}$ -Lipschitz continuous for any  $\theta \in \mathbb{R}^m$ .
We next make an assumption on the bias and variance of the stochastic gradient  $g_{i}(\theta ;\xi)$
Assumption 2. For every task  $i \in [K]$ , the gradient  $g_i(\theta; \xi)$  is the unbiased estimate of  $g_i(\theta)$ . Moreover, the gradient variance is bounded by  $\mathbb{E}_{\xi}[\| g_i(\theta; \xi) - g_i(\theta)\|^2] \leq \sigma_i^2$ .
Assumption 3. Assume there exists a constant  $C_g > 0$  such that  $\| G(\theta)\| \leq C_g$ .
The bounded gradient condition in Assumption 3 is necessary to ensure the boundedness of the multi-gradient estimation error, as also adopted by [12, 13].
5.2 Convergence Analysis with Nonconvex Objectives
We first upper-bound the CA distance for our proposed method.
Proposition 1. Suppose Assumptions 1-3 are satisfied. If we choose  $\beta_{t,s} = c / \sqrt{s}$  and  $S > 1$ , then
$$
\| \mathbb {E} _ {\zeta , w _ {t, S} | \theta_ {t}} [ G (\theta_ {t}; \zeta) w _ {t, S} + \lambda g _ {0} (\theta_ {t}; \zeta) ] - G (\theta_ {t}) w _ {t, \lambda} ^ {*} - \lambda g _ {0} (\theta_ {t}) \| \leq \sqrt {\left(\frac {2}{c} + 2 c C _ {1}\right) \frac {2 + l o g (S)}{\sqrt {S}}},
$$
where  $C_1 = \mathcal{O}(K(1 + \lambda))$  and  $c$  is a constant.
Proposition 1 shows that CA distance is decreasing with the number  $S$  of iterations on  $w$  updates. Then, by selecting a properly large  $S$ , we obtain the following general convergence result.
Theorem 1 (SDMGrad). Suppose Assumption 1-3 are satisfied. Set  $\alpha_{t} = \alpha = \Theta((1 + \lambda)^{-1}K^{-\frac{1}{2}}T^{-\frac{1}{2}})$ ,  $\beta_{t,s} = c / \sqrt{s}$ , where  $c$  is a constant, and  $S = \Theta((1 + \lambda)^{-2}T^{2})$ . Then the outputs of the proposed SDMGrad algorithm satisfy
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| G (\theta_ {t}) w _ {t, \lambda} ^ {*} + \lambda g _ {0} (\theta_ {t}) \| ^ {2} ] = \widetilde {\mathcal {O}} ((1 + \lambda^ {2}) K ^ {\frac {1}{2}} T ^ {- \frac {1}{2}}), \tag {10}
$$
where  $\tilde{\mathcal{O}}$  omits the order of  $\log T$ .
Theorem 1 establishes a general convergence guarantee for SDMGrad along the multi-gradient direction  $d^{*} = g_{w_{\lambda}^{*}} + \lambda g_{0}$ . Building on Theorem 1, we next show that with different choices of the regularization parameter  $\lambda$ , two types of convergence results can be obtained in the following two corollaries.
Corollary 1 (Constant  $\lambda$ ). Under the same setting as in Theorem 1, choosing a constant-level  $\lambda > 0$  we have  $\frac{1}{T}\sum_{t=0}^{T-1}\mathbb{E}[\|G(\theta_t)w_t^*\|^2] = \widetilde{\mathcal{O}}(K^{\frac{1}{2}}T^{-\frac{1}{2}})$ , where  $w_t^* \in \arg \min_{w \in \mathcal{W}}\frac{1}{2}\|G(\theta_t)w\|^2$ . To achieve an  $\epsilon$ -accurate Pareto stationary point, each objective requires  $\mathcal{O}(K^3\epsilon^{-6})$  samples in  $\xi(\xi')$  and  $\mathcal{O}(K\epsilon^{-2})$  samples in  $\zeta$ , respectively. Meanwhile, the CA distance takes the order of  $\widetilde{\mathcal{O}}(\epsilon)$ .
Corollary 1 covers the MGDA case when  $\lambda = 0$ . In this setting, the sample complexity  $\mathcal{O}(\epsilon^{-6})$  improves those of MoCo [13] and MoDo [14] by an order of  $\epsilon^{-4}$  and  $\epsilon^{-2}$ , respectively, while achieving an  $\epsilon$ -level CA distance.
Corollary 2 (Increasing  $\lambda$ ). Under the same setting as in Theorem 1 and choosing  $\lambda = \Theta(T^{\frac{1}{2}})$ , we have  $\frac{1}{T}\sum_{t=0}^{T-1}\mathbb{E}[\|g_0(\theta_t)\|^2] = \mathcal{O}(K^{\frac{1}{2}}T^{-\frac{1}{2}})$ . To achieve an  $\epsilon$ -accurate stationary point, each objective requires  $\mathcal{O}(K^2\epsilon^{-4})$  samples in  $\xi(\xi')$  and  $\mathcal{O}(K\epsilon^{-2})$  samples in  $\zeta$ , respectively. Meanwhile, the CA distance takes the order of  $\widetilde{\mathcal{O}}(\sqrt{K})$ .
Corollary 2 analyzes the case with an increasing  $\lambda = \Theta (T^{\frac{1}{2}})$ . We show that SDMGrad converges to a stationary point of the objective  $L_{0}(\theta) = \sum_{i}\widetilde{w}_{i}L_{i}(\theta)$  with an improved sample complexity, but with a worse constant-level CA distance. This justifies the flexibility of our framework that the  $\lambda$  can balance the worst local improvement of individual objectives and the target objective  $L_{0}(\theta)$ .
Convergence under objective sampling. We provide a convergence analysis for SDMGrad-OS.
Theorem 2 (SDMGrad-OS). Suppose Assumption 1-3 are satisfied. Define  $\gamma = \frac{K}{n}$ ,  $\alpha_{t} = \alpha = \Theta((1 + \lambda^{2})^{-\frac{1}{2}}\gamma^{-\frac{1}{2}}K^{-\frac{1}{2}}T^{-\frac{1}{2}})$ ,  $\beta_{t,s} = c / \sqrt{s}$  where  $c$  is a constant, and  $S = \Theta((1 + \lambda^{2})^{-2}\gamma^{-2}T^{2})$ . Then, by choosing a constant  $\lambda$ , the iterates of the proposed SDMGrad-OS algorithm satisfy
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| G (\theta_ {t}) w _ {t} ^ {*} \| ^ {2} ] = \widetilde {\mathcal {O}} (K ^ {\frac {1}{2}} \gamma^ {\frac {1}{2}} T ^ {- \frac {1}{2}}).
$$
Theorem 2 establishes the convergence guarantee for our SDMGrad-OS algorithm, which achieves a per-objective sample complexity of  $\mathcal{O}(\epsilon^{-6})$  comparable to that of SDMGrad, but with a much better efficiency due to the objective sampling, as also validated by the empirical comparison in Table 4.
5.3 Lower sample complexity but constant-level CA distance
Without the requirement on the  $\epsilon$ -level CA distance, we further improve the sample complexity of our method to  $\mathcal{O}(\epsilon^{-2})$ , as shown in the following theorem.
Theorem 3. Suppose Assumptions 1-3 are satisfied. Set  $S = 1$ ,  $\alpha_{t} = \alpha = \Theta (K^{-\frac{1}{2}}T^{-\frac{1}{2}})$ ,  $\beta_{t} = \beta = \Theta (K^{-1}T^{-\frac{1}{2}})$  and  $\lambda$  as constant. The iterates of the proposed SDMGrad satisfy
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| G (\theta_ {t}) w _ {t} ^ {*} \| ^ {2} ] = \mathcal {O} (K T ^ {- \frac {1}{2}}).
$$
Theorem 3 shows that to achieve an  $\epsilon$ -accurate Pareto stationary point, our method requires  $T = \mathcal{O}(K^2\epsilon^{-2})$ . In this case, each objective requires a number  $\mathcal{O}(K^2\epsilon^{-2})$  of samples in  $\xi(\xi')$  and  $\zeta$ .
Convergence under objective sampling. We next analyze the convergence of SDMGrad-OS.
Theorem 4. Suppose Assumptions 1-3 are satisfied. Set  $S = 1$ ,  $\gamma = \frac{K}{n}$ ,  $\alpha_{t} = \alpha = \Theta(K^{-\frac{1}{2}}\gamma^{-\frac{1}{2}}T^{-\frac{1}{2}})$ ,  $\beta_{t} = \beta = \Theta(K^{-1}\gamma^{-1}T^{-\frac{1}{2}})$  and  $\lambda$  as a constant. The iterates of the proposed SDMGrad-OS algorithm satisfy
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| G (\theta_ {t}) w _ {t} ^ {*} \| ^ {2} ] = \mathcal {O} (K \gamma T ^ {- \frac {1}{2}}).
$$
Theorem 4 shows that to achieve an  $\epsilon$ -accurate Pareto stationary point, our algorithm requires  $T = \mathcal{O}(\gamma^2 K^2 \epsilon^{-2})$ , and each objective requires a number  $\mathcal{O}(\gamma^2 K^2 \epsilon^{-2})$  of samples in  $\xi(\xi')$  and  $\zeta$ .
In this section, we first describe the implementation details of our proposed methods. Then, we demonstrate the effectiveness of the methods under a couple of multi-task supervised learning and reinforcement settings. The experimental details and more empirical results such as the two-objective toy example, consistency with GD and MGDA, and ablation studies over  $\lambda$  can be found in the Appendix A.
6.1 Practical Implementation
Double sampling. In supervised learning, double sampling (i.e., drawing two samples simultaneously for gradient estimation) is employed, whereas in reinforcement learning, single sampling is used because double sampling requires to visit the entire episode twice a time, which is much more time-consuming.
Gradient normalization and rescale. During the training process, the gradient norms of tasks may change over time. Thus, directly solving the objective in eq. (8) may trigger numerical problems.
Inspired by CAGrad [9], we normalize the gradient of each task and rescale the final update  $d$  by multiplying a factor of  $\frac{1}{1 + \lambda}$  to stabilize the training.
Projected gradient descent. The computation of the projection to the probability simplex we use is the Euclidean projection proposed by [33], which involves solving a convex problem via quadratic programming. The implementation used in our experiments follows the repository in [34], which is very efficient in practice.
Method	Segmentation	Depth	MR ↓	Δm% ↓
mIoU ↑	Pix Acc ↑	Abs Err ↓	Rel Err ↓
STL	74.01	93.16	0.0125	27.77
LS	75.18	93.49	0.0155	46.77	8.50	22.60
SI	70.95	91.73	0.0161	33.83	11.50	14.11
RLW [35]	74.57	93.41	0.0158	47.79	11.25	24.38
DWA [36]	75.24	93.52	0.0160	44.37	8.50	21.45
UW [26]	72.02	92.85	0.0140	30.13	7.75	5.89
MGDA [7]	68.84	91.54	0.0309	33.50	12.00	44.14
PCGrad [8]	75.13	93.48	0.0154	42.07	8.75	18.29
GradDrop [29]	75.27	93.53	0.0157	47.54	7.75	23.73
CAGrad [9]	75.16	93.48	0.0141	37.60	7.25	11.64
IMTL-G [37]	75.33	93.49	0.0135	38.41	5.25	11.10
MoCo [13]	75.42	93.55	0.0149	34.19	4.00	9.90
MoDo [14]	74.55	93.32	0.0159	41.51	10.75	18.89
Nash-MTL [10]	75.41	93.66	0.0129	35.02	2.75	6.82
FAMO [38]	74.54	93.29	0.0145	32.59	7.75	8.13
SDMGrad	74.53	93.52	0.0137	34.01	6.25	7.79
Table 2: Multi-task supervised learning on Cityscapes dataset.
6.2 Supervised Learning
For the supervised learning setting, we evaluate the performance on the Cityscapes [39] and NYUv2 [40] datasets. The former dataset involves 2 pixel-wise tasks: 7-class semantic segmentation and depth estimation, and the latter one involves 3 pixel-wise tasks: 13-class semantic segmentation, depth estimation and surface normal estimation. Following the experimental setup of [9], we embed a MTL method MTAN [36] into our SDMGrad method, which builds on SegNet [41] and is empowered by a task-specific attention mechanism. We compare SDMGrad with Linear Scalarization (LS) which minimizes the average loss, Scale-invariant (SI) which minimizes the average logarithmic loss, RLW [35], DWA [36], UW [26], MGDA [7], PCGrad [8], GradDrop [29], CA-Grad [9], IMTL-G [37], MoCo [13], MoDo [14], Nash-MTL [10], and FAMO [38]. Following [42, 9, 13, 10, 38], we compute two metrics reflecting the overall performance: (1)  $\Delta \mathbf{m}\%$ , the average per-task performance drop versus the single-task (STL) baseline  $b$  to assess method  $m$ :  $\Delta m\% = \frac{1}{K} \sum_{k=1}^{K} (-1)^{l_k} (M_{m,k} - M_{b,k}) / M_{b,k} \times 100$ , where  $K$  is the number of metrics,  $M_{b,k}$  is the value of metric  $M_k$  obtained by baseline  $b$ , and  $M_{m,k}$  obtained by the compared method  $m$ .  $l_k = 1$  if the evaluation metric  $M_k$  on task  $k$  prefers a higher value and 0 otherwise. (2) Mean Rank (MR): the average rank of each method across all tasks.
We search the hyperparameter  $\lambda \in \{0.1, 0.2, \dots, 1.0\}$  for our SDMGrad method and report the results in Table 2 and Table 3. Each experiment is repeated 3 times with different random seeds and the average is reported. It can be seen that our proposed method is able to obtain better or comparable results than the baselines, and in addition, can strike a better performance balance on multiple tasks than other baselines. For example, although MGDA achieves better results on Surface Normal, it performs the worst on both Segmentation and Depth. As a comparison, our SDMGrad method can achieve more balanced results on all tasks.
6.3 Reinforcement Learning
For the reinforcement learning setting, we evaluate the performance on the MT10 benchmarks, which include 10 robot manipulation tasks under the Meta-World environment [43]. Following the experiment setup in [9, 13, 10], we adopt Soft Actor-Critic (SAC) [44] as the underlying training algorithm. We compare SDMGrad with Multi-task SAC [43], Multi-headed SAC [43], Multi-task SAC + Task Encoder [43], PCGrad [8], CAGrad [9], MoCo [13], Nash-MTL [10] and FAMO [38]. We search  $\lambda \in \{0.1, 0.2, \dots, 1.0\}$  and provide the success rate and average training time (in sec
Method	Segmentation	Depth	Surface Normal	MR ↓	Δm% ↓
mIoU ↑	Pix Acc ↑	Abs Err ↓	Rel Err ↓	Angle Distance ↓	Within t° ↑
Mean	Median	11.25	22.5	30
STL	38.30	63.76	0.6754	0.2780	25.01	19.21	30.14	57.20	69.15
LS	39.29	65.33	0.5493	0.2263	28.15	23.96	22.09	47.50	61.08	11.33	5.59
SI	38.45	64.27	0.5354	0.2201	27.60	23.37	22.53	48.57	62.32	10.33	4.39
RLW [35]	37.17	63.77	0.5759	0.2410	28.27	24.18	22.26	47.05	60.62	13.89	7.78
DWA[36]	39.11	65.31	0.5510	0.2285	27.61	23.18	24.17	50.18	62.39	10.11	3.57
UW [26]	36.87	63.17	0.5446	0.2260	27.04	22.61	23.54	49.05	63.65	9.89	4.05
MGDA[7]	30.47	59.90	0.6070	0.2555	24.88	19.45	29.18	56.88	69.36	7.33	1.38
PCGrad [8]	38.06	64.64	0.5550	0.2325	27.41	22.80	23.86	49.83	63.14	10.44	3.97
GradDrop [29]	39.39	65.12	0.5455	0.2279	27.48	22.96	23.38	49.44	62.87	9.44	3.58
CAGrad [9]	39.79	65.49	0.5486	0.2250	26.31	21.58	25.61	52.36	65.58	6.44	0.20
IMTL-G [37]	39.35	65.60	0.5426	0.2256	26.02	21.19	26.20	53.13	66.24	5.67	-0.76
MoCo [13]	40.30	66.07	0.5575	0.2135	26.67	21.83	25.61	51.78	64.85	6.33	0.16
MoDo [14]	35.28	62.62	0.5821	0.2405	25.65	20.33	28.04	54.86	67.37	8.89	0.49
Nash-MTL [10]	40.13	65.93	0.5261	0.2171	25.26	20.08	28.40	55.47	68.15	3.33	-4.04
FAMO [38]	38.88	64.90	0.5474	0.2194	25.06	19.57	29.21	56.61	68.98	4.22	-4.10
SDMGrad	40.47	65.90	0.5225	0.2084	25.07	19.99	28.54	55.74	68.53	2.33	-4.84
ons) per episode in Table 4. As shown, SDMGrad achieves the second best success rate among the compared baselines.
We also validate the efficiency of the proposed objective sampling strategy against other acceleration strategies including CAGrad-Fast [9], Nash-MTL with updating once per  $\{50, 100\}$  iterations [38]. Following [9], we choose the task sampling size  $n = 4$ . As shown in Table 4, our SDMGrad-OS with objective sampling achieves approximately  $1.4 \times$  speedup on MT10, while achieving a success rate comparable to that of SDMGrad. We also observe that although our SDMGrad-OS requires more time than CAGrad-Fast [9] and Nash-MTL [38] (every 100), it reaches a higher or comparable success rate.
Table 3: Multi-task supervised learning on NYU-v2 dataset.
Method	Metaworld MT10
success rate (mean ± stderr)	time
SAC STL (upper bound)	0.90 ± 0.03	-
Multi-task SAC [43]	0.49 ± 0.07	-
Multi-task SAC + Task Encoder [43]	0.54 ± 0.05	-
Multi-headed SAC [43]	0.61 ± 0.04	-
Nash-MTL* [10]	0.91 ± 0.03	-
PCGrad [8]	0.72 ± 0.02	11.6
CAGrad [9]	0.83 ± 0.05	13.5
MoCo [13]	0.75 ± 0.05	11.5
Nash-MTL [38]	0.80 ± 0.13	87.4
FAMO [38]	0.83 ± 0.05	4.2
SDMGrad	0.84 ± 0.10	13.6
CAGrad-Fast [9]	0.82 ± 0.04	8.6
Nash-MTL [38] (every 50)	0.76 ± 0.10	9.7
Nash-MTL [38] (every 100)	0.80 ± 0.12	9.3
SDMGrad-OS	0.82 ± 0.08	9.7
SDMGrad-OS (S=1)	0.80 ± 0.12	6.8
Table 4: Multi-task reinforcement learning on Metaworld MT10 benchmarks. Nash-MTL\* [10] denotes the results reported in the original paper. Nash-MTL [38] denotes the reproduced result.
In this paper, we propose a new and flexible direction-oriented multi-objective problem formulation, as well as two simple and efficient MOO algorithms named SDMGrad and SDMGrad-OS. We establish the convergence guarantee for both algorithms in various settings. Extensive experiments validate the promise of our methods. We anticipate that our new problem formulation and the proposed algorithms can be applied in other learning applications with multiple measures, and the analysis can be of independent interest in analyzing other stochastic MOO algorithms.
A.1 Toy Example
To demonstrate our proposed method can achieve better or comparable performance under stochastic settings, we provide an empirical study on the two-objective toy example used in CAGrad [9]. The two objectives  $L_{1}(x)$  and  $L_{2}(x)$  shown in Figure 1 are defined on  $x = (x_{1}, x_{2})^{\top} \in \mathbb{R}^{2}$ ,
$$
L _ {1} (x) = f _ {1} (x) g _ {1} (x) + f _ {2} (x) h _ {1} (x)
$$
$$
L _ {2} (x) = f _ {1} (x) g _ {2} (x) + f _ {2} (x) h _ {2} (x),
$$
where the functions are given by
$$
f _ {1} (x) = \max  (\tanh  (0. 5 x _ {2}), 0)
$$
$$
f _ {2} (x) = \max  \left(\tanh  (- 0. 5 x _ {2}), 0\right)
$$
$$
g _ {1} (x) = \log \left(\max  \left(| 0. 5 (- x _ {1} - 7) - \tanh  (- x _ {2}) |, 0. 0 0 0 0 0 5\right)\right) + 6
$$
$$
g _ {2} (x) = \log \left(\max  \left(| 0. 5 (- x _ {1} + 3) - \tanh  (- x _ {2}) + 2 |, 0. 0 0 0 0 0 5\right)\right) + 6
$$
$$
h _ {1} (x) = \left((- x _ {1} + 7) ^ {2} + 0. 1 (- x _ {1} - 8) ^ {2}\right) / 1 0 - 2 0
$$
$$
h _ {2} (x) = \left((- x _ {1} - 7) ^ {2} + 0. 1 (- x _ {1} - 8) ^ {2}\right) / 1 0 - 2 0.
$$
(a) Mean objective
(e) GD
(b) Objective 1
(f) PCGrad
(c) Objective 2
(g) CAGrad
(d) MGDA
(h) SDMGrad(Ours)
Figure 1: A two-objective toy example.
We choose 3 initializations
$$
x _ {0} \in \left\{(- 8. 5, 7. 5), (- 8. 5, 5), (9, 9) \right\}
$$
for different methods and visualize the optimization trajectories in Figure 1. The starting point of every trajectory in Figure 1d-Figure 1h is given by the  $\bullet$  symbol, and the color of every trajectory changes gradually from red to yellow. The gray line illustrates the Pareto front, and the  $\star$  symbol denotes the global optimum. To simulate the stochastic setting, we add zero-mean Gaussian noise to the gradient of each objective for all the methods except MGDA. We adopt Adam optimizer with learning rate of 0.002 and 70000 iterations for each run. As shown, GD can get stuck due to the dominant gradient of a specific objective, which stops progressing towards the Pareto front. PCGrad and CAGrad can also fail to converge to the Pareto front in certain circumstances.
A.2 Consistency Verification
We conduct the experiment on the multi-task classification dataset Multi-Fashion+MNIST [45]. Each image contained in this dataset is constructed by overlaying two images randomly sampled from MNIST [46] and FashionMNIST [47] respectively. We adopt shrunked Lenet [48] as the shared base-encoder and a task-specific linear classification head for each task. We report the training losses obtained from different methods over 3 independent runs in Figure 2. As illustrated, the performance of SDMGrad with large  $\lambda$  is similar to GD, and the performance when  $\lambda$  is small resembles MGDA. With properly tuned  $\lambda$ , lower average training loss can be obtained. Generally, the results confirm the consistency of our formulation with the direction-oriented principle.
Figure 2: Consistency verification on Multi-Fashion+MNIST dataset.
The Multi-Fashion+MNIST[45] includes images constructed from FashionMNIST[47] and MNIST[46]. First, select one image from each dataset randomly, then transform the two images into a single image with one put in the top-left corner and the other in bottom-right corner. The dataset contains 120000 training images and 20000 test images. We use SGD optimizer with learning rate 0.001 and train for 100 epochs with batch size 256. We use multi-step scheduler with scale factor 0.1 to decay learning rate every 15 epochs. The projected gradient descent is performed with learning rate of 10 and momentum of 0.5 and 20 gradient descent steps are applied.
A.3 Supervised Learning
We implement the methods based on the library released by [10]. Following [9, 13, 10], we train our method for 200 epochs, using Adam optimizer with learning rate 0.0001 for the first 100 epochs and 0.00005 for the rest. The batch size for Cityscapes and NYU-v2 are 8 and 2 respectively. We compute the averaged test performance over the last 10 epochs as final performance measure. The inner projected gradient descent is performed with learning rate of 10 and momentum of 0.5 and 20 gradient descent steps are applied. The experiments on Cityscapes and NYU-v2 are run on RTX 3090 and Tesla V100 GPU, respectively. We also report additional experiment results over different  $\lambda$  and  $S = 1$  in Table 5 and Table 6.
A.4 Reinforcement Learning
Following [9, 13, 10], we conduct the experiments based on MTRL codebase[49]. We train our method for 2 million steps with batch size of 1280. The inner projected gradient descent is performed with learning rate of 10 for MT10 benchmark and 20 gradient descent steps are applied. The method is evaluated once every 10000 steps and the best average test performance over 10 random seeds over the entire training process is reported. We search  $\lambda \in \{0.1, 0.2, \dots, 1.0\}$  for MT10 benchmark and the highest success rate is achieved when  $\lambda = 0.6$ . For our objective sampling strategy, the number of sampled objectives is a random variable obeying binomial distribution whose expectation is  $n$ .
Method	Segmentation	Depth	Δm% ↓
mIoU ↑	Pix Acc ↑	Abs Err ↓	Rel Err ↓
STL	74.01	93.16	0.0125	27.77
SDMGrad (λ = 0.1)	72.56	92.68	0.0156	40.89	18.65
SDMGrad (λ = 0.2)	74.79	93.30	0.0149	32.46	8.62
SDMGrad (λ = 0.3)	74.53	93.52	0.0137	34.01	7.79
SDMGrad (λ = 0.4)	75.10	93.48	0.0137	35.66	9.11
SDMGrad (λ = 0.5)	74.63	93.46	0.0131	38.99	11.09
SDMGrad (λ = 0.6)	74.42	93.22	0.0138	38.79	12.30
SDMGrad (λ = 0.7)	75.06	93.42	0.0158	39.98	17.24
SDMGrad (λ = 0.8)	74.99	93.40	0.0155	39.65	16.30
SDMGrad (λ = 0.9)	75.60	93.50	0.0134	43.52	15.39
SDMGrad (λ = 1.0)	74.50	93.47	0.0142	42.80	16.41
SDMGrad (λ = 10)	74.17	93.13	0.0154	41.77	18.36
SDMGrad (λ = 0.3, S = 1)	75.41	93.62	0.0139	38.83	12.22
Table 5: Additional supervised learning experiments on Cityscapes dataset.
Method	Segmentation	Depth	Surface Normal	Δm%↓
mIoU ↑	Pix Acc ↑	Abs Err ↓	Rel Err ↓	Angle Distance ↓	Within t° ↑
Mean	Median	11.25	22.5	30
STL	38.30	63.76	0.6754	0.2780	25.01	19.21	30.14	57.20	69.15
SDMGrad (λ = 0.1)	40.23	66.01	0.5360	0.2268	25.03	19.99	28.45	55.80	68.65	-3.86
SDMGrad (λ = 0.2)	39.23	65.67	0.5315	0.2189	25.13	20.02	28.12	55.71	68.46	-3.66
SDMGrad (λ = 0.3)	40.47	65.90	0.5225	0.2084	25.07	19.99	28.54	55.74	68.53	-4.84
SDMGrad (λ = 0.4)	40.68	66.53	0.5248	0.2199	25.21	20.01	27.69	55.72	68.58	-4.14
SDMGrad (λ = 0.5)	41.08	66.82	0.5184	0.2116	25.65	20.68	26.70	54.27	67.46	-3.33
SDMGrad (λ = 0.6)	41.20	66.86	0.5258	0.2175	25.85	21.03	26.47	53.51	66.82	-2.39
SDMGrad (λ = 0.7)	41.00	66.31	0.5224	0.2202	25.60	20.64	27.64	54.30	67.15	-3.16
SDMGrad (λ = 0.8)	39.88	66.13	0.5406	0.2266	26.20	21.57	25.67	52.33	65.65	-0.09
SDMGrad (λ = 0.9)	41.03	67.16	0.5314	0.2271	25.89	20.97	27.22	53.58	66.48	-2.17
SDMGrad (λ = 1.0)	39.94	66.27	0.5224	0.2155	26.51	21.95	25.15	51.54	64.94	-0.06
SDMGrad (λ = 10)	39.81	66.11	0.5352	0.2232	27.05	22.57	24.53	50.24	63.59	1.82
SDMGrad (λ = 0.3, S = 1)	39.63	65.43	0.5296	0.2140	25.66	20.83	27.18	53.93	67.05	-2.34
To compare with CAGrad-Fast[9], we choose  $n = 4$  for MT10 benchmark. We cite the reported success rates of all baseline methods in Table 4, but independently run each experiment 5 times to calculate the average running time. All experiments on MT10 are run on RTX 2080Ti GPU. We also report additional experiments results over  $S = 1$  on MT10 in Table 7.
Table 6: Additional supervised learning experiments on NYU-v2 dataset.
Method	Metaworld MT10
success (mean ± stderr)	time
SDMGrad	0.84 ± 0.10	13.6
SDMGrad (S=1)	0.83 ± 0.05	11.2
SDMGrad-OS	0.82 ± 0.08	9.7
SDMGrad-OS (S=1)	0.80 ± 0.12	6.8
Table 7: Additional reinforcement learning experiments on Metaworld MT10 benchmarks.
B Notations for Technical Proofs