Original: In Defense of the Unitary Scalarization for Deep Multi-Task Learning
Revised: In Defense of the Unitary Scalarization for Deep Multi-Task Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Recent multi-task learning research argues against unitary scalarization , wheretraining simply minimizes the sum of the task losses.
Revised: Recent multi-task learning research argues against unitary scalarization , wheretraining simply minimizes the sum of the task losses.

Original: Several ad-hoc multi-task optimization algorithms have instead been proposed, inspired by various hypothesesabout what makes multi-task settings difﬁcult.
Revised: Several ad-hoc multi-task optimization algorithms have instead been proposed, inspired by various hypothesesabout what makes multi-task settings difﬁcult.

Original: The majority of these optimizersrequire per-task gradients, and introduce signiﬁcant memory, runtime, and implementation overhead.
Revised: The majority of these optimizersrequire per-task gradients, and introduce signiﬁcant memory, runtime, and implementation overhead.

Original: We show that unitary scalarization, coupled with standardregularization and stabilization techniques from single-task learning, matches orimproves upon the performance of complex multi-task optimizers in popular supervised and reinforcement learning settings.
Revised: We show that unitary scalarization, coupled with standardregularization and stabilization techniques from single-task learning, matches orimproves upon the performance of complex multi-task optimizers in popular supervised and reinforcement learning settings.

Original: We then present an analysis suggestingthat many specialized multi-task optimizers can be partly interpreted as forms ofregularization, potentially explaining our surprising results.
Revised: We then present an analysis suggestingthat many specialized multi-task optimizers can be partly interpreted as forms ofregularization, potentially explaining our surprising results.

Original: We believe our resultscall for a critical reevaluation of recent research in the area.
Revised: We believe our resultscall for a critical reevaluation of recent research in the area.

Original:  Introduction
Revised:  Introduction

Original: Multi-Task Learning (MTL) [5] exploits similarities between tasks to yield models that are moreaccurate, generalize better and require less training data.
Revised: Multi-Task Learning (MTL) [5] exploits similarities between tasks to yield models that are moreaccurate, generalize better and require less training data.

Original: Owing to the success of MTL on traditionalmachine learning models [3, 16, 22] and of deep single-task learning across a variety of domains, agrowing body of research has focused on deep MTL.
Revised: Owing to the success of MTL on traditionalmachine learning models [3, 16, 22] and of deep single-task learning across a variety of domains, agrowing body of research has focused on deep MTL.

Original: The most straightforward way to train a neuralnetwork for multiple tasks at once is to minimize the sum of per-task losses.
Revised: The most straightforward way to train a neuralnetwork for multiple tasks at once is to minimize the sum of per-task losses.

Original: Adopting terminologyfrom multi-objective optimization, we call this approach unitary scalarization .
Revised: Adopting terminologyfrom multi-objective optimization, we call this approach unitary scalarization .

Original: While some work shows that multi-task networks trained via unitary scalarization exhibit superiorperformance to independent per-task models [28, 34], others suggest the opposite [29, 53, 57].
Revised: While some work shows that multi-task networks trained via unitary scalarization exhibit superiorperformance to independent per-task models [28, 34], others suggest the opposite [29, 53, 57].

Original: As a result, many explanations for the difﬁculty of MTL have been proposed, each motivating anew Specialized Multi-Task Optimizer (SMTO)
Revised: As a result, many explanations for the difﬁculty of MTL have been proposed, each motivating anew Specialized Multi-Task Optimizer (SMTO)

Original: These works typically claim thatthe proposed SMTO outperforms unitary scalarization, in addition to relevant prior work.
Revised: These works typically claim thatthe proposed SMTO outperforms unitary scalarization, in addition to relevant prior work.

Original: However,
Revised: However,

Original: SMTOs usually require access to per-task gradients either with respect to the shared parameters,or to the shared representation.
Revised: SMTOs usually require access to per-task gradients either with respect to the shared parameters,or to the shared representation.

Original: Therefore, their reported performance gain comes at signiﬁcantcomputation and memory cost, the overhead scaling linearly with the number of tasks.
Revised: Therefore, their reported performance gain comes at signiﬁcantcomputation and memory cost, the overhead scaling linearly with the number of tasks.

Original: By contrast,unitary scalarization requires only the average of the gradients across tasks, which can be computedvia a single backpropagation.
Revised: By contrast,unitary scalarization requires only the average of the gradients across tasks, which can be computedvia a single backpropagation.

Original: Existing SMTOs were introduced to solve challenges related to the optimization of the deep MTLproblem.
Revised: Existing SMTOs were introduced to solve challenges related to the optimization of the deep MTLproblem.

Original: We instead postulate that the reported weakness of unitary scalarization is linked toexperimental variability or to a lack of regularization, leading to the following contributions:
Revised: We instead postulate that the reported weakness of unitary scalarization is linked toexperimental variability or to a lack of regularization, leading to the following contributions:

Original: • A comprehensive experimental evaluation ( § 4) of recent SMTOs on popular multi-task benchmarks, showing that no SMTO consistently outperforms unitary scalarization in spite of the
Revised: • A comprehensive experimental evaluation ( § 4) of recent SMTOs on popular multi-task benchmarks, showing that no SMTO consistently outperforms unitary scalarization in spite of the

Original: added complexity and overhead.
Revised: added complexity and overhead.

Original: In particular, either the differences between unitary scalarization
Revised: In particular, either the differences between unitary scalarization

Original: and SMTOs are not statistically signiﬁcant, or they can be bridged by standard regularization
Revised: and SMTOs are not statistically signiﬁcant, or they can be bridged by standard regularization

Original: and stabilization techniques from the single-task literature.
Revised: and stabilization techniques from the single-task literature.

Original: Our reinforcement learning (RL)experiments include optimizers previously applied only to supervised learning.•
Revised: Our reinforcement learning (RL)experiments include optimizers previously applied only to supervised learning.•

Original: An empirical and technical analysis of the considered SMTOs, suggesting that they reduceoverﬁtting on the multi-task problem and hence act as regularizers ( § 5).
Revised: An empirical and technical analysis of the considered SMTOs, suggesting that they reduceoverﬁtting on the multi-task problem and hence act as regularizers ( § 5).

Original: We conduct an ablationstudy and provide a collection of novel and existing technical results that support this hypothesis.•
Revised: We conduct an ablationstudy and provide a collection of novel and existing technical results that support this hypothesis.•

Original: We will publicly release a uniﬁed PyTorch [49] codebase, currently provided in the supplementarymaterial, for all SMTOs in the experimental evaluation.
Revised: We will publicly release a uniﬁed PyTorch [49] codebase, currently provided in the supplementarymaterial, for all SMTOs in the experimental evaluation.

Original: We believe that our results suggest that the considered SMTOs can be often replaced
Revised: We believe that our results suggest that the considered SMTOs can be often replaced

Original: by lessexpensive techniques.
Revised: by lessexpensive techniques.

Original: We hope that
Revised: We hope that

Original: these
Revised: these

Original: surprising results stimulate the search for a deeper
Revised: surprising results stimulate the search for a deeper

Original: understanding of MTL
Revised: understanding of MTL

Original:  Related Work
Revised:  Related Work

Original: Before diving into details of speciﬁc SMTOs in Section 5, we provide a high-level overview of thedeep MTL research.
Revised: Before diving into details of speciﬁc SMTOs in Section 5, we provide a high-level overview of thedeep MTL research.

Original: MTL is represented by a vast body of literature, including Caruana’s pioneeringwork [6] exploring hard parameter sharing , i.e., sharing neural network parameters between all taskswith, possibly, a separate part of the model for each task.
Revised: MTL is represented by a vast body of literature, including Caruana’s pioneeringwork [6] exploring hard parameter sharing , i.e., sharing neural network parameters between all taskswith, possibly, a separate part of the model for each task.

Original: Hard parameter sharing is still the major
Revised: Hard parameter sharing is still the major

Original: MTL approach adopted in natural language processing [9, 12], computer vision [45], and speechrecognition [52].
Revised: MTL approach adopted in natural language processing [9, 12], computer vision [45], and speechrecognition [52].

Original: In this work, we implicitly assume that each parameter update employs informationfrom all tasks.
Revised: In this work, we implicitly assume that each parameter update employs informationfrom all tasks.

Original: However, not all works satisfy this assumption, either due to a large number oftasks [4, 35], or simply as an implementation decision [25, 36].
Revised: However, not all works satisfy this assumption, either due to a large number oftasks [4, 35], or simply as an implementation decision [25, 36].

Original: In this setting, MTL resembles otherproblems dealing with multiple tasks, i.e., continual [31], curriculum [46], and meta-learning [24],which are not the focus of this work.
Revised: In this setting, MTL resembles otherproblems dealing with multiple tasks, i.e., continual [31], curriculum [46], and meta-learning [24],which are not the focus of this work.

Original: Many works strive to improve the performance of deep multi-task models.
Revised: Many works strive to improve the performance of deep multi-task models.

Original: One line of researchhypothesizes that conﬂicting per-task gradient directions lead to suboptimal models, and thus focuseson explicitly removing conﬂicts across task gradients [11, 27, 40, 41, 61, 64].
Revised: One line of researchhypothesizes that conﬂicting per-task gradient directions lead to suboptimal models, and thus focuseson explicitly removing conﬂicts across task gradients [11, 27, 40, 41, 61, 64].

Original: Some authors postulatethat loss imbalances across tasks hinder learning, proposing loss reweighting methods [10, 29, 39].
Revised: Some authors postulatethat loss imbalances across tasks hinder learning, proposing loss reweighting methods [10, 29, 39].

Original: Finally, Sener and Koltun [53] and Navon et al.
Revised: Finally, Sener and Koltun [53] and Navon et al.

Original: [47] propose that tasks compete for model capacityand interpret MTL as multi-objective optimization in order to cope with inter-task competition.
Revised: [47] propose that tasks compete for model capacityand interpret MTL as multi-objective optimization in order to cope with inter-task competition.

Original: Here, we focus on algorithms that explicitly rely on per-task gradients to try to outperform unitaryscalarization ( § 5).
Revised: Here, we focus on algorithms that explicitly rely on per-task gradients to try to outperform unitaryscalarization ( § 5).

Original: Research on multi-task architectures [19, 45] or MTL algorithms exclusivelymotivated by deterministic loss reweighting [18, 29, 42] are orthogonal to our work.
Revised: Research on multi-task architectures [19, 45] or MTL algorithms exclusivelymotivated by deterministic loss reweighting [18, 29, 42] are orthogonal to our work.

Original: Both topics areinvestigated by a recent survey on pixel-level multi-task computer vision problems [60], which foundthat the minimization of tuned weighted sums of losses is empirically competitive with deterministicloss reweighting and MGDA in the considered settings.
Revised: Both topics areinvestigated by a recent survey on pixel-level multi-task computer vision problems [60], which foundthat the minimization of tuned weighted sums of losses is empirically competitive with deterministicloss reweighting and MGDA in the considered settings.

Original: In addition to the common supervised settings, we also consider multi-task RL, whose research canbe grouped into three categories: the ﬁrst adds auxiliary tasks providing additional inductive biases tospeed up learning [26] on a target task.
Revised: In addition to the common supervised settings, we also consider multi-task RL, whose research canbe grouped into three categories: the ﬁrst adds auxiliary tasks providing additional inductive biases tospeed up learning [26] on a target task.

Original: The second, based on policy distillation, uses per-task teachermodels to provide labels for a multi-task model or per-task policies as regularizers [48, 50, 56].
Revised: The second, based on policy distillation, uses per-task teachermodels to provide labels for a multi-task model or per-task policies as regularizers [48, 50, 56].

Original: Thethird directly learns a shared policy [28], possibly via an SMTO [64].
Revised: Thethird directly learns a shared policy [28], possibly via an SMTO [64].

Original: We focus on the third category,whose literature reports varying performance for unitary scalarization (better [28] or worse [64] thanper-task models), indicating confounding factors in evaluation pipelines and further motivating ourwork.
Revised: We focus on the third category,whose literature reports varying performance for unitary scalarization (better [28] or worse [64] thanper-task models), indicating confounding factors in evaluation pipelines and further motivating ourwork.

Original: PopArt [23, 59] performs scale-invariant value function updates in order to address differencesin returns across environments, showing improvements in the multi-task setting while still usingunitary scalarization.
Revised: PopArt [23, 59] performs scale-invariant value function updates in order to address differencesin returns across environments, showing improvements in the multi-task setting while still usingunitary scalarization.

Original: PopArt does not require per-task gradients but introduces additional hyperparameters.
Revised: PopArt does not require per-task gradients but introduces additional hyperparameters.

Original: In our work, we address the differences in rewards by normalizing them at the replay bufferlevel.
Revised: In our work, we address the differences in rewards by normalizing them at the replay bufferlevel.

Original: However, we believe both unitary scalarization and SMTOs might equally beneﬁt from PopArt.
Revised: However, we believe both unitary scalarization and SMTOs might equally beneﬁt from PopArt.

Original:  Multi-Task Learning Optimizers
Revised:  Multi-Task Learning Optimizers

Original: We will now describe the deep MTL training problem and common algorithms employed for itssolution.
Revised: We will now describe the deep MTL training problem and common algorithms employed for itssolution.

Original: Let ( X, Y ) ∈ R
Revised: Let ( X, Y ) ∈ R

Original: d × n × R o × n be the training set, composed of n d -dimensional points ando -dimensional labels.
Revised: d × n × R o × n be the training set, composed of n d -dimensional points ando -dimensional labels.

Original: In addition, L i : R o × n × R o × n → R denotes the loss for the i -th task, θ ∈ R Sthe parameter space, T := { 1 , . . .
Revised: In addition, L i : R o × n × R o × n → R denotes the loss for the i -th task, θ ∈ R Sthe parameter space, T := { 1 , . . .

Original: , m } the set of m tasks.
Revised: , m } the set of m tasks.

Original: The goal of MTL is to learn a single(generally task-aware) parametrized model f : R S × R d × n × T → R o × n that performs well on all tasks T .
Revised: The goal of MTL is to learn a single(generally task-aware) parametrized model f : R S × R d × n × T → R o × n that performs well on all tasks T .

Original: The parameter space is often split into a set of shared parameters across tasks (generally themajority of the architecture), denoted θ (cid:107) , and (possibly empty) task-speciﬁc parameters, denoted θ ⊥ ,so that θ :=
Revised: The parameter space is often split into a set of shared parameters across tasks (generally themajority of the architecture), denoted θ (cid:107) , and (possibly empty) task-speciﬁc parameters, denoted θ ⊥ ,so that θ :=

Original: [ θ (cid:107) , θ ⊥ ] T .
Revised: [ θ (cid:107) , θ ⊥ ] T .

Original: In this context, the model f often takes on an encoder-decoder architecture,where the encoder g learns a shared representation across tasks, and the decoders h
Revised: In this context, the model f often takes on an encoder-decoder architecture,where the encoder g learns a shared representation across tasks, and the decoders h

Original: i are task-speciﬁcpredictive heads: f ( θ , X, i )
Revised: i are task-speciﬁcpredictive heads: f ( θ , X, i )

Original: =
Revised: =

Original: h i ( g ( θ (cid:107) , X ) , θ ⊥ ) .
Revised: h i ( g ( θ (cid:107) , X ) , θ ⊥ ) .

Original: In this case, we denote by z = g ( θ (cid:107) , X ) ∈ R r
Revised: In this case, we denote by z = g ( θ (cid:107) , X ) ∈ R r

Original: ×
Revised: ×

Original: nthe r -dimensional shared representation of X .
Revised: nthe r -dimensional shared representation of X .

Original: The training problem for MTL is typically formulated as the sum of the per-task losses [11, 53, 64]:
Revised: The training problem for MTL is typically formulated as the sum of the per-task losses [11, 53, 64]:

Original: [Equation]
Revised: [Equation]

Original: Unitary Scalarization
Revised: Unitary Scalarization

Original: The obvious way to minimize the multi-task training objective in equation(1) is to rely on a standard gradient-based algorithm.
Revised: The obvious way to minimize the multi-task training objective in equation(1) is to rely on a standard gradient-based algorithm.

Original: While, for simplicity, we focus on standardgradient descent rather than mini-batch stochastic gradient descent, the notation can be adapted byreplacing the dataset size n by the mini-batch size b .
Revised: While, for simplicity, we focus on standardgradient descent rather than mini-batch stochastic gradient descent, the notation can be adapted byreplacing the dataset size n by the mini-batch size b .

Original: Equation (1) corresponds to a linear scalarizationwith unitary weights under a multi-objective interpretation of MTL; hence, we call the directapplication of gradient descent on equation (1) unitary scalarization .
Revised: Equation (1) corresponds to a linear scalarizationwith unitary weights under a multi-objective interpretation of MTL; hence, we call the directapplication of gradient descent on equation (1) unitary scalarization .

Original: For vanilla gradient descent, thiscorresponds to taking a step in the opposite direction as the one given by the sum of per-task gradients:∇ θ L MT =
Revised: For vanilla gradient descent, thiscorresponds to taking a step in the opposite direction as the one given by the sum of per-task gradients:∇ θ L MT =

Original: (cid:80) i ∈T ∇ θ L i .
Revised: (cid:80) i ∈T ∇ θ L i .

Original: Per-task gradients are not required, as it sufﬁces to directly compute thegradient of the sum L MT .
Revised: Per-task gradients are not required, as it sufﬁces to directly compute thegradient of the sum L MT .

Original: Hence, when relying on deep learning frameworks based on reverse-modedifferentiation, such as PyTorch [49], the backward pass is performed once per iteration (rather than
Revised: Hence, when relying on deep learning frameworks based on reverse-modedifferentiation, such as PyTorch [49], the backward pass is performed once per iteration (rather than

Original: [Figure]
Revised: [Figure]

Original: m times).
Revised: m times).

Original: Furthermore, the memory cost is a factor m less than most SMTOs, which require accessto each ∇ θ L i .
Revised: Furthermore, the memory cost is a factor m less than most SMTOs, which require accessto each ∇ θ L i .

Original: As a consequence, unitary scalarization is simple, fast, and memory efﬁcient.
Revised: As a consequence, unitary scalarization is simple, fast, and memory efﬁcient.

Original: Ourexperiments demonstrate that, when possibly coupled with single-task regularization such as earlystopping, (cid:96) 2 penalty or dropout layers [55], this simple optimizer is strongly competitive with SMTOs.
Revised: Ourexperiments demonstrate that, when possibly coupled with single-task regularization such as earlystopping, (cid:96) 2 penalty or dropout layers [55], this simple optimizer is strongly competitive with SMTOs.

Original: MGDA Sener and Koltun [53] point out that equation (1) can be cast as a multi-objective optimization problem with the following objective: L MT ( θ ) :=
Revised: MGDA Sener and Koltun [53] point out that equation (1) can be cast as a multi-objective optimization problem with the following objective: L MT ( θ ) :=

Original: [ L 1 ( θ ) , . . .
Revised: [ L 1 ( θ ) , . . .

Original: , L m ( θ )] T .
Revised: , L m ( θ )] T .

Original: A commonlyemployed solution concept in multi-objective optimization is Pareto optimality.
Revised: A commonlyemployed solution concept in multi-objective optimization is Pareto optimality.

Original: A point θ ∗ iscalled Pareto-optimal if, for any another point θ † such that ∃ i ∈ T : L i ( θ † ) <
Revised: A point θ ∗ iscalled Pareto-optimal if, for any another point θ † such that ∃ i ∈ T : L i ( θ † ) <

Original: L i ( θ ∗ ) , then∃ j ∈ T : L j ( θ † ) >
Revised: L i ( θ ∗ ) , then∃ j ∈ T : L j ( θ † ) >

Original: L j ( θ ∗ ) .
Revised: L j ( θ ∗ ) .

Original: Sener and Koltun [53] rely on Multiple-Gradient Descent Algorithm(MGDA) [14] to reach a Pareto-stationary point for shared parameters θ (cid:107) .
Revised: Sener and Koltun [53] rely on Multiple-Gradient Descent Algorithm(MGDA) [14] to reach a Pareto-stationary point for shared parameters θ (cid:107) .

Original: MGDA takes steps ina direction that decreases the loss of all tasks at once [14, 17], which can be found by solving thefollowing optimization problem:
Revised: MGDA takes steps ina direction that decreases the loss of all tasks at once [14, 17], which can be found by solving thefollowing optimization problem:

Original: [Equation]
Revised: [Equation]

Original: whose dual takes the following form (corresponding to the formulation from Désidéri [14]):
Revised: whose dual takes the following form (corresponding to the formulation from Désidéri [14]):

Original: [Equation]
Revised: [Equation]

Original: In other words, MGDA takes a step in a direction g given by the negative convex combinationof per-task gradients, whose coefﬁcients are given by solving equation (3).
Revised: In other words, MGDA takes a step in a direction g given by the negative convex combinationof per-task gradients, whose coefﬁcients are given by solving equation (3).

Original: In practice, per-taskgradients are rescaled before applying MGDA:
Revised: In practice, per-taskgradients are rescaled before applying MGDA:

Original: the original authors’ implementation [53] relieson ∇ θ (cid:107) L i ← ∇ θ (cid:107) L i / (cid:13)(cid:13)(cid:13) ∇ θ (cid:107) L i (cid:13)(cid:13)(cid:13)
Revised: the original authors’ implementation [53] relieson ∇ θ (cid:107) L i ← ∇ θ (cid:107) L i / (cid:13)(cid:13)(cid:13) ∇ θ (cid:107) L i (cid:13)(cid:13)(cid:13)

Original: L i ( θ ) .
Revised: L i ( θ ) .

Original: The convergence of MGDA to a Pareto-optimal point is stillguaranteed after normalization [14].
Revised: The convergence of MGDA to a Pareto-optimal point is stillguaranteed after normalization [14].

Original: IMTL Impartial Multi-Task Learning (IMTL) [41] is presented as an SMTO that is not biasedagainst any single task.
Revised: IMTL Impartial Multi-Task Learning (IMTL) [41] is presented as an SMTO that is not biasedagainst any single task.

Original: It is composed of two complementary algorithmic blocks: IMTL-L, acting ontask losses, and IMTL-G, acting on per-task gradients.
Revised: It is composed of two complementary algorithmic blocks: IMTL-L, acting ontask losses, and IMTL-G, acting on per-task gradients.

Original: IMTL-G follows the intuition that a multi-taskoptimizer should proceed along a direction g = − (cid:80) i
Revised: IMTL-G follows the intuition that a multi-taskoptimizer should proceed along a direction g = − (cid:80) i

Original: α i ∇ θ (cid:107) L i that equally represents per-taskgradients.
Revised: α i ∇ θ (cid:107) L i that equally represents per-taskgradients.

Original: This is formulated analytically by requiring that the cosine similarity between g and each∇ θ (cid:107) L i be the same.
Revised: This is formulated analytically by requiring that the cosine similarity between g and each∇ θ (cid:107) L i be the same.

Original: To prevent the resulting problem from being underdetermined, Liu et al.
Revised: To prevent the resulting problem from being underdetermined, Liu et al.

Original: [41]add the constraint (cid:80) i ∈T α i = 1 , resulting in a problem that admits a closed-form solution for g :
Revised: [41]add the constraint (cid:80) i ∈T α i = 1 , resulting in a problem that admits a closed-form solution for g :

Original: [Equation]
Revised: [Equation]

Original: IMTL-L, instead, aims to reweight task losses so that they are all constant over time, and equalto 1 .
Revised: IMTL-L, instead, aims to reweight task losses so that they are all constant over time, and equalto 1 .

Original: In order to limit oscillations of the scaling factors, the authors propose to learn them jointly with the network by minimizing a common objective via gradient descent.
Revised: In order to limit oscillations of the scaling factors, the authors propose to learn them jointly with the network by minimizing a common objective via gradient descent.

Original: In particular, givens i ∈ R ∀ i ∈ T , Liu et al.
Revised: In particular, givens i ∈ R ∀ i ∈ T , Liu et al.

Original: [41] derive the following form for the joint minimization problem:min s , θ [ (cid:80) i ( e s i
Revised: [41] derive the following form for the joint minimization problem:min s , θ [ (cid:80) i ( e s i

Original: L i ( θ ) − s i )] .
Revised: L i ( θ ) − s i )] .

Original: As proved by Liu et al.
Revised: As proved by Liu et al.

Original: [41], IMTL-L only has a rescaling effect onthe update direction of IMTL-G.
Revised: [41], IMTL-L only has a rescaling effect onthe update direction of IMTL-G.

Original: Unlike IMTL-G and the other SMTOs presented in this section,
Revised: Unlike IMTL-G and the other SMTOs presented in this section,

Original: IMTL-L rescaling is designed to affect the updates for task-speciﬁc parameters θ ⊥ as well.
Revised: IMTL-L rescaling is designed to affect the updates for task-speciﬁc parameters θ ⊥ as well.

Original: PCGrad Let us write cos( x , z ) for the cosine similarity between vectors x and z .
Revised: PCGrad Let us write cos( x , z ) for the cosine similarity between vectors x and z .

Original: Yu et al.
Revised: Yu et al.

Original: [64]postulate that multi-task convergence is severely slowed down if the following three conditions(named the tragic triad ) hold at once: (i) conﬂicting gradient directions: cos( ∇ θ (cid:107) L i , ∇ θ (cid:107) L j ) <
Revised: [64]postulate that multi-task convergence is severely slowed down if the following three conditions(named the tragic triad ) hold at once: (i) conﬂicting gradient directions: cos( ∇ θ (cid:107) L i , ∇ θ (cid:107) L j ) <

Original: 0for some i, j ∈ T ; (ii) differing gradient magnitudes: (cid:13)(cid:13) ∇ θ (cid:107) L i (cid:13)(cid:13)
Revised: 0for some i, j ∈ T ; (ii) differing gradient magnitudes: (cid:13)(cid:13) ∇ θ (cid:107) L i (cid:13)(cid:13)

Original: (cid:29) (cid:13)(cid:13) ∇ θ (cid:107) L j (cid:13)(cid:13) for some i, j ∈ T ; and(iii) the unitary scalarization L MT has high curvature along ∇ θ (cid:107) L MT .
Revised: (cid:29) (cid:13)(cid:13) ∇ θ (cid:107) L j (cid:13)(cid:13) for some i, j ∈ T ; and(iii) the unitary scalarization L MT has high curvature along ∇ θ (cid:107) L MT .

Original: The PCGrad [64] SMTO ispresented as a solution to the tragic triad, targeted at the ﬁrst condition.
Revised: The PCGrad [64] SMTO ispresented as a solution to the tragic triad, targeted at the ﬁrst condition.

Original: Consistent with the previoussections, let us denote the update direction by g .
Revised: Consistent with the previoussections, let us denote the update direction by g .

Original: Furthermore, let [ x ] + := max( x , 0 ) .
Revised: Furthermore, let [ x ] + := max( x , 0 ) .

Original: Given per-taskgradients ∇ θ (cid:107) L i , PCGrad iteratively projects each task gradient onto the normal plane of all thegradients with which it conﬂicts:
Revised: Given per-taskgradients ∇ θ (cid:107) L i , PCGrad iteratively projects each task gradient onto the normal plane of all thegradients with which it conﬂicts:

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: x ) (cid:13)(cid:13)(cid:13)
Revised: x ) (cid:13)(cid:13)(cid:13)

Original: [Table]
Revised: [Table]

Original: (cid:107) L j (cid:13)(cid:13)(cid:13)(cid:35)
Revised: (cid:107) L j (cid:13)(cid:13)(cid:13)(cid:35)

Original: + ∇ θ (cid:107) L j ∀ j ∈ T \ { i } (cid:35) ∀ i ∈ T , g = − (cid:88)
Revised: + ∇ θ (cid:107) L j ∀ j ∈ T \ { i } (cid:35) ∀ i ∈ T , g = − (cid:88)

Original: i ∈T g i , (5) where the iterative updates of g i with respect to ∇ θ (cid:107) L j are performed in random order.
Revised: i ∈T g i , (5) where the iterative updates of g i with respect to ∇ θ (cid:107) L j are performed in random order.

Original: GradDrop Chen et al.
Revised: GradDrop Chen et al.

Original: [11] focus on conﬂicting signs across task gradient entries, arguing thatsuch conﬂicts lead to gradient “tug-of-wars".
Revised: [11] focus on conﬂicting signs across task gradient entries, arguing thatsuch conﬂicts lead to gradient “tug-of-wars".

Original: The GradDrop SMTO [11], presented as a solution tothis problem, proposes to randomly mask per-task gradients ∇ θ (cid:107)
Revised: The GradDrop SMTO [11], presented as a solution tothis problem, proposes to randomly mask per-task gradients ∇ θ (cid:107)

Original: L
Revised: L

Original: i so as to minimize such conﬂicts.
Revised: i so as to minimize such conﬂicts.

Original: Speciﬁcally, GradDrop computes the “positive sign purity" p j for the task gradient’s j -th entry andthen masks the j -th entry of each per-task gradient with probability increasing with p j , if the entry isnegative, or decreasing with p j , if the entry is positive.
Revised: Speciﬁcally, GradDrop computes the “positive sign purity" p j for the task gradient’s j -th entry andthen masks the j -th entry of each per-task gradient with probability increasing with p j , if the entry isnegative, or decreasing with p j , if the entry is positive.

Original: Let us write p :=
Revised: Let us write p :=

Original: [ p 1 , . .
Revised: [ p 1 , . .

Original: . , p S ] , where S is thedimensionality of the parameter space (see § 3), (cid:12) for the Hadamard product and 1 a for the indicatorvector on condition a .
Revised: . , p S ] , where S is thedimensionality of the parameter space (see § 3), (cid:12) for the Hadamard product and 1 a for the indicatorvector on condition a .

Original: Given a vector u i , uniformly sampled in [ 0 , 1 ] at each iteration, GradDroptakes a step in the direction given by:
Revised: Given a vector u i , uniformly sampled in [ 0 , 1 ] at each iteration, GradDroptakes a step in the direction given by:

Original: [Equation]
Revised: [Equation]

Original: ) 
Revised: ) 

Original:  , with p = 1(cid:32) 1 + (cid:80) i ∈T ∇ θ (cid:107) L i (cid:80) i ∈T (cid:12)(cid:12) ∇ θ (cid:107) L i (cid:12)(cid:12) (cid:33) .
Revised:  , with p = 1(cid:32) 1 + (cid:80) i ∈T ∇ θ (cid:107) L i (cid:80) i ∈T (cid:12)(cid:12) ∇ θ (cid:107) L i (cid:12)(cid:12) (cid:33) .

Original:  Experimental Evaluation
Revised:  Experimental Evaluation

Original: Relying on a uniﬁed experimental pipeline, we present an empirical evaluation on common MTLbenchmarks of unitary scalarization ( § 3), of the popular SMTOs presented in § 3, and of the recent
Revised: Relying on a uniﬁed experimental pipeline, we present an empirical evaluation on common MTLbenchmarks of unitary scalarization ( § 3), of the popular SMTOs presented in § 3, and of the recent

Original: RLW algorithms [39] due to their similarities with PCGrad and GradDrop (see § 5.2).
Revised: RLW algorithms [39] due to their similarities with PCGrad and GradDrop (see § 5.2).

Original: In particular,we benchmark against the two RLW instances that showed the best average performance in theoriginal paper: RLW with weights sampled from a Dirichlet distribution (“RLW Diri.”), and RLWwith weights sampled from a Normal distribution (“RLW Norm.”).
Revised: In particular,we benchmark against the two RLW instances that showed the best average performance in theoriginal paper: RLW with weights sampled from a Dirichlet distribution (“RLW Diri.”), and RLWwith weights sampled from a Normal distribution (“RLW Norm.”).

Original: Whenever appropriate, we employ“Unit.
Revised: Whenever appropriate, we employ“Unit.

Original: Scal.”
Revised: Scal.”

Original: as shorthand for unitary scalarization.
Revised: as shorthand for unitary scalarization.

Original: We ﬁrst present supervised learning experiments( § 4.1), and then evaluate on a popular reinforcement learning benchmark ( § 4.2).
Revised: We ﬁrst present supervised learning experiments( § 4.1), and then evaluate on a popular reinforcement learning benchmark ( § 4.2).

Original: Our experiments indicate that the performance of unitary scalarization has been consistently underestimated in the literature.
Revised: Our experiments indicate that the performance of unitary scalarization has been consistently underestimated in the literature.

Original: By showing the variability between runs and by relying on standardregularization and stabilization techniques from the single-task literature, we demonstrate that no
Revised: By showing the variability between runs and by relying on standardregularization and stabilization techniques from the single-task literature, we demonstrate that no

Original: SMTO consistently outperforms unitary scalarization across the considered settings .
Revised: SMTO consistently outperforms unitary scalarization across the considered settings .

Original: This resultholds in spite of the added complexity and computational overhead associated with most SMTOs.
Revised: This resultholds in spite of the added complexity and computational overhead associated with most SMTOs.

Original: Furthermore, in supervised learning, most methods drive the training loss of all tasks in the proximityof the respective global minima.
Revised: Furthermore, in supervised learning, most methods drive the training loss of all tasks in the proximityof the respective global minima.

Original: This suggests that the main difﬁculty of MTL is not associated withthe optimization of its training objective, but rather to incorporating adequate regularization (cf.
Revised: This suggests that the main difﬁculty of MTL is not associated withthe optimization of its training objective, but rather to incorporating adequate regularization (cf.

Original: 4.1 Supervised Learning
Revised: 4.1 Supervised Learning

Original: All the architectures employed in the supervised learning experiments conform to the encoder-decoderstructure detailed in § 3.
Revised: All the architectures employed in the supervised learning experiments conform to the encoder-decoderstructure detailed in § 3.

Original: Whenever suggested by the original authors for this context, the SMTO
Revised: Whenever suggested by the original authors for this context, the SMTO

Original: Unit.
Revised: Unit.

Original: Scal.
Revised: Scal.

Original: IMTL
Revised: IMTL

Original: MGDA
Revised: MGDA

Original: GradDrop
Revised: GradDrop

Original: PCGrad
Revised: PCGrad

Original: RLW Diri.
Revised: RLW Diri.

Original: RLW Norm.
Revised: RLW Norm.

Original: A v e r a g e T a s k A cc u r a c
Revised: A v e r a g e T a s k A cc u r a c

Original: y
Revised: y

Original: (a) Avg. task test accuracy: mean and 95 % CI (10 runs).
Revised: (a) Avg. task test accuracy: mean and 95 % CI (10 runs).

Original: Unit.
Revised: Unit.

Original: Scal.
Revised: Scal.

Original: IMTL
Revised: IMTL

Original: MGDA
Revised: MGDA

Original: GradDrop
Revised: GradDrop

Original: PCGrad
Revised: PCGrad

Original: RLW Diri.
Revised: RLW Diri.

Original: RLW Norm.
Revised: RLW Norm.

Original: E p o c h r un t i m e
Revised: E p o c h r un t i m e

Original: [ s ] implementations rely on per-task gradients with respect to the last shared activation, ∇ z , ratherthan on the usually more expensive ∇ θ L i .
Revised: [ s ] implementations rely on per-task gradients with respect to the last shared activation, ∇ z , ratherthan on the usually more expensive ∇ θ L i .

Original: In particular, this is the case for MGDA, IMTL and
Revised: In particular, this is the case for MGDA, IMTL and

Original: GradDrop.
Revised: GradDrop.

Original: See appendix B for details concerning each individual algorithm.
Revised: See appendix B for details concerning each individual algorithm.

Original: Surprisingly, several
Revised: Surprisingly, several

Original: MTL works [11, 39, 41, 64] report validation results, making it easier to overﬁt.
Revised: MTL works [11, 39, 41, 64] report validation results, making it easier to overﬁt.

Original: Instead, followingstandard machine learning practice, we select a model on the validation set, and later report testmetrics for all benchmarks.
Revised: Instead, followingstandard machine learning practice, we select a model on the validation set, and later report testmetrics for all benchmarks.

Original: Validation results are also available in appendix D. Appendix C.1 reportsdataset descriptions, the computational setup, hyperparameter and tuning details.
Revised: Validation results are also available in appendix D. Appendix C.1 reportsdataset descriptions, the computational setup, hyperparameter and tuning details.

Original: 4.1.1 Multi-MNIST
Revised: 4.1.1 Multi-MNIST

Original: We present results on the Multi-MNIST [53] dataset, a simple two-task supervised learning benchmark.
Revised: We present results on the Multi-MNIST [53] dataset, a simple two-task supervised learning benchmark.

Original: We employ a popular architecture from previous work [53, 64] (see appendix C.1), where a singledropout layer [55] (with dropout probability 0 . 5 ) is employed in both the encoder and the decoder.
Revised: We employ a popular architecture from previous work [53, 64] (see appendix C.1), where a singledropout layer [55] (with dropout probability 0 . 5 ) is employed in both the encoder and the decoder.

Original: (cid:96) 2regularization did not improve validation performance and was therefore omitted.
Revised: (cid:96) 2regularization did not improve validation performance and was therefore omitted.

Original: Figure 1 reports theaverage task test accuracy, and the training time per epoch.
Revised: Figure 1 reports theaverage task test accuracy, and the training time per epoch.

Original: For each run, the test model was selectedas the model with the largest average task validation accuracy across the training epochs.
Revised: For each run, the test model was selectedas the model with the largest average task validation accuracy across the training epochs.

Original: Appendix Dpresents the results of Figure 1 in tabular form, as well as the average task validation accuracy perepoch.
Revised: Appendix Dpresents the results of Figure 1 in tabular form, as well as the average task validation accuracy perepoch.

Original: As seen from the overlapping conﬁdence intervals, none of the considered algorithms clearlyoutperforms the others.
Revised: As seen from the overlapping conﬁdence intervals, none of the considered algorithms clearlyoutperforms the others.

Original: However, GradDrop displays higher experimental variability.
Revised: However, GradDrop displays higher experimental variability.

Original: Furthermore,
Revised: Furthermore,

Original: Figure 6(b) shows that the sums of the task cross-entropy losses is driven nearly to zero by most methods.
Revised: Figure 6(b) shows that the sums of the task cross-entropy losses is driven nearly to zero by most methods.

Original: Finally, Figure 1(b) shows that unitary scalarization also has among the lowest training times.
Revised: Finally, Figure 1(b) shows that unitary scalarization also has among the lowest training times.

Original: 4.1.2 CelebA
Revised: 4.1.2 CelebA

Original: We now show results for the CelebA [43] dataset, a challenging 40 -task multi-label classiﬁcation problem.
Revised: We now show results for the CelebA [43] dataset, a challenging 40 -task multi-label classiﬁcation problem.

Original: We employ the same architecture as many previous studies [39, 41, 53, 64] (see appendix C.1).
Revised: We employ the same architecture as many previous studies [39, 41, 53, 64] (see appendix C.1).

Original: We tuned (cid:96) 2 regularization terms λ for all SMTOs in the following grid: λ ∈ { 0 , 10 − 4 , 10 − 3 } .
Revised: We tuned (cid:96) 2 regularization terms λ for all SMTOs in the following grid: λ ∈ { 0 , 10 − 4 , 10 − 3 } .

Original: Thebest validation performance was attained with λ = 10 − 3 for unitary scalarization, IMTL and PCGrad,and with λ = 10 − 4 for MGDA, GradDrop, and RLW.
Revised: Thebest validation performance was attained with λ = 10 − 3 for unitary scalarization, IMTL and PCGrad,and with λ = 10 − 4 for MGDA, GradDrop, and RLW.

Original: Validation performance was further stabilizedby the addition of several dropout layers (see Figure 5), with dropout probabilities from 0 .
Revised: Validation performance was further stabilizedby the addition of several dropout layers (see Figure 5), with dropout probabilities from 0 .

Original: 25 to 0 .
Revised: 25 to 0 .

Original: We present an ablation study on the effect of regularization on this experiment in § 5.1.
Revised: We present an ablation study on the effect of regularization on this experiment in § 5.1.

Original: Figure 9(appendix D.2) shows that regularization improves the peak average validation performance for allthe considered methods.
Revised: Figure 9(appendix D.2) shows that regularization improves the peak average validation performance for allthe considered methods.

Original: Analogously to our Multi-MNIST results, Figure 2 plots the distribution ofthe training time per epoch, and the average test task accuracy.
Revised: Analogously to our Multi-MNIST results, Figure 2 plots the distribution ofthe training time per epoch, and the average test task accuracy.

Original: As with Multi-MNIST, the test modelfor each run was the one with maximal average validation task accuracy across epochs.
Revised: As with Multi-MNIST, the test modelfor each run was the one with maximal average validation task accuracy across epochs.

Original: In otherwords, if the peak is attained before the last epoch, we perform early stopping: as shown in Figure7(a) in appendix D this is the case for most methods.
Revised: In otherwords, if the peak is attained before the last epoch, we perform early stopping: as shown in Figure7(a) in appendix D this is the case for most methods.

Original: Due to the large number of tasks, Figure 2(b)shows relatively large runtime differences across methods.
Revised: Due to the large number of tasks, Figure 2(b)shows relatively large runtime differences across methods.

Original: PCGrad is the slowest (roughly 35 timesslower than unitary scalarization).
Revised: PCGrad is the slowest (roughly 35 timesslower than unitary scalarization).

Original: In fact, amongst the considered algorithms, it is the only one thatcomputes per-task gradients over the parameters ( ∇ θ L i ∀ i ∈ T ) at each iteration.
Revised: In fact, amongst the considered algorithms, it is the only one thatcomputes per-task gradients over the parameters ( ∇ θ L i ∀ i ∈ T ) at each iteration.

Original: GradDrop, MGDAand IMTL have overhead factors (compared to unitary scalarization) ranging from roughly 1 . 

Revised: GradDrop, MGDAand IMTL have overhead factors (compared to unitary scalarization) ranging from roughly 1 .

Original: to
Revised: to

Original: [Figure]
Revised: [Figure]

Original: v
Revised: v

Original: [Figure]
Revised: [Figure]

Original: r a g e T a
Revised: r a g e T a

Original: [Figure]
Revised: [Figure]

Original: k A cc u r a c y
Revised: k A cc u r a c y

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: which proves to be the most cost-effective
Revised: which proves to be the most cost-effective

Original: algorithm
Revised: algorithm

Original: 4 due to the relatively small size of z for the employed architecture.
Revised: 4 due to the relatively small size of z for the employed architecture.

Original: The overhead of RLW isnegligible: roughly 5% .
Revised: The overhead of RLW isnegligible: roughly 5% .

Original: Nevertheless, due to largely overlapping conﬁdence intervals in Figure 2(a),none of the methods consistently outperforms unitary scalarization.
Revised: Nevertheless, due to largely overlapping conﬁdence intervals in Figure 2(a),none of the methods consistently outperforms unitary scalarization.

Original: In fact, owing to our adoption ofexplicit regularization techniques (see § 5.1) its average performance is superior to that reported in theliterature [41, 53].
Revised: In fact, owing to our adoption ofexplicit regularization techniques (see § 5.1) its average performance is superior to that reported in theliterature [41, 53].

Original: As with Multi-MNIST, Figure 8(a) demonstrates that the cross-entropy loss ofeach task can be driven near to its global optimum by most optimizers.
Revised: As with Multi-MNIST, Figure 8(a) demonstrates that the cross-entropy loss ofeach task can be driven near to its global optimum by most optimizers.

Original: 4.1.3 Cityscapes
Revised: 4.1.3 Cityscapes

Original: In order to complement the multi-task classiﬁcation experiments for Multi-MNIST and CelebA,we present results for Cityscapes [13], a dataset for semantic understanding of urban street scenes.
Revised: In order to complement the multi-task classiﬁcation experiments for Multi-MNIST and CelebA,we present results for Cityscapes [13], a dataset for semantic understanding of urban street scenes.

Original: We rely on a common encoder architecture from the literature [39, 41] (see appendix C.1), a singledropout layer is contained in the task-speciﬁc heads [39].
Revised: We rely on a common encoder architecture from the literature [39, 41] (see appendix C.1), a singledropout layer is contained in the task-speciﬁc heads [39].

Original: As for CelebA, unitary scalarization, IMTL,and PCGrad beneﬁt from more regularization than the other optimizers: we employ λ = 10 − 5 forthese three algorithms, as it resulted in better validation performance on the majority of metrics,and λ = 0 for the remaining methods.
Revised: As for CelebA, unitary scalarization, IMTL,and PCGrad beneﬁt from more regularization than the other optimizers: we employ λ = 10 − 5 forthese three algorithms, as it resulted in better validation performance on the majority of metrics,and λ = 0 for the remaining methods.

Original: Figure 3 shows test results for two metrics per task, andthe distribution of the training time per epoch.
Revised: Figure 3 shows test results for two metrics per task, andthe distribution of the training time per epoch.

Original: The test model for each run and each metric wasselected as the one associated to the best (maximal or minimal, depending on the metric) validationresult across epochs.
Revised: The test model for each run and each metric wasselected as the one associated to the best (maximal or minimal, depending on the metric) validationresult across epochs.

Original: We therefore perform per-run early stopping for most methods (see Figure 17).
Revised: We therefore perform per-run early stopping for most methods (see Figure 17).

Original: As with Multi-MNIST and CelebA, no training algorithm clearly outperforms unitary scalarization(signiﬁcant overlaps across conﬁdence intervals exist), which is again the least expensive method.
Revised: As with Multi-MNIST and CelebA, no training algorithm clearly outperforms unitary scalarization(signiﬁcant overlaps across conﬁdence intervals exist), which is again the least expensive method.

Original: Incontrast with a popular hypothesis [10, 29, 41], this holds in spite of relatively large loss imbalances.
Revised: Incontrast with a popular hypothesis [10, 29, 41], this holds in spite of relatively large loss imbalances.

Original: In fact, the loss for the depth task is roughly 10 times smaller than that of the segmentation task:see ﬁgures 17(f)-17(g).
Revised: In fact, the loss for the depth task is roughly 10 times smaller than that of the segmentation task:see ﬁgures 17(f)-17(g).

Original: Nevertheless, both losses are rapidly driven towards their respective globalminima.
Revised: Nevertheless, both losses are rapidly driven towards their respective globalminima.

Original: Unlike CelebA (see Figure 2(b)), IMTL, MGDA and GradDrop are signiﬁcantly slower thanunitary scalarization (factors from 1 . 6 to 2 . 3 ), due to the relatively (compared to the parameter space)large size of z in the employed architecture.
Revised: Unlike CelebA (see Figure 2(b)), IMTL, MGDA and GradDrop are signiﬁcantly slower thanunitary scalarization (factors from 1 . 6 to 2 . 3 ), due to the relatively (compared to the parameter space)large size of z in the employed architecture.

Original: PCGrad, instead, appears to be less expensive ( 30%more than the baseline), demonstrating the beneﬁts of working on ∇ θ L i on this model.
Revised: PCGrad, instead, appears to be less expensive ( 30%more than the baseline), demonstrating the beneﬁts of working on ∇ θ L i on this model.

Original: 4.2 Reinforcement Learning
Revised: 4.2 Reinforcement Learning

Original: For RL experiments, we use Meta-World [63] and the Soft Actor-Critic [20] implementation from [54].
Revised: For RL experiments, we use Meta-World [63] and the Soft Actor-Critic [20] implementation from [54].

Original: Unlike § 4.1, the employed network architecture (see appendix C.1) is fully shared across tasks.
Revised: Unlike § 4.1, the employed network architecture (see appendix C.1) is fully shared across tasks.

Original: Therefore, all SMTO implementations for these experiments rely on per-task gradients with respectto network parameters ∇ θ L i (see § 5).
Revised: Therefore, all SMTO implementations for these experiments rely on per-task gradients with respectto network parameters ∇ θ L i (see § 5).

Original: Among the SMTOs we consider, PCGrad is the only onedeveloped with the RL setting in mind.
Revised: Among the SMTOs we consider, PCGrad is the only onedeveloped with the RL setting in mind.

Original: For fairness and completeness, we add all the other SMTOsfrom the supervised learning experiments, and are the ﬁrst to test these optimizers in the RL setting.
Revised: For fairness and completeness, we add all the other SMTOsfrom the supervised learning experiments, and are the ﬁrst to test these optimizers in the RL setting.

Original: To stabilize learning, we increase the replay buffer size, a well known technique in single-task RL,add actor l 2 regularization, and modify the reward normalization employed by Sodhani et al.
Revised: To stabilize learning, we increase the replay buffer size, a well known technique in single-task RL,add actor l 2 regularization, and modify the reward normalization employed by Sodhani et al.

Original: The unitary scalarization performance reported by Yu et al.
Revised: The unitary scalarization performance reported by Yu et al.

Original: [64] is considerably lower than thatof Sodhani et al.
Revised: [64] is considerably lower than thatof Sodhani et al.

Original: [54], which we believe is due to the lack of reward normalization in the former.
Revised: [54], which we believe is due to the lack of reward normalization in the former.

Original: Sodhani et al.
Revised: Sodhani et al.

Original: [54] keep a moving average of rewards in the environment, with a hyperparameter
Revised: [54] keep a moving average of rewards in the environment, with a hyperparameter

Original: [Figure]
Revised: [Figure]

Original: A v e r a g
Revised: A v e r a g

Original: [Figure]
Revised: [Figure]

Original: u cc e ss r a t e
Revised: u cc e ss r a t e

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: repetitions)
Revised: repetitions)

Original: [Figure]
Revised: [Figure]

Original: A v e r a g
Revised: A v e r a g

Original: e s u cc e ss r a t e
Revised: e s u cc e ss r a t e

Original: [Figure]
Revised: [Figure]

Original: controlling the speed of the moving average.
Revised: controlling the speed of the moving average.

Original: As we show in Figure 16, the learning algorithm issensitive to that hyperparameter.
Revised: As we show in Figure 16, the learning algorithm issensitive to that hyperparameter.

Original: Moreover, such normalization might make similar transitions havedrastically different rewards stored in the replay buffer.
Revised: Moreover, such normalization might make similar transitions havedrastically different rewards stored in the replay buffer.

Original: To alleviate these issues, we store the rawrewards in the buffer, and normalize only when a mini-batch is sampled.
Revised: To alleviate these issues, we store the rawrewards in the buffer, and normalize only when a mini-batch is sampled.

Original: Figure 4 reports the best average success rate across the updates and the runtime for 10,000 updates.
Revised: Figure 4 reports the best average success rate across the updates and the runtime for 10,000 updates.

Original: In addition to these summary statistics, reported for consistency with § 4.1, the learning curves areshown in appendix E. Our MT10 (10 tasks) results in Figure 4(a) show that by stabilizing the baselineusing standard RL techniques, unitary scalarization performs on par with other SMTOs, mirroringour ﬁndings in § 4.1.
Revised: In addition to these summary statistics, reported for consistency with § 4.1, the learning curves areshown in appendix E. Our MT10 (10 tasks) results in Figure 4(a) show that by stabilizing the baselineusing standard RL techniques, unitary scalarization performs on par with other SMTOs, mirroringour ﬁndings in § 4.1.

Original: This is in contrast with the previous literature, which reported that PCGradoutperforms unitary scalarization [54, 64].
Revised: This is in contrast with the previous literature, which reported that PCGradoutperforms unitary scalarization [54, 64].

Original: Figure 4(b) presents results on MT50 (50 tasks): similarlyto MT10, none of the SMTOs signiﬁcantly outperforms unitary scalarization, with PCGrad’s averagebeing slightly above unitary scalarization.
Revised: Figure 4(b) presents results on MT50 (50 tasks): similarlyto MT10, none of the SMTOs signiﬁcantly outperforms unitary scalarization, with PCGrad’s averagebeing slightly above unitary scalarization.

Original: We speculate that the stochastic loss rescaling performedby PCGrad
Revised: We speculate that the stochastic loss rescaling performedby PCGrad

Original: (see Proposition 3) reduces the differences in task return scales, and expect that methodslike PopArt [59] would have a similar effect without requiring access to per-task gradients.
Revised: (see Proposition 3) reduces the differences in task return scales, and expect that methodslike PopArt [59] would have a similar effect without requiring access to per-task gradients.

Original: While wedid not tune hyperparameters for MT50 (we employed those found for MT10), it would be mucheasier to do that for unitary scalarization due to its lower runtime.
Revised: While wedid not tune hyperparameters for MT50 (we employed those found for MT10), it would be mucheasier to do that for unitary scalarization due to its lower runtime.

Original: In fact, Figure 4(d) shows thata single unitary scalarization run takes roughly 15 hours, whereas PCGrad, MGDA and GradDroprequire more than a week.
Revised: In fact, Figure 4(d) shows thata single unitary scalarization run takes roughly 15 hours, whereas PCGrad, MGDA and GradDroprequire more than a week.

Original: Similarly to MT10, actor regularization pushes the average performanceof unitary scalarization higher (see in appendix E.2).
Revised: Similarly to MT10, actor regularization pushes the average performanceof unitary scalarization higher (see in appendix E.2).

Original: Overall, as in the supervised learning setting,unitary scalarization performs comparably to SMTOs despite being simpler and less demanding inboth memory and compute.
Revised: Overall, as in the supervised learning setting,unitary scalarization performs comparably to SMTOs despite being simpler and less demanding inboth memory and compute.

Original: IMTL was unstable on this RL benchmark and all of the runs crasheddue to numerical overﬂow.
Revised: IMTL was unstable on this RL benchmark and all of the runs crasheddue to numerical overﬂow.

Original: We hence omit IMTL results from the main body of the paper and showits results in Figure 13 in appendix E, which also describes a possible explanation.
Revised: We hence omit IMTL results from the main body of the paper and showits results in Figure 13 in appendix E, which also describes a possible explanation.

Original: We hypothesizethat the instability of IMTL is due to lack of bounds on scaling coefﬁcients.
Revised: We hypothesizethat the instability of IMTL is due to lack of bounds on scaling coefﬁcients.

Original: See appendix C.2 forhyperparameter settings and ablation studies.
Revised: See appendix C.2 forhyperparameter settings and ablation studies.

Original:  Regularization in Specialized Multi-Task Optimizers
Revised:  Regularization in Specialized Multi-Task Optimizers

Original: The empirical results presented in § 4 motivate the need to carefully analyze existing SMTOs.
Revised: The empirical results presented in § 4 motivate the need to carefully analyze existing SMTOs.

Original: Wemake an initial attempt in this direction by viewing their effects through the lens of regularization.
Revised: Wemake an initial attempt in this direction by viewing their effects through the lens of regularization.

Original: Let us deﬁne a regularizer as a technique to reduce overﬁtting [15].
Revised: Let us deﬁne a regularizer as a technique to reduce overﬁtting [15].

Original: We ﬁrst show that the SMTOsconsidered in § 4 empirically act as regularizers via an ablation study ( § 5.1).
Revised: We ﬁrst show that the SMTOsconsidered in § 4 empirically act as regularizers via an ablation study ( § 5.1).

Original: We then take a closerlook at their behavior, presenting technical results that support their alternative interpretation asregularizers ( § 5.2).
Revised: We then take a closerlook at their behavior, presenting technical results that support their alternative interpretation asregularizers ( § 5.2).

Original: Unless otherwise stated, we assume that MTL methods apply only to θ (cid:107) and thatstandard gradient-based updates are employed for tasks-speciﬁc parameters θ ⊥ .
Revised: Unless otherwise stated, we assume that MTL methods apply only to θ (cid:107) and thatstandard gradient-based updates are employed for tasks-speciﬁc parameters θ ⊥ .

Original: We furthermoreadopt the following shorthands: L i ( θ ) for L i ( f ( θ , X, i ) , Y ) , and ∇ θ L i for ∇ θ L i ( f ( θ , X, i ) , Y ) .
Revised: We furthermoreadopt the following shorthands: L i ( θ ) for L i ( f ( θ , X, i ) , Y ) , and ∇ θ L i for ∇ θ L i ( f ( θ , X, i ) , Y ) .

Original: 5.1 Ablation Study
Revised: 5.1 Ablation Study

Original: We repeat the experiment from § 4.1.2 and remove explicit regularization: no dropout layers areadded to the encoder-decoder architecture, and λ = 0 for all optimizers.
Revised: We repeat the experiment from § 4.1.2 and remove explicit regularization: no dropout layers areadded to the encoder-decoder architecture, and λ = 0 for all optimizers.

Original: In addition, we examine thebehavior of two different (cid:96) 2 -regularized instances of unitary scalarization: λ = 10 − 4 for “Unit.
Revised: In addition, we examine thebehavior of two different (cid:96) 2 -regularized instances of unitary scalarization: λ = 10 − 4 for “Unit.

Original: Scal.(cid:96) 2 ”, λ = 2 × 10 − 3 for “Unit.
Revised: Scal.(cid:96) 2 ”, λ = 2 × 10 − 3 for “Unit.

Original: Scal.
Revised: Scal.

Original: (cid:96) 2 + ”.
Revised: (cid:96) 2 + ”.

Original: Figure 5 shows that SMTOs behave similarly to an (cid:96) 2 - penalized unitary scalarization.
Revised: Figure 5 shows that SMTOs behave similarly to an (cid:96) 2 - penalized unitary scalarization.

Original: Importantly, SMTOs delay overﬁtting, requiring less early stoppingcompared to unitary scalarization to obtain comparable performance.
Revised: Importantly, SMTOs delay overﬁtting, requiring less early stoppingcompared to unitary scalarization to obtain comparable performance.

Original: In other words, early stoppingis sufﬁcient for unitary scalarization to perform on par with SMTOs.
Revised: In other words, early stoppingis sufﬁcient for unitary scalarization to perform on par with SMTOs.

Original: Finally, overﬁtting is furtherreduced by “Unit.
Revised: Finally, overﬁtting is furtherreduced by “Unit.

Original: Scal.
Revised: Scal.

Original: Reg.”, which plots the regularized unitary scalarization from § 4.1.2, withdropout layers and a weight decay of λ = 10 − 3 .
Revised: Reg.”, which plots the regularized unitary scalarization from § 4.1.2, withdropout layers and a weight decay of λ = 10 − 3 .

Original: Further results are presented in appendices D.2, D.3.
Revised: Further results are presented in appendices D.2, D.3.

Original: 5.2 Technical Results
Revised: 5.2 Technical Results

Original: [Figure]
Revised: [Figure]

Original: postpone the onset of overﬁtting, mirroring the effect of (cid:96) 2 regularization on unitary scalarization.
Revised: postpone the onset of overﬁtting, mirroring the effect of (cid:96) 2 regularization on unitary scalarization.

Original: All the methods considered in § 5.1 regularizemore than unitary scalarization.
Revised: All the methods considered in § 5.1 regularizemore than unitary scalarization.

Original: While RLWwas shown to reduce overﬁtting by the originalauthors [39, theorem 2], we now provide acollection of novel and existing technicalresults that potentially explain the regularizingbehavior of each of the other algorithms,complementing the presentation from § 3.
Revised: While RLWwas shown to reduce overﬁtting by the originalauthors [39, theorem 2], we now provide acollection of novel and existing technicalresults that potentially explain the regularizingbehavior of each of the other algorithms,complementing the presentation from § 3.

Original: Inparticular, we show that MGDA, IMTL and
Revised: Inparticular, we show that MGDA, IMTL and

Original: PCGrad have a larger convergence set thanunitary scalarization, reducing the chances toland on sharp local minima [15].
Revised: PCGrad have a larger convergence set thanunitary scalarization, reducing the chances toland on sharp local minima [15].

Original: Furthermore,
Revised: Furthermore,

Original: GradDrop and PCGrad introduce signiﬁcant stochasticity, which is often linked to the sameeffect [30, 33].
Revised: GradDrop and PCGrad introduce signiﬁcant stochasticity, which is often linked to the sameeffect [30, 33].

Original: We hope these observations will steer further research.
Revised: We hope these observations will steer further research.

Original: MGDA Let us denote the convex hull of a set A by Conv ( A ) .
Revised: MGDA Let us denote the convex hull of a set A by Conv ( A ) .

Original: We now recall a well-known propertyof MGDA [14] and relate it to the behavior of unitary scalarization.
Revised: We now recall a well-known propertyof MGDA [14] and relate it to the behavior of unitary scalarization.

Original: Proposition 1.
Revised: Proposition 1.

Original: The MGDA SMTO [53] converges to a superset of the convergence points of unitaryscalarization.
Revised: The MGDA SMTO [53] converges to a superset of the convergence points of unitaryscalarization.

Original: More speciﬁcally, it converges to any point θ ∗ (cid:107)
Revised: More speciﬁcally, it converges to any point θ ∗ (cid:107)

Original: such that: 0 ∈ Conv ( {∇ θ ∗ (cid:107) L i | i ∈ T } ) .
Revised: such that: 0 ∈ Conv ( {∇ θ ∗ (cid:107) L i | i ∈ T } ) .

Original: See appendix B.1 for a simple proof.
Revised: See appendix B.1 for a simple proof.

Original: As a consequence of Proposition 1, MGDA does not necessarilyreach a stationary point for L MT (that is, a point for which (cid:80) i ∈T ∇ θ (cid:107) L i = 0 ) or for any of the losses
Revised: As a consequence of Proposition 1, MGDA does not necessarilyreach a stationary point for L MT (that is, a point for which (cid:80) i ∈T ∇ θ (cid:107) L i = 0 ) or for any of the losses

Original: L i ( ∇ θ (cid:107) L i = 0 ).
Revised: L i ( ∇ θ (cid:107) L i = 0 ).

Original: For example, any point θ (cid:107) for which two per-task gradients point in oppositedirections is Pareto stationary.
Revised: For example, any point θ (cid:107) for which two per-task gradients point in oppositedirections is Pareto stationary.

Original: On account of the well-known [15] relationship between underoptimizing (e.g., early stopping [7, 38]) and overﬁtting, proposition 1 supports the interpretation of
Revised: On account of the well-known [15] relationship between underoptimizing (e.g., early stopping [7, 38]) and overﬁtting, proposition 1 supports the interpretation of

Original: MGDA as a regularizer for equation (1).
Revised: MGDA as a regularizer for equation (1).

Original: Empirical evidence that MGDA under-optimizes is providedin Figure 8(a), appendix D.3, and Figure 5, which shows over-regularization.
Revised: Empirical evidence that MGDA under-optimizes is providedin Figure 8(a), appendix D.3, and Figure 5, which shows over-regularization.

Original: Proposition 1 can betrivially extended to the recent Nash-MTL, which shares the same convergence set [47, Theorem 5.4]
Revised: Proposition 1 can betrivially extended to the recent Nash-MTL, which shares the same convergence set [47, Theorem 5.4]

Original: IMTL We now show that aggregating per-task gradients so that their cosine similarity is the same(equation (4)) yields a constrained steepest-descent algorithm (Proposition 2).
Revised: IMTL We now show that aggregating per-task gradients so that their cosine similarity is the same(equation (4)) yields a constrained steepest-descent algorithm (Proposition 2).

Original: This view on theupdate step of IMTL leads to a novel analysis of its convergence points (corollary 1).
Revised: This view on theupdate step of IMTL leads to a novel analysis of its convergence points (corollary 1).

Original: Proofs can befound in appendix B.2.
Revised: Proofs can befound in appendix B.2.

Original: We will denote by Aff ( A ) the afﬁne hull of a set A .
Revised: We will denote by Aff ( A ) the afﬁne hull of a set A .

Original: Proposition 2.
Revised: Proposition 2.

Original: IMTL by Liu et al.
Revised: IMTL by Liu et al.

Original: [41] updates θ (cid:107) by taking a step in the steepest descent directionwhose cosine similarity with per-task gradients is the same across tasks.
Revised: [41] updates θ (cid:107) by taking a step in the steepest descent directionwhose cosine similarity with per-task gradients is the same across tasks.

Original: Corollary 1.
Revised: Corollary 1.

Original: IMTL by Liu et al.
Revised: IMTL by Liu et al.

Original: [41] converges to a superset of the Pareto-optimal points for θ (cid:107)(and hence of the convergence points of the unitary scalarization).
Revised: [41] converges to a superset of the Pareto-optimal points for θ (cid:107)(and hence of the convergence points of the unitary scalarization).

Original: More speciﬁcally, it converges toany point θ ∗ (cid:107) such that:
Revised: More speciﬁcally, it converges toany point θ ∗ (cid:107) such that:

Original: [Equation]
Revised: [Equation]

Original: As seen for MGDA, corollary 1 implies that, even if the employed model f has the capacity toreach the minimal loss on L MT , IMTL may stop before reaching a stationary point.
Revised: As seen for MGDA, corollary 1 implies that, even if the employed model f has the capacity toreach the minimal loss on L MT , IMTL may stop before reaching a stationary point.

Original: Recalling therelationship between under-optimizing and overﬁtting [15], this supports the interpretation of IMTLas a regularizer for equation (1).
Revised: Recalling therelationship between under-optimizing and overﬁtting [15], this supports the interpretation of IMTLas a regularizer for equation (1).

Original: This is empirically shown in Figures 5, 8(a), 11.
Revised: This is empirically shown in Figures 5, 8(a), 11.

Original: In particular, unitaryscalarization reaches the same average performance of IMTL but requires earlier stopping.
Revised: In particular, unitaryscalarization reaches the same average performance of IMTL but requires earlier stopping.

Original: PCGrad
Revised: PCGrad

Original: We provide an alternative characterization of the PCGrad update rule, highlighting itsstochasticity in the context of its interpretation as loss rescaling [39, 41].
Revised: We provide an alternative characterization of the PCGrad update rule, highlighting itsstochasticity in the context of its interpretation as loss rescaling [39, 41].

Original: See appendix B.3 for a proof.338 Proposition 3.
Revised: See appendix B.3 for a proof.338 Proposition 3.

Original: PCGrad is equivalent to a dynamic, and possibly stochastic, loss rescaling for θ (cid:107) .
Revised: PCGrad is equivalent to a dynamic, and possibly stochastic, loss rescaling for θ (cid:107) .

Original: At each iteration, per-task gradients are rescaled as follows:
Revised: At each iteration, per-task gradients are rescaled as follows:

Original: [Equation]
Revised: [Equation]

Original: (cid:21) .
Revised: (cid:21) .

Original: Furthermore, if |T | > 2 , d ji is a random variable, and the above range contains its support.
Revised: Furthermore, if |T | > 2 , d ji is a random variable, and the above range contains its support.

Original: The results from proposition 3 can be easily extended to GradVac [61], which generalizes PCGrad’sprojection onto the normal vector to arbitrary target cosine similarities between per-task gradients.
Revised: The results from proposition 3 can be easily extended to GradVac [61], which generalizes PCGrad’sprojection onto the normal vector to arbitrary target cosine similarities between per-task gradients.

Original: When |T | > 2 , PCGrad corresponds to a stochastic loss re-weighting.
Revised: When |T | > 2 , PCGrad corresponds to a stochastic loss re-weighting.

Original: As such, PCGrad bears manysimilarities with Random Loss Weighting (RLW) [39].
Revised: As such, PCGrad bears manysimilarities with Random Loss Weighting (RLW) [39].

Original: RLW proposes to sample scalarization weightsfrom standard probability distributions at each iteration, and proves that this leads the better generalization [39, theorem 2].
Revised: RLW proposes to sample scalarization weightsfrom standard probability distributions at each iteration, and proves that this leads the better generalization [39, theorem 2].

Original: Indeed, it is well-known that adding noise to stochastic gradient estimationsleads the optimization towards ﬂatter minima, and that such minima may reduce overﬁtting [30, 33].
Revised: Indeed, it is well-known that adding noise to stochastic gradient estimationsleads the optimization towards ﬂatter minima, and that such minima may reduce overﬁtting [30, 33].

Original: In line with the main technical results by Yu et al.
Revised: In line with the main technical results by Yu et al.

Original: [64], we now restrict our focus to two-task problems,which allow for an easy description of PCGrad’s convergence points.
Revised: [64], we now restrict our focus to two-task problems,which allow for an easy description of PCGrad’s convergence points.

Original: The result is largely basedon [64, theorem 1]: we relax some of the assumptions and provide a proof in appendix B.3.
Revised: The result is largely basedon [64, theorem 1]: we relax some of the assumptions and provide a proof in appendix B.3.

Original: Corollary 2.
Revised: Corollary 2.

Original: If |T | = 2 , PCGrad will stop at any point where cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 .
Revised: If |T | = 2 , PCGrad will stop at any point where cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 .

Original: Furthermore, if L 1 and L 2 are differentiable, and ∇ θ (cid:107) L MT is L-Lipschitz with L > 0 , PCGrad with step sizet <
Revised: Furthermore, if L 1 and L 2 are differentiable, and ∇ θ (cid:107) L MT is L-Lipschitz with L > 0 , PCGrad with step sizet <

Original: 1 L converges to a superset of the convergence points of the unitary scalarization.
Revised: 1 L converges to a superset of the convergence points of the unitary scalarization.

Original: Corollary 2 implies that, when |T | = 2 , PCGrad may under-optimize equation (1) as MGDAand IMTL.
Revised: Corollary 2 implies that, when |T | = 2 , PCGrad may under-optimize equation (1) as MGDAand IMTL.

Original: In particular, if cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 , then 0 ∈ Conv ( {∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 } ) (seeproposition 1).
Revised: In particular, if cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 , then 0 ∈ Conv ( {∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 } ) (seeproposition 1).

Original: We believe that PCGrad’s stochasticity and enlarged convergence set potentiallyexplain its regularizing effect.
Revised: We believe that PCGrad’s stochasticity and enlarged convergence set potentiallyexplain its regularizing effect.

Original: GradDrop While the motivation behind GradDrop is to avoid entry-wise gradient conﬂicts acrosstasks, the main property of the method is to drive the optimization towards “joint minima": points thatare stationary for all the individual tasks at once [11, proposition 1].
Revised: GradDrop While the motivation behind GradDrop is to avoid entry-wise gradient conﬂicts acrosstasks, the main property of the method is to drive the optimization towards “joint minima": points thatare stationary for all the individual tasks at once [11, proposition 1].

Original: In other words: ∇ θ (cid:107) L i = 0 ∀ i ∈
Revised: In other words: ∇ θ (cid:107) L i = 0 ∀ i ∈

Original: T .
Revised: T .

Original: While this property is desirable, we show that it holds beyond GradDrop, and independently ofthe gradient directions.
Revised: While this property is desirable, we show that it holds beyond GradDrop, and independently ofthe gradient directions.

Original: Under strong assumptions on the model capacity, the above property wouldtrivially hold for unitary scalarization (proposition 5, appendix B.3).
Revised: Under strong assumptions on the model capacity, the above property wouldtrivially hold for unitary scalarization (proposition 5, appendix B.3).

Original: Proposition 4 shows that it holdsfor a simple randomized version of unitary scalarization, which we name Random Grad Drop (RGD).
Revised: Proposition 4 shows that it holdsfor a simple randomized version of unitary scalarization, which we name Random Grad Drop (RGD).

Original: Proposition 4.
Revised: Proposition 4.

Original: Let L RGD ( θ (cid:107) ) :
Revised: Let L RGD ( θ (cid:107) ) :

Original: = (cid:80) i ∈T u i L
Revised: = (cid:80) i ∈T u i L

Original: i ( θ (cid:107) ) , where u i ∼ Bernoulli ( p ) ∀ i ∈ T and p ∈ (0 , 1] .
Revised: i ( θ (cid:107) ) , where u i ∼ Bernoulli ( p ) ∀ i ∈ T and p ∈ (0 , 1] .

Original: The gradient ∇ θ (cid:107) L RGD is always zero if and only if ∇ θ (cid:107) L i =
Revised: The gradient ∇ θ (cid:107) L RGD is always zero if and only if ∇ θ (cid:107) L i =

Original: 0 ∀ i ∈ T .
Revised: 0 ∀ i ∈ T .

Original: In other words, the resultfrom [11, proposition 1] can be obtained without any information on the sign of per-task gradients.
Revised: In other words, the resultfrom [11, proposition 1] can be obtained without any information on the sign of per-task gradients.

Original: Proposition 4 (see appendix B.4 for a simple proof) shows that an inexpensive sign-independentstochastic scalarization shares GradDrop’s main reported property.
Revised: Proposition 4 (see appendix B.4 for a simple proof) shows that an inexpensive sign-independentstochastic scalarization shares GradDrop’s main reported property.

Original: L RGD can be directly castan instance of RLW, and hence as a regularization method [30, 33].
Revised: L RGD can be directly castan instance of RLW, and hence as a regularization method [30, 33].

Original: Furthermore, Figure 12 inappendix D.4 shows that the empirical results of GradDrop on CelebA [43] are closely matchedby a sign-agnostic gradient masking, partly undermining the conﬂicting gradients assumption.
Revised: Furthermore, Figure 12 inappendix D.4 shows that the empirical results of GradDrop on CelebA [43] are closely matchedby a sign-agnostic gradient masking, partly undermining the conﬂicting gradients assumption.

Original: Webelieve that the above results, along with the authors’ original experiments showing that GradDropdelays overﬁtting on CelebA [11, ﬁgure 3], suggest that GradDrop behaves as a regularizer.
Revised: Webelieve that the above results, along with the authors’ original experiments showing that GradDropdelays overﬁtting on CelebA [11, ﬁgure 3], suggest that GradDrop behaves as a regularizer.

Original:  Conclusions
Revised:  Conclusions

Original: This paper made two main contributions.
Revised: This paper made two main contributions.

Original: First, we evaluated popular SMTOs using a single experimental pipeline, including previously unpublished results of MGDA, IMTL, RLW, and GradDrop inthe RL setting.
Revised: First, we evaluated popular SMTOs using a single experimental pipeline, including previously unpublished results of MGDA, IMTL, RLW, and GradDrop inthe RL setting.

Original: Surprisingly, our evaluation showed that none of the SMTOs consistently outperformunitary scalarization, the simplest and least expensive method.
Revised: Surprisingly, our evaluation showed that none of the SMTOs consistently outperformunitary scalarization, the simplest and least expensive method.

Original: Second, in order to explain our surprising results, we postulate that SMTOs act as regularizers and present an analysis that supports ourhypothesis.
Revised: Second, in order to explain our surprising results, we postulate that SMTOs act as regularizers and present an analysis that supports ourhypothesis.

Original: We believe our work calls for further reevaluation of progress in developing principledand efﬁcient MTL algorithms.
Revised: We believe our work calls for further reevaluation of progress in developing principledand efﬁcient MTL algorithms.

Original: We conclude by addressing the limitations of our work.
Revised: We conclude by addressing the limitations of our work.

Original: While we covered a wide range of popularbenchmarks, we do not exclude the existence of settings where unitary scalarization underperforms:discovering them is an interesting direction for future work.
Revised: While we covered a wide range of popularbenchmarks, we do not exclude the existence of settings where unitary scalarization underperforms:discovering them is an interesting direction for future work.

Original: Furthermore, our experimental resultswere obtained via grid searches under limited compute resources: some of the methods mightbeneﬁt from further ﬁne-tuning.
Revised: Furthermore, our experimental resultswere obtained via grid searches under limited compute resources: some of the methods mightbeneﬁt from further ﬁne-tuning.

Original: Nevertheless, we remark that ﬁne-tuning will be easier for unitaryscalarization due to its shorter runtimes.
Revised: Nevertheless, we remark that ﬁne-tuning will be easier for unitaryscalarization due to its shorter runtimes.

Original: Finally, we presented the regularization hypothesis only as apartial explanation of our results: we hope it will steer further analysis and consequently improve theunderstanding of MTL.
Revised: Finally, we presented the regularization hypothesis only as apartial explanation of our results: we hope it will steer further analysis and consequently improve theunderstanding of MTL.

Original: Checklist
Revised: Checklist

Original: For all
Revised: For all

Original: authors...
Revised: authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] see § 6.
Revised: [Yes] see § 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] due to
Revised: [Yes] due to

Original: space constraints, we provide a discussion in appendix A.
Revised: space constraints, we provide a discussion in appendix A.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] we provide full proofs
Revised: [Yes] we provide full proofs

Original: in the Appendix, and refer to them in the main body of the paper.
Revised: in the Appendix, and refer to them in the main body of the paper.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] we provide thecode and the instructions in the supplemental material.
Revised: [Yes] we provide thecode and the instructions in the supplemental material.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,
Revised: g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] see appendix C.1.
Revised: [Yes] see appendix C.1.

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] appendix C.3 describes licenses of all
Revised: [Yes] appendix C.3 describes licenses of all

Original: benchmarks and implementations we used for our work.
Revised: benchmarks and implementations we used for our work.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes]

Original: we include the code and the instructions on how to replicate the experiments into the
Revised: we include the code and the instructions on how to replicate the experiments into the

Original: supplemental material.
Revised: supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Societal Impact
Revised: A Societal Impact

Original: Due to the object of its study, our work does not have a direct societal impact.
Revised: Due to the object of its study, our work does not have a direct societal impact.

Original: However, as anymachine learning paper, it can potentially negatively effect the society through automation and lossof jobs.
Revised: However, as anymachine learning paper, it can potentially negatively effect the society through automation and lossof jobs.

Original: While it is hard to anticipate any particular risk, as any technology, if not regulated properly,it might lead to growing social and economic inequality.
Revised: While it is hard to anticipate any particular risk, as any technology, if not regulated properly,it might lead to growing social and economic inequality.

Original: On the positive side, our work might have a positive environmental impact since it advocates forsimpler and more economical methods which will reduce energy consumption in data centers.
Revised: On the positive side, our work might have a positive environmental impact since it advocates forsimpler and more economical methods which will reduce energy consumption in data centers.

Original: Finally,simpler methods are usually easier to understand, which is beneﬁcial in terms of explainability, animportant factor for real-life applications.
Revised: Finally,simpler methods are usually easier to understand, which is beneﬁcial in terms of explainability, animportant factor for real-life applications.

Original: B Supplement to the Overview of Multi-Task Optimizers
Revised: B Supplement to the Overview of Multi-Task Optimizers

Original: This section presents the proofs and the technical results omitted from section 5, along with a description of the use of per-task gradients with respect to the last shared activation for encoder-decoderarchitectures (usually less expensive than per-task gradients with respect to shared parameters).
Revised: This section presents the proofs and the technical results omitted from section 5, along with a description of the use of per-task gradients with respect to the last shared activation for encoder-decoderarchitectures (usually less expensive than per-task gradients with respect to shared parameters).

Original: B.1 MGDA
Revised: B.1 MGDA

Original: Proposition 1.
Revised: Proposition 1.

Original: The MGDA SMTO [53] converges to a superset of the convergence points of unitaryscalarization.
Revised: The MGDA SMTO [53] converges to a superset of the convergence points of unitaryscalarization.

Original: More speciﬁcally, it converges to any point θ ∗ (cid:107)
Revised: More speciﬁcally, it converges to any point θ ∗ (cid:107)

Original: such that: 0 ∈ Conv ( {∇ θ ∗ (cid:107) L i | i ∈ T } ) .
Revised: such that: 0 ∈ Conv ( {∇ θ ∗ (cid:107) L i | i ∈ T } ) .

Original: Proof.
Revised: Proof.

Original: As shown by Désidéri [14], equation (3) is a simplex-constrained norm-minimization problem.
Revised: As shown by Désidéri [14], equation (3) is a simplex-constrained norm-minimization problem.

Original: In other words, the argument of the minimum is the projection of 0 onto the feasible set.
Revised: In other words, the argument of the minimum is the projection of 0 onto the feasible set.

Original: Therefore: g = 

Revised: Therefore: g =

Original: ⇐⇒ 

Revised: ⇐⇒

Original: ∈ Conv ( {∇ θ (cid:107) L i | i ∈ T } ) .
Revised: ∈ Conv ( {∇ θ (cid:107) L i | i ∈ T } ) .

Original: It then sufﬁces to point out that (cid:80) i ∈T ∇ θ (cid:107) L i =
Revised: It then sufﬁces to point out that (cid:80) i ∈T ∇ θ (cid:107) L i =

Original: ⇐⇒ (cid:80) i ∈T 1 |T | ∇ θ (cid:107) L i = 

Revised: ⇐⇒ (cid:80) i ∈T 1 |T | ∇ θ (cid:107) L i =

Original: ⇒ 0 ∈
Revised: ⇒ 0 ∈

Original: Conv ( {∇ θ (cid:107) L i | i ∈ T } ) to conclude the proof.
Revised: Conv ( {∇ θ (cid:107) L i | i ∈ T } ) to conclude the proof.

Original: Due to the cost of computing per-task gradients, Sener and Koltun [53] propose MGDA-UB, whichreplaces the gradients wrt the parameters ∇ θ (cid:107) L i with the gradients wrt the shared activation ∇ z L iin the computation of the coefﬁcients of g = − (cid:80)
Revised: Due to the cost of computing per-task gradients, Sener and Koltun [53] propose MGDA-UB, whichreplaces the gradients wrt the parameters ∇ θ (cid:107) L i with the gradients wrt the shared activation ∇ z L iin the computation of the coefﬁcients of g = − (cid:80)

Original: i α
Revised: i α

Original: i ∇ θ (cid:107) L i .
Revised: i ∇ θ (cid:107) L i .

Original: This yields an upper bound on theobjective of equation (3), thus restricting the set of points the algorithm convergences to.
Revised: This yields an upper bound on theobjective of equation (3), thus restricting the set of points the algorithm convergences to.

Original: Ratherthan directly relying on ∇ θ (cid:107) L i , g can then be obtained by computing the gradient of (cid:80) i ∈T α i L i viareverse-mode differentiation, hence saving memory and compute.
Revised: Ratherthan directly relying on ∇ θ (cid:107) L i , g can then be obtained by computing the gradient of (cid:80) i ∈T α i L i viareverse-mode differentiation, hence saving memory and compute.

Original: Corollary 3.
Revised: Corollary 3.

Original: The MGDA-UB SMTO by Sener and Koltun [53] converges to any point such that:0 ∈ Conv ( {∇ z L i | i ∈ T } ) .
Revised: The MGDA-UB SMTO by Sener and Koltun [53] converges to any point such that:0 ∈ Conv ( {∇ z L i | i ∈ T } ) .

Original: Furthermore, if ∂ z ∂ θ (cid:107) is non-singular, it converges to a superset of theconvergence points of the unitary scalarization.
Revised: Furthermore, if ∂ z ∂ θ (cid:107) is non-singular, it converges to a superset of theconvergence points of the unitary scalarization.

Original: Proof.
Revised: Proof.

Original: The ﬁrst part of the proof proceeds as the proof of proposition 1, noting that the MGDA-UBupdate is associated to the following problem:
Revised: The ﬁrst part of the proof proceeds as the proof of proposition 1, noting that the MGDA-UBupdate is associated to the following problem:

Original: [Equation]
Revised: [Equation]

Original: In order to show that a stationary point of the unitary scalarization satisﬁes 0 ∈ Conv ( {∇ z ∗ L i | i ∈
Revised: In order to show that a stationary point of the unitary scalarization satisﬁes 0 ∈ Conv ( {∇ z ∗ L i | i ∈

Original: T } ) , we will assume ∂ z ∂ θ (cid:107) is non-singular, as done by Sener and Koltun [53, theorem 1].
Revised: T } ) , we will assume ∂ z ∂ θ (cid:107) is non-singular, as done by Sener and Koltun [53, theorem 1].

Original: Then, relying on the chain rule, the result follows from:
Revised: Then, relying on the chain rule, the result follows from:

Original: [Equation]
Revised: [Equation]

Original: B.2 IMTL
Revised: B.2 IMTL

Original: Proposition 2.
Revised: Proposition 2.

Original: IMTL by Liu et al.
Revised: IMTL by Liu et al.

Original: [41] updates θ (cid:107) by taking a step in the steepest descent directionwhose cosine similarity with per-task gradients is the same across tasks.
Revised: [41] updates θ (cid:107) by taking a step in the steepest descent directionwhose cosine similarity with per-task gradients is the same across tasks.

Original: Proof.
Revised: Proof.

Original: First, equation (4) solves the linear system in α :=
Revised: First, equation (4) solves the linear system in α :=

Original: [ α 1 , . . .
Revised: [ α 1 , . . .

Original: , α m ] given by:
Revised: , α m ] given by:

Original: [Equation]
Revised: [Equation]

Original: which corresponds to ﬁnding a point of A (cid:48)
Revised: which corresponds to ﬁnding a point of A (cid:48)

Original: = Aff ( (cid:8)
Revised: = Aff ( (cid:8)

Original: [Equation]
Revised: [Equation]

Original: L i | i ∈ T (cid:9) ) which is orthogonal to A :=
Revised: L i | i ∈ T (cid:9) ) which is orthogonal to A :=

Original: Aff
Revised: Aff

Original: [Equation]
Revised: [Equation]

Original: To see this, it sufﬁces to point out that any point orthogonal to A is alsoorthogonal to the vector subspace spanned by differences of vectors belonging to A .
Revised: To see this, it sufﬁces to point out that any point orthogonal to A is alsoorthogonal to the vector subspace spanned by differences of vectors belonging to A .

Original: As this subspacehas m − 1 dimensions, any vector orthogonal to
Revised: As this subspacehas m − 1 dimensions, any vector orthogonal to

Original: [Equation]
Revised: [Equation]

Original: for each i ∈ T \ { 1 } isorthogonal to the entire subspace.
Revised: for each i ∈ T \ { 1 } isorthogonal to the entire subspace.

Original: Second, consider the problem of ﬁnding a point in A that is orthogonal to the afﬁne subspace A .
Revised: Second, consider the problem of ﬁnding a point in A that is orthogonal to the afﬁne subspace A .

Original: Inother words, we seek the projection of 0 onto A .
Revised: Inother words, we seek the projection of 0 onto A .

Original: Recalling the deﬁnition of A , we can write:
Revised: Recalling the deﬁnition of A , we can write:

Original: [Equation]
Revised: [Equation]

Original: The solution of equation (7) is always collinear to the solution of equation (4).
Revised: The solution of equation (7) is always collinear to the solution of equation (4).

Original: In fact, if a vectorg ∈ A (cid:48) is orthogonal to the afﬁne subspace A , then γ g = (cid:18) γ (cid:80)
Revised: In fact, if a vectorg ∈ A (cid:48) is orthogonal to the afﬁne subspace A , then γ g = (cid:18) γ (cid:80)

Original: [Equation]
Revised: [Equation]

Original: ∇
Revised: ∇

Original: [Equation]
Revised: [Equation]

Original: ∇ θ
Revised: ∇ θ

Original: [Equation]
Revised: [Equation]

Original: isorthogonal to A as well, and γ =
Revised: isorthogonal to A as well, and γ =

Original: [Equation]
Revised: [Equation]

Original: Finally, equation (7) differs from equation (3) in two aspects: α is not constrained to be non-negative(hence the convex hull is replaced by the afﬁne hull), and the task vectors are normalized.
Revised: Finally, equation (7) differs from equation (3) in two aspects: α is not constrained to be non-negative(hence the convex hull is replaced by the afﬁne hull), and the task vectors are normalized.

Original: Therefore,equation (7) is the dual of:
Revised: Therefore,equation (7) is the dual of:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The proposition then follows by comparing equation (8) with equation (2), and recalling that IMTL-Lonly adds a scaling factor to the chosen update direction.
Revised: The proposition then follows by comparing equation (8) with equation (2), and recalling that IMTL-Lonly adds a scaling factor to the chosen update direction.

Original: Corollary 1.
Revised: Corollary 1.

Original: IMTL by Liu et al.
Revised: IMTL by Liu et al.

Original: [41] converges to a superset of the Pareto-optimal points for θ (cid:107)(and hence of the convergence points of the unitary scalarization).
Revised: [41] converges to a superset of the Pareto-optimal points for θ (cid:107)(and hence of the convergence points of the unitary scalarization).

Original: More speciﬁcally, it converges toany point θ ∗ (cid:107) such that:
Revised: More speciﬁcally, it converges toany point θ ∗ (cid:107) such that:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Inspecting equation (8), which yields a collinear point to the IMTL update, reveals that
Revised: Inspecting equation (8), which yields a collinear point to the IMTL update, reveals that

Original: IMTL might converge to non Pareto-stationary points: due to the restrictive equality constraints,the minimizer of equation (8) might be 0 even if a descent direction exists.
Revised: IMTL might converge to non Pareto-stationary points: due to the restrictive equality constraints,the minimizer of equation (8) might be 0 even if a descent direction exists.

Original: Furthermore, its dual,equation (7), implies that:
Revised: Furthermore, its dual,equation (7), implies that:

Original: [Equation]
Revised: [Equation]

Original: which, noting that Conv ( A ) ⊆
Revised: which, noting that Conv ( A ) ⊆

Original: Aff ( A ) for any A , concludes the proof.
Revised: Aff ( A ) for any A , concludes the proof.

Original: Similarly to MGDA-UB, Liu et al.
Revised: Similarly to MGDA-UB, Liu et al.

Original: [41] advocate using ∇ z L i in place of ∇ θ (cid:

Revised: [41] advocate using ∇ z L i in place of ∇ θ (cid:

Original: ) L i while solvingequation (4), typically reducing the cost of computing the coefﬁcients of g = − (cid:80) i
Revised: ) L i while solvingequation (4), typically reducing the cost of computing the coefﬁcients of g = − (cid:80) i

Original: α i ∇ θ
Revised: α i ∇ θ

Original: [Equation]
Revised: [Equation]

Original: L
Revised: L

Original: [Equation]
Revised: [Equation]

Original: Corollary 4.
Revised: Corollary 4.

Original: When employing the approximation of problem (4) that relies on ∇ z L i , IMTL by Liuet al.
Revised: When employing the approximation of problem (4) that relies on ∇ z L i , IMTL by Liuet al.

Original: [41] converges to 0 ∈ Aff (cid:16)(cid:110) ∇ z L i (cid:107)∇ z L i (cid:107) | i ∈ T (cid:111)(cid:17) .
Revised: [41] converges to 0 ∈ Aff (cid:16)(cid:110) ∇ z L i (cid:107)∇ z L i (cid:107) | i ∈ T (cid:111)(cid:17) .

Original: If ∂ z ∂ θ (cid:107) is non-singular, this is a superset of ofthe convergence points of the unitary scalarization.
Revised: If ∂ z ∂ θ (cid:107) is non-singular, this is a superset of ofthe convergence points of the unitary scalarization.

Original: Proof.
Revised: Proof.

Original: Following the proof of proposition 2, the following problem yields a collinear point to the∇ z
Revised: Following the proof of proposition 2, the following problem yields a collinear point to the∇ z

Original: L i -approximate IMTL update:
Revised: L i -approximate IMTL update:

Original: [Equation]
Revised: [Equation]

Original: Therefore:
Revised: Therefore:

Original: [Equation]
Revised: [Equation]

Original: Finally, assuming ∂ z ∂ θ (cid:107) is non-singular, we can replicate the procedure in the proof of corollary 3 toget:
Revised: Finally, assuming ∂ z ∂ θ (cid:107) is non-singular, we can replicate the procedure in the proof of corollary 3 toget:

Original: [Equation]
Revised: [Equation]

Original: ∇ z L i (
Revised: ∇ z L i (

Original: cid:107)∇ z L i (cid:107) =
Revised: cid:107)∇ z L i (cid:107) =

Original: ⇒ 

Revised: ⇒

Original: ∈ Conv (cid:18)(cid:26) ∇ z L i (
Revised: ∈ Conv (cid:18)(cid:26) ∇ z L i (

Original: cid:107)∇ z L i (cid:107) | i ∈ T (cid:27)(cid:19) ⇒ 

Revised: cid:107)∇ z L i (cid:107) | i ∈ T (cid:27)(cid:19) ⇒

Original: ∈ Aff (cid:18)(cid:26) ∇ z L i (
Revised: ∈ Aff (cid:18)(cid:26) ∇ z L i (

Original: cid:107)∇ z L i (cid:107) | i ∈ T (cid:27)(cid:19) , which shows that Aff (cid:16)(cid:110) ∇ z L i (
Revised: cid:107)∇ z L i (cid:107) | i ∈ T (cid:27)(cid:19) , which shows that Aff (cid:16)(cid:110) ∇ z L i (

Original: cid:107)∇ z L i (cid:107) | i ∈ T (cid:111)(cid:17) contains the convergence points of the unitary scalarization.
Revised: cid:107)∇ z L i (cid:107) | i ∈ T (cid:111)(cid:17) contains the convergence points of the unitary scalarization.

Original: B.3 PCGrad
Revised: B.3 PCGrad

Original: Proposition 3.
Revised: Proposition 3.

Original: PCGrad is equivalent to a dynamic, and possibly stochastic, loss rescaling for θ (cid:107) .
Revised: PCGrad is equivalent to a dynamic, and possibly stochastic, loss rescaling for θ (cid:107) .

Original: At each iteration, per-task gradients are rescaled as follows:
Revised: At each iteration, per-task gradients are rescaled as follows:

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if |T | > 2 , d ji is a random variable
Revised: Furthermore, if |T | > 2 , d ji is a random variable

Original: [Equation]
Revised: [Equation]

Original: and the above range contains its support.
Revised: and the above range contains its support.

Original: Proof.
Revised: Proof.

Original: We start by pointing out that:
Revised: We start by pointing out that:

Original: [Equation]
Revised: [Equation]

Original: As g i is obtained by iterative projections of ∇ θ (cid:107) L i onto the normals of ∇ θ (cid:107) L j ∀ j ∈ T \ { i } , and thenorm of a vector can only decrease or remain unvaried after projections, we can write the coefﬁcientof the last g i update as:
Revised: As g i is obtained by iterative projections of ∇ θ (cid:107) L i onto the normals of ∇ θ (cid:107) L j ∀ j ∈ T \ { i } , and thenorm of a vector can only decrease or remain unvaried after projections, we can write the coefﬁcientof the last g i update as:

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if |T | > 2 the contraction factor
Revised: Furthermore, if |T | > 2 the contraction factor

Original: [Equation]
Revised: [Equation]

Original: for the norm of g i depends on the ordering ofthe projections, which is stochastic by design [64].
Revised: for the norm of g i depends on the ordering ofthe projections, which is stochastic by design [64].

Original: Therefore, d ij a random variable whose support iscontained in
Revised: Therefore, d ij a random variable whose support iscontained in

Original: [Equation]
Revised: [Equation]

Original: Finally, exploiting the deﬁnition of d ij , we can re-write equation (5) as:
Revised: Finally, exploiting the deﬁnition of d ij , we can re-write equation (5) as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 
Revised: 

Original:  , from which the result trivially follows.
Revised:  , from which the result trivially follows.

Original: Corollary 2.
Revised: Corollary 2.

Original: If |T | = 2 , PCGrad will stop at any point where cos( ∇ θ
Revised: If |T | = 2 , PCGrad will stop at any point where cos( ∇ θ

Original: [Equation]
Revised: [Equation]

Original: L 1 , ∇ θ (cid:107) L 2 ) = − 1 .
Revised: L 1 , ∇ θ (cid:107) L 2 ) = − 1 .

Original: Furthermore, if L 1 and L 2 are differentiable, and ∇ θ (cid:107) L MT is L-Lipschitz with L > 0 , PCGrad with step sizet <
Revised: Furthermore, if L 1 and L 2 are differentiable, and ∇ θ (cid:107) L MT is L-Lipschitz with L > 0 , PCGrad with step sizet <

Original: 1 L converges to a superset of the convergence points of the unitary scalarization.
Revised: 1 L converges to a superset of the convergence points of the unitary scalarization.

Original: Proof.
Revised: Proof.

Original: Let us start from the ﬁrst statement, which does not require any assumption on the losslandscape.
Revised: Let us start from the ﬁrst statement, which does not require any assumption on the losslandscape.

Original: From proposition 3, we get:
Revised: From proposition 3, we get:

Original: [Equation]
Revised: [Equation]

Original: which shows that, in case of conﬂicting gradient directions, gradient norms are rebalanced proportionally to the angle between them.
Revised: which shows that, in case of conﬂicting gradient directions, gradient norms are rebalanced proportionally to the angle between them.

Original: For cos( ∇ θ
Revised: For cos( ∇ θ

Original: [Equation]
Revised: [Equation]

Original: L 1 , ∇ θ
Revised: L 1 , ∇ θ

Original: [Equation]
Revised: [Equation]

Original: L 2 )
Revised: L 2 )

Original: = − 1 , the above evaluates to:
Revised: = − 1 , the above evaluates to:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst part of the result then follows by pointing out that, if cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 , then∇ θ (cid:107) L 1 = −∇ θ (cid:107) L 2 , and hence g = 0 .
Revised: The ﬁrst part of the result then follows by pointing out that, if cos( ∇ θ (cid:107) L 1 , ∇ θ (cid:107) L 2 ) = − 1 , then∇ θ (cid:107) L 1 = −∇ θ (cid:107) L 2 , and hence g = 0 .

Original: We remark that a similar proof appears in [64, theorem 1 andproposition 1].
Revised: We remark that a similar proof appears in [64, theorem 1 andproposition 1].

Original: However, our derivation relaxes the author’s assumptions on L MT and is thereforeapplicable to the training of neural networks.
Revised: However, our derivation relaxes the author’s assumptions on L MT and is thereforeapplicable to the training of neural networks.

Original: Finally, given the assumptions on differentiability and smoothness, we need to prove that PCGrad converges to the stationary points of the unitary scalarization: this directly follows from [64, proposition1].
Revised: Finally, given the assumptions on differentiability and smoothness, we need to prove that PCGrad converges to the stationary points of the unitary scalarization: this directly follows from [64, proposition1].

Original: B.4 GradDrop
Revised: B.4 GradDrop

Original: Proposition 5.
Revised: Proposition 5.

Original: Let us assume, as often demonstrated in the single-task case [1, 44], that the multi-tasknetwork has the capacity to interpolate the data on all tasks at once: min θ L MT =
Revised: Let us assume, as often demonstrated in the single-task case [1, 44], that the multi-tasknetwork has the capacity to interpolate the data on all tasks at once: min θ L MT =

Original: (cid:80) i ∈T min θ L i ,and that its training by gradient descent attains such global minimum.
Revised: (cid:80) i ∈T min θ L i ,and that its training by gradient descent attains such global minimum.

Original: Then, if inf θ L i > −∞ ∀ i ∈
Revised: Then, if inf θ L i > −∞ ∀ i ∈

Original: T , unitary scalarization converges to a joint minimum.
Revised: T , unitary scalarization converges to a joint minimum.

Original: Proof.
Revised: Proof.

Original: It sufﬁces to point out that if L MT ( θ ∗ ) =
Revised: It sufﬁces to point out that if L MT ( θ ∗ ) =

Original: (cid:80) i ∈T min θ L i , then the globally optimal loss isattained for all tasks.
Revised: (cid:80) i ∈T min θ L i , then the globally optimal loss isattained for all tasks.

Original: In other words L i ( θ ∗ ) =
Revised: In other words L i ( θ ∗ ) =

Original: min
Revised: min

Original: θ L i ∀ i ∈ T , and hence ∇ θ ∗ L i = 0 ∀
Revised: θ L i ∀ i ∈ T , and hence ∇ θ ∗ L i = 0 ∀

Original: i ∈ T(joint minimum).
Revised: i ∈ T(joint minimum).

Original: Furthermore, running gradient descent on min θ L MT corresponds to the unitaryscalarization ( § 3), which concludes the proof.
Revised: Furthermore, running gradient descent on min θ L MT corresponds to the unitaryscalarization ( § 3), which concludes the proof.

Original: Proposition 4.
Revised: Proposition 4.

Original: Let L RGD ( θ (cid:107) ) :
Revised: Let L RGD ( θ (cid:107) ) :

Original: = (cid:80) i ∈T u i L
Revised: = (cid:80) i ∈T u i L

Original: i ( θ (cid:107) ) , where u i ∼ Bernoulli ( p ) ∀ i ∈ T and p ∈ (0 , 1] .
Revised: i ( θ (cid:107) ) , where u i ∼ Bernoulli ( p ) ∀ i ∈ T and p ∈ (0 , 1] .

Original: The gradient ∇ θ (cid:107) L RGD is always zero if and only if ∇ θ (cid:107) L i =
Revised: The gradient ∇ θ (cid:107) L RGD is always zero if and only if ∇ θ (cid:107) L i =

Original: 0 ∀ i ∈ T .
Revised: 0 ∀ i ∈ T .

Original: In other words, the resultfrom [11, proposition 1] can be obtained without any information on the sign of per-task gradients.
Revised: In other words, the resultfrom [11, proposition 1] can be obtained without any information on the sign of per-task gradients.

Original: Proposition 4 can be proved by adapting the proof from Chen et al.
Revised: Proposition 4 can be proved by adapting the proof from Chen et al.

Original: [11, proposition 1]: it sufﬁces toreplace f ( P ) with the Bernoulli parameter p , which is non-negative by deﬁnition.
Revised: [11, proposition 1]: it sufﬁces toreplace f ( P ) with the Bernoulli parameter p , which is non-negative by deﬁnition.

Original: In our opinion, thisseriously undermines the conﬂicting gradient hypothesis that motivated GradDrop.
Revised: In our opinion, thisseriously undermines the conﬂicting gradient hypothesis that motivated GradDrop.

Original: For the reader’sconvenience, we now provide a straightforward and self-contained proof.
Revised: For the reader’sconvenience, we now provide a straightforward and self-contained proof.

Original: Proof.
Revised: Proof.

Original: Let us start from the statement on ∇ θ (cid:107) L RGD .
Revised: Let us start from the statement on ∇ θ (cid:107) L RGD .

Original: If ∇ θ (cid:107) L i = 0 ∀ i ∈ T , then ∇ θ (cid:107) L RGD =
Revised: If ∇ θ (cid:107) L i = 0 ∀ i ∈ T , then ∇ θ (cid:107) L RGD =

Original: 0with probability one.
Revised: 0with probability one.

Original: On the other hand, if ∃ j : ∇
Revised: On the other hand, if ∃ j : ∇

Original: [Equation]
Revised: [Equation]

Original: L j (cid:54) = 0 , then:
Revised: L j (cid:54) = 0 , then:

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality comes from the fact that ∇ θ (cid:107) L RGD = ∇ θ (cid:107) L j is only one of the manyinstances of a non-null ∇ θ (cid:107) L RGD .
Revised: where the ﬁrst inequality comes from the fact that ∇ θ (cid:107) L RGD = ∇ θ (cid:107) L j is only one of the manyinstances of a non-null ∇ θ (cid:107) L RGD .

Original: Let sign ( x ) stand for the element-wise sign operator applied on x .
Revised: Let sign ( x ) stand for the element-wise sign operator applied on x .

Original: On encoder-decoder architectures,similarly to MGDA and IMTL (see appendices B.1 and B.2), the authors do not apply GradDropon ∇ θ (cid:107) L i , but rather on a the usually less expensive ∇ z L i .
Revised: On encoder-decoder architectures,similarly to MGDA and IMTL (see appendices B.1 and B.2), the authors do not apply GradDropon ∇ θ (cid:107) L i , but rather on a the usually less expensive ∇ z L i .

Original: In more detail, they compute the
Revised: In more detail, they compute the

Original: GradDrop sign purity scores p from equation (6) on (cid:80) ni
Revised: GradDrop sign purity scores p from equation (6) on (cid:80) ni

Original: =1 ( sign ( z ) (cid:12) ∇ z L i )
Revised: =1 ( sign ( z ) (cid:12) ∇ z L i )

Original: [ i ] ∈ R r , and then applyequation (6) on the ∇ z L i gradients, yielding a vector g z ∈ R n × r .
Revised: [ i ] ∈ R r , and then applyequation (6) on the ∇ z L i gradients, yielding a vector g z ∈ R n × r .

Original: Then, relying on reverse-modedifferentiation, the update direction in the space of the parameters θ (cid:107) is obtained via a Jacobian-vectorproduct: g = − (cid:16) ∂ z ∂ θ (cid:107) (cid:17) T g z .
Revised: Then, relying on reverse-modedifferentiation, the update direction in the space of the parameters θ (cid:107) is obtained via a Jacobian-vectorproduct: g = − (cid:16) ∂ z ∂ θ (cid:107) (cid:17) T g z .

Original: Such a computation replaces the similar ∇ θ (cid:107) L MT = (cid:16) ∂ z ∂ θ (cid:107) (cid:17) T ∇ z L MTfrom the unitary scalarization.
Revised: Such a computation replaces the similar ∇ θ (cid:107) L MT = (cid:16) ∂ z ∂ θ (cid:107) (cid:17) T ∇ z L MTfrom the unitary scalarization.

Original: C Experimental Setting, Reproducibility
Revised: C Experimental Setting, Reproducibility

Original: We now present details concerning the experimental settings from § 4, including details on theemployed open-source software, dataset information, hardware speciﬁcations, and hyper-parameters.
Revised: We now present details concerning the experimental settings from § 4, including details on theemployed open-source software, dataset information, hardware speciﬁcations, and hyper-parameters.

Original: C.1 Supervised Learning
Revised: C.1 Supervised Learning

Original: All the experiments were run under Ubuntu 18.04 LTS, on a single GPU per run (using two 8-GPUmachines in total).
Revised: All the experiments were run under Ubuntu 18.04 LTS, on a single GPU per run (using two 8-GPUmachines in total).

Original: Timing experiments were all run on Nvidia GeForce GTX 1080 Ti GPUs, with an
Revised: Timing experiments were all run on Nvidia GeForce GTX 1080 Ti GPUs, with an

Original: Intel Xeon E5-2650 CPU.
Revised: Intel Xeon E5-2650 CPU.

Original: The remaining experiments were run on either Nvidia GeForce RTX 

Revised: The remaining experiments were run on either Nvidia GeForce RTX

Original: Ti GPUs or Nvidia GeForce GTX 1080 Ti GPUs, respectively using an Intel Xeon Gold 6230 CPUor an Intel Xeon E5-2650 CPU.
Revised: Ti GPUs or Nvidia GeForce GTX 1080 Ti GPUs, respectively using an Intel Xeon Gold 6230 CPUor an Intel Xeon E5-2650 CPU.

Original: C.1.1 MultiMNIST
Revised: C.1.1 MultiMNIST

Original: Multi-MNIST, originally introduced by Sabour et al.
Revised: Multi-MNIST, originally introduced by Sabour et al.

Original: [51] and as modiﬁed by Sener and Koltun[53], is a simple two-task supervised learning benchmark dataset constructed by uniformly sampling
Revised: [51] and as modiﬁed by Sener and Koltun[53], is a simple two-task supervised learning benchmark dataset constructed by uniformly sampling

Original: MNIST [37] images, and placing one in the top-left corner, the other in the bottom-right corner.
Revised: MNIST [37] images, and placing one in the top-left corner, the other in the bottom-right corner.

Original: Eachof the two overlaid images corresponds to a 10-class classiﬁcation task.
Revised: Eachof the two overlaid images corresponds to a 10-class classiﬁcation task.

Original: Using the above procedure,we generate the Multi-MNIST training set from the ﬁrst 50000 MNIST training images, the validationset from the last 10000 training images, and the test set from the original MNIST test set.
Revised: Using the above procedure,we generate the Multi-MNIST training set from the ﬁrst 50000 MNIST training images, the validationset from the last 10000 training images, and the test set from the original MNIST test set.

Original: Forconsistency with the experimental setup of Sener and Koltun [53], we employ a modiﬁed encoderdecoder version of the LeNet architecture [37].
Revised: Forconsistency with the experimental setup of Sener and Koltun [53], we employ a modiﬁed encoderdecoder version of the LeNet architecture [37].

Original: Speciﬁcally, the last layer is omitted from the encoder,and two fully-connected layers are employed as task-speciﬁc predictive heads.
Revised: Speciﬁcally, the last layer is omitted from the encoder,and two fully-connected layers are employed as task-speciﬁc predictive heads.

Original: The cross-entropyloss is used for both tasks.
Revised: The cross-entropyloss is used for both tasks.

Original: All methods are trained for 100 epochs using Adam [32] in the stochasticgradient setting, with an initial learning rate of η = 10 − 2 (tuned in η ∈ { 10 − 3 , 10 − 2 , 10 − 1 } andyielding the best validation results for all considered algorithms), exponentially decayed by 0 .
Revised: All methods are trained for 100 epochs using Adam [32] in the stochasticgradient setting, with an initial learning rate of η = 10 − 2 (tuned in η ∈ { 10 − 3 , 10 − 2 , 10 − 1 } andyielding the best validation results for all considered algorithms), exponentially decayed by 0 .

Original: 95 aftereach epoch, and a mini-batch size of 256 .
Revised: 95 aftereach epoch, and a mini-batch size of 256 .

Original: C.1.2 CelebA
Revised: C.1.2 CelebA

Original: The CelebA [43] dataset consists of 200 , 000 headshots (with standard training, validation and testsplits) associated with the presence or absence of 40 attributes.
Revised: The CelebA [43] dataset consists of 200 , 000 headshots (with standard training, validation and testsplits) associated with the presence or absence of 40 attributes.

Original: In the MTL literature, is commonlytreated as a 40 -task classiﬁcation problem, each task being a binary classiﬁcation problem foran attribute.
Revised: In the MTL literature, is commonlytreated as a 40 -task classiﬁcation problem, each task being a binary classiﬁcation problem foran attribute.

Original: As commonly done in previous work [41, 53, 64], we employ an encoder-decoderarchitecture where the encoder is a ResNet-18 [21] (without the ﬁnal layer), and the per-task decodersare linear classiﬁers.
Revised: As commonly done in previous work [41, 53, 64], we employ an encoder-decoderarchitecture where the encoder is a ResNet-18 [21] (without the ﬁnal layer), and the per-task decodersare linear classiﬁers.

Original: The cross-entropy loss is used for all tasks.
Revised: The cross-entropy loss is used for all tasks.

Original: The training is performed fromscratch for 50 epochs using Adam, with a mini-batch size of 128 and a per-epoch exponential decayfactor of 0 .
Revised: The training is performed fromscratch for 50 epochs using Adam, with a mini-batch size of 128 and a per-epoch exponential decayfactor of 0 .

Original: As common on this network-dataset combination [11, 39], the initial learning rate isη = 10 − 3 for all methods except for MGDA and IMTL, for which η = 5 × 10 − 4 yielded a bettervalidation performance.
Revised: As common on this network-dataset combination [11, 39], the initial learning rate isη = 10 − 3 for all methods except for MGDA and IMTL, for which η = 5 × 10 − 4 yielded a bettervalidation performance.

Original: As done by the respective authors, for PCGrad, RLW and GradDrop we usethe same learning rate as the unitary scalarization [11, 39, 64].
Revised: As done by the respective authors, for PCGrad, RLW and GradDrop we usethe same learning rate as the unitary scalarization [11, 39, 64].

Original: C.1.3 Cityscapes
Revised: C.1.3 Cityscapes

Original: We rely on the version of the dataset pre-processed by Liu et al.
Revised: We rely on the version of the dataset pre-processed by Liu et al.

Original: [42], which consists of 2 , 975training and 500 test images and presents two tasks: semantic segmentation on 7 classes, and depthestimation.
Revised: [42], which consists of 2 , 975training and 500 test images and presents two tasks: semantic segmentation on 7 classes, and depthestimation.

Original: We further split the original training set into a validation set of 595 images, employedto tune hyper-parameters, and a training set of 2380 images.
Revised: We further split the original training set into a validation set of 595 images, employedto tune hyper-parameters, and a training set of 2380 images.

Original: Consistently with recent work [39],we rely on a dilated ResNet-50 architecture pre-trained on ImageNet [62] for the encoder, and onthe Atrous Spatial Pyramid Pooling [8] as decoders.
Revised: Consistently with recent work [39],we rely on a dilated ResNet-50 architecture pre-trained on ImageNet [62] for the encoder, and onthe Atrous Spatial Pyramid Pooling [8] as decoders.

Original: While more powerful encoders might lead tobetter performance on Cityscapes, like the SegNet [2] used in [27, 40, 47], we aim to provide a faircomparison of MTL optimizers, rather than maximize overall task performance.
Revised: While more powerful encoders might lead tobetter performance on Cityscapes, like the SegNet [2] used in [27, 40, 47], we aim to provide a faircomparison of MTL optimizers, rather than maximize overall task performance.

Original: Cross-entropy lossis employed on the semantic segmentation task, whereas the (cid:96) 1 loss is used for the depth estimation.
Revised: Cross-entropy lossis employed on the semantic segmentation task, whereas the (cid:96) 1 loss is used for the depth estimation.

Original: The training is performed by using Adam with a mini-batch size of 32 for 100 epochs, with an initialstep size η = 5 × 10 − 4 resulting in the best validation performance for all algorithms, exponentiallydecayed by 0 .
Revised: The training is performed by using Adam with a mini-batch size of 32 for 100 epochs, with an initialstep size η = 5 × 10 − 4 resulting in the best validation performance for all algorithms, exponentiallydecayed by 0 .

Original: 95 at each epoch.
Revised: 95 at each epoch.

Revised: Cityscapes is a heterogeneous MTL problem: it contains tasks of different types whose validationmetrics cannot be averaged to perform model selection.

Revised: Considering the lack of an established model selection procedure in this context, we potentially evaluate a different model for each metric,chosen as the one with the best validation performance across the epochs.

Revised: This procedure maximizesper-task performance, at the cost of increased inference time.

Revised: If inference time is a priority, analternative model selection procedure could rely on relative task improvement [27, 40, 47], assumingthat per-metric improvements are to be weighted linearly.

Revised: Nevertheless, any consistently appliedmodel selection scheme serves the main goal of our work: evaluating all SMTOs on a fair ground.

Original: C.2 Reinforcement Learning
Revised: C.2 Reinforcement Learning

Original: Similarly to the supervised learning experiments, we ran all the experiments under Ubuntu 18.04 LTSusing one GPU per run (using six 8-GPU machines in total).
Revised: Similarly to the supervised learning experiments, we ran all the experiments under Ubuntu 18.04 LTSusing one GPU per run (using six 8-GPU machines in total).

Original: Timing experiments were all run using
Revised: Timing experiments were all run using

Original: NVIDIA GeForce RTX 2080 Ti GPUs, with an Intel Xeon Gold 6230 CPU.
Revised: NVIDIA GeForce RTX 2080 Ti GPUs, with an Intel Xeon Gold 6230 CPU.

Original: The main bulk of theremaining experiments was run on Nvidia GeForce RTX 2080 Ti GPUs with either Intel Xeon Gold6230 or Intel Xeon Silver 4216.
Revised: The main bulk of theremaining experiments was run on Nvidia GeForce RTX 2080 Ti GPUs with either Intel Xeon Gold6230 or Intel Xeon Silver 4216.

Original: We utilised NVIDIA GeForce RTX 3080 GPUs with Intel Xeon
Revised: We utilised NVIDIA GeForce RTX 3080 GPUs with Intel Xeon

Original: Gold 6230 CPUs for a small fraction of experiments.
Revised: Gold 6230 CPUs for a small fraction of experiments.

Original: We use Meta-World’s MT10/MT50 for our experiment.
Revised: We use Meta-World’s MT10/MT50 for our experiment.

Original: The benchmark consists of ten/ﬁfty tasks inwhich a simulated robot manipulator has to perform various actions, e.g., pressing a button, openinga door, or pushing the block.
Revised: The benchmark consists of ten/ﬁfty tasks inwhich a simulated robot manipulator has to perform various actions, e.g., pressing a button, openinga door, or pushing the block.

Original: We use Sodhani et al.
Revised: We use Sodhani et al.

Original: [54] for most of the hyperparameters and listthem in Table 1.
Revised: [54] for most of the hyperparameters and listthem in Table 1.

Original: We use bold font where we use a hyperparameter different from Sodhani et al.
Revised: We use bold font where we use a hyperparameter different from Sodhani et al.

Original: Similarly to Sodhani et al.
Revised: Similarly to Sodhani et al.

Original: [54], we use the V 1 version of Metaworld for our experiments 1 .
Revised: [54], we use the V 1 version of Metaworld for our experiments 1 .

Original: Sodhaniet al.
Revised: Sodhaniet al.

Original: [54] use a shared entropy loss weight α for PCGrad and separate α for unitary scalarization 2 .
Revised: [54] use a shared entropy loss weight α for PCGrad and separate α for unitary scalarization 2 .

Original: Inour experiments, use shared α for all of the methods for fairness.
Revised: Inour experiments, use shared α for all of the methods for fairness.

Original: Since it is a single number (ratherthan a vector), we used unitary scalarization to update α for all SMTOs apart from PCGrad whichwas already implemented in [54].
Revised: Since it is a single number (ratherthan a vector), we used unitary scalarization to update α for all SMTOs apart from PCGrad whichwas already implemented in [54].

Original: We use the same network architecture as in Sodhani et al.
Revised: We use the same network architecture as in Sodhani et al.

Original: [54], i.e. a three-layered feedforwardfully-connected network with 400 hidden units per layer for both, the actor and the critic.
Revised: [54], i.e. a three-layered feedforwardfully-connected network with 400 hidden units per layer for both, the actor and the critic.

Original: The actoris shared across all tasks as well as the critic.
Revised: The actoris shared across all tasks as well as the critic.

Original: To normalize rewards, we keep track of ﬁrst and second moments in the buffer and normalise therewards by their standard deviation: r (cid:48) i
Revised: To normalize rewards, we keep track of ﬁrst and second moments in the buffer and normalise therewards by their standard deviation: r (cid:48) i

Original: =
Revised: =

Original: r i / ˆ σ i , where ˆ σ i is the sample standard deviation of therewards for environment i .
Revised: r i / ˆ σ i , where ˆ σ i is the sample standard deviation of therewards for environment i .

Original: Sodhani et al.
Revised: Sodhani et al.

Original: [54] average the gradient for unitary scalarisation and pcgrad, whereas our SMTOimplementations sum the gradients, i.e. effectively using larger learning rates (apart from MGDA thatassures that all the aggregation weights sum to 1).
Revised: [54] average the gradient for unitary scalarisation and pcgrad, whereas our SMTOimplementations sum the gradients, i.e. effectively using larger learning rates (apart from MGDA thatassures that all the aggregation weights sum to 1).

Original: We tried reducing the learning rate for SMTOsthat sum (RLW Norm., RLW Diri., and GradDrop) both for MT10 and MT50, but it worked worsefor these methods and we kept the default learning rate for them as well.
Revised: We tried reducing the learning rate for SMTOsthat sum (RLW Norm., RLW Diri., and GradDrop) both for MT10 and MT50, but it worked worsefor these methods and we kept the default learning rate for them as well.

Original: We had to use a smallerlearning rate for IMTL, because with the default one it crashed at the beginning of training due tonumerical overﬂow.
Revised: We had to use a smallerlearning rate for IMTL, because with the default one it crashed at the beginning of training due tonumerical overﬂow.

Original: Smaller learning rate did not prevent it from crashing, but this happened muchlater.
Revised: Smaller learning rate did not prevent it from crashing, but this happened muchlater.

Original: We tried 10 6 , 2 × 10 6 , and 4 × 10 6 for the replay buffer size with the last being superior in termsof stability.
Revised: We tried 10 6 , 2 × 10 6 , and 4 × 10 6 for the replay buffer size with the last being superior in termsof stability.

Original: Additionally, for l 2 actor regularization, we tried 0 .
Revised: Additionally, for l 2 actor regularization, we tried 0 .

Original: 0001 and 0 .
Revised: 0001 and 0 .

Original: 0003 with the latterbeing slightly superior for the baseline.
Revised: 0003 with the latterbeing slightly superior for the baseline.

Original: We tried the same options for other SMTOs, and picked thebest option for each of the method.
Revised: We tried the same options for other SMTOs, and picked thebest option for each of the method.

Original: For MGDA, no regularisation works best, most likely due to astrong regularization effect of the method itself, which is mirrored by our supervised learning results.
Revised: For MGDA, no regularisation works best, most likely due to astrong regularization effect of the method itself, which is mirrored by our supervised learning results.

Original: PCGrad and Graddrop work best with the regularization coefﬁcient of 0 . 0001 .
Revised: PCGrad and Graddrop work best with the regularization coefﬁcient of 0 . 0001 .

Original: Both RLW variantsuse the same coefﬁcient as the baseline ( 0 . 0003 ).
Revised: Both RLW variantsuse the same coefﬁcient as the baseline ( 0 . 0003 ).

Original: For MT50, we took the best MT10 hyperparameters, and we believe one could obtain even betterresults for unitary scalarisation since it is much faster to tune compared to other SMTOs (e.g. 15hours for unitary scalarisation vs 9 days for PCGrad).
Revised: For MT50, we took the best MT10 hyperparameters, and we believe one could obtain even betterresults for unitary scalarisation since it is much faster to tune compared to other SMTOs (e.g. 15hours for unitary scalarisation vs 9 days for PCGrad).

Original: C.3 Software Acknowledgments and Licenses
Revised: C.3 Software Acknowledgments and Licenses

Original: Our codebase is built upon several prior works: [53], [42], [39] and [54]: all of them were releasedunder a MIT license.
Revised: Our codebase is built upon several prior works: [53], [42], [39] and [54]: all of them were releasedunder a MIT license.

Original: We also acknowledge Tseng [58], upon which we built some of our code.
Revised: We also acknowledge Tseng [58], upon which we built some of our code.

Original: Multi-MNIST is based on MNIST dataset that is released under Creative Commons AttributionShare Alike 3.0 license.
Revised: Multi-MNIST is based on MNIST dataset that is released under Creative Commons AttributionShare Alike 3.0 license.

Original: The code for generating Multi-MNIST dataset was taken from Senerand Koltun [53] released under MIT license.
Revised: The code for generating Multi-MNIST dataset was taken from Senerand Koltun [53] released under MIT license.

Original: CelebA dataset has a custom license allowing noncommercial research purposes.
Revised: CelebA dataset has a custom license allowing noncommercial research purposes.

Original: More details can be found on the project website: http://mmlab.ie. .edu.hk/projects/CelebA.html .
Revised: More details can be found on the project website: http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html .

Original: Cityscapes also has a custom license allowing noncommercial research purposes.
Revised: Cityscapes also has a custom license allowing noncommercial research purposes.

Original: The full text of the license can be found on the project website: https://www.cityscapes-dataset.com/license/ .
Revised: The full text of the license can be found on the project website: https://www.cityscapes-dataset.com/license/ .

Original: Metaworld, used for RL experiments is releasedunder MIT license.
Revised: Metaworld, used for RL experiments is releasedunder MIT license.

Original: D Supplementary Supervised Learning Experiments
Revised: D Supplementary Supervised Learning Experiments

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: 2 coeff.

Original: This section presents supervised learning resultsomitted from § 4.1.
Revised: This section presents supervised learning resultsomitted from § 4.1.

Original: In particular, we show additional plots for the experiments of § 4.1, thenpresent an analysis of the regularising effect of
Revised: In particular, we show additional plots for the experiments of § 4.1, thenpresent an analysis of the regularising effect of

Original: SMTOs in the absence of single-task regularization ( § D.2), and conclude with an ablationstudy on GradDrop’s dependency on the sign ofper-task gradients ( § D.4).
Revised: SMTOs in the absence of single-task regularization ( § D.2), and conclude with an ablationstudy on GradDrop’s dependency on the sign ofper-task gradients ( § D.4).

Original: D.1 Addendum
Revised: D.1 Addendum

Original: This section complements the plots presentedin § 4.1.
Revised: This section complements the plots presentedin § 4.1.

Original: In particular, we show the test andruntime results in table form, along with thebehavior of the validation metrics and of thetraining loss over the training epochs.
Revised: In particular, we show the test andruntime results in table form, along with thebehavior of the validation metrics and of thetraining loss over the training epochs.

Original: Plots for
Revised: Plots for

Original: Multi-MNIST, CelebA, and Cityscapes are reported in Figures 6, 7 and 17, respectively.
Revised: Multi-MNIST, CelebA, and Cityscapes are reported in Figures 6, 7 and 17, respectively.

Original: The
Revised: The

Original: Multi-MNIST plots show that all optimizers are [Table] relatively stable and drive each task’s loss to- [Table]
Revised: Multi-MNIST plots show that all optimizers arerelatively stable and drive each task’s loss towards its global minimum of 0 .

Original: wards its global minimum of 0 .

Original: The behavior of

Original: [Table] the CelebA training loss demonstrates heavier [Table] regularization (compare with the unregularized
Revised: The behavior ofthe CelebA training loss demonstrates heavierregularization (compare with the unregularizedplot in Figure 8(a)).

Original: [Table]

Original: plot in Figure 8(a)).

Original: Except IMTL and MGDA,for which the tuned values of the weight decayprevent overﬁtting, the other optimizers displayvery similar validation and training curves, andstart overﬁtting around epoch 30 .
Revised: Except IMTL and MGDA,for which the tuned values of the weight decayprevent overﬁtting, the other optimizers displayvery similar validation and training curves, andstart overﬁtting around epoch 30 .

Original: Consideringthat most SMTOs required less regularization(see § 4.1.2), the results are consistent with our [Table] interpretation of SMTOs as regularizers in § 5.[Table]
Revised: Consideringthat most SMTOs required less regularization(see § 4.1.2), the results are consistent with ourinterpretation of SMTOs as regularizers in § 5.

Original: The Cityscapes plots display a certain instability [Table] across training epochs, as demonstrated by  the [Table]
Revised: The Cityscapes plots display a certain instabilityacross training epochs, as demonstrated by thevarious peaks and valleys in the metrics.

Original: various peaks and valleys in the metrics.

Original: Never-

Original: [Table] theless, in spite of a factor 10 difference in scale, [Table] both training losses are rapidly driven towards
Revised: Nevertheless, in spite of a factor 10 difference in scale,both training losses are rapidly driven towards0 by most optimizers.

Original: [Table]

Original: 0 by most optimizers.

Original: [Table]

Original: D.2 Unregularized Experiments
Revised: D.2 Unregularized Experiments

Original: [Table] Figures 5, 8(a) and 8(b) respectively report theaverage task validation accuracy, the multi-tasktraining loss, and the multi-task validation lossat each training epoch.
Revised: Figures 5, 8(a) and 8(b) respectively report theaverage task validation accuracy, the multi-tasktraining loss, and the multi-task validation lossat each training epoch.

Original: The regularizing effect
Revised: The regularizing effectof SMTOs compared to unitary scalarization isshown by: (i) the delay of the onset of overﬁtting on the validation data in ﬁgure 5, (ii) the

Original: [Table]
Revised: [Figure]

Original: of SMTOs compared to unitary scalarization isshown by: (i) the delay of the onset of overﬁtting on the validation data in ﬁgure 5, (ii) thereduction of the convergence rate on the trainingloss in ﬁgure 8(a) (compare with ﬁgure 7(b)),and (iii) the fact that validation and training losses remain positively correlated for larger numbersof epochs.
Revised: [Figure]

Original: In fact, the behavior of both the training and validation loss for the SMTOs closelyparallels that of (cid:96) 2 -regularized unitary scalarization, with differing degrees of regularization. Wefurther note that unregularized IMTL displays a certain instability (compare with the regularizedversion in ﬁgure 7(a)). The addition of dropout layers further reduces overﬁtting, improves stability (reduced conﬁdenceintervals) and pushes the average validation curve upwards, motivating its use on all optimizers for the
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Revised: r

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: g

Original: [Figure]
Revised: [Figure]

Original: k A
Revised: T

Original: [Figure]
Revised: [Figure]

Original: u r
Revised: k

Original: [Figure]
Revised: [Figure]

Original: c
Revised: u r

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: experiments of § 4.1.2.
Revised: reduction of the convergence rate on the trainingloss in ﬁgure 8(a) (compare with ﬁgure 7(b)),and (iii) the fact that validation and training losses remain positively correlated for larger numbersof epochs. In fact, the behavior of both the training and validation loss for the SMTOs closelyparallels that of (cid:96) 2 -regularized unitary scalarization, with differing degrees of regularization. Wefurther note that unregularized IMTL displays a certain instability (compare with the regularizedversion in ﬁgure 7(a)). The addition of dropout layers further reduces overﬁtting, improves stability (reduced conﬁdenceintervals) and pushes the average validation curve upwards, motivating its use on all optimizers for theexperiments of § 4.1.2.

Original: Nevertheless, conﬁdence intervals in Figure 5 still overlap due to the instabilityof the unregularized unitary scalarization.
Revised: Nevertheless, conﬁdence intervals in Figure 5 still overlap due to the instabilityof the unregularized unitary scalarization.

Original: Figure 10 provides a more detailed comparison over 20repetitions, conﬁrming that the combined use of dropout layers and (cid:96) 2 regularization improves averageperformance and reduces the empirical variance for unitary scalarization.
Revised: Figure 10 provides a more detailed comparison over 20repetitions, conﬁrming that the combined use of dropout layers and (cid:96) 2 regularization improves averageperformance and reduces the empirical variance for unitary scalarization.

Original: Furthermore, Figure 9shows that regularization improves the peak average validation performance for all algorithms,demonstrating the need of tuning λ also for SMTOs.
Revised: Furthermore, Figure 9 shows that regularization improves the peak average validation performance for all algorithms,demonstrating the need of tuning λ also for SMTOs.

Original: We conclude by pointing out that even withoutregularization, when carefully tuned, the maximal performance over epochs of unitary scalarizationis comparable to SMTOs in Figure 5. 
Revised: We conclude by pointing out that even withoutregularization, when carefully tuned, the maximal performance over epochs of unitary scalarizationis comparable to SMTOs in Figure 5. [Figure]

Original: D.3 Multi-Task Optimizers and Under-Optimization
Revised: l t i

Original: The experiments in § 5.1 and appendix D.2 show that SMTOs empirically act as regularizers forequation (1).
Revised: [Figure] a i n i n

Original: The analysis from § 5.2 provides a partial explanation for the results: due to their larger
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: the CelebA dataset: violin plots (20 runs) for the best avg.
Revised: A v e r a g e T a s k A cc u r a c y

Original: task validation accuracy over epochs.
Revised: D.3 Multi-Task Optimizers and Under-Optimization

Original: The width at a given value represents the proportion of runs yielding that result.
Revised: The experiments in § 5.1 and appendix D.2 show that SMTOs empirically act as regularizers forequation (1).

Original: Regularization improves the average performance while decreasing its variability. convergence sets, MGDA, IMTL and PCGrad might under-optimize equation 1 compared to unitaryscalarization.
Revised: The analysis from § 5.2 provides a partial explanation for the results: due to their largerconvergence sets, MGDA, IMTL and PCGrad might under-optimize equation 1 compared to unitaryscalarization.

Original: In order to empirically evaluate our analysis, we estimate (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 , the normof the unitary scalarization update on shared parameters θ (cid:107) , for all optimizers throughout the unregularized CelebA experiment ( § 5.1).
Revised: In order to empirically evaluate our analysis, we estimate (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 , the normof the unitary scalarization update on shared parameters θ (cid:107) , for all optimizers throughout the unregularized CelebA experiment ( § 5.1).

Original: Large magnitudes for (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 towards convergencewould indicate that SMTOs steer optimization far from stationary points of unitary scalarization,resulting in under-optimization.
Revised: Large magnitudes for (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 towards convergencewould indicate that SMTOs steer optimization far from stationary points of unitary scalarization,resulting in under-optimization.

Original: For Figure 11, we compute the update norm on the mini-batch loss every 100 updates, and report theper-epoch average.
Revised: For Figure 11, we compute the update norm on the mini-batch loss every 100 updates, and report theper-epoch average.

Original: Most SMTOs have a smaller update magnitude than unitary scalarization in theﬁrst 15 epochs.
Revised: Most SMTOs have a smaller update magnitude than unitary scalarization in theﬁrst 15 epochs.

Original: However, towards convergence, (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 is larger for SMTOs comparedto unitary scalarization.
Revised: However, towards convergence, (cid:13)(cid:13)(cid:80) i ∈T ∇ θ (cid:107) L i (cid:13)(cid:13) 2 is larger for SMTOs comparedto unitary scalarization.

Original: In particular, IMTL and MGDA, both of which were shown to have alarger convergence set in § 5.2, display the largest norm.
Revised: In particular, IMTL and MGDA, both of which were shown to have alarger convergence set in § 5.2, display the largest norm.

Original: The additional stochasticity of RLW and
Revised: The additional stochasticity of RLW and

Original: GradDrop also appears to lead to larger norm values than unitary scalarization, yet to a lesser degree.
Revised: GradDrop also appears to lead to larger norm values than unitary scalarization, yet to a lesser degree.

Original: Given that SMTOs incur a larger loss than unitary scalarization (see Figure 8(a)) in the later epochs,we can conclude that SMTOs guide optimization towards regions of the parameter space whichunder-optimize the loss, providing empirical support for our analysis. 
Revised: Given that SMTOs incur a larger loss than unitary scalarization (see Figure 8(a)) in the later epochs,we can conclude that SMTOs guide optimization towards regions of the parameter space whichunder-optimize the loss, providing empirical support for our analysis. [Figure]

Original: D.4 Sign-Agnostic GradDrop
Revised: D.4 Sign-Agnostic GradDrop

Original: We will now present an ablation study on GradDrop, investigating the effect of the sign of per-taskgradients on the SMTO’s performance.
Revised: We will now present an ablation study on GradDrop, investigating the effect of the sign of per-taskgradients on the SMTO’s performance.

Original: Speciﬁcally, we compare the performance of GradDropwith a sign-agnostic version of its stochastic gradient masking (which we refer to as “Sign-Agnostic
Revised: Speciﬁcally, we compare the performance of GradDropwith a sign-agnostic version of its stochastic gradient masking (which we refer to as “Sign-Agnostic

Original: [Figure]

Original: GradDrop"), whose update direction is deﬁned as follows:
Revised: GradDrop"), whose update direction is deﬁned as follows:

Original: [Figure]

Original: [Equation]
Revised: [Equation]

Original: where u i , ∇ z L i ∈ R n × r and, for all i ∈ T , u i is i.i.d.
Revised: where u i , ∇ z L i ∈ R n × r and, for all i ∈ T , u i is i.i.d.

Original: according to u i [ j, k ] ∼ Bernoulli ( p ) ∀ j ∈{ 1 , . . . , n } , k ∈ { 1 , . . .
Revised: according to u i [ j, k ] ∼ Bernoulli ( p ) ∀ j ∈{ 1 , . . . , n } , k ∈ { 1 , . . .

Original: , r } .
Revised: , r } .

Original: Differently from a similar study carried out by Chen et al.
Revised: Differently from a similar study carried out by Chen et al.

Original: [11],we tuned the hyper-parameter of the sign-agnostic masking in the following range: p ∈{ 0 . 1 , 0 .
Revised: [11],we tuned the hyper-parameter of the sign-agnostic masking in the following range: p ∈{ 0 . 1 , 0 .

Original: The experimental setup complies with the one described in appendix C.1.
Revised: The experimental setup complies with the one described in appendix C.1.

Original: Figure 12, plotting testand validation results for the CelebA dataset [43], shows that the performance of Sign-Agnostic
Revised: Figure 12, plotting testand validation results for the CelebA dataset [43], shows that the performance of Sign-Agnostic

Original: GradDrop closely matches the original algorithm.
Revised: GradDrop closely matches the original algorithm.

Original: Therefore, sign conﬂicts across per-task gradientsdo not seem to play a signiﬁcant role in GradDrop’s performance.
Revised: Therefore, sign conﬂicts across per-task gradientsdo not seem to play a signiﬁcant role in GradDrop’s performance.

Original: [Figure]
Revised: [Figure]

Original: E Supplementary Reinforcement Learning ExperimentsE.1 Addendum
Revised: E Supplementary Reinforcement Learning Experiments

Revised: [Figure] E.1 Addendum

Revised: [Figure]

Original: This section presents additional plots for the RL experiments in § 4.2.
Revised: This section presents additional plots for the RL experiments in § 4.2.

Original: Speciﬁcally, Figure 13 re-plots
Revised: Speciﬁcally, Figure 13 re-plots

Original: Figure 4(a) and 4(b) with the omitted IMTL results, while Figure 14 shows the learning curves omittedfrom § 4.2.
Revised: Figure 4(a) and 4(b) with the omitted IMTL results, while Figure 14 shows the learning curves omitted [Figure] [Figure] [Figure]

Original:  As pointed out in § 4.2, none of the IMTL runs successfully terminated due to numericalinstability.
Revised: from § 4.2. As pointed out in § 4.2, none of the IMTL runs successfully terminated due to numericalinstability.

Original: Indeed, Liu et al.
Revised: Indeed, Liu et al.

Original: [41] show that, in supervised settings, coefﬁcients do not ﬂuctuate muchacross epochs [41, Figure 4, appendix B] and never become negative.
Revised: [41] show that, in supervised settings, coefﬁcients do not ﬂuctuate muchacross epochs [41, Figure 4, appendix B] and never become negative.

Original: By contrast, up to 50% ofthe scaling coefﬁcients α are negative in our experiments, thus reversing subtask gradient directions.
Revised: By contrast, up to 50% ofthe scaling coefﬁcients α are negative in our experiments, thus reversing subtask gradient directions.

Original: MGDA, which constrains the weights, is more stable and is comparable to unitary scalarization.
Revised: MGDA, which constrains the weights, is more stable and is comparable to unitary scalarization.

Original: Inorder to avoid incomplete curves and unfair calculations of the mean, Figure 14 plots the highestvalue ever achieved by any seed as a dashed horizontal line.
Revised: Inorder to avoid incomplete curves and unfair calculations of the mean, Figure 14 plots the highestvalue ever achieved by any seed as a dashed horizontal line.

Original: The IMTL results in Figure 13, instead,report the best average success rate of each seed until its termination.[Figure] [Figure] [Figure]
Revised: The IMTL results in Figure 13, instead,report the best average success rate of each seed until its termination.

Original: cc

Original: [Figure]

Original: E.2 Ablation studies
Revised: E.2 Ablation studies

Original: Figure 18 presents our ablations for MT10 experiments.
Revised: Figure 18 presents our ablations for MT10 experiments.

Original: Due to computational constraints, we ranablations on the unitary scalarization and PCGrad since these are the two methods previously testedin the RL setting.
Revised: Due to computational constraints, we ranablations on the unitary scalarization and PCGrad since these are the two methods previously testedin the RL setting.

Original: Figure 15 shows ablation studies on the effect of regularization on MT10 and MT50.
Revised: Figure 15 shows ablation studies on the effect of regularization on MT10 and MT50.

Original: In spite of CIoverlaps, actor l 2 regularization pushes the average higher on both benchmarks, motivating our use ofregularization for the experiments in § 4.2.
Revised: In spite of CIoverlaps, actor l 2 regularization pushes the average higher on both benchmarks, motivating our use ofregularization for the experiments in § 4.2.

Original: Furthermore, the gap between the averages tends to widenwith the number of updates on MT50, suggesting improved stabilization.
Revised: Furthermore, the gap between the averages tends to widenwith the number of updates on MT50, suggesting improved stabilization.

Original: E.3 Sensitivity to Reward Normalization
Revised: E.3 Sensitivity to Reward Normalization

Original: Figure 16 shows that multitask agent performance is highly sensitive to the reward normalizationmoving average hyperparameter 3 motivating our buffer normalization in Section 4.2.
Revised: Figure 16 shows that multitask agent performance is highly sensitive to the reward normalizationmoving average hyperparameter 3 motivating our buffer normalization in Section 4.2.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Use PCGrad parameters for the baseline (shared alpha)
Revised: Use PCGrad parameters for the baseline (shared alpha)

Original: Increase replay buffer size
Revised: Increase replay buffer size

Original: Normalise reward at the buffer level
Revised: Normalise reward at the buffer level

Original: Regularize the actor
Revised: Regularize the actor

Original: Hyperparameters from [Sodhani et al, 2021]
Revised: Hyperparameters from [Sodhani et al, 2021]