Rotograd: Dynamic Gradient Homogenization for Multitask Learning
Authors: Adrián Javaloy
Isabel Valera
Venue:  (2021)
Keywords: multitask learning
deep learning
gradnorm

Abstract: GradNorm (Chen et al., 2018) is a broadly used gradient-based approach for training multitask networks, where different tasks share, and thus compete during learning, for the network parameters. GradNorm eases the fitting of all individual tasks by dynamically equalizing the contribution of each task to the overall gradient magnitude. However, it does not prevent the individual tasks’ gradients from conflicting, i.e., pointing towards opposite directions, and thus resulting in a poor multitask performance. In this work we propose Rotograd, an extension to GradNorm that addresses this problem by dynamically homogenizing not only the gradient magnitudes but also their directions across tasks. For this purpose,Rotograd adds a layer of task-specific rotation matrices that aligns all the task gradients. Importantly, we then analyze Rotograd (and its predecessor) through the lens of game theory, providing theoretical guarantees on the algorithm stability and convergence. Finally, our experiments on several real-world datasets and network architectures show that Rotograd outperforms previous approaches for multitask learning.



Under review as a conference paper at ICLR 2021
ROTOGRAD: DYNAMIC GRADIENT HOMOGENIZATION FOR MULTITASK LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
GradNorm (Chen et al. 2018) is a broadly used gradient-based approach for train- ing multitask networks, where different tasks share, and thus compete during learning, for the network parameters. GradNorm eases the fitting of all individ- ual tasks by dynamically equalizing the contribution of each task to the overall gradient magnitude. However, it does not prevent the individual tasks' gradients from conflicting, i.e., pointing towards opposite directions, and thus resulting poor multitask performance. In this work propose Rotograd, an extension to GradNorm that addresses this problem by dynamically homogenizing not only the gradient magnitudes but also their directions across tasks. For this purpose, Rotograd adds a layer of task-specific rotation matrices that aligns all the task gradients. Importantly, we then analyze Rotograd (and its predecessor) through the lens of game theory, providing theoretical guarantees on the algorithm stabil- ity and convergence. Finally, our experiments on several real-world datasets and network architectures show that Rotograd outperforms previous approaches for multitask learning.
INTRODUCTION
While single-task learning is broadly-used and keeps achieving state-of-the-art results in different domains- beating human performance, there are countless scenarios where we would like to solve several related tasks, we encounter overfitting problems, or the available data is scarce. Multitask learning is promising field of machine learning aiming to solve or at least, alleviate- the aforementioned problems that single-task networks suffer from (Caruana (993)..
Many multitask architectures have emerged throughout the past years (Kokkinos) 2017 Maninis al. 2019 Dong al., 2015 He et al., (017)) yet most of them fall under the umbrella of hard pa- rameter sharing (Caruana 1993). This architecture is characterized by two components: i) shared backbone which as an encoder, and produces an intermediate representation shared across tasks based on the input; and ii) set of task-specific modules that act as decoders and, using this inter- mediate representation input, produce an specialized output for each of the tasks.
These networks have proven to be powerful, efficient, and in many occasions, capable to improve the results of their single-task counterparts. However, they can be difficult to train. Competition among tasks for the shared resources can lead to poor results where the resources are dominated by a subset of tasks. Internally, this can be traced sum over task gradients of the form respect to the backbone parameters. In this setting two undesirable scenarios may occur, as depicted Figure la i) a subset of tasks dominate the overall gradient evaluation due to magnitude differ- ences across task gradients; and ii) individual per-task gradients point towards different directions in the parameter space, cancelling each other out.
Ee V LL with
This problem can be seen particular "sharing information with unrelated tasks might actually hurt performance" (Ruder) 2017). Under the assumption that the problem comes from task unrelatedness, one solution use only related tasks. Several works have explored this direction (Thrun & Oullivvnn 1996 Zamir et al. 2018), for example Standley et al. (2019) for recent in-depth study task relatedness.
of negative transfer, a phenomenon describing when
Another way of tackling the problem is by weighting the task dynamically during training. the motivation being that with this approach the task contribution can manipulated will and

---

Under review as a conference paper at ICLR 2021
u
u
>L,(%, Vi)
(a) Magnitude disparity.
hd
Y
Yx Lx (8x, Xx)
(c) Hard parameter sharing model.
(b) Conflicting directions.
Figure I: Examples of: (a) neglecting one vector due magnitude difference; (b) partial cancella- tion of two vectors with equal magnitude; (c) a hard parameter sharing architecture.
good balance between gradient magnitudes can be found. Two important works on this direction are GradNorm (Chen et al. 2018), which we will cover in detail later, and Kendall al. (2018), which adopts probabilistic approach to the problem. For example Guo et al. (2018). adapts the weights based on their "difficulty" to solved to prioritize harder tasks.
Furthermore, series of papers attempting to solve the conflicting gradients problem in different settings have recently emerged. For example, Flennerhag et al. (2019) deals with similar problem in the context of meta-learning, and Yu et al. proposes two different heuristic solutions for rein- forcement learning. In the context of multitask learning, Levi & Ullman (2020) adopts an image- dependent solution that uses top-down network, and Maninis al. (2019) applies adversarial training to deal with conflicting gradients, making use of double-backpropagation which poorly
scales.
The key contributions of this paper are the following:
The introduction of Rotograd, an extension of GradNorm that homogenizes both the mag- nitude and the direction of the task gradients during training. Different experiments support this statement, showing the performance gain provided by Rotograd.
A novel interpretation of GradNorm (and Rotograd) as gradient-play Stackelberg game, which allow us to draw theoretical guarantees regarding the stability of the training process long as GradNorm (Rotograd) is asymptotically slower learner than the network's optimizer.
A new implementation of GradNorm where is it applied to the intermediate representation instead of a subset of the shared parameters. This way remove the process of choosing this subset. Besides, we give some guarantees regarding the the norm of the gradient with respect to all the shared parameters, and empirically show that results are comparably good to the usual way of applying GradNorm.
BACKGROUND
2.1 MULTITASK LEARNING
Let us consider K different tasks that we want to simultancously learn using a gradient-based ap- proach. All of them share the same input dataset X E NDD where each row æn E R is D-dimensional sample. Additionally, each task has its own outputs Y, E Nx 1x, and loss function gh Vv x Y + R.
As mentioned in Section T and illustrated in Figure Ic the model is composed of shared backbone and task-specific modules. Upon invoking the model, first the input X E TR is passed through the backbone, ho : X - Z, which produces a common intermediate representation of the input, Z E Bxd where B is the batch size and the size of the shared intermediate space. Then, each lask-specific module, for : 2 + Ys. using the intermediate representation Z as input, predicts the desired outcome for its designated task, Y
2

---

Under review as a conference paper at ICLR 2021
As the focus of this work concerns only the shared parameters, simplify the notation garding the task-specific pipeline- that is, the branching paths shown in Figure Ic by writing the single-point k-th loss evaluation time e (t), instead of ethh (fot (z()). and denoting by (t( = E (t) the total loss for the k-th task.
When it comes to training, it is necessary to single loss function in order to backpropagate through it. In our setting, however, there is a loss function & per task. common approach is to combine these losses by taking weighted sum, that the training objective is to minimize the loss function [(t) == Σx wk La Therefore, the common parameters 0 are updated following the direction of the gradient
K
VoL(t)= = Σ Wk (oL.(1)
(1)
As we noted in Section Equation turns out to be the main source of problems since the loss functions-and thus the tasks-have been artificially coupled. The magnitude of these task-specific gradients represent how much they contribute the parameter update: if some of them relatively small (large). their tasks will be overlooked (prioritized) during the training procedure, as Figure TTa exemplifies. More subtly if gradients from different tasks point completely different regions of the parameter space, they may cancel each other out, resulting in direction update potentially harmful for these tasks, as shown in Figure Tb
2.2 STACKELBERG GAMES
Let us now introduce some game theoretical concepts necessary for the following sections. A Stack- elberg game (Fiez al. 2019) is type of asymmetric game where two players play alternately. The first player is the follower F, whose sole objective is to minimize its own loss function. The other player is the leader L who also attempts to minimize its loss function, but having access to additional information regarding which will be the best response the follower could employ the next turn. In mathematical terms.
Ceader: min (((11,,))] ø E argmin F(z,, v()) x,x,;
MEX,
(2)
Follower: min F(m,,yy) x,EX,
where and xyy are the actions of the leader and the follower, respectively.
Another important concept in game theory is that of a local equilibrium point. layman's terms, a local equilibrium point in a game between two players is one in which both players have reached state that satisfies them, meaning that there no available move improving any of the player scores and further actions taken. In case
interested in the following definition:
Definition 2.1 (differential Stackelberg equilibrium). The pair points xf E X1, . X,,, where r = r<xf) ) is implicitly defined by F2 = = 0, is a differential Stackelberg equilibrium V ((xt,(((r)))) r(=))) = 0, and Vi, C(xj, r(=ī)) is positive definite.
As final remark, note that in this particular concept of equilibrium both players reach critical point (local optimum). Even more, when both players optimize their objectives using a gradient-basee approach and the leader is slower learner-meaning having an asymptotically smaller learning rate, both players reach a local minimum on the equilibrium point (Fiez et al. 2019).
3 THE GAME OF GRADNORM
GradNorm (Chen et al. 2018) was conceived with a simple idea in mind: equalizing the contribution of each task to during training such that they all are learned at the same rate. This idea, observed in Equation] I means making the gradient magnitudes relatively equal across all tasks. GradNorm achieves this by considering the task weights w/ parameters that modified during training.
3

---

we refer Under review as a conference paper at ICLR 2021
These parameters trained alongside the original network parameters via gradient descent, yet their gradients are obtained by minimizing a specific loss function:
: K Vl x lrx(0)=
grgr (wk, 0) == well V
(3)
where W is a subset of the shared network parameters 8, the weights are corrected such that Ex W/ K always, is hyperparameter and [t) balances the learning speed per task, and the average gradient magnitude (that is, the target magnitude) is treated as a constant.
Using the concepts from Section 2.2 in this new point of view the follower would be the player whose objective to optimize the pa- rameters of the backbone, whereas GradNorm would play the role of the leader, optimizing the task weights in order to equalize the gradient magnitudes:
reinterpret GramNorm
Stackelherg game. Namely,
Leader: min
Σ Clvead(wig) (wis, O), Follower: min Σ (0). (w
(4)
Note that there is fundamental difference between Equation and the extra information available to the leader is not explicitly written in its optimization problem. This is due to the more subtle nature of this bit of information. The actual objective of the leader is not to equalize the magnitudes at the current step t. but at + 1, since that is when the effect of updating wk will take effect. Therefore, the extra information that the leader possesses is that the follower updates its parameters in a smooth way and the gradient magnitude between consecutive steps are similar.
This is, to the best of our knowledge, the first time GradNorm has been interpreted as a (stochastic) basedd Stackelberg game. The main advantage of this point of view is that, based the results presented by Fiez et al. (2019), we can provide theoretical guarantees regarding the con- vergence of the learning dynamics between the leader (GradNorm) and the follower (network's optimizer). Specifically, adapted from Fiez et al. (2019):
Proposition 3.1. Suppose a finite-fime (stochastic) gradient-based Stackelberg where the learning rate of the leader tends to zero on a faster timescale than the follower, that is, on(1) = ola (1)). Then, the game will with high probubility converge to neighborhood of stable differential Stack- elberg equilibrium.
This previous statement means in terms of the training dynamics of the network- that, long as the leader is the slow learner, the parameters of the leader will stabilize at some point and the network parameters will end up in local optimum, In other words, the training will be stable.
ROTOGRAD
4.1 Z AS THE CRITICAL SPACE
As additional remark Section 3. we believe that it is worth noting that GradNorm is applied to only subset of the network parameters W C 0 for efficiency reasons. The selection of such subset is left the practitioner and, the best of our knowledge, there exists no intuition regarding which parts of the backbone are preferred to be selected as W. In this work, we propose to apply GradNorm on the last shared representation of the network, which as shown in Figure Ic to as Z. In other words, we assume that W = Z.
Since the size of Z tends to be orders of magnitude smaller than the total number of shared pa- rameters 0, that is, B x d << [0), this choice saves computation. Moreover, we argue that this selection is the most logical one since the intermediate representation Z is the last shared represen- tation across all tasks, shown in Figure Ic] In order to further motivate this selection of W we provide and proof proposition in Appendix A. stating that, under the assumption that V Z has linearly independent columns (which is mild since 0 >> d). the magnitude of the gradient with respect to the (arbitrarily many) shared parameters 0 is bounded on both sides, which is in stark contrast with the lack of guarantees when W is selected (strict) subset of 0.
4

---

Under review as a conference paper at ICLR 2021
+6.0%.)))
X
10
V22
+, Y2)
2200
Yx
fg(Yx, (x)
Figure Extended hard parameter sharing model with task-specific representations.
4.2 ROTOGRAD: A BETTER LEADER
In Section saw that GradNorm's 's objective is to equalize the gradient magnitude that, in the next step, the shared parameters are updated according to the necessities of all the tasks. GradNorm does address the conflicting gradients problem (Figure lbb in the total gradient computation (Equation [. which can be as detrimental as the one of magnitude disparity (Figure Ta Fortunately, the reinterpretation of GradNorm fix this problem. More specifically, we introduce a new leader that aims to homogenize both the gradient magnitudes and directions for all tasks. We refer to the resulting algorithm as Rotograd.
Stackelberg game presented in Section 2.2 allow us to easily
Rotograd extends GradNorm by additionally expanding the model's architecture as shown in Fig- ure 2. As before, E R Bxd represents the shared intermediate representation, and now Z BaX denotes specifiec intermediate representations of Z. one per task. This new space Z is shifted--and-rotated version of Z, such that the i-th instance of transformed as 2 Re + da. where R2 E (O(d) is rotation matrix and de E Fd translation vector.
For the k-th task at step f, Rotograd optimizes the weights we as GradNorm, and finds the rigid motion that minimizes the distance between the the next set of shared points {2] (2 to the next set of points we would have reached if only that task were being trained {a the additional knowledge of the leader plays role, it knows the next evaluation points given the current ones, concept that we represent using two oracle functions F (z))) and (z for the multi-task and single-task next points, respectively.
12 J=r- ft=1- Note that
This new objective function can be mathematically expressed as
B
Lo (RR, du, 0) = B
[R(O(2z) + du -
(5)
resulting in the Stackelberg game:
Leader: minimize (wh,Ri )x
[Itoo (Rx, dx, 0) + I VErad(wk: 0)]
(6)
Follower: minimize
Σ
we Ln)).
4.3 SOLVING THE NEW OBJECTIVE FUNCTION
At first glance solving the newly-formulated problem Lyot seems quite challenging; we cannot write down the oracle functions, and matrix. In this section we show how to approximately solve this problem in closed form.
constrained optimization problem since R& is an orthogonal
Recall that, the current step t. we can obtain any current intermediate point in the shared space Rf d' To simplify notation, let
as 2 heg (æf), and its task-specific equivalent as 2, us denote the gradient of the k-th task with respect to z* at step f. by = V (z k.t and the gradient of the weighted loss with respect to at step gf vzl(z,) Σx W.R
As mentioned before, we cannot in general write the oracle functions in closed form. However, since assume that the network parameters optimized using first-order optimization method, we can approximate the oracle functions by assuming that the next point will follow the gradient
These could he simulated by training the model with all and single task at each step, respectively.
5

---

Under review as a conference paper at ICLR 2021
direction, that is, approximate the oracles
O 2 Sgh and
Ox(2, 2.t Bgg
where we treat the step-size 3 as an unknown variable.
By plugging this approximation the oracles into the loss function in Equation S and using the law of cosines we can approximately solve min T. RR, d2, 0) by solving instead
min EO((())
Rag 9, 11 and
min dy dR
1
[| d2.
2, 2.."
(8)
B
which are two least-square problems with closed form solutions Rig and dz. Details in Appendix A.2
Note that the first problem in Equation looks for the orthogonal matrix that minimizes the gradient conflict between the weighted gradients of and the gradients of the k-th task 9; which was our main objective for this extension. The second problem looks for the best way of translating the current points in the common space 2 to the current points that were plugged into the task specific k.t
module for the k-th task 2;
Similarly what we did in Section we can apply again Proposition 3.1 thus providing theoretical guarantees on the stable behaviour of the algorithm as long as Rotograd has stronger learning rate decay than the network's optimizer. order for this to hold, we should adopt a gradient descent ap- proach and, since we have an approximation to the solution of the problem, we set the gradient of our parameters to the difference between our solution and its current value. For example, in the case of applying SGD to the parameter Rx, the update rule would be of the form R Rt a a(It 127) where, afterwards, we should make sure that R is rotation matrix.
An overview of the proposed Rotograd algorithm can found in Algorithm
Algorithm Forward and update methods for the k-task using Rotograd.
1: procedure FORWARD_ K(z{)
2: return R*2 d2
Transform to task-specific space Z
3: end procedure
4:
5: procedure UPDATE ROT K({
g Zi 99 (1)
kit k.
6:
Rg + argmin R. π + i=1
|Ra 9, $1.1
P. Solve conflicting gradient problem
7:
di argmind, 1 2i=1
| dx 2 1
Solve translation problem
8:
9:
d + dj aad d;)
Update rule for dę b Update rule for Rx
10:
R + R a - R&)
11:
12:
U,S.VT + SVD(RE+*)
13:
R²(1 +UV
D Ensure that R8+ is a rotation
14:
end procedure
EXPERIMENTS
5.1 QUALITATIVE RESULTS
In order to get better understanding of Rotograd, we provide two simple synthetic examples that show both, its effect on the shared intermediate space Z, and its effect on the final predictions.
First, we consider two different settings where we use two-layer feed-forward NN with 10 hidden units to solve two regression tasks. The first setting problem, illustrated in Figure 38 shows an example where not correcting for conflicting gradients may completely prevent from learning (i.e., none of the tasks is optimized), while Rotograd converges to the global optima of both tasks. The second setting in Figure 3b shows that, due differences in the gradient magnitudes, only one of

---

Under review as a conference paper at ICLR 2021
Correct
04
Wrong
(a) Regression example
(b) Regression example
(c) Classification example.
Figure 3: Figures (a) and (b) show the trajectories followed when using Rotograd in the spaces Z (blue) and Z (brown), versus not using it (red). Circular markers mean the final point of the trajectory, whereas the triangle and star mark the global optimum of each task. Figure (c) shows the deep logistic regression classifier learned using Rotograd predict binary label and its (Correct) and its negated (Wrong).
the tasks is learned when Rotograd is not applied. It is worth porth-mentioning well that Rotograd is still subject to the nature of stochastic optimization, and thus being subject to fall into local optima, shown in Figure 3b
As an extreme example, we consider two classification tasks, which aim to predict opposite labels. As before, we
ferd-forward backbone without task-specific networks, and logistic output for each task. In this case, the two tasks, and therefore their gradients with respect to the network parameter, in conflict. Thus, without additional model flexibility (i.e., without the network cannot learn any of the tasks. In contrast, Rotograd is able to rotate the space and learn that one function is the opposite of the other, learning an accurate classifier, as shown Figure 3c
rotation layer)
5.2 QUANTITATIVE RESULTS
Dataset and tasks. Similar to Sener & Koltun (2018), we employ as dataset a multitask version of MNIST (LeCun et al.) 1998) that has been expanded to contain instead two digits, one in each side of the picture. Regarding the tasks, we learn five different tasks: i) classify the left-side digit; ii) classify the right-side digit; iii) predict the sum of the two digits; iv) predict their product; and v) predict the number of pixels that are activated in the picture. The idea behind these tasks is no other than having a set of tasks that are correlated, highly correlated, and uncorrelated.
Architecture. We use as backbone a modified version of LeNet (LeCun et al., 1998) where the number of parameters has been greatly reduced in order to not have enough parameters to learn all the tasks perfectly, and thus, try to force them to collaborate in order to improve their results. The size of Z is d 25. For the task-specific parts make use of a single layer per task, followed by sigmoid activation function in the case of solving a classification problem. Regarding loss functions, we use mean-squared error for the regression tasks and negative log- likelihood for the classification ones. Furthermore, we divide all losses by their loss at the initial step to normalize them.
Training. We train all models for 150 epochs using batch size of 1024 and Adam (Kingma & Ba 2014) as optimizer. All experiments are averaged over ten different runs. The learning rate of the network parameters are set to 0.001 and the rest of hyperparameters are selected via grid search.
Metrics. We employ the mean-squared eтror for regression tasks and accuracy for classification tasks. Besides, as in Vandenhende et al. (2019), we include an additional metric that summarizes the overall performance of the multitask network:
1 K
Mm.k Ma,k
Am
Σ
1)
(9)
K
Ma
7

---

Under review as a conference paper at ICLR 2021
Exp. decay
Left f
Right f
Sum
Multiply 4
Density
Am f
0.9
89.46 00.57
85.90 00.65
1.64 0.25
133.97 = 7.20
0.26 0.06
0.18 0.06 0.06
0.99
88.99 00.81
84.92 + 00.97
4.58 + 0.16
135.19 + 4.35
0.24 0.05
0.18 0.06
0.999
85.51 = 01.55
80.09 + 01.86
4.79 :: 0.18
142.07 = 3.86
0.24
0.15 4 0.06
0.9999
84.56 = 01.63
79.46 : 01.80
4.81
+ 0.23
142.64 = 4.07
0.24 0.06
0.15 0.06
1.0
83.55 01.69
79.31 + 02.43
4.88 = 0.20
143.41 = 3.17
0.25 +: 0.13
0.13 111300.00 0.09
R R2
64.73 = 04.83
58.55 05.94
6.11 0.26
193.93 = 9.53
0.89 0.21
-0.23 0.21
Table I: Task performance on MNIST for different leader's learning speeds. Note that multitask performance gradually worsens as the leader learns faster, being the case where no iterative updates are performed clearly unstable.
LeNet
Method
Left f
Right ^
Sum
Multiply |
Density
Am. f
uniform
96.40
00.17
94.99 00.16
1.74 1: 0.39
38.07 1.21
0.28 :: 0.07
0.13 1: 0.20
original
rotograd
96.45
00.14
94.89 00.16
1.67 + 0.33
38.70 + 1.89
0.27 = 0.08
0.15 + 0.18
wilform
90.15
00.53
86.65
00.41
5.14 + 0.33
149.21 + 5.97
0.51 + 0.02
0.06 + 0.11
reduced
rotograd
89.01 00.87
84.62 = 01.19
4.54 + 0.19
134.95 = 5.92
0.23 = 0.04
0.18 + 0.06
Table 2: Task performance on MNIST for Rotograd and a baseline method using the same architec- ture with different number of parameters. As observed, the effect of negative transfer diminishes as the model capacity increases.
where M k and M., are the performance on the k-task for the multitask and single task networks, respectively, and la is 1 if lower value means better performance and 0 otherwise. This metric simply represents the average performance gained per task.
5.2. ABLATION STUDY
In order to better understand different aspects of the proposed algorithm, we perform a simple abla- tion study where only the quantities of interest are modified.
Slow-learner leader. To understand the negative effect that having fast-learner leader can have in the results, we set the initial learning rate of the leader to 0.02 and, at each iteration, exponential decay is applied to it, being the decay rate the quantity that changes across experiments. Moreover, and as an extreme case, we consider the case where the parameters of Rotograd are directly set to their approximated closed-form solutions (R = RR), which would further justify the use of iterative updates introduced in Section
Table T show the task performance for different types of learning speeds. Results show that there is clear trend relating the multitask performance and the decay rate applied to the leader. As this value gets closer and closer to one, the leader becomes effectively a faster learner than the follower, and instabilities in the training process (which is easily observed in the classification tasks' variance) result worst performance. As seen in the last row of Table (and in Appendix(A.3 when iterative updates are omitted the training becomes extremely unstable.
Model capacity. As the number of shared parameters increases, it is sensible to think that the model will have better time solving the same set of tasks, it has more capacity store the same information and shared parameters are more likely not to be shared. Therefore, we expect the effect of negative transfer to be less noticeable on bigger models. To verify this, we run Rotograd well as a baseline method where uniform weights on two different architectures: the original LeNet (LeCun al. 1998); and the reduced version mentioned at the beginning of the section. Results shown in Table 2 confirm this intuition since results on the original architecture are statistically identical for both methods, yet the reduced version negative transfer has more impact and Rotograd is able to alleviate its effect.
Note that this is equivalent using SGD as optimizer and setting the learning rate one.
8

---

Under review as a conference paper at ICLR 2021
Method
Left f
Right 1
Sum
Multiply
Density 4
Am f
single task
93.50
00.47
90.65 00.46
6,44 4.63
159.08 6.16
1.62 + 1.78
uniform
90.15
00.53
86.65 00.41
5.14 0.33
149.21 5.97
0.51 + 0.02
0.06 0.11
rotograd
89.01 + 00.87
84.62 + 01.19
4.54
134.95 + 5.92
0.23 +: 0.04
0.18 0.06
rotograd-sgd
87.30 + 00.36
83.48 + 00.69
5.08 0.31
161.86
8.71
0.30 0.09
0.12
gradnorm Z
89.15 + 00.58
85.32 : 00.72
5.07 +.30
144.91
8.51
0.31 0.07
0,14 0.05
gradnorm W
89.60 + 00.52
85.72 + 00.70
5.13
151.58
+ 8.01
0.44 0.28
d-grad
88.71 + 00.69
84.96 - 00.96
4.93 - 0.38
144.54
+ 9.88
0.60 = 0.54
0,08 - 0.16
adversarial
89.37 + 00.60
85.58 + 00.53
5.60 + 0.45
160.00
+ 6.98
1.65 = 1.06
-0.25 = 0.43
Table 3:
Test results on the MNIST dataset for different methods. The arrow the each column's title describes whether higher is better or the other opposite.
5.2.2 COMPARING DIFFERENT METHODS
We consider two different baselines: which will be also used to compute ) and uniform, which simply assumes uniform weights and there is no leader involved. Besides, we consider two different versions of GradNorm and Rotograd. First, to understand the effect of using W Z instead of subset of 0, we include these two methods: gradnorm Z and gradnorm W, respectively. Second, to understand the effect of using Ry - -Ry as update direction instead of the usual gradient, we compare the proposed Rotograd with rotograd-sgel, a variation that also optimizes Equation X but relying on automatic differentiation to compute the update direction. Finally, we compare with two different methods: adversarial (Maninis et al.] (2019). which uses adversarial training to correct the gradient directions and two- layer ford-forward neural network as discriminator; and d-grad, an adaption of the method proposed by Yu et al.] for hard-parameter sharing that simply adds learnable affine transformation per task before each task-specific module.
single task, which learns each of the tasks independently (and
Figure 3 summarizes the results obtained. We observe how Rotograd outperforms the rest of meth- ods with regards to the overall improvement performance Am.. Rotograd is able to drastically in- crease the performance
the regression tasks with just a slight trade-off on the classification ones. Not only we can say that there is an improvement, but also that approximately solving the problem in Equation 73 in closed-form
important for as rotograd-sgd is far from Rotograd's results. When it comes to GradNorm. Table T shows that there is no disadvantage on applying the algorithm to Z. thus removing the selection of the hyperparameter W. In Appendix [A.3 and A.4 we show further results regarding the metrics and cosine similarity evolution during traning. as well as some preliminary results on a larger dataset such as ChestX-rayl4 14 (Rajpurkar et al., (017).
Finally, it is worth mentioning that d-grad, and adversarial perform considerably worse. Our in- tuition for these results are that: i) d-grad adds more parameters to each task ssecispeifiic modules, but since the per-task capacity is not the problem, it does not result into much of an improvement; and ii) adversarial suffers from training instability, as it is common with other adversarial train- ing techniques such as Generative Adversarial Networks (Arjovsky & Bottou 2017). Additional experiments on the ChestX-ray 14 dataset are included in Appendix [A.4] where we observed simi- lar patterns in the results for the different methods, being the results of GradNorms and Rotograd comparable, as seems that such dataset does not suffer from the conflicting gradient problem.
6 CONCLUSIONS
In this work, we have proposed a novel learning algorithm for multitask learning, Rotograd, which aims to minimize negative transfer among tasks by homogenizing the gradient magnitudes and di- rections across tasks. Our starting point here has been GradNorm, which we have reinterpreted as Stackelberg game where GradNorm act as the leader and the network optimizer as the follower. With this novel interpretation, we have proposed Rotograd, and derived stability and convergence guarantees for both Rotograd and GradNorm.
As future work, we plan to extend Rotograd to related scenarios, such as soft parameter sharing networks and meta-learning, which could gratefully benefit from using our algorithm. We also leave for future research trying Rotograd in heavier experiments that are affected by negative transfer, as it could happen in the field of computer vision.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint 1eXiv:1701.04622 2017.
Richard Caruana. Multitask learning: A knowledge-based source of inductive bias. Proceedings of the Tenth International Conference on Machine Learning, pp. 41-48. Morgan Kaufmann, 1993.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International Conference on Machine Learning, pp. 794-803 PMLR, 2018.
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. Multi-task learning for mul- tiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp.
2015.
Tanner Fiez, Benjamin Chasnov, and Lillian Radiff Convergence of learning dynamics in stack- elberg games, arXiv preprint arXiv: 906.01217, 2019.
Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. Meta-learning with warped gradient descent. arXiv preprint arXiv: 1909.00025, 2019.
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei. Dynamic task priori- tization for multitask learning. In Proceedings of the European Conference Computer Vision (ECCV), pp. 270-287, 2018.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969 2017.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7482-7491, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rX.v:1412.6980, 2014.
lasonas Kokkinos. Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6129-6138, 2017.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to 1998.
document recognition. Proceedings of the IEEE,
Hila Levi and Shimon Ullman. Multi-task learning by top-down control network. arXiv preprint arXiv:2002.03335. 2020.
Kokis-Ko Maninis, Ilija Radosavovic, and lasonas Kokkinos. Attentive single-tasking of multi- ple tasks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1851-1860, 2019.
Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang. Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, et al. Chexnet: Radiologist-level pncumo- nia detection on chest x-rays with deep learning. arXiv preprint arXiv: 2017.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint 2017.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-oojective optimization. Advances in Neural Information Processing Systems, pp. 527-538, 2018.
Trevor Standley, Amir R Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? arXiv preprint arviv: MrXiv:19955 0755035533, 2019.
10

---

Under review as a conference paper at ICLR 2021
Sebastian Thrun and Joseph O' O'Sullivan. Discovering structure in multiple learning tasks: The to algorithm. ICML, volume 96, pp. +899-97, 1996.
Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Luc Van Gool. Branched multi-task networks: deciding what layers to share. arXiv preprint arXiv: 19t.:9994.02920, 2019.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Multi-task reinforcement learning without interference.
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3712-3722, 2018.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 BOUNDING THE MAGNITUDE GRADIENT W.R.T THE SHARED PARAMETERS
Proposition A.I. for every task lae V I. (4)1l2 C, where C is constant value, and the rank of V d, then:
C
II 1[Vo2))1l2 S VoLa & 22 for all h: =, K (10)
where (Vo2)t is the left inverse of v,Z, that is, (Va2)+ VoZ
Proof. First, we prove the upper bound, which is a direct consequence of the chain rule and the submultiplicative property of the 2-norm:
||wx (2Lx = I|wa 2 (t)ll & 11V2212 C.
(11)
To obtain the lower bound we use the reasoning as before, now with the addition that, as rank(Va d, the left inverse matrix (V+Z)+ exists:
||w Z (t)ll2 = [V V VoZVZ L
S ll(V,2) + |1, w/ V (z)ll
11(Vo VoLa
which implies that
C
Volx
(12)
I[Vo2)+1l2
Q.E.D.
A.2 APPROXIMATION OF ROTOGRAD'S LOSS
In Section 4.2 we had the optimization problem
L (Rk, dg. 0)
Σ [+((022)) + d, Ox(2
(13)
and claim that, by plugging in the approximation the oracles
O(2)) 2 - -3gt. and On(z,tt) R =
R.t.
(14)
we could get closed form approximation for the solution of the problem in Equation 13 Here
[ougging[4[inte are going to derive this approximation. First, by plugging 14]into 13 we can split the latter into three different problems:
B
L (Rk, dx, 0) = [R + dr (2(
1
Σ
B
i=1
1
||a((1| - -B9!) dz - (2, 3g. 112
B
1
[1(R22) +d 2./1) 3( Ragi)ll
B
B
1 B
Σ
|| 2 + cs - 2 2/..1112
(15)
B
+
3 1
|1 R1oil12
(16)
B
1=1
+ 23
1 B
(R + dj.
- z, ,g. -Rag))
k.t k,t
(17)
1=1
12

---

Under review as a conference paper at ICLR 2021
Note that Equation 16 only depends the rotation matrix R₃ and it is least-square problem, for which there closed form solution. In particular, it can be shown that the solution for Equation I6 is R UVT, where U and V are the two orthogonal matrices obtained in the Singular Value Decomposition of the d x matrix obtained of multiplying the task-specific gradients 91 with larget ones g. that is, l .... gg) x [9] k. 92 ... 9
R = UEVT. Note that in order to ensure that
the matrix a rotation, we should make sure that its determinant is one.
Similarly, it is straight-forward to show that the solution of the least-square problem for the transla- tion dg with respect Equation is given by dz zkt R where * and o are the averages over the batch of 2, and z/, respectively. So, instead of having in Rotograd dz as our translation parameter, we can have two parameters z* and z, that we update using the gradient update z* * and ž², respectively, and use the transformation 2 Rz x) + z" when doing the forward pass.
Finally, note that we can afford ignore the eгтоr term from Equation 17 as long as our solutions in Equation 16 and 15 are good enough. The reason is that this term is nothing more but the scalar product between the residuals of the two previous problems. The error on this term will never be bigger than the maximum error committed in the other two problems.
A.3 FURTHER EXPERIMENTAL RESULTS
In this section we present two additional plots regarding the experiments with MNIST. Specifically, FigureHajjhowss the validation metries during training for one run of Rotograd and uniform, showing how Rotograd outperforms the rest of the tasks in the regression metrics (being more stable in the density task), while having almost the same accuracy of the classification tasks (limited the model capacity). More importantly, Figure shows the cosine similarity between the task gradient Vz(((z) and the averaged gradient (=((z) (taking the median of the population), that is, how similar is the direction these two gradients point towards. This value is between -1 and 1, being that they point to opposite directions, that they are orthogonal (which the most often case in high dimensional spaces), and 1 that they are pointing to the same direction.
Figure 4b clearly showcases the effectiveness of Rotograd in regards of homogenizing the direction of the lask gradients while providing a stable training. While keeping almost the same cosine similarity on the classification tasks, Rotograd highly increases the cosine similarity of the regression ones, being worth mentioning the of the density task, which is pretty much orthogonal (i.e., unrelated) to the rest tasks, achieving a cosine similarity of almost 0.5 when Rotograd is applied.
13

---

Under review as a conference paper at ICLR 2021
val.meittc.onneltyym
vOLumerni__Ht MFL.c_L
vLL ww_.uuuupyyyeeeee
100
20-3
700
100
03
86
www.menii..iihh..rcc
sumLIns
501
0.8
0.5
0.4
40
0.2
20
80
80
100
140
60
100
120
140
(a) Metrics over validation set.
cosnee coEne.smiantty www.arttitty........mmo.... median density
codtie. www.sssarrrtymeeeei.... mecien..eftt
conine wwtp::/meeeeemteeeee..
0.5
0.46
0.
0.4
0.2
0.58
02
0.31
right
cosine similanty sum
088
0.44
0.4
0.4
20
80
20
100
120
140
20
50
80
100
120
140
D4
(b) Cosine similarity during training.
Figure 4: Tensorboard plots with the results of for uniform, gradnorm W, gradnorm Z, rotograd-sgd, and rotograd. Figure (a) shows the validation metrics, where rotograd outperforms in the regression tasks. Figure (b) shows the (median) cosine similarity between the task gradient Vz (z) and the averaged gradient V vz((2), clearly showing how rotograd reduces the direction conflict among task gradients.
the MNIST experiments during training
A.4 CHEST
In Table 4 we show the results obtained in a similar manner as in the MNIST case. We use the same model and dataset as in Rajpurkar al. (2017), using single linear layer for the task-specific modules, and we train using only a 20 % of the data.
14

---

Under review as a conference paper at ICLR 2021
Measure
single tock
uniforn
rotograd
rotograd-sdd
gradnorn Z
gradworm W
d-grad
atelectasis
74.58 + 0.91
77.25
0.77
77.61 + 1.06
68.98 1.87
78.28
0.61
77.85 + 0.63
72.05 + 1.32
cardiomegaly
62.74
3.51
84.43
85.77
effusion
84.92 + 0.72
85.57 + 0.29
85.88
+ 0.48
77.14 1.19
85.82 + 0.40
85.56 + 0.64
79.07 + 1.30
infiltratinn
63.89 +: 1.35
67.65
1: 0.39
67.96
0.33
63.12 0.68
67.62 0.39
67.31 +: 0.49
64.35 1 0.72
52.43 1.30
76.83
1.19
77.50
0.84
77.73 144
78.30
1.61
63.09
1.67
nodule
53.37 1: 2.10
70.03
1: 1.34
70.50
- 0.70
56.22 = 0.98
70.33 2 1.75
70.76
+ 1.33
58.70 = 1.01
pneumonia
56.17 2.78
1.89
72.10
1.13
63.11 1.75
71.35 - 1.00
71.74
66.14
1.67
pneumothorax
67.08 1.90
76.18 2.33
78.73
1.08
2.54
77.19 1.90
77.52
2.55
69.41
1.92
consolidution
70.14
78.28 0.53
0.55 78.38
71.57 = 0 18
77.01 + 1.27
75.17 - 0.61
82.35
emplysema
49.35 +: 1.69
78.57 2.95
80.48 +: 1.96
59.61 2.18
81.80 2 1.57
82.22 1 1.38
65.55 + 1.56
librosis
64.69 2.06
76.47 1.12
77.63
1.71
65.10 2.40
78.07
1.36
77.69
1.44
69.34 0.55
pleural thick.
55.02
0.86
71.16
2.55
72.38
64.59 1.06
hernia
68.58 - 6.36
83.50 1.29
85 09
2 2.99
62.53
7.81
88.87
1.93 19.90
89.23 + 1.81
70.23 + 14.97
Am
22.15
1.37
22.30
1.40
3.75 + 1.17
3.18
24.02
1.28
9.30 2.43
Table Task performance results on the ChestX-ray 14 dataset for different methods. All tasks are classification tasks.
15


Reviews:
[{"review_id": "ZqxUamCQxpR", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In the paper, Rotograd is proposed as a new gradient-based approach for training multi-task deep neural networks based on GradNorm. GradNorm is first formulated as a Stackelberg game, where the leader aims at normalizing the gradient of different tasks and the follower aims at optimizing the collective weighted loss objective. Under this formulation, one can utilize theoretical guarantees of the Stackelberg game by making the leader have a learning rate that decays to zero faster than the follower. To further account for the different gradient directions, a learnable rotation and translation are applied to the representation of each task, such that the transformed representation match that of the single-task learning. By adding an additional term accounting for learning this rotation, the leader in the Stackelberg game will minimize the loss to homogenize both the gradient magnitude and match the representation to single-task learning as close as possible. \n\nIn general, I find the direction of gradient homogenization for multi-task learning very important and interesting. The paper provides an interesting perspective through the Stackelberg game formulation, which provides a framework for selecting the learning rate of GradNorm type of gradient homogenization methods. The other contribution of the paper is a learnable task-specific rotation that aligns the task gradients with single-task learning. The proposing of a learnable rotation matrix seems an interesting idea, although I am not sure if it has been proposed previously for multi-task learning. \n\nI find the first contribution of formulating the problem as a Stackelberg game to be interesting and novel. However, in terms of the second contribution, I have some concerns about whether it makes the most sense by aligning the transformed representation with that of single-task learning. For MTL, one of the key benefits is learning a better representation by sharing it across different tasks to encourage helpful transfer between the tasks; by constraining the transformed representation to be as close to the single-task learning representation, it might limit the transfer between tasks since the representation are constrained to be equivalent to that learned by single-task learning. I think it is helpful to think about using rotation invariant representations for aligning the gradient directions, but it is questionable to align it to that of the single-task learning. \n\nAnother major concern is about the experimental results, full experiments are only conducted on one real-world dataset. The experiment on the second dataset seems to be very preliminary, which might not be sufficient to justify the proposed method empirically. Also on the second dataset, it seems the two different implementations of Rotograd have a large discrepancy in the results, which might need more investigation about why this happens. Meanwhile, many ablation studies seem to be missing. I am mostly interested to see experiments that validate the Stackelberg game formulation, for example by using different learning rates for the leader and the follower. Also, it would be interesting to see how the proposed Rotograd compares with pure GradNorm on gradient direction alignment. Overall, I feel the experiments are not complete for validating the effectiveness of the method. \n\nSome minor points: the description of d-grad method seems to be missing. Also, Yu et. al [2020] also deals with gradient aligning for MTL which could be considered as a baseline to compare with. \n\nYu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2020). Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782. \n\n--------After author's response----------\n\nI am not fully convinced by the explanation of the motivation behind rotation matrix, in particular why it is aligning with the single-task learning, which is counter-intuitive. The authors provided more ablation studies, however, the evaluation on datasets is still quite preliminary with some questions remaining (such as why there is a discrepancy between the two versions of Rotograd on the second dataset). Therefore I am keeping my original score.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Q8wnabSju2z", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper proposes an MTL method that encourages the gradients on shared parameters to have similar directions across different tasks. The motivation is to reduce conflicts between gradients of different tasks, so that training can proceed more smoothly, and fit multiple tasks more easily. The paper introduces a new way of thinking about this kind of method, i.e., through the lens of Stackelberg games, which could be useful in reasoning about the convergence of such methods. The method is shown to perform favorably against related methods, especially in regression settings.\n\n\n\nStrong points:\n\nMinimizing gradient conflict is a well-motivated way to reduce negative transfer.\n\nThe algorithm description is detailed, and should be straightforward for others to implement.\n\nStackelberg games are an interesting framework for thinking about methods like GradNorm and Rotograd that adaptively guide MTL training.\n\n\nWeak points:\n\nThe theory is interesting at a high-level, but it is not clear that it provides insights on what makes Rotograd work. In the paper, one main takeaway from the Stackelberg games framework is that the methods converge if the leader’s learning rate is asymptotically smaller than the follower’s. This takeaway is implemented by decaying the leader’s learning rate, but it is not shown that this is a key point required for Rotograd to work. I would not be surprised if the results were unaffected if this decay were removed. If this point is really important, it should be illustrated in ablation studies. More broadly, since the point does not only apply to Rotograd, this ablation could also be done on Gradnorm and other methods. Such ablations would be one way to connect the theory to the methods.\n\nAnother main takeaway from the theory is that the rotation matrices and translation vectors should be updated with gradient descent, instead of simply replacing them each step. Intuitively, the algorithm would still make sense and be simpler if R and d were simply replaced. Experiments showing that the gradient-descent update rule is necessary would help show the value of the theory.\n\nSimilarly, the value of Proposition 4.1 is not clear. Is it to prove stability? Does this have some particular connection to Rotograd, or is it a useful fact about hard parameter-sharing methods in general?\n\nThere is one ablation “rotograd-sgd”, but it is not clear how exactly it works: Can it simply update R and d however it wants, or is Eq. 9 still used to regularize the updates in some way?\n\nBy adding the rotation matrices, it’s possible that information that would be useful to share across tasks is instead stored in these task-specific matrices. That is, conflict between tasks can beneficially lead to more general representations. Restricting R to be a rotation instead of any matrix is one step towards limiting the amount of information leakage into task-specific parameters. Is there a conceptual reason to expect that the benefits from reducing conflicts will outweigh this leakage?\n\nThe experiments are on an intentionally very small architecture, where one of the main issues is expressivity, which gives Rotograd an edge over methods that do not include an additional task-specific matrix. \n\nIn Section 5.1, does the method without Rotograd do poorly because there are no task-specific networks in that case?\n\nAlthough Rotograd is motivated to reduce negative transfer, Table 1 shows that Rotograd does not reduce negative transfer, but rather improves positive transfer. That is, uniform does better than rotograd in the tasks where single-task is better than multi-task, but rotograd does better than uniform in the tasks where uniform is already better than single-task. This makes me think that the benefits of Rotograd are not coming from reducing negative transfer, but from somewhere else.\n\nIs there an explanation for why Rotograd does not work as well for multi-class classification tasks (i.e., performs worse than all other methods for Left and Right)? Is it because the task-specific heads have larger output sizes? E.g., could it be better to have a separate rotation matrix for each class? Figure 4 in A.3 confirms that there is an issue here: the cosine similarity is not higher for rotograd for the classification tasks.\n\nOverall, from the limited scope of the experiments it is not clear that Rotograd would provide practical advantages over competing methods. The ChestXray experiments show that although Rotograd does not hurt much, it does not help overall compared too uniform.\n\nThat said, it would be still be interesting to see whether insights from Stackelberg games could lead to practical improvements for this problem.\n\n\nMinor comments:\n\nThe writing has some issues. These issues don’t make the work unclear, but they are a bit distracting. Some example suggestions for fixing distracting word choice: “palliate” -> “alleviate”, “spoiled” -> “noted”, “we have not being able to propose Rotograd, but also to derive” -> “we have proposed Rotograd, and derived”. There is also frequent non-standard mixing of em dashes with spaces and commas.\n\n“$[r_k(t)]^\\alpha$ is a hyperparameter” -> “$\\alpha$ is a hyperparameter” The hyperparameter is \\alpha, correct?\n\n----\n\nUpdate: \n\nI am very happy to see the new experiments that validate the implications of the Stackelberg games theory. The main drawback of the paper is that it is not clear that direction homogenization could lead to practical improvements for multi-task learning. The additional experiments in Table 2 are useful, and suggest that much of the benefit comes from the greater expressivity due to task-specific matrices.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "8xR_2doyjng", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an extension of Gradnorm to address task conflicting due to discordant gradient direction. Specially it introduces a rotation matrix to rotate the hidden representation from the last shared layer. The authors put the proposed method in the context of game theory to show stability and convergence of the training, which might be of merit. \n\nThe writing of the paper doesn’t meet the publication standard, needing major work to improve. There are many typos and awkward sentences, hindering understanding of their work. Also, there are many places that need clarification, for example, in Proposition 4.1, the inverse of the gradient of Z with respective to \\theta needs to be calculated. So, what is the shape of this gradient matrix? How it is necessarily to be a square matrix? What ||\\Delta_{\\theta} Z|| represents? the F-norm? There is lack of adequate explanation of the motivation behind the objective in Eq. (6). By reading the paper, I have no idea about the two oracle functions, and why they are defined in the way shown in Eq. (8). \n\nEq. (3) is inaccurate, not aligning with that proposed in the GradNorm paper for the computation of L_{grad}^k.\n\nEq. (9) is problematic. Why R_k z_i^t does not appear in the objective function of the first optimization problem? If this is because z_i^{k,t} = R_k z_i^t + d_k, then the objective in the second optimization problem would be just 0. \n\nWhy operating on z instead of the gradient in Gradnorm can resolve the discordant gradient issue among tasks is not properly justified. \n\nThe reported empirical results are weak and do not support this method works as claimed.  \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is proposing a novel representation of the GradNorm. GradNorm is presented as a Stackelberg game and its theory is used to understand and improve the convergence of the GradNorm. Moreover, in addition to the magnitude normalization, a direction normalization objective is added to the leader and a rotation matrix and a translation is used for this alignment. The paper is reviewed by three knowledgable reviewers and they unanimously agree on the rejection. Here are the major issues raised by the reviewers and the are chair:
- The motivation behind the rotation matrix layers is not clear. It should be motivated in more detail and explained better with additional illustrations and analyses.
- Empirical study is weak. More state of the art approaches from MTL should be included and more realistic datasets should be included.
- The proposed method is not properly explained with respect to existing methods. There are MTL methods beyond GradNorm like PCGrad and MGDA (MTL as MOO). These methods also fix directions. Hence, it is not clear what is the relationship of the proposed method with these ones.

I strongly recommend authors to improve their paper by fixing these major issues and submit to the next venue.
Decision: Reject