Measuring and Harnessing Transference in Multi-Task Learning
Authors: Chris Fifty
Ehsan Amid
Zhe Zhao
Tianhe Yu
Rohan Anil
Chelsea Finn
Venue:  (2021)
Keywords: multitask learning

Abstract: Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in particular, identifying the tasks that would benefit from co-training remains a challenging design question. In this paper, we analyze the dynamics of information transfer, or transference, across tasks throughout training. Specifically, we develop a similarity measure that can quantify transference among tasks and use this quantity to both better understand the optimization dynamics of multi-task learning as well as improve overall learning performance. In the latter case, we propose two methods to leverage our transference metric. The first operates at a macro-level by selecting which tasks should train together while the second functions at a micro-level by determining how to combine task gradients at each training step. We find these methods can lead to significant improvement over prior work on three supervised multi-task learning benchmarks and one multi-task reinforcement learning paradigm.

Under review as a conference paper at ICLR 2021
MEASURING AND HARNESSING TRANSFERENCE IN MULTI-TASK LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Multi-task learning leverage information learned by task benefit the training of other tasks. Despite this capacity, naïve formulations often degrade performance and in particular, identifying the tasks that would benefit from co- training remains a challenging design question. In this paper, we analyze the dynamics of information transfer, or transference, across tasks throughout train- ing. Specifically, we develop similarity measure that can quantify transference among tasks and use this quantity to both better understand the optimization dy- namics of multi-task learning as well as improve overall learning performance. In the latter case, we propose two methods to leverage our transference metric. The first operates at macro-level by selecting which tasks should train together while the second functions at a micro-level by determining how to combine task gradients at each training step. We find these methods can lead to significant im- provement over prior work on three supervised multi-task learning benchmarks and one multi-task reinforcement learning paradigm.
INTRODUCTION
Deciding if two or more objectives should be trained together in a multi-task model, as well as choosing how that model's parameters should shared, is an inherently complex issue often left to human experts (Zhang & Zang|20177 However, human's 's understanding of similarity is motivated by their intuition and experience rather than prescient knowledge of the underlying structures learned by neural network. To further complicate matters, the benefit or detriment induced from co-training relies on many non-trivial decisions including, but not limited to, dataset characteristics, model architecture, hyperparameters. capacity, and convergence (Wu et al. 2020; [Vandenhende ct al., 2019; Standley et al.) 2019, Sun et al. 2019). As a result, a quantifiable measure which conveys the effect of information transfer in neural network would be valuable to practitioners and researchers alike to better construct or understand multi-task learning paradigms (Baxter 2000) Ben-David & Schuller 2003).
The training dynamics specific to multitask neural networks, namely cross-task interactions at the shared parameters (Zhao al., 2018), are difficult predict and only fully manifest at the comple- tion of training. Given the cost, both with regards to time and resources, of fully training deep neural network, an exhaustive search over the 2 possible combinations of m tasks determine ideal task groupings can be infeasible. This search is only complicated by the irreproducibility inher- in traversing loss landscape with high curvature; an effect which appears especially pronounced in multi-tass learning paradigms (Yu et al. 2020; Standley et al. 2019).
In this paper, we aim to take step towards quantifying transference, or the dynamics of informa- tion transfer, and understanding its effect on multi-task training efficiency. As both the input data and state of model convergence are fundamental to transference (Wu et al.] (0020). we develop a parameter-free approach measure this effect at per-minibatch level of granularity. Moreover, our quantity makes no assumptions regarding model architecture, and is applicable to any paradigm in which shared parameters updated with respect to multiple task losses.
By analyzing multi-task training dynamics through the lens of transference, we present the follow- ing observations. First, information transfer is highly dependent model convergence and varies significantly throughout training. Second, and perhaps surprisingly, excluding certain task gradients from the multi-task gradient update for select minibatches can improve learning efficiency. Our

---

Under review as a conference paper at ICLR 2021
Transference ColebA
Transferecce in MeaaWorld
Goudien Sindaa
ruchh
pesshettorrop top
(a)
Figure 1: Transference in (a) CelebA for a subset of attributes; (b) Meta-Workee for "push", "reach", "press button top", and "open window". To determine task groupings, we compute the transference of each task i on all other tasks j. i.c. Z [1) and average over time. For the purpose of illustration, we normalize the transference along each axis. Notice the majority of the tasks in (a) concen- trate around single value for each attribute. Tasks which exhibit transference above this value are considered to have relatively high transference. For instance, A3 exhibits higher-than-average trans- ference on Ag. A1, and As. A similar effect is observed in (b), with "close window" manifesting high transference on "push" and "reach".
analysis suggests this is due to large variation in loss landscapes for different tasks as illustrated in Figure [ Building on these observations, we propose two methods utilize transference in multi- task learning algorithms to choose which tasks to train together as well as determining which gradients to apply at each minibatch. Our experiments indicate the former can identify promising task groupings, while the latter can improve learning performance over prior methods.
In summary, our main contributions are three-fold: we (1) propose the first measure (to our knowl- edge) which quantifies information transfer among tasks in multi-task learning: (2) demonstrate how transference can be used as a heuristic to select task groupings; (3) present a method which leverages minibatch-lever transference to augment network performance.
RELATED WORK
Multi-Task Formulation. The most prevalent formulation of MTL is hard parameter sharing of hidden layers (Ruder 2017 Caruana 1993). In this design, a subset the hidden layers are typically shared among all tasks, and task-specific layers are stacked top of the shared base to output a prediction value. Each task assigned a weight, and the loss the entire model is a linear combination of each task's loss multiplied by its respective loss weight. This particular design enables parameter efficiency by sharing hidden layers across tasks, reduces overfitting, and can facilitate transfer learning effects among tasks (Ruder) 2017 Baxter 2000 /Zhang & Yang 2017).
Information Sharing. Prevailing wisdom suggests tasks which are similar or share a similar un- derlying structure may benefit from co-training in a multi-task system (Caruana [993 1997). A plethora of multi-task methods addressing what to share have been developed, such as Neural Ar- chitecture Search (Guo al.
2020; Sun et al., 2019) Vandenhende et al. 2019 Rusu et al. 2016
Huang et al. 2018 Lu et al. 2017) and Soft-Parameter Sharing (Misra et al. 2016 Duong et al,, 2015 Yang & Hospedales] 2016), improve multi-task performance. Though our measure of trans-
ference is complementary with these methods, we direct our focus towards which tasks should be trained together rather than architecture modifications to maximize the benefits of co-training.
While deciding which tasks to train together has traditionally been addressed with costly cross- validation techniques or high variance human intuition, recent advances have developed increas- ingly efficient algorithms to assess co-training performance. [Swirszez & Lozano (2012) and Bingel & Søgaard (2017) approximate multi-task performance by analyzing single-tast learning character- istics. An altogether different approach may leverage recent advances in transfer learning focused on understanding task relationships (Zamir et al. 2018 Achille et al. 2019b] Dwwivedi & Roig 2019:
Zhuang al. 2020 Achille al. 2019a); however, Standley al. (2019) show transfer learning
2

---

Under review as a conference paper at ICLR 2021
Trandomence
Eppchs
Transforenze Middle Eppch
Transfennce Spucts
Gaadien: Updote
055
-1.
-0.3
B2
Ae
A
Figure 2: Effect of convergence on transference for CelebA attributes Ag, A7, and As. Transference highly dynamic and changes over the course of training. See Section 5.2 for more details.
algorithms which determine task similarity do not carry over to the multi-task learning domain and instead propose a multi-task specific framework.
Training Dynamics. Significant effort has also been invested to improve the training dynamics of MTL systems. In particular, dynamic loss reweighing has achieved performance superior to using fixed loss weights found with extensive hyperparameter search (Kendall et al.) 2018 Guo et al, 2018 Liu et al., 2019; Chen et al.] 2017, [Sener & Koltun| 2018) [Lnn et al. 2019). Another set of methods seeks to mitigate the optimization challenges multi-task learning by manipulating the task gradients in a number of ways such as (1) modifying the direction of task gradients with the underlying assumption that directional inconsistency of gradients on the shared parameters are detrimental to model convergence and performance (Zhao et al. 2018 Suteu & Guo 2019). and (2) altering both the direction and the magnitude of the task gradients Yu al. 2020] Chen al. 2020
Wang et al. 20200b) Instead of directly modifying the task gradients during optimization, our work builds upon these approaches by quantifying how a gradient update to the shared parameters would affect training loss and choosing the combination of gradients which maximizes positive information transfer.
Looking into the Future. Looking what could happen determine what should happen has been used extensively in both the meta-learning (Finn et al., 2017 Nichol et al. [0188] Brinkmeyer et al.)
2019 Grant et al. 22088] Kim et al., (018)) well optimization domains (Nesterov 1983 Hinton & Plaut 1987 Zhang et al. 2019 Izmailov et al. 2018; Johnson & Zhang 2013). Lookahead meta- learning algorithms focusing
validation loss have also been used to improve generalization in
multi-task learning systems (Wang et al. 2020a). and our work further adapts this central concept to
multi-task learning to both quantify and improve information transfer.
TRANSFERENCE IN MULTI-TASK LEARNING
Within the context of hard-paramedd sharing paradigm, tasks collaborate build a shared feature representation which is then specialized by individual task-specific heads to output a prediction. Accordingly, tasks implicitly transfer information each other by updating this shared feature rep- resentation with successive gradient updates. We can then view transference, or information transfer in multi-task learning, as the effect of a task's gradient update to the shared parameters on another task's ' loss during training.
While the the shared parameter update using a task's gradient, often but not always, increases the losses of the other tasks in the network, we find the extent to which these losses change to be highly task specific. This indicates certain tasks interact more constructively than others. Further, we notice this effect to be reproducible and nearly unchanged across successive training runs with varying parameter initializations. Motivated by these observations, we derive quantitative measure of transference, describe how it can be used to determine which tasks should be trained together, and provide empirical analysis of these claims. Later, we will build upon these ideas to derive a new multi-task learning algorithm.
3.1 MEASURING TRANSFERENCE
Consider m-multitask loss function parameterized by (0.) U {0, [m]] where 0, represents the shared parameters and 0g represents the task E [m]] specific parameters. Let
L 0z. (0:))
Σ L,(X', 0.),
(E(m)
3

---

Under review as a conference paper at ICLR 2021
Performance of Reach
Nales
0,0
Performance Push
w/ poss button
w/ open window
0.5
0.0
500
1000 Number Erv Sarg
1500
2000
Figure 3: (Top) Performance of reach when co-trained with "push" and one of "press button top", "close window", or "open window". (Bottom) performance of "push" when co-trained with reach and one of "press button top", "close window", "open window". In both cases, the best perfor- mance is achieved by co-training with "close window". Performance is degraded when co-trained with "open window".
denote the total loss where L, represents the non-negative loss of task i. For simplicity of notation, we set the loss weight of each task to be equal to 1, though our construction generalizes to arbitrary weightings.
For given training batch X at time-step t, we can first update the task specific parameters (0]+1) using standard gradient updates. can now define the quantity 0t+11 to represent the updated shared parameters after a gradient step with respect to the tasks in the non-empty subset Ø C & [m]] Assuming SGD for simplicity, we have
g++1 = 0- -1 EnnEVoon Lo 00,
iEE
This quantity allows us to calculate lookahead loss using the updated shared parameters while keeping the task-specific parameters as well as the input batch unchanged across different subsets of task gradients. That is, in order to assess the effect of the gradient update of tasks in task j. we can compare the loss of task j before and after applying the gradient update on the shared parameters with respect to Ç. In order to eliminate the scale discrepancy among different task losses, we consider the ratio of a task's 's loss before and after the gradient step on the shared parameters a scale invariant measure of relative progress. We then define asymmetric measure for calculating the transference of the tasks in at a given time-step on single task j as
given
L2(2', g1 ( 15+1)
Ze
(I)
L,(X', 05, 09)
Notice that a positive value of - indicates that the update on the shared parameters results in lower loss task j than the original parameter values, while negative value of Z indicates that the shared parameter update is antagonistic for this task's performance. Also, note that for ç { our definition of transference encompasses a notion of self-transference, i.e. the effect of a task's s gradient update on its own loss. This quantity is particularly useful baseline determine whether a subset of gradient updates can result in improved performance when compared with a task's own eefftrransferenc. As discuss in the next section, transference provides effective guideline for choosing the subset of tasks to train together in a multi-task setting.
3.2 TASK GROUPINGS BASED ON TRANSFERENCE
Before using transference to develop a multi-task training augmentation, we aim evaluate if measure of transference is meaningful in practice. To do this, we empirically test whether transfer- ence is predictive of whether group tasks should be trained together. We consider two multi-task learning problems based on the CclebA dataset (Liu et al. 2015)) and the Meta-Worid benchmark (Yu) al., 2019). Compiling transference scores into radar chart, we use Figure To identify groupings of tasks which exhibit beneficial or antagonistic transference. We then evaluate if our heuristic led us to select ideal task groupings by comparing against all other possible task groupings. Unlike prior approaches, our method requires only single training run and is minimally complex, only making additional forward and backward passes through the network which can be done in parallel.
Note that the hackward pass is computed only once and the gradients are calculated (0) U {0 e [m]]-
4

---

Under review as a conference paper at ICLR 2021
Group
Group 2
Co-trained with
Ao
As
A1
Aa
(All Tasks}
92.65
96.76
96.27
87.98
95.14
An
88.00
94.92
A1
92.55
96.77
96.48
87.93
95.20
A2
92.56
96.77
96.51
88.04
95.02
Ag
93.10
06.84
96.56
87.97
95.11
A4
87.92
95.10
As
88.11
95.15
AE 92.48
92.63
96.54
96.14
88. 48.46
95.20
A7
96.66
96.56
As
92.71
96.69
96.34
Table 1: Test Accuracy on CelebA, Group and Group are co-trained with the task(s) in the left column. Group most benefits from co-training with A3 while Group 6 most benefits from co-training with A6.
We first consider a multi-task classification problem by selecting attribute from the CelebA dataset and computing their transference when trained together in single model. Specifically, we compute the transference of each task on all other tasks j the network, i.e. 2 (1)--) While transference computed at a per-minibatch level, we can average the transference across mini- batches to compute epoch-level transference metric. Integrating across the number of steps in training then provides us with an overall (scalar) transference score. Figure (igure[aa shows the transfer- ence score among the 9 attributes in the CelebA dataset. For purposes of illustration, we normalize the transference scores on each task by dividing the values by the task's self-transference. Thus, self-transference becomes for all tasks.
As illustrated in Figure [a), two clusters of strong mutual transference manifest: (1) {Ao. As. As, As} and (2) (As. A7. As). We draw this grouping by choosing subsets of tasks which induce relatively high mutual transference. For instance, A3 demonstrates significantly higher transference on Ao, A4, and As. when compared with the transference of A1, Ag, A6, A7, and As on these tasks. In Table we construct Group I and Group 2 from interpreting Figure [[a) and co-train both groups with all other attributes as shown in the left column of Table |
We find the inclusion A3 in Group (Ao. A₁. and Aş) results in the highest accuracy when compared to co-training with any other attribute. Similarly, As is the best attribute to co-train with A, and Ag in Group 2.
We also consider multi-task reinforcement learning (RL) problem using the Meta-Worke bench- mark (Yu et al., 2019), which contains 50 qualitatively different robotic manipulation tasks. We select five tasks from Meta- World task suite, namely "reach", "push", "press button top", "open window" and "close window". We train these five tasks together using the soft actor critic (SAC) (Haarnoja et al. 2018)) algorithm with the weights of the critic and the policy shared across all tasks. We compute the transference on the critic loss to produce Figure [b). We include more details on the multi-task RL experiment Appendix A.2
Meta-World
Figure [b) indicates that "open window" exhibits relatively low transference with all tasks while "close window" exhibits especially high transference with "push" and "reach". Accordingly, we group "push" and "reach" together and then compute the efficacy of these tasks when co-training with "press button top". "open window", and "close window". As shown in Figure [3 and sug- gested by transference, the success rate of "reach" converges more quickly to significantly higher value when is co-trained with "close window", and marginally faster when co-trained with "press button top", as compared to co-training with "open window". This effect is only magnified when we assess the performance of "push". : For "push", its performance in terms of success rates and data efficiency is greatly increased when co-trained with either "close window" or "press button top" when compared to co-training with "open window"..
In summary, transference can be used as heuristic determine task groupings. A set of tasks which exhibit relatively high transference tend to train effectively together, while tasks which ex- hibit relatively low transference with this set should be excluded. Using this method, our empirical analysis suggests transference is capable of identifying beneficial and antagonistic task groupings in both supervised and reinforcement learning paradigms.
"To avoid possible biases implications conveyed by the definition of the attributes, we omit their names.
5

---

Under review as a conference paper at ICLR 2021
Parametir update in early suge
Parameter updati in early stago
Parameier update later stage
slod
right gradintt step
14
cumbired gradient
Maileach Train
2.72
10
2.0
2.0
3.0
2.70
0.5
1.5
Interpolation
Interpolation
Intepelation
(b)
(c)
Figure 4: Total loss in MultiFashion along linear interpolation between the current and updated shared parameter values. We extend the interpolation
magnitude 3x the original update to
illustrate the curvature along this direction. The dashed vertical line crosses the loss curves the
original update. In all cases, a step along the left gradient is better.
INCREASED TRANSFER MTL
As shown in Section 3.2 the transference measure defined in Eq. ① is effective "macro-level" quantity to recognize the tasks that may benefit from co-training. In this section, we extend the utility of our transference measure beyond determining task groupings by incorporating it directly into the training dynamics of multi-task learning. In particular, we present a parameter-free algorithm which selects the combination of task gradients in each step of training that most increases transference among all tasks. Let us define total transference for the subset of tasks & at time-step I. as
(x(X,
(X,t1 gh++1 1,t1)
z
Σ ]E[m]
BEE = Σ
(I JELm)
(2)
L, 09)
Total transference provides cumulative measure of relative progress across all tasks as a result of applying gradient update to the shared parameters with respect subset of tasks &. Perhaps surprisingly, & = [m]] is not always the set of tasks which most increases transference. Rather, we find that this particular update can often result in worse transference than a gradient update using a subset of tasks, effect especially pervasive the beginning of training.
With this motivation in mind, we present increased transfer multi-task learning (IT-MTL), a parameter-free augmentation to multi-task learning which chooses the gradient that most increases transference. Specifically, IT-MTL chooses the shared parameter update using a subset tasks which induce the highest transference in given minibatch. Formally, define C P((m)) 2 where PSS) denotes the power-set of set S S Although the possible number of task combinations is exponentially large in the number of tasks m, practice and as found in our experiments in Sec- tion 5 carefully chosen subset of tasks of size | = ((m) provides compelling results. Specif- ically, choosing J as the set of m-many leave-one-on
subsets, i.e. [m] - {i) for all i [m]) plus the set of all tasks [ma]] works well in practice. IT-MTL proceeds by calculating the total transference defined in Eq. (2) for each subset of tasks Ş E J and then applies the gradient update to the shared parameters using the subset that induces the highest total transference. Task-specific parameters are updated as normal. The full algorithm is provided in Algorithm
To further illuminate the intuition behind IT-MTL, we present deeper analysis into the loss land- scape of MultiFashion (Lin et al. 2019) in Figure 4 This figure provides insight into several cases where a single task gradient update on the shared parameters is more beneficial than the gradient us- ing the full set of tasks. Figure 4[a) exemplifies the case where high curvature in the direction the right task gradient significantly increases the total loss. Similarly, the combined gradient marginally decreases the total loss while the left gradient significantly decreases total loss. In a related instance, and illustrated in Figure 4b). high curvature in the combined gradient direction, but relatively low curvature in the direction of the left and right gradient, will also lead to single task gradient exhiditing higher transference than the combined gradient.
II other words, for subset & E J and & Ø, either particular task participates in the gradient update, i.c.
i E. not

---

Under review as a conference paper at ICLR 2021
Algorithm Increased Transfer Multi Task Learning
1: Initialize network weights: {0. [ [m]]
Set candidate subsets: J C ((m]]) for O..... :
do
4:
00.00 0.) Vi [m] Compute per-task loss: L,(X*, ,0,
P typical forward pass
Update task-specific parameters: 0,-
= 0 Vo Vi [m]
for & E I do
0 -1) "iEe Ve, La(2' 02 0))
Z Exe(m)) (1
8:
1a(20 00,00)
end for
10: Select max transfer task combination: &* arg max; 2g
11: Update shared parameters: 0f+1 = 02 1j Cieer Vu, Lx 00,00)
12:
end for
While the first two cases of high curvature occur predominantly during the early rounds of training, third case which occurs throughout training is shown in Fe). In this instance, the right task's gradient marginally decreases its own loss but significantly increases the loss of the left task. This causes the combined gradient to only marginally decrease the total loss. On the other hand, the left gradient most increases transference. As result, only using the left gradient significantly improves the total loss. Additional information regarding this analysis can found Appendix A.2.1
4.1 FIRST ORDER APPROXIMATION OF THE INCREASED TRANSFER MTL METHOD
The IT-MTIL procedure requires multiple baakwdd passes to calculate the lookahead losses of tasks in J. This may become computationally prohibitive for large models, especially as the num- ber of candidate tasks in J grows. In this section, we derive a simple first order approximation of IT- MTL which requires only single hewarddbackwart pass. Unlike Algorithm [ the approximation does not update the task-specific parameters before computing the update to the shared parameters, effectively moving line 5 in Algorithm to line 11. Ignoring the learning rate 17 for simplicity, first order Taylor series expansion transference in Eq. (1) yields:
L 0 ali 0
L,(X', 69
L 0² 07)
1, 82 09, "o,' 07,
L,( 1,(2'00 0% 09)
Vo. L, C2 03), Eise Vo (x ,", 01)) (,( 00,,09
where () denotes inner product. Thus, total transference defined in Eq. 2 can be written as
Vo, (x 0,) Li(Xtt 02, 09) 7Eg
Σ
Vo, Vox 00,00)
jE[mJ
jE(m)
which can be rewritten as
-Vo.
log L2( (2",0'000) 00.00)
Σ
Va, Lit"",00
jE(m]
sEg
Vo, log II L2(X* 0r 09),
Vo, Li(* 00 0,)
jE(m)
iEg
log-product loss
Our IT-MTL approximation computes alignment between the gradients of the candidate tasks with the gradient of the first quantity in the inner product, which we call the "log-producct loss. The gradient of the subtasks with the strongest alignment to the gradient of the log-product loss is used to make the final update to the shared parameters. Note in the approximate procedure, the gradients are calculated once, and the approximation has computational complexity similar to that of gradient correction methods such as PCGrad (Yu et al. 2020) and GradNorm (Chen et al. 2017).
7

---

Under review as a conference paper at ICLR 2021
4.2 AFFINITY WITH GRADIENT PROJECTION METHODS
IT-MTL can be combined with related work which modifies gradient direction and/or magnitude. The modified gradient can be added to the set in Algorithm ass additional candidate gradient for the current minibatch. If the modified gradient increases the total transference more so than the gradient of the candidate tasks J. used to update the shared parameters. We explore this idea in our experiments by composing J = {total loss, (ttal loss)} to select between the typical multitask gradient and the PCGrad gradient.
EXPERIMENTS
Motivated by our analysis in Section 4 we study the utility of transference in selecting the com- bination of gradients which increases transference for each minibatch. Unlike our evaluation of transference Section 3.2 on datastes with large number of tasks, IT-MTL is most computation- ally efficient when the number of tasks is small. Accordingly, we focus our evaluation on datasets with either 2 or tasks and perform our analysis on MultiMNIST, multitask variant of the MNIST dataset (LeCun et al. 1998); MultiFashion, multitask variant of the MNISTFashion dataset (Xiao) jet al. 2017); and NYUv2 (Silberman et al., 2012). Further, we found the rate at which a single- task gradient is chosen over the combined gradient to be higher during the beginning of training. With this observation as motivation, we evaluate how information transfer changes with respect to convergence.
5.1 INCREASED TRANSFER MULTI-TASK LEARNING EVALUATION
To assess the efficacy of IT-MTL, we evaluate its performance on the MulüMNIST and MultiFashion datasets. To increase comparability, we run our experiments on the same datasets as (Lin al., 2019) with a multitask variant of LeNet (LeCun et al. 1998). Full experimental details are provided in Appendix A.2.T] and (Lin et al. 2019) can be referenced for dataset construction. The code used in generating experimental results is attached to the supplementary material part of our submission.
For both datasets, we compare against the corresponding single-task baselines, equal weight multi- task learning, PCGrad (Yu et al.] (020)) MGDA-UB (Sener & Koltun 2018), and Uncertainty Weighing (UW) (Kendall et al. 2018). Table 2
summarises our experimental results. Notably, find IT-MTL improves both left and right image accuracy over the -qugh-weight MTL baseline on both MultiFashion and MultiMNIST by choosing the gradient combination which most increases transference. Aside from this augmentation, there is no difference between these two models. More- over, the IT augmentation can be combined with prior approaches to dynamically reweigh the tasks directly modify the gradient by choosing the gradient combination which most increases trans- ference. In particular, our method, and its corresponding approximation described in Section [.] combined with uncertainty weights and PCGrad (IT-UW-PCGrad) achieves very strong performance on both datasets.
To further evaluate the robustness of IT-MTL, we assess its performance on the more challenging NYUv2 dataset with a Multi-Tate Attention Network architecture (MTAN) (Liu et al.) 2019). The
MultiFashion
MultiNNIST
Method
Left Image Acc
Right Image Ace
Left Image Acc
Right Image Ace
Single Task Models
79.81
0.39
78.64 = 0.62
89.40 + 0.17
87.80 + 0.22
MTL
78,80 : 0.35
77.92 = 0.32
89.01
= 0.22
86. + 0.15
PCGrad
78.17 0.48
77.20 = 0.56
88,92 0.21
86.61 + 0.22
TE-MTL
79.30 2: 0.30
78.17 = 0.36
89.12 0.21
86.35 + 0.23
MGDA-UB
79.25 0.69
78.62 - 0.48
89.38 1 0.16
86.81 + 0.46
UW-MTL
80.93
0.27
80.11 = 0.15
90.77 -0.77=+0.11 0.13
88.36
+ 0.12
UW-PCGrad
80.81
80.19
0.19
90.77 0.11
IT-UW-MIL
80.94 2: 0.22
80.31
1 0.21
90.71
+ 0.18
88.51
+: 0.13
TPUW-PCGraS
0.15
90.94
IT-UW-PCGrad
80.99 0.17
80.25
0.21
90.92
0.09
88.60
0.14
Table Test Accuracy on MultiMNIST and MultiFashion classification datasets. The test accuracy is averaged over 10 samples. We report standard error, and best results are highlighted in bold. The approximation described Section 4.1 is denoted with 4, Our results indicate the augmentation improve the performance of traditional MTL, uncertainty weights (Kendall et al. 2018). and PCGrad (Yu et al. 2020).
8

---

Under review as a conference paper at ICLR 2021
Sogmentation
Depth
Surface Normal
Method
(Higher Better)
(Lower Better)
Angle Distance
(Lower Better)
Within 1 (Higher Better)
mioU
Acc
Abs Err
Mean
Median
11.25
22.5
Split, Wide
15.89
51.19
0.6494
0.2804
33.69
28.91
18.54
39.91
52.02
Split, Deep
13:03
41.47
0.7836
0.3326
38.28
36.55
9.50
27.11
39.63
Dense
16.06
52.73
0.6488
0.2871
33.58
28.01
20.07
41.50
53.35
Cross-Stich
14.71
50.23
0.2871
33.56
28.58
20.08
40.54
51.97
MTAN
20.91
56.45
0.6111
0.2592
31.36
27.14
18.71
41.94
$5.04
IT-MIL
21.36
56.48
0.5921
0.2516
31.21
26.92
19.74
42.41
55.37
PCGrad
22.93
57.79
0.6224
0.2687
30.90
26.81
19.64
42.56
55.63
IT-PCGrad 23.55
58.48
0.5926
0.2539
30.40
26.05
21.11
43.96
56,79
IT-PCGradd 23.43
57.38
0.5909
0.24.98
30.47
26.08
21.44
43.96
56.63
Table 3: 13-class semantic segmentation, depth estimation, and surface normal prediction results
on the NYUv2 validation dataset. Performance of (Split, Wide), (Split, Deep), Dense, and Cross- Stitch MMisra et al. 2016) reported in (Liu et al., 2019). The symbol denotes the approximation
described in Section
dataset is composed of RGB-D images of indoor scenes and supports modeling of 13-class semantic segmentation, true depth estimation, and surface normal prediction. We follow the procedure of (Liu et al. 2019) and directly utilize their framework to evaluate the performance of IT-MTL. For computational efficiency, we form J == {semantic + depth, semantic + normal, depth + normal, semantic + depth + normal) } in IT-MTAN and J = {semantic + depth + normal, PCGrad gradi- ent} in IT-PCGrad. Table 3 summarizes our experimental findings. We find IT-MTAN improves modeling performance across all measurements for segmentation, depth, and surface normal tasks as compared with the MTAN baseline. IT-PCGrad and the approximation IT-PCGrad demonstrate similar improvements when compared with the ECGrad-MTAN baseline. This result indicates the benefit of IT-MTIL can hold for complex neural network architectures on challenging real world dataset.
5.2 EFFECT OF CONVERGENCE ON TRANSFERENCE
In this section, we return our focus to CelebA to analyze the effects of model convergence on infor- mation transfer. As shown in Figure transference is a highly dynamic process that is significantly affected by model convergence. In particular, we find the transference As on Ag to be nearly iden- tical to that of As 's self- sefftraasserrenee during the first two epochs of training. This result indicates the information encapsulated in the gradient update of Ag on the shared parameters is as effective at minimizing the loss of Ag its own gradient update. However, this effect is dampened throughout training with positive transference only manifesting at the beginning of training.
Interpreting our results in the context of CelebA, the model may learn the location of certain at- tributes in the beginning of training which are highly transferable to other related attributes. Once this fundamental structure is leamne,, gradients may encode increasingly task-specific information leading to lower positive information transfer among tasks. These observations lend weight to the development of flexible sharing architectures, in particular those which can quickly adapt chang- ing information transfer dynamics in the shared parameters throughout training.
6 CONCLUSION
In this work, we take first step towards quantifying information transfer in multi-task lcarning. We develop measure quantify transference and leverage this quantity to determine which tasks should be trained together as well as develop method which improves multi-task learning efficiency and performance.
Nonetheless, the method is not without its shortfalls. Using transference to select task groupings does not account for regularization-lik effects inherent in multi-task learning. As a result, although a specific set of task groupings may exhibit high transference, there will be cases when this grouping sub-par. Moreover, the transference radar charts are open to interpretation. While the charts provide flexibility in determining task groupings identifying tasks which detract from co-training, they do not unequivocally produce a final ranking. With regards to IT-MTL, training time scales linearly with respect to the number of tasks if the lookahead loss computation is not in parallel.
In spite of these detriments, we hope our analysis of information transfer in multi-task learning encourages further analysis into its training dynamics. Future work transference can incorpo- rate this measure into a continuous learning algorithm, or guide the development of flexible architectures to further improve multi-task learning performance.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, al. Tensorflow: system for large- scale machine learning. In /2th {USENIX} symposium on operating systems design and imple- mentation ((OSDI} 16), pp. 265-283, 2016.
Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Char- less C Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1430-6439 2019a.
Alessandro Achille, Giovanni Paolini, Glen Mbeng. and Stefano Soatto. The information complexity of learning tasks, their structure and their distance. arXiv preprint arXiv:1904.03292, 2019b.
Jonathan Baxter. A model inductive bias learning. Journal of artificial intelligence research, 12: 149-198, 2000.
Shai Ben- Ben-David and Reba Schuller. Exploiting task relatedness for multiple task learning. In Learn- ing Theory and Kernel Machines, pp. 567-580. Springer, 2003.
Joachim Bingel and Anders Søgaard. Identifying beneficial task relations for multi-task learning in deep neural networks. arXiv preprint arXiv: 1702.08700. 2017.
Lukas Brinkmeyer, Rafael Rego Drumond, Randolf Scholz, Josif Grabocka, and Lars Schmidt- Thieme. Chameleon: Learning model initializations across tasks with different schemas. arXiv preprint arXiv:1909. 13576, 2019.
Rich Caruana. Multitask learning. Machine Learning. 28, pp. 41- 1997.
Richard Caruana. Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the Tenth International Conference on Machine Learning, pp. 41-48. Morgan Kaufmann, 1993.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gra- dient normalization for adaptive loss balancing in deep multitask networks. arXiv preprint arXiv:1711.02257, 2017.
Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. Advances in Neural Information Processing Systems, 33, 2020.
François Chollet et al. Keras: The python deep learning library. ascl, pp. ascl-1806, 2018
Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. Low resource dependency parsing: Cross- lingual parameter sharing in neural network parser. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Pupers), pp. 845-850, 2015.
Kshitij Dwivedi and Gemma Roig. Representation similarity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12387-12396. 2019.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 1126-1135. JMLR. org, 2017.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-learning as hicrarchical bayes, 2018.
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei. Dynamic task priori- tization for multitask learning. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 270-287, 2018.
Pengsheng Guo, Chen Yu Lee, and Daniel Ulbricht. Learning to branch for multi-tase task learning. arXiv preprint arXiv:2006.0/88 2020.
10

---

Under review as a conference paper at ICLR 2021
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint arXiv:1801.01290. 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Geoffrey E Hinton and David C Plaut. Using fast weights to deblur old memories. In Proceedings of the ninth annual conference of the Cognitive Science Society, pp. 177-186, 1987.
Siyu Huang, Xi Li, Zhi-Qi Cheng, Zhongfei Zhang, and Alexander Hauptmann. Gnas. 2018 ACM Multimedia Conference on Multimedia Conference - MM '18, 2018.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv: 2018.
Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems, pp. 315-323, 2013.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7482-7491, 2018.
Taesup Kim, Jaesik Yoon, Ousmane Dia. Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta-learning. arXiv preprint arXiv: 03836, 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69880 2014.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2277) 22224, 1998.
Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-tase task learning. In Advances in Neural Information Processing Systems, pp. 12037-12047, 2019.
Shikun Liu, Edward Johns, and Andrew J Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1871- 1880, 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. 2015 IEEE International Conference on Computer Vision (ICCV), Dec 2015.
Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully- adaptive feature sharing in multi-task networks with applications in person attribute classification. 2017 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), Jul 2017.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3994 4004, 2016.
Yu Nesterov. A method of solving a convex programming problem with convergence rate ((1/k²). In Sov. Math. Dokl, volume 27, pp. 372-376, 1983.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02000 2018
Sebastian Ruder. overview of multi-task learning in deep neural networks. arXiv preprint arXiv: 2017.
Andrei A Rusu, Neil C Rabinowitz. Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: arXiv:1606.0440 04667, 2016.
11

---

Under review as a conference paper at ICLR 2021
Ozan Sener and Vladlen Koltun. Multi-task learning multi-objective optimization. In Advances in Neural Information Processing Systems, pp. 527-538, 2018.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from rgbd images. In European conference on computer vision, pp. 746-760. Springer, 2012.
Trevor Standley, Amir R Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? arXiv preprint arXiv:/905.000 975500753, 2019.
Ximeng Sun, Rameswar Panda, and Rogerio Feris. Adashare: Learning what share for efficient deep multi-task learning. arXiv preprint arXiv: 2019.
Mihai Suteu and Yike Guo. Regularizing deep multi-task networks using orthogonal gradients. arXiv preprint 1x1v::112.006444 2019.
Grzegorz Swirszez and Aurelie C Lozano. Multi-level lasso for sparse multi-takk regression. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pp. 361-368, 2012.
Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Luc Van Gool. Branched multi-task networks: deciding what layers to share. arXiv preprint arXiv:1904.0220 2019.
Zirui Wang, Zachary Lipton, and Yulia Tsvetkov. On negative interference in multilingual models: Findings and a meta-lcarning treatment. arXiv preprint arXiv:2010.03017 2020a.
Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating
and improving multi-task optimization in massively multilingual models. arXiv preprint arXiv:2010.05874, 2020b.
Sen Wu, Hongyang R Zhang, and Christopher Ré. Understanding and improving information trans- fer in multi-task learning. arXiv preprint XXi::2005.00944 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST novel image dataset for bench- marking machine learning algorithms. arXiv preprint 1.0V:1770..7747, 2017.
Yongxin Yang and Timothy M. Hospedales. Trace norm regularised deep multi-task learning. arXiv preprint arXiv: 002666666000800 2016.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning, 2019.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782 2020.
Amir R. Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. 2018 IEEE/CVF Conference, Jun 2018.
Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton. Lookahead optimizer: k steps forward, step back. In Advances in Neural Information Processing Systems, pp. 9597-9608, 2019.
Yu Zhang and Qiang Yang. A survey on multi-task learning, arXiv preprint arXiv:1007.08114, 2017.
Xiangyun Zhao, Haoxiang Li, Xiaohui Shen, Xiaodan Liang, and Ying Wu. A modulation module for multi-task learning with applications in image retrieval. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 401-416, 2018.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey transfer learning. Proceedings of the IEEE, 2020.
12

---

Under review as a conference paper at ICLR 2021
A
APPENDIX
A.1
EXPERIMENTAL DETAILS
In this section, we detail our experimental methodology with the goal of facilitating reproducibility. The code used to produce experimental results can be found by accessing the Supplementary Material section our OpenReview submission.
A.1.1 CELEBA
Our experiments on CelebA are generated using a combination Keras (Chollet et al. 2018) and TensorFlow (Abadi al. 2016) and access the CelebA dataset publicly available Tensor- Flow datasets https: / www. tensorf low. www.orr///ooo barg/daaaaeteee cata log/ hetg///eebb_ee attributes from the subset of 40 annotated attributes for our analysis.
We selected
The encoder architecture is based loosely on ResNet18 (He et al. 2016) with a shallow feed forward network decoder. A learning rate of 0.001 is used for 40 epochs with the learning rate halved 30 epochs. The model uses a momentum optimizer with momentum set to 0.9 and a batch size of 256. We maintain 5-epoch moving average of the task accuracies and report the highest average 5-epoch moving accuracy achieved during training.
We found our model exhibits similar, if slightly improved, performance over the ResNet18 variant used in (Sener & Koltun 2018) that was trained for 100 epochs; however given transference computes an update to the shared parameters, we adopted an architecture with less shared parameter capacity and more task-specific capacity to improve training time without sacrificing performance.
Basch Normu
Encoder
Snidee Pakking
Input
Fincc Norm
Aug Puol 2x03
Image
Rel.ar
Rel.an
ReLa
Read tnskets! Padding saz
Kered Rieeni Puilding
Seidectd Pucdlop: seex
Decoder
FC Layor 1000
Layer 500
Encoded Image
RcLa
ReLa
Attribute Prediction
PCC
Figure 5: CelebA Encoder and Decoder used for measuring transference and determining which tasks should train together.
A.2 META-WORLD WORLD
We use the five tasks: "reach", "push", "press button top", "open window", and "close window" from Meta- World eu al. 2019). We use fully-comnected layers with 400 hidden units for both the pol- icy and the critic with weights shared across all tasks. For each iteration, for each environment and train the policy and the critic for 600 steps with a batch size 128 per task. We the soft actor critic (SAC) (Haarooja al. 2018) our RL algorithm and adopt the default
collect 600 data points
13

---

Under review as a conference paper at ICLR 2021
Table Chosen hyperparameters for MultiMNISTTFashion experiments.
Dataset
Method
Ir
loss weight
GradNorm
MTL
le
0.5
PCGrad
le
0.5
MultiFashion
IT-MTL
le
0.5
UW-MTL
le
IT-UW-MTL
le
MTL
le
0.5
PCGrad
le
0.5
MultMNNSST
IT-MTL
le
0.5
UW-MTL IT-UW-MTL
le
1e
hyperparameters used in the public repository of SAC http:///ttttgtg..bbbbbb... com/ www//aallberrellry..eleooloo--ee
soft learniannnn nn at hash 5999a5550ol1723300c4ce7777776363 We compute the transfer- ence on the critic loss
(Q ()sD -(Q(s.a)
where s and a denote the state and action, Q denotes the target Q network, D denotes the off-policy dataset collected by the agent, and & denotes the parameter of the critic Q network.
A.2.1 MULTIMMNIST/FASSIINN
Our experimental results on Multi-MNIST and Multi-Fasien multi-Fashhion are generated using combination of Keras and TensorFlow. We evaluate on the datasets released by Lin et al. (2019) further split of the training dataset into validation set for final dataset splits of 700k720K7200 train/valid/test.
The model architecture is loosely based on LeNet (LeCun et al. 1998) with a fully convolutional decoder and shallow feed-forward neural net decoder. A visual depiction is presented in (figure 6 The model uses a momentum optimizer with momentum set to 0.9 and a batch size 256. The lookahead loss is computed by simulating the full momentum update to the shared parameters rather than the SGD update described in Section 3.] The learning rate of the MTL baseline was selected validation dataset over (1e 4, 5e 4, 5e 3. 1e 5e 2) using schedule which halves the learning rate every 30 epochs. coarse grid search of the task-specific weights with left image weight
right image weight yielded left weight right weight 0.5. IT-MTI, Uncertainty Weight, and PCGrad used the same hyperparameters as the baseline. GradNorm was found to be much more sensitive hyperparameters, and these were tuned via random search between [le 6, le 2] for the learning rate and [ee 6, 5.0] for the spring constant. The parameters we used for each experiment are listed in Table
Due to non-trivial inter-run variance, we ran each experiment to completion 6 times, dropped the worst performance, and averaged the remaining 5 runs to produce the results shown in Table ? Moreover, we report the average accuracy of the final 5 epochs to further improve comparability.
FC Lyper 130c50
FC Layer 50:30
Fetaayer Snv
ReLo
Relu
Left Image
More Dood (2x2))
Max Pool 202)
Input Image
Cney Kernekd Filienct Stridascist
Rela
Cane
Rolu
Keraeis
Filues:20 Stridcctsl
FC Layer 1ayx::720t.50 50
FC Lyar 5n.50
FC Lyarr 50ut0
Right Image
Figure 6: LeNet used in experiments. Note, the fully convolutional encoder and shallow neural net decoder.
14

---

Under review as a conference paper at ICLR 2021
A.3 NYUv2
We clone the MTAN repository released by (Liu et al., 2019 (https://githhn) www.coo//ooooooo com/lorennt: mt at hash
to empirically test method and IT- PCGrad choosing between the combined gradient (i.e. gradient with respect to depth + semantic
normals loss) with the PCGrad gradient. The optimization uses Adam (Kingma & Ba 2014). and
the lookahead loss is computed by simulating the full Adam update to the shared parameters rather
than the SGD update described in Section 3.1
We run all MTAN experiments with default hyperparameters and settings with the exception of reducing the number of steps in PCGrad and IT-PCGrad 100 find significant overfittigg begins after this stage. Results from Split, Wide; Split, Deep; Dense; and Cross-Stitch results are
taken from Liu al. 2019
Expanded Analysis MultiFashion 1-D Loss Landscapes
Parameter
Update UntrseeerUupdliiee
Parameter Update Epoch
Parameter Updatt in Fpoch
lcape
0.723
10-
0.7200
0.712-
10
0.715
0.710
2.0
10
0.725
30-
0.734
10-
0.720
0.0
0.5
1.0
2.0
255
0.5
15
255
3.0
0.0
1.0
1.5
2.0
3.0
Interssttion
Interpatation
Inteppllation
Figure Expanded 1-d loss landscapes along each gradient direction.
Left Loss Landscape
Right Loss Landscape
Combined Loss Landscape
(8)
(b)
(c)
Figure 8: 2-Dimensional loss landscape of the (a) left digit loss, (b) right digit loss, and (c) the combined (average of left and right) loss MultiFashion in a step where IT-MTL chooses the left gradient for the shared parameter update. The symbol indicates the location of the minimum value of the loss in the local landscape and shows the projected coordinates of the current parameter.
15

---

Under review as a conference paper at ICLR 2021
A.4
Loss LANDSCAPE EXPANDED ANALYSIS
Figure was created by halting training in a given epoch immediately after either the left or the right task gradient update manifests higher transference than the 1/2(left + right) (i.e. combined) gradient update. We then applied the parameter update to the shared parameters using SGD with momentum create linear interpolation between the current parameter values and the parameter values following an update. We extend this interpolation 3x past the typical update to measure the curvature along the direction of the update.
In Figure 7 we compute the loss along linear interpolation of the left gradient, the right gradient, and the combined gradient direction with each column corresponding to the total loss plot presented in Figure 4 For instance, Column 1, Row 2 plots the left and right loss along the left gradient step the leftmost plot in Figure 4 and Column 2, Row plots the left and right loss along the right
gradient step for the center plot Figure 4
In Figure S we plot the 2-dimensional loss landscape of the left and right loss as well as the com- bined loss for MultiFashion. To generate the plots, we first sample two random directions in the parameter space and then scale the norms of these directions to be equal to the norm of the parame- ters. Next, interpolate the parameters along these two directions in the range [-0.1, +0.1] times the norm of the parameters.
The left image loss depicts smooth landscape whereas the right image loss is highly non-smooth. Notice that the level sets of the combined (i.e. average) loss is higher than those of the left loss. For this step, IT-MTTL chooses the left gradient for the shared parameter update which aligns with the curvature discrepancy between the right image loss and the left image loss.
16


Reviews:
[{"review_id": "wUi1y7VbCQ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary] This paper studies the problem of task relationship/transference in multi-task learning, by introducing a quantifiable measurement based on relative loss updates.  A (nonsymmetric) task transference between task $i$ and task $j$ then can be computed by measuring the relative change of training loss of task $j$, with the updated shared parameters from the training loss of task $i$. By finding a subset of tasks that achieves maximal total transference over every single task, multi-task learning performance can be further improved.\n\n[Strength] Understanding multi-task relationships and transference is important to achieve a good multi-task learning performance and on multi-task architecture design. The proposed solution based on the relative loss update is intuitive, clean, and simple to implement.\n\n[Weakness] \n1. **Task selection.** To achieve a maximal performance improvement, it seems that we have to compute task transference from all possible combinations of task grouping, which is exponential based on the number of tasks. The authors claim that \"a carefully chosen subset of tasks with $|\\mathcal{J}| =  \\mathcal{O}(m)$ provides reasonable improvements\", but I cannot find any details on how these tasks are selected. Besides, in the experiment section, the number of tasks evaluated is no more than 3, and the improvements are quite marginal (mostly within 0.5%). \n\n2. **Visualisations are unclear.** In Fig. 1, task transference is visualised by the pairing of all possible combination of two different tasks. But $\\xi$ selects a subset of tasks, rather than just an individual task. So how can we know the transference for each individual task just by the transference for all tasks? Or in this case, $\\xi$ is specifically designed to be a single task? The authors should elaborate on these details.\n\n2. **Notations are unclear.** $\\xi$ represents a non-empty subset of all task groupings. So why introduce another notation $\\mathcal{J}$ in Algorithm 1 to include all possible task groupings, this looks quite redundant to me.\n\n\nI hope the authors could elaborate on these questions and include more relevant details in the paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "lEzlQVATkKz", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the transferability in multi-task learning. They propose a metric, transference, to evaluate how tasks affect each other during multi-task training, and a method called IT-MTL which utilizes this metric to compute and improve lookahead loss changes. Although the proposed metric and method are interesting from a scientific point of view, there are a few key downsides (as the author themselves summarized in the conclusion) that require further investigation/improvements.\n\nPros:\n1. The idea of using change in losses to capture inter-task transferability is new and could have potential impact in the community.\n\nCons:\n1. The biggest concern is efficiency. The proposed method requires calculating lookahead loss for a set of combinations of tasks. While the author claim it only require O(m) possibilities empirically, it is not guaranteed for other datasets/settings. In fact, even with O(m) complexity could still be prohibitive for large models. To reveal the full picture, I recommend the author to demonstrate the actual time required for training for each method, and conduct an ablation analysis on the efficiency-performance trade-off.\n2. From Table 2, I can only observe marginal improvements over prior methods. Considering the extra computational cost, it is hard to justify the effectiveness of the method itself.\n3. The proposed method does not account for optimizers with ema gradients such as Adam, which may diminish the effect of selecting subset of tasks.\n\nOther comments:\n1. The idea of using lookahead method on shared layers in multi-task learning has been recently explored in [1]. Although they use a different optimization process, the high-level idea of improving transference/validation loss is shared. On the other hand, dropping gradients is also recently proposed [2]. Though concurrent, it still might be good to mention in the final version.\n2. I would also recommend exploring some greedy strategies for the proposed method to be efficient. In other words, how to greedily reduce the search space of combination of tasks.\n3. This is not very important, but what will happen if we directly use the transference metric to reweight tasks? For example, in multi-source transfer learning, can we utilize the metric proposed to decide what tasks to use?\n\n[1] On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment. Wang et al., EMNLP 2020. \n\n[2] Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout. Chen et al., NeurIPS 2020.", "extra_scores": {}, "extra_text": {}}, {"review_id": "OWlldBBKTGD", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes some interesting observations and proposes a novel measure of transference. Based, the proposed measure, this paper proposes a task-grouping method and novel training algorithm for MTL. However, there are some flaws in the proposed method and the proposed methods seems not technically sound. My concerns are listed as follows.\n\n1. The definition of the transference measure is problematic. The measure is depended on the empirical loss. However, in MTL, the negative transfer or positive transfer is proposed with respect to the generalization loss. The increasing empirical loss may not lead to an increasing generalization loss. This paper should give theoretical support to show that the proposed measure can indicate the influence on generalization loss.\n\n2. This paper does not provide experimental sufficient support for the superiority of the proposed method. For example, this paper is closely related to the previous work [1]. However, this paper has not compared with it. The authors should add the comparison with [1].\n\n3. The paper is hard to follow. Some important details are not clear. For example, how to do task grouping based on the measure of transference are not clearly written.\n\n[1]. Yu T, Kumar S, Gupta A, et al. Gradient surgery for multi-task learning. NeurIPS, 2020.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CLU7x9zNo9q", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors derived a quantitative measure of transference among tasks in the setting of multi-task learning. Based on this measure, two methods were subsequently proposed, one to find group of tasks that may benefit from collaborative training, the other to combine task gradients at each training step. \n\nDespite there are typos in a few places, the paper is good written and easy to follow. \n\nNo comparison with any baselines was done to show the advantage of the proposed method to identify task groups. One such baseline can be first training each task separately, then for each trained network fixing the shared parameters, resetting and adapting the task specific parameters for other tasks, and cross comparing the performance of all obtained models to identify task group for which collaborative training could be beneficial for individual tasks in the group. \n\nI was not able to find any guidelines in the paper for how to construct the candidate set J. So, it would be most likely exhaustive search? I guess because of this, as the authors mentioned, their method is not scalable, only can handle problems containing small number of tasks. The empirical results are very weak. With considering the variance, I doubt any of the improvement in Table 2 is statistically significant.  Such weak results do not justify the significantly elevated computational cost. Based on the results included in the paper, I do not think the proposed method has any practical significance. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method to quantify transference, which is a measure of information transfer across tasks, for multi-task learning framework. Specifically, the transference is measured as the change in the loss for a specific task after performing a gradient update for another. The proposed transference measure is used to both understand the optimization dynamics of MTL and improve the MTL performance, either by grouping tasks or combining task gradients based on the transference. The method is validated on multiple datasets and is shown to bring in some performance gains over the base MTL model (PCGrad, UW-MTL). 

The majority of the reviewers were negative about this paper (4, 4, 5), while one reviewer gave it a positive rating (6). The reviewers in general agreed that the idea of measuring transference as the change in the loss with gradient updates is novel and intuitive. Yet, the reviewers had common concerns on the 1) weak performance improvements, and the 2) high-cost of computing the transference. While computing the transference requires additional computations with linear time complexity, which may be problematic with a large number of tasks, the performance gains using it were rather marginal (less than 0.5% over the baselines).  Another common concern from the reviewers was its insufficient experimental validation, as a comparative study against existing works that perform task grouping is missing. Both the authors and reviewers actively participated in the interactive discussion. However, the reviewers found that the two critical limitations persist even after the authors’ feedback, and in a subsequent internal discussion, they reached a consensus that the paper is not yet ready for publication. 

Thus, although the proposed method is novel and appears to be promising, it may need more developments to make it both more effective and efficient. Moreover, there should be more in-depth analysis of its time-efficiency, and other benefits (e.g. interpretability) that could be achieved with the proposed transference measure. Finally, while there exist many works on learning both hard or soft task grouping, the authors do not reference or compare against them. To name a few, [Kang et al. 11] propose how to learn the discrete task groupings, [Kumar and Daume III 12] propose to learn a soft grouping between tasks, [Lee et al. 16] propose to learn soft grouping based on asymmetric knowledge transfer direction across the tasks, and [Lee et al. 18] proposes the extension of [Lee et al. 16] to a deep learning framework. I suggest the authors to discuss and compare against the above mentioned works, and fortify the related work section by searching for more classical works on multi-task learning. 

- [Kang et al. 11] Learning with Whom to Share in Multi-task Feature Learning, ICML 2011
- [Kumar and Daume III 12] Learning Task Grouping and Overlap in Multi-task Learning, ICML 2012
- [Lee et al. 16] Asymmetric Multi-task Learning based on Task Relatedness and Confidence, ICML 2016  
- [Lee et al. 18] Deep Asymmetric Multi-task Feature Learning, ICML 2018.
Decision: Reject