AUXILIARY TASK UPDATE DECOMPOSITION: THE GOOD, THE BAD AND THE NEUTRAL
Authors: Lucio M. Dery
Yann Dauphin
David Grangier
Venue: ICLR 2021 Poster (2021)
Keywords: pre-training
multitask learning
deeplearning
gradient decomposition

Abstract: While deep learning has been very beneficial in data-rich settings, tasks with smaller training set
often resort to pre-training or multitask learning to leverage data from other tasks. In this case,
careful consideration is needed to select tasks and model parameterizations such that updates from
the auxiliary tasks actually help the primary task. We seek to alleviate this burden by formulating a model-agnostic framework that performs fine-grained manipulation of the auxiliary task gradients. We propose to decompose auxiliary updates into directions which help, damage or leave the primary task loss unchanged. This allows weighting the update directions 
differently depending on their impact on the problem of interest. We present a novel and efficient algorithm for that
purpose and show its advantage in practice. Our method leverages efficient automatic differentiation 
procedures and randomized singular value decomposition for scalability. We show that our framework is 
generic and encompasses some prior work as particular cases. Our approach consistently outperforms strong and widely used baselines when leveraging out-of-distribution data for Text and Image classification tasks.

Published as a conference paper at ICLR 2021
AUXILIARY TASK UPDATE DECOMPOSITION: THE GOOD, THE BAD AND THE NEUTRAL
Lucio M. Dery
Yann Dauphin Google Research
David Grangier Google Research
Department of Computer Science Carnegie Mellon University Pittburgh, PA, USA
ABSTRACT
While deep learning has been very beneficial in data-rich settings, tasks with smaller training set often resort pre-training multitask learning to leverage data from other tasks. In this case, careful consideration is needed to select tasks and model parameterizations such that updates from the auxiliary tasks actually help the primary task. We seek to alleviate this burden by formulating a model- agnostic framework that performs fine-gr fin-grained manipulation of the auxiliary task gradients. We propose to decompose auxiliary updates into directions which help, damage leave the primary task loss unchanged. This allows weighting the up- date directions differently depending on their impact on the problem of interest. We present novel and efficient algorithm for that purpose and show its advantage in practice. Our method leverages efficient automatic differentiation procedures and randomized singular value decomposition for scalability. We show that our framework is generic and encompasses some prior work as particular cases. Our approach consistently outperforms strong and widely used baselines when lever- aging out-of-distribution data for Text and Image classification tasks.
INTRODUCTION
Multitask learning (Caruana 1997) and pretraining (Devlin et al.] 2018 Caron al.] 2019) have transformed machine learning by allowing downstream tasks with small training sets to benefit from statistical regularities from data-rich related tasks (Collobert Weston 2008 Zhang et al. 2014
Liu et al.) 2019, Kornblith et al. 2019). Despite these advances, leveraging the mixing of tasks is still an art left to the practitioner. When one is interested in a primary task, unclear how to select helpful auxiliary tasks, an appropriate parameter sharing architecture and a good way to filter out auxiliary data which might be detrimental to the primary tasks. Without careful choices, pre- training might hurt end-task performance (Gururangan al. 2020 or have limited impact (Raghu et al. 2019).
Prior work has examined these problems and proposed solutions, either to choose auxiliary tasks depending on their impact on the primary task (Du et al. 2018 Lin et al. 2019) or to equalize
the impact of updates across tasks (Sener & Koltun 2018 Chen et 2018 Hessel et al. 2019). Recently, several approaches (Sinha et al.) 2018 Suteu & Guo 2019 Yu et al. (2200) have been proposed that attempt to minimize interference between the updates across tasks. Our work builds on this direction, but unlike these previous approaches, we do not consider a symmetric view of multi-tast learning in the sense that our goal not to train a model performing well on all tasks. Instead, we focus on improving generalization for single task, the primary task, and the other tasks, the auxiliary tasks are considered only through their impact on the problem of interest.
For that purpose, we introduce framework which decomposes the gradient updates from the aux- iliary tasks according to their impact on the primary task. We analyze the auxiliary task gradients in the subspace spanned by the primary task per-example gradients. This allows us to decompose auxiliary gradients into into three components components that help, interfere or have no impact on the primary task according to the Taylor expansion of the expected primary loss. This decompo- sition allows us to re- re-wttght each component differently prior to the update. Our framework enables us treat each auxiliary update differently depending its impact the task of interest and it

---

Published as a conference paper at ICLR 2021
encompasses prior methods such as classical multitask learning (Caruana 1997) or more novel gra- dient surgery techniques (Yu et al. 2020). To achieve a tractable approach, we introduce an efficient, robust algorithm (ATTITTUD, Auxiliary Task Training with Influence from Target Task Update Di- rection) to estimate the subspace spanned by the primary task gradients in an online manner and decompose the auxiliary updates appropriately. As result, we can integrate our approach with the stochastic training of large neural networks in various contexts.
The contribution of our work is four-fold. To our knowledge, this paper proposes the first approach to adapt auxiliary gradients using a decomposition built from the span of the primary task Jaco- bian. In order to scale this approach to deep neural nets, we contribute a tractable and efficient algorithm called ATTITTUD that leverages insights from randomized linear algebra and automatic differentiation such as the R-operator (Pearlmutter 1994). As our third contribution, we show that the ene-grained manipulation of the auxiliary task gradients under ATTITTUD, represents a unified framework that encompasses several previous approaches to asymmetrical task learning as special cases. Finally, we demonstrate the efficacy of our approach in both data-rich and data-starved pri- mary tasks, over both images and textual data.
RELATED WORK
Methods to leverage data outside of the task of interest have been popular in machine learning since the inception of multitask learning (Caruana 1997; Ruder 2017; Vandenhende et al.) 2020).
These methods address multiple task simultaneously and have been successful in various application
domains (Collobert & Weston 2008; Zhang et al. 2014 Misra et al. 2016). The optimization problem induced by multitask learning difficult and solutions have been proposed for the various difficulties, including dealing with task gradients of different magnitude (Sener & Koltun, 2018
Chen al., 2018 [Hessll et al.] (019)), or interfering with each others (Sinha al. [018]] /uteu & Guo 2019 eu et (020)) The specific problem of interference has been studied extensively in the context of continual learning. Continual learning visits task in sequence and update interference is particularly problematic as it yields newer tasks to damage previously mastered tasks. In particular, a family of methods to project the gradient of the new tasks to be orthogonal to the gradient of the Chaudhry et al. 2018 Farajtabar
previous tasks has been proposed (Lopez- Paz & Ranzato 2017 ct al. 2019).
Different from many previous approaches, we are not interested in addressing multiple tasks per se. In our setting, only the primary task matters and the other auxiliary task have the sole role of improving generalization on the primary task. This is the setting considered by Du et al. (018))] Lin jct al. (2019), who favor auxiliary tasks whose gradient directions are helpful to the primary task. Unlike these works that use coarse properties like the cosine similarity between averaged gradients, our approach allows finegrriiedd gradient manipulation within a subspace. Also, in our case, we do not distinguish between the different auxiliary tasks. Instead, we aim at correcting every auxiliary gradient in the same manner to improve the loss on the primary task. This type of gradient correction related to Yu al. 2220)) which considers projecting multi-task gradients such that the directions of disagreement are removed. This method is actually special case of our framework.
Our work also shares some similarities with data selection and domain adaptation approaches. In this case, the training data comes from a single task but its distribution is different from the valida- tion/test distribution (Moore & Lewis 2010 Axelrod et al. 201T Ngiam al. 2018). This classical problem has recently been addressed by sampling training points whose gradient aligns well with the expected validation gradient (Wang et al.) 22220b a). Instead of sampling individual points based on an estimated distribution of how helpful they will be to the primary task, our work avoids the use (and inherent challenges) of this reinforcement learning approach by operating on batch gradients of groups of points.
Our primary task/auxiliary task setting is also related to the pre-training then fine-tuning paradigm in which the auxiliary tasks are visited first (pre-training) to give an initialization for training on the primary task (fine-tuning). These methods have been very successful in settings where primary task data are rare. In particular, it is common to first rely on unsupervised task over very large datasets prior to fine tuning over a supervised task (Devlin et al., 20188) Liu et al. 2019 Kornblith
et al. 2019 Yang et al. 2019 Song al. 2019 Caron et al. 2018).
2

---

Published as a conference paper at ICLR 2021
9
9
9
Figure I: Example gradient manipulation in the 2-D x plane with ATTITUD. ATTITUD can operate in any n-dimensional subspace. Left: Primary task gradient Sprim decomposed along the 3 Dimensions æ, y and z. Mid: Decomposed Auxiliary task gradient Jaux- We label the com- ponent of 9axx as positive since it agrees (in direction) with the x component of gprim- Since the y component of Jaux is in the opposite direction as that of gprim- this is assigned negative label. Right: Corresponds to Jour obtained by applying Taa (1.0, 1.0, -10). We flip the conflicting gradient direction to agree with our primary task. This is just one configuration achievable under framework.
3 AUXILIARY TASK UPDATE DECOMPOSITION
This section introduces a new method to improve generalization on a primary task T using training data from auxiliary tasks T (Ti..... ., TT).. where IE denote the parameters shared by all tasks. Our approach leverages gradient updates from the auxiliary tasks, but unlike the traditional approach, we decompose these gradients maximize their usefulness to T* Precisely, we decom- pose the auxiliary task gradients into directions which decrease first-order approximation of the primary task loss, increase or have no effect. This decomposition allows weighting these three directions differently when learning from the auxiliary tasks.
In order to decompose the auxiliary gradient, we must collect more fine- tree-graind statistics about the primary task. At each training step, we collect the gradient of the loss with respect to for individual examples from the primary task, { C Vi }. The span of these vectors,
S = Span V Ceprii Vij
defines subspace in which any linear combination of primary task gradients lies, including the gradient of the expected primary task loss, i.c. Jarim B(V C.rimm, E S. We denote the size of the subspace, [S] K. This is upper-bounded by the number of examples m. used to construct S. If we define the orthogonal complement of S as S any vector S s is therefore orthogonal to gprims i.e. V 9prim This means that adding such a vector to the parameters has no impact on the expected primary task loss, according the order-1 Taylor expansion of Cerim i.c.
Cpeim (0 + +u) Carim (0) + Aprim = Crrim(()) (8).
We propose to project auxiliary task gradients onto S and S This allow to distinguish between directions of the auxiliary task updates which impact the primary task loss and those which do not. If we denote the averaged auxiliary task gradient
Iaur E V (V this gradient as Jaux - + g where _ E S is the portion of the gradient that lies in the span of the primary task example gradients and 9 E S is the portion that lies outside of it. Since orthogonal to the average primary task gradient and parameter updates along the
decompose
gous E S-, direction of gul. are expected to have limited impact on the primary task loss. On the other hand, potentially improve damage the averaged primary task
updates along the direction of git loss. This component deserves more careful treatment.
For that purpose, introduce {u,, 1,... K} an orthonormal basis of S. In this basis, measure if the components of 9 agree or disagree with gprin- We say that the two gradients agree along un iif sign(g wx) sign(gprim u.) This means that we can decompose gt gous Fou
3

---

Published as a conference paper at ICLR 2021
where 9 refers the projection of g onto the basis vectors where g and Iprim agree. By this decomposition, Joux helps the primary task, Jaux task, Jaux" 9prim < O.
9prim >> O, while 9 interfere with the primary
Guided by the primary task, we can therefore decompose the auxiliary task gradient as
Joux Jaux + Imux + Jaux
which is described on Fig Our approach proposes to re-weight differently the components of Janes i.e.
Jamx m1. 77- 17m9u
(2)
where Fax (7L1 77. 17- are nyper-parameters adjusting the auxiliary gradient according to the impact on the main task. we also wish to include the primary task gradient in descent, as with multitasking, we can introduce Tprim scalar control variable to modulate its weighting.
consequence of introducing nau is that specific configurations lead us to gradient updates that are guaranteed to do no harm to both tasks. This is captured by Theorem below.
Theorem 1. Let caux(2)) and C (0.) represent the full batch losses of the auxiliary tasks and primary task respectively at step We assume the gradients of L and Carim are Lipschitz con-
timuous with constant L > O. Following the update rule Bu+1 = 0₂ - a Jawx where a S is the learning rate, we are guaranteed:
Caux (0c+1) S Laux (0:)
(prim(0e+)) S Crrim (0:)
If n- = 0 and 11, 2
Proof. See Appendix A
This theorem focuses on single update and guarantees progress both auxiliary and primary tasks. However, our asymmetric scenario is not interested in improving the auxiliary tasks per se and amenable to more aggressive settings. Ideally we want gradient updates during pre-training with T to not only do-no-harm T when applied downstream but also to be along descent directions that are maximally beneficial to T* We can consider 7/- 0 as in Fig I Reversing the direction of Joue by setting 17- < preserves the descent guarantee on Cprim (0++1) ) but no longer ensures descent on (f++1). There are other interesting settings for control parameters. One recover the original gradient Jaus with 7 the primary task gradient span with 7 0.0, ignore those which conflict with the main task by setting 7 = 0.0.
1/+ = 1.0. One can choose to drop gradients orthogonal to
Relationships to other approaches Our framework is generic and encompasses other ap- proaches as a particular case. One can train solely on the primary task by selecting nax = (0.0, 0.0, 0.0) and aeiim 1.0. Classical multitasking corresponds naux (1.0, 1.0, 1.0) and Mriin > 0.0, while classical pre-training corresponds to performing first phase with nau (1.0, 1.0, 1.0) and Mprin 0.0. Interestingly, our formulation introduces novel variants of pre-training. for instance, one can consider pre-training with only auxiliary gradients help- ful to the primary task, naur (0.0, 1.0, 0.0) and mpiim 0.0, followed by fine-tuning with
nax (0.0, 0.O, 0.0) and Mprim 1.0.
Our approach also instantiates PCGrad Yu et al., 2020) as a particular case. This method was introduced to address the issue of conflicting gradients in multitask settings. PCGrad orthogonalizes the gradients of each task and removes confficting gradients. To recover PCGrad under our approach, specific choice of our decomposition in the 1-D subspace spanned by the gprim- PCGrad then removes components of goux that conflict with gprim which is equivalent to
note that equivalent
Maxx (Olaury O'uzxy 0.0) and Naue prim
4 IMPLEMENTATION
Equation 2 requires selecting a basis for the span of primary task gradients. Multiple choices are possible to define the basis {wz). represent the span at each optimization time-step. This choice
4

---

Published as a conference paper at ICLR 2021
important since the components of guu are labeled positive negative depending how they agree with the projection of the averaged primary task gradient onto the same basis. A natural choice to select the basis as the singular vectors of the matrix of primary task per-example gradients J E Rm=DD also know as the Jacobian. To improve efficiency and prevent over-fitting on a few examples, consider the span defined by the, k < [S), largest principal vectors of J Using the principal vectors as directions of descent instead of the mean induces a more robust algorithm since the mini-batch average gradient is susceptible to outliers and skew from replicated data-points. To the best of our knowledge, we are the first to propose using the singular vectors of J as directions of descent. We leave the theoretical implications of this algorithm future work but note that its variance reduction properties may induce generalization benefits (Namkoong & Duchi 2017).
We also consider alternative choices of bases as baselines, including the canonical parameter ba- sis. This choice will examine the sign of every parameter update to verify whether it agrees with Sprim- Whilst Theorem holds irrespective of the choice of basis, its proof reveals that the amount of progress made each loss depends the choice of basis. Specifically, the reduction in Cerim (prim(0+41)) Laux (0++1) after a gradient step along Jau is proportional to the fraction the norms of gprim and Jaux captured by the subspace spanned by choice of basis. To justify our use of the top singular values of J", we evaluate this fraction for different choice basis in our experiments (see Appendix CC.
We are interested in applying our approach to the training of large neural networks and must consider scalable algorithmic solution. As stochastic optimization is prevalent in this setting, we construct subspace S from a mini-batch of primary task data. Similarly, the expected gradients gprim and goux are defined over mini-batch. Instead of computing the singular value decomposition (SVD) of (VoLprim, Vii exactly, we rely on a randomized upproximation (Halko et al. 2011 Rokhlin al. 2010 Nakatsukasa, 2017). This method does not require instantiating the vectors { /oC Vi) and only needs a low dimensional projection onto random subspace. This is advantageous for high dimensional cases, i.e. when the number of model parameters is large. In our case, this method also allows to benefit from memory-efficient computation of Jacobian Vector product using the R-operator (Pearlmutter, 1994) offered by automatic differentiation packages (Baydin et al. 2015) like Pytorch (Paszke et al. [2017). This means that we can compute SVD with limited computa- tional and memory burden, albeit without sacrificing approximation accuracy (Nakatsukasa 2017). Additionally, we do not recompute the basis at every optimization step but at every n steps, which is efficient when training with small updates, e.g. when small learning rates and gradient clipping are used (Pascanu et al. 2013) (see Appendix C for more details about n!).
We study the impact of these choices in practice in Section 6 Putting it all together results in the ATTITTUD algorithm, Auxiliary Task Training with Influence from Target Task Update Direction, shown as Algorithm | The sub-procedure randomi www.:/oik.iwoooooa.........a pendix as Algorithm
is detailed in Ap-
Algorithm 1: ATTITTUU: Construct Auxiliary Task Surrogate Gradient
Require: Jauxs J' Auxiliary task average gradient, primary task Jacobian
Require: Naux = (Ty "-,,"),,>>> ) Auxiliary task control parameters
Require: k Size of subspace
Iprim = 1 Em= Jí.
V + randomizedd www.:ommzzzoooooaaaaarrroooo
Pprim- Pau Ve(gin))) Vi(gnux)
1/ is the hadamard product operator
""" P Paixr Panx = -(1 [pomnopa 2 20/
(1Prmmopu <
Paux:
Paux
11 Calculate the docomposition components
gonst I = (p "V, (pmx)) TV
// Calculate the out of span component
g Iaur (gitue + Imux)
Janr (Tl - + (1+ g + (n_ 9
Return: Jaux Auxiliary task surrogate gradient
5

---

Published as a conference paper at ICLR 2021
5 EXPERIMENTAL SETUP
We compare ATTITTUD with previous methods on variety of tasks and domains. We rely both text and image classification tasks to conduct our analysis. We also present ablation experiments explain the impact of hyper-parameter selection. We make code for ATTITTUD and related experiments available on github.
Text Classification. We apply our method on binary sentiment classification. We consider the Amazon Helpfulness (McAuley et 2015) and Imdb Movie Review (Maas et al., 2201 tasks. The Amazon Helpfulness task splits text reviews into 115k/5k/25k documents for train-validation-tes split whilst the Imdb Review dataset has 20k/5k/25k split. The Imdb Review task also has 50k unlabeled reviews as extra data which we utilize.
For our models we build on top of Gururangan et al. Adaptive Pre-training (TAPT). TAPT further pre-trains a generic model, Roberta (Liu et al., 2019). by performing Masked Language Modelling, MIM,, (Devlin et al. 2018) on the task specitic data (ignoring the labels) before doing supervised learning with the same data. We replicate (Gururangan et al. (2020)'s 's experimental setup and re-use their tayee--paameeter for our experiments. We use the TAPT task as our auxiliary task. We extend TAPT to use our method by modifying the TAPT gra- dient with guidance from the supervised sepeeviseddleeriingg task gradients. As baselines, compare against TAPT and cross- cross-TAPT: where we swap the masked language modelling pre- pre-tra training data for the two tasks. Cross-TAPT is setting where one uses -ittt--iisttiuttionn data for pre-training.
2020 's work where they introduce Task-
Neppicate|Guuruannnnn
Image Classification. We apply our method to both high-resource and limited-data image classi- fication tasks. We the C0 dataset (Krizhevsky et al. 2009) to explore the high-resource setting. We follow [Rosenbaum et al. (2017) and treat each of the 20 super-classes / coarse labels of Cifar100 as a separate task. In our asymmetrical task setting, each of the 20 tasks is treated as primary task, whilst the remaining 95 classes are grouped into a single auxiliary task. Thus, for each coarse label, we have an auxiliary wa- classification task and 5-way primary classification task. Moving forward, we refer to this setting as MultiCifar100)
We use a down-sampled version of Cifar10 (Krizhevsky et al., 2009) as a low-resource setting- Specifically, we rely on Cat-vs-Dog for the primary task and use the remaining 8 classes for the auxiliary task. Our auxiliary task is therefore an 8-way classification task where each class has 5,000 examples. We restrict the Cat and Dog classes to only 50 training examples from each class. We use the low-resource setting to compare against other methods and for our ablation study.
For these vision experiments, we use WideResNet-22 architecture (Zagoruyko & Komodakis 2016) with a depth of k/ = 4. We compare our method to 4 different baselines : no pre-training, vanilla pre-trainin pre-training, multitasking and PCGrad (Yu et al. 2020). Our architecture is more standard and allows gradient descent optimization unlike the routing network of Rosenbaum et al. (2077) and (Yu et al. 2020), which requires reinforcement learning for training.
Medical Imaging Transfer. We apply our method to cross-domain transfer for low-resource medi- cal image classification. Specifically, we use 5k training examples from the ChexPert Dataset (Irvin) let al. 2019) as our primary task and seek to identify 5 different thoracic pathologies: atelectasis, cardiomegaly, consolidation, edema and plcural effusion. This setup has been used in several cross- domain pretraining studies (Raghu et al. 2019 [Jaswal et al. (2019). Note that since we do not have access to the test set for this task, we use the validation set (231 images) as proxy test set, and sample 100 images from the training data as a new validation set. We rely on generic photographs (Imagenet) as an auxiliary task (Deng et al., 2009). We use Tiny Imagenet Dataset (Le & Yang. 2015), subset of Imagenet which consists of 500 examples each from 200 classes, instead of train- ing on full Imagenet. All approaches are applied to the Resnet18 model (He et al., (016)) trained with Adam (Kingma & Ba 2014).
For all our experiments, we select the auxiliary task control parameters naux within {(1.0, 1.0, -L0), (1.0, 1.0, 0.0), (1.0, 0.0, -10),, (1.0, 0.0, 0.0)} for ease of interpretability. For settings where we compare against multi-tasking, we select Upiim within a small subset of the set- tings that worked best with multitasking baseline experiments. These choices limit the overhead of hyper-parameter search but still allow us to show the empirical advantage of our method. In
"oode available here http:://gthhhh.ccom///cy//tAATTt

---

Published as a conference paper at ICLR 2021
Imdb
Imdb Amazon MLM
Amazon
Amazon + Imdb MLM
Roberta
95.4 + 0.14
67.0 + 0.50
TAPT
96.1 + 0.11
95.1 + 0.10
70.3 +: 0.87
67.8 +: 0,46
Ours
96.1 + 0.09
95.4:10.03
70.1 + 1.13
68.5
Table 1: Results on Text Classification measured by Fl. Experiments are averaged over 5 runs.
all our experiments, we provide all methods with similar hyper-parameler search budgets, e.g. for we ran grid search with 16 configurations for regular pretraining, 16 config- urations for PCGrad and 12 configurations for ATTITUD, More experimental details are available in Appendix C
RESULTS AND DISCUSSION
Text Classification. Table T shows the results for text classification. When the same data used both for the auxiliary task of MLM and the primary classification task, TAPT and ATTITTUD both bring a similar improvement over Roberta (Imdb, Amazon columns). For the Cross-TAPT setting where different data is used for the auxiliary task and the primary task (Imdb + Amazon MLM, Amazon + Imdb MLM columns), TAPT does not perform as well as ATTITTUD. This highlights the advantage of ATTITTUD when the auxiliary task data distribution differ from the primary task distribution.
Image Classification. Our results presented in Table 2 Both for MultiCifar 100 (high resource setting) and Cifarl0-Cat-vs-Der (low resource setting), ATTTTUD shows a strong improvement baselines. In general, find that primary-task aware pre-training (Multitasking, PCGrad, Ours) is better than vanilla pre-training which also performs better than having no pre-training at all. For MultiCifar 100, we find that using noux = (1.0, 1.0, 1.0), Mprin 0.1 worked best for 11 out of the 20 Cifar100 super-classes tasks. Note that Tau = (1.0, 1.0, -100) is an aggressive but novel configuration we introduce. Multitask learning and PCGrad produce better models on 6 and 3 tasks respectively. In the low-resource Cat-vs-Dop setting ATTITUD produces bigger boost performance compared to baselines, with the best performing configuration being Fax = (1.0, 0.0, 0.0), Arrim 0.01. We posit that this configuration is successful because removal of the in-span components makes overfitting less likely. Applying the out-of-span components means the model learns features that do not harm the loss of the current mini-batch but could be useful later. Note that our best performing configurations are all novel and never an instantiation PCGrad.
Method
MultiCifar100
Cifarl0-Cat-vs-Dogs
No-PPretraining
57.6
53.6 + 2.26
Vanilla Pre-training
70.2
64.5 +- 1.26
PCGrad
75.6
64.2 + 1.10
Multitask
75.5
65.3 + 1.35
Ours
76.1
67.1
Table Average Accuracy on MultiCifarl00 and Cat-vs-Dog Cifar10 tasks. Cat-vs-Dog experi- ments are averaged over 5 runs
Method
Average AUC Across Pathologies
No-Pretraie No-Pretraining
78.3 + 0.87
Pretrained-Resse
81.4 + 1.34
Pretrained -eieeedddeeNeettee + Ours
83.3:10.71
Table 3: Results on ChexPert-5k task measured by average AUC (Area Under Roc-Curve). All experiments are averaged over 5 runs.
Medical Imaging Transfer. Table 3 shows our results on the ChexPert multi-label classification task. Per-pathology breakdowns are in Appendix C. Doing pre-training all performs worst.

---

Published as a conference paper at ICLR 2021
Our method outperforms using pre-trained Resnet18 model over Imagenet. We apply the end- task-aware ATTITUD over 100k ImageNet images after the initial pretraining and we reach 83.3% AUC, an improvement over 81.4%.
Subspace
Canonical
Random
Unit_vvg-rad
Randomized_SVD
Average Acc. 51.42 2.09
58.72 + 2.68
59.13 + 2.08 62.214.00
Experiment conducted on Cat-vr-Dog Cifar10 dataset for different choices of subspace basis. We use k = 5 for Random and Landomized_SVD This ablation uses smaller hyper- parameter budget than Table
Table 4:
Ablation Study. Our approach relies on the top-k singular vectors from randomized_svd to define the basis identify the positive and negative component of the auxiliary task gradient, see Section This method is more accurate than several alternatives; see Table Namely, we compare our choice to random, the basis spanned by randomly chosen orthogonal vectors in RD,, unit avg- grad, the basis spanned by the average primary task gradient, and canonical, the per- per-parameter basis. This ablation was performed under a more limited tuning budget (we cross-validated on configurations 1,0) and (1, 1,] only) than the full Cat-vs-Dog experiments from Table
Number Samples Extimat
Accuracy Dimensenalty
0.775
2td=2.0055
std 0207
01165
0.725
Burenge Accunee
0.675
High Resource
Low Resource
0.60
0.50
0.000
0.575
166 32
100
1.2
16
32
Number of Samples for tatimating 5-O subupecs
monalonnllty Subs Estimazedd from samples
Figure 2: Averaged across 5 random initializations. Left vary the number of samples used to estimate 5-d subspace up maximimum of 100 (the total number of training examples in this low-resource setting). Right. We compare the effect of the dimensionality of the subspace in the low-resource (50 examples each for Cat. Dog classes) and high-resource (1000 examples each per class).
We also examine the number of samples to estimate the principal directions of the per-example primary task gradient. Larger sample sizes involve more computation but have limited benefit on average accuracy. Large sample sizes however reduce variance, as shown in Figure 2 (left). This is as expected since using more samples gives a higher fidelity estimate the top-k singular vectors.
Another parameter of our algorithm is the size of our subspace, k. In general, we observe that in low-resource settings, better to operate on the auxiliary task gradient in smaller dimensional subspace. The opposite holds for high-resource settings. This can be seen in Figure (right). Whilst using a larger dimensional subspace captures a richer description of the J*, it also creates the risk of over-fitting especially in limited data setting. This trade-off therefore has to be validated per-task basis.
CONCLUSIONS
In this work, we propose a new approach to training a model with additional help from an auxil- iary task. Our method decomposes the gradients of the auxiliary task according three directions, with positive, negative and neutral impact on the primary task. This decomposition allows a flex- ible re-weighting of the auxiliary task components and give rise family of training strategies,
8

---

Published as a conference paper at ICLR 2021
which encompasses novel and existing approaches. We leverage insights from randomized linear algebra and automatic differentiation to scale the approach to large deep networks. Experiments in multitasking, pretraining and domain transfer over vision and text classification task demonstrate the empirical benefit of our framework.
REFERENCES
Amittai Axelrod, Xiaodong He, and Jianfeng Gao. Domain adaptation via pseudo in-domain data selection. In EMNLP, 2011.
Atilim Gunes Baydin, Barak Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: survey, 2015.
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsu- pervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 132-149, 2018
Mathilde Caron, Piotr Bojanowski, Julien Mairal, and Armand Joulin. Unsupervised pre-training of image features on non-curated data. In Proceedings of the IEEE International Conference on Computer Vision, pp.
2019.
Rich Caruana. Multitask learning. Machine Learning, 1997.
Arslan Chaudhry, Marc Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. 1//.00//0 2018.
Zhao Chen, Vijay Badrinarayanan, Chen-Yo Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International Conference on Machine Learning, 2018.
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In International Conference on Machine Learning. 2008.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 2018.
Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar, Razvan Pascanu, and Balaji Laksh- minarayanan. Adapting auxiliary losses using gradient similarity. CoRR, abs/1812.02224. 2018.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for contin- ual learning. arXiv preprint arXiv:1910.07100 2019.
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don't stop pretraining: Adapt language models to domains and tasks. Pro- ceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. htLht//:// wttp.o....rgg/1...53333/////
doi: 10.
acl main. 740
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness:
Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, 53 (2):217-288, 2011.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Matteo Hessel, Hubert Soyer, Lasse Espeholt, Wojeiech Czamnekk, Simon Schmitt, and Hado van Hasselt. Multi-task deep reinforcement learning with popart. In Proceedings of the AAAI Confer- on Artificial Intelligence, volume 33, 2019.

---

Published as a conference paper at ICLR 2021
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-llcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 590-597, 2019.
Amit Kumar Jaiswal, Prayag Tiwari, Sachin Kumar, Deepak Gupta, Ashish Khanna, and Joel JPC Rodrigues. Identifying pneumonia in chest x-rays: A deep learning approach. Measurement, 145: 511-518, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Simon Korblithh, Jonathon Shlens, and Quoc V. Le. Do better imagenet models transfer better? 2019 IEEE/CVF Conference Computer Vision and Pattemn Recognition (CVPR), 2019. doi: 10.1 10.100990pp22019002777.. CRLhttp:://o doi. wtw...orgg1....... htw../d/....//......////...... 2019. 00277
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS CS 231N, 2015.
Xingyu Lin, Harjatin Baweja, George Kantor, and David Held. Adaptive auxiliary task weighting for reinforcement learning. In Advances in Neural Information Processing Systems, pp. 4772-4783, 2019.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprini urXiv:19077 1692, 2019.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, 2017.
Andrew Maas, Raymond Daly, Peter Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th amnual meeting of the association for computational linguistics; Human language technologies, pp. 142-150, 2011.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-basee rec- ommendations styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp. 43-52, 2015.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In Conference on Computer Vision and Pattern Recognition, CVPR, 2016.
Robert C Moore and William Lewis. Intelligent selection of language model training data. ACL, 2010.
Yuji Nakatsukasa. Accuracy of singular vectors obtained by projection-based svd methods. BIT Numerical Mathematics,
2017.
Hongseok Namkoong and John Duchi. Variance-based regularization with convex ob- jectives. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Gamett (eds.), Advances in Neural Information Processing Systems 30, pp. 2971-2980. Curran Associates, Inc., 2017. URL 6890 variance based- requlariiaaaaaee
www..iccpppp.......eeeeppppppppe with convex w.t.--coooee--beeeeiiiioooo .
Jiquan Ngiam, Daiyi Peng, Vijay Vasudevan, Simon Kornblith, Quoe V. Le, and Ruoming Pang. Domain adaptive transfer learning with specialist models. CVPR, 2018.
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. the difficulty of training recurrent neural networks. In International conference on machine learning, pp. 1310-1318, 2013.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
10

---

Published as a conference paper at ICLR 2021
Barak A Pearlmutter. Fast exact multiplication by the hessian. Neural computation, 6(1):147-160, 1994.
Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understanding transfer learning for medical imaging. In Advances in neural information processing systems, pp. 3347-3357, 2019.
Vladimir Rokhlin, Arthur Szlam, and Mark Tygert. A randomized algorithm for principal com- ponent analysis. SIAM Journal on Matrix Analysis and Applications, 31(3): 100 1124, Jan 2010. ISSN 1095-7162. 08C736417
doi: 10.1137/080736417, URL htt.ttp://o......orrrr/ http://twx.c..rr/1....111
Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing networks: Adaptive selection of non-linear functions for multi-task learning. arXiv preprint arXiv:
2017. 2017.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv:1706.0055 05098,, 2017.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, 2018.
Ayan Sinha, Zhao Chen, Vijay Badrinarayanan, and Andrew Rabinovich. Gradient adversarial train- ing of neural networks. arXiv preprint arXiv:1806.000 08028, 2018.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Yan Liu. Mass: Masked sequence sequence pre-training for language generation. arXiv preprint 190ivv1905.024 2019.
Mihai Suteu and Yike Guo. Regularizing deep multi-task networks using orthogonal gradients. arXiv preprint arXiv: 2019.
Simon Vandenhende, Stamatios Georgoulis, Mare Proesmans, Dengxin Dai, and Lue Van Gool. Revisiting multi-task learning in the deep learning era, 2020.
Wei Wang, Ye Tian, Jiquan Ngiam, Yinfei Yang, Isaac Caswell, and Zarana Parekh. Learning multi-domain curriculum for neural machine translation. In ACL, 2020a.
Xinyi Wang, Hieu Pham, Paul Michel, Antonios Anastasopoulos, Jaime Carbonell, and Graham Neubig. Optimizing data usage via differentiable rewards. In ACL, 2020b.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pp. 5753-5763, 2019.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.
Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782, 2020.
Sergey Zagoruyko and Nikos Komodakis. 1605.00 2016.
Wide residual networks.
arXiv preprint
Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In European conference on computer vision. Springer, 2014.

---

Published as a conference paper at ICLR 2021
A PROOF OF THEOREM 1
Theorem. Let (0,) and (prim(()) represent the full batch losses of the auxiliary tasks and pri- mary task respectively at step We assume the gradients [ and C are Lipschitz continuous with constant > O. Following the update rule 8 = 8 a 9ame where a S 1 is the learning 1.
rate, we guaranteed:
C caux(0et1) - Laux (0s) C (0c+1) S LDrim (0z)
n- = 0 and 171, 77 0
Proof. Let V; E IR KxD be the orthonormal matrix whose rows span the per-example primary task m
gradients J' timestep t. The projections of the average primary task gradient 9prim -i= J and average auxiliary task gradient Jou at iteration are
Pprim Vi (gprim))
T
Paux Vz (gmax)
Porim and Paix will agree on some directions (same sign on those components). We use the operator [x]+ to mark these directions of agreement. This operator preserves components that agree and
sets those that disagree to zero. As an example given Pprim [1. 1, -1] and Paux [1, 3, 10), [Pprin]+ [1, 0] and [Pou]++ [1, 3, 0j. For directions that disagree (different signs of the respective components), we introduce the operator [xr]-. . In the above example [Pprim] [0, 0. -1] and [Pau]] - [0, O, 10). Note that operators defined by comparing two vectors Iı and 22. Our operators have the following properties by definition
[x] [x]+
and
[x]
[x]] [on)a 1 [+2]=
From Equation 2
Jaur = 7/ J + 17- Jous + T.n.
We can re-write this in terms of [x] as:
Jave 1/+ Paur] 71 Panx] 11.1 (gaux Panx
We now proceed to show the effect of the gradient descent update below on (oux(Or+1) and Lriim (prlm(0r+1).
0 02 Faux
(3)
How does this update affect the loss on the primary task loss Crrim (ft+1)?
Cprim (0.41) = L Laux (ft - a Iaux)
RJ C.rim (0.) a: (gam)) Sprim (First order Taylor Expansion)
T
Crrim (0t)
[powd]+ + [poux] + 171.
gprim
T
Crim (0.) 77 [Poal++ + n [Poxx]- + 1
[Pwrim++ + (ppriml
Crim (0.) 7 [paut]? I [prim]++ [Daux] 17r (prin))
+ n- (Daur] !T [pprmm)+ + [Pon) (Pormn)))
77+ [Paual I [ppril++ + 7- [Ponx] TT [Pprinl Criim (0.)
< Crrim (0t) (if < 0, 9 2 0)
12

---

100 examples from the training set as validation and test all 1000 test Published as a conference paper at ICLR 2021
Note that in going from line in the proof above, we use the fact that (garr) Sprim 0 since Jaur lies outside the subspace and Sprim lies inside it. For the last step of the proof, we use the observations below:
[Pacc]]
[prina + 2 0 since these directions agree in sign
Pax
Pprim
since these directions disagree in sign
Pax
Pprim
0 by the property of the [xc] operator
Poux
Pprin] + same motivation as above
How does Equation 3 affeec the auxiliary task loss (aux(0(+1))
Laux (Be+1) = Laux(B - a Jawx)
(0e) (gaux)
T
Jaux
(First order Taylor Expansion)
Laux (0,) a( +7 muut) (gaiu Toux Faux)
(O)) - aannn_llga000n aluee 112 + 7 1l1 + 77 91-11mmm (Cross terms cancel due to orthogonality)
S c (0t) 111 0)
Thus, choosing 7/- 0 ensures that we are minimizing both (a,)(0.) and Cprim We can combine this with the constraint on a S 1 to derive convergence guarantees after some T steps as in optimization literature.
B RANDOMIZED MATRIX THEORY
Algorithm 2: randomized lowrank approx: Construct low rank approximation
Require: J <mmX, Input Matrix
Require : k: Rank of subspace
IT A/0 7) E R
C -T.J
V Gram Schmid(hh))t(C)
Return: : V e DR Low rank approximation of J
The Gram Schmidt procedure orthogonalizes the rows of an input matrix.
C MORE EXPERIMENTAL DETAILS
Image Classification For MultiCifar unlike [Rosenbaum et al. (2017); Yu al. (2020) who use 500-100 train-test split for examples under each fine-gran uine-granned CIFAR 100 label, we include
validation set and therefore opt for 400-100-100 train-validation-tes split. We test all 1000 test examples per class.
For Cat-vs-Dog, examples per-class.
For Image Classification experiments, we perform pre-training with a learning rate of Ic-4 for all experiments and finetuning learning rate of 5e-4. These values were selected after coarse hyper- parameter search. In both pre-training and finetuning settings, we decay the learning rate by 0.5 the validation loss has not improved over 4 epochs, up till minimum learning rate of le-5. we use the Adam Optimizer (Kingma & Ba 2014) with 3 = (0.9, 0.999). We clip all gradient norms to 1.0 before performing gradient descent. We cross-validated dropout rates within the set {0.05, 0.1, 0.2, 0.3) for both pre-training and finetuning steps. We cross validate apinm based on the relative sizes of primary and auxilary task datasets. All experiments averaged over 5 random seeds. For all our Vision experiments, we either recompute our subspace basis every n or n = 10 iterations. We with the two choices showing similar
find that n is not important the other
13

---

Published as a conference paper at ICLR 2021
performance when the other arper-parameters (leaming rate and gradient norm clipping) are fixed to reasonable values.
Due to the fact that Yue al (PCGrad) treat all tasks symmetrically, which is different from primary-auxiliary setting, we introduced an extra parameter, pprim. for PCGrad to account for weighting the primary task. We cross validated values of Oprrm (0.1, 0.05, 0.01, 0.001).
Medical Imaging Transfer Table 4 presents a more detailed breakdown of the ChexPert task. For 50k examples from Imagenet, our best performing configuration was foxxx (1.0, 0.0, 1.0).. We did not use the primary task gradient directly for pre-training so Vpprim = 0.0 for all cases. For ATTITUD, use the same learning rates as in the Image classification setup above. For the No- No-Pretraining and Vanilla pretraining we cross-validated the learning rates for both finetuning and pre-training from the set {le-3, le-4). We cross-validated the same list of dropout values above.
Method
No-Pretraining
Pretrain Imgnet
Pretrained Ours (50k)
Pretrained + Ours (100k)
Atelectasis
76.0 +: 1.82
79.0 + 3.66
$1.6:11.38
01.8
Cardiomegaly
74.9 + 2.34
75.8 + 4.04
78.0 = 2.13
80.7=1.79
Consolidation
83.2 + 2.26
85.3 1.86
85.611..32
84.9 + 1.36
Edema
79.5 + 1.27
82.6 + 0.76
85.2::: 1.23
84.7 + 1.78
P. Effusion
77.9 + 1.88
84.4
83.4 +: 1.80
84.3
Table 5:
Results ChexPert-5k tasks measured by average AUC (Area Under Roc-Curve)
Text Classification For our NLP experiments, we tried limiting the number of layers we applied ATTITUD to. We achieved good performance without applying ATTITUD to the word embedding layers (these were updated with untouched auxiliary task gradients). We cross-validated Tprinn (0.01, 0.05, 0.0025). For all our NLP experiments, we either recompute our subspace basis every n or n = times
For all experiments involving ATTITUD, We cross-validate the following choices of the subspace size E {5, 10, 20} from steps for vision experiments and every 4 steps for NLP experiments. We run all experiments for maximum of 150 pretraining epochs and 500 finetuning epochs. We performed early stopping for all experiments if no improvement after 10 consecutive epochs.
R using m E {32, 64). We recompute the subspace every 10
blation of Fraction of Norm within Subspace The left pane of Figure reinforces our intuition and confirms that our choice of the top-k singular vectors (randomized svd) gives the best accuracy as averaged across seeds. random the basis spanned by k randomly chosen orthogonal vectors in RDD unit_avg grad is the basis spanned by the average primary task gradient whilst canonical uses the per-parameter basis. Note that 5 for random and rundomized_sve whilst for unit_avg grad and canonical, i and D respectively. We use the fraction of the norm of sample gradients within subspace as indicators of how semantically meaningful that choice of subspace is. We expect that a semantically meaningful choice of basis will achieve better generalization performance because it captures the essential parts of the gradient with & < D canonical trivially captures all D, generalizes poorly. Notice that
the norm of the sampled gradient vectors but because only small fractions of the norms of sample primary and auxiliary task average gradients lie in the subspace for random and unit avg avg-aradg grad, whilst significant fractions lie randomized svd.
14

---

Published as a conference paper at ICLR 2021
660
540
Kung Sol
Nearo Tral
201 Banok ths Bo may
La oal Lharatie valuassa tho
400
Figure 3: Experiment conducted on Cat-vr-Dog Cifar10 dataset. Left Averaged accuracy across seeds of different choices of basis. Our choice, randomized svd performs best. Right We look at the fraction of the norm of J within each subspace (dashed line). We also do for randomly sampled mini batch of the primary task (solid line).
15


Reviews:
[{"review_id": "r1CUpI3QzsX", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\nLeveraging the power of the data-rich related tasks have been studied (e.g., pre-training and multitask learning). This paper points out that careful utilization of auxiliary task is required to gain enhanced performance in primary tasks. In order to prevent harming the performance of primary tasks, they suggest the method to decompose auxiliary updates into three directions which have positive, negative and neutral impact on the primary task.\n\n##########################################################################\n\nReasons for score: \n\nIn this paper, it is highly interesting to see how to use a decomposition from the span of the primary task Jacobian to adapt auxiliary gradients and validate the proposed methodology on image and textual data. Even though this is an interesting setting and the technical solutions presented in the paper look reasonable, the idea seems to be pretty incremental as it stacks multiple existing techniques without many innovations. \n\n##########################################################################\n\nPros: \n1. The proposed methodology utilizes  automatic differentiation procedures and randomized singular value decomposition for efficient scalability. \n2. The proposed framework allows the model to treat each auxiliary update independently by its impact on the task of interest, which seems to be interesting.\n\n##########################################################################\n\nCons: \nAuthors need to perform more qualitative and quantitative analysis on the datasets to vilify the effectiveness of the proposed methodology.\n\n##########################################################################\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "aI_k3y3S0yt", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary] This paper studies auxiliary learning, and propose a decomposition method on the auxiliary gradient into several components, and to select relevant/useful decomposed gradients to maximize the assistance of the primary task.\n\n[Strength] How to adjust auxiliary tasks in a beneficial way is always a challenging problem in multi-task learning. The proposed solution on gradient decomposition is novel and interesting. The improved performance based on the proposed method seems to be non-trivial.\n\n[Weakness] This paper however has some significant weaknesses which I will outline below.\n-- Missing details. The subspace of the primary task gradient is composed of all training samples in the primary task, based on the definition of $\\mathcal{S}$. However, for any reasonable-size training dataset which contains at least 10k training samples, this process seems to be extremely expensive to compute? The authors have introduced randomised approximation on decomposition step, but accumulate gradients seem to be already taking a lot of computation. I hope the authors could justify this.\n\n-- Unfair experimental setting. My biggest concern is on hyper-parameter tuning on each component of the decomposed auxiliary task gradient. In Eq. 1, the authors decompose each auxiliary task gradient into three components: i) one is lying in the subspace of primary task gradient, ii) one is orthogonal to the primary task gradient ii) the final one is in conflict to the primary task gradient. By selecting different weightings on each component, the authors claim that this formulation can be degraded into prior auxiliary learning methods. I agree that this formulation is general. However, all of these component weightings are selected by hand, and seem to be very different across different datasets and tasks. \n\nThis gives a very unfair comparison to previous methods, simply because these methods are included in one of the hyper-parameter sets of corresponding weighting values. I have noticed that the authors have constrained the search space into 4 sets of weighting, but this does not justify this problem. Ideally, these weighting should be automatically computed during training, and varied based on the dataset.\n\nIn addition, the weighting for the primary task also varies across different datasets. So I am wondering whether these task weightings are consistent in baselines, and did authors perform a similar hyper-parameter search on baseline methods as well?\n\nOther minor issues:\nThis formulation *cannot* be used as pre-training with only auxiliary gradients which are helpful to the primary task, since we do not know the primary task gradient beforehand. Even we know the primary task gradient, (0, 1, 0) is simply putting more weighting on the existing primary task gradient, so it's the same as to tune up the learning rate.\n\n=======================\n\nRated up after authors' clarification.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CuBHK5kZR_I", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe authors present a general formulation of different settings in multitask learning (including pretraining regimes), in a setting where the goal is to get best performance for a pre-specified primary task and additional auxiliary tasks. The main idea is to divide the gradients on the auxiliary task into 2 subspaces: a subspace where the gradients influence performance of the primary task and a subspace where they only influence the auxiliary task without changing the loss on the primary task. Within the subspace that does have influence on the primary task, it is easy to compute directions that have a positive or negative effect on the primary task, which allows to create different learning schemes given the gradients that point toward: i) auxiliary influence only, ii) positive influence on auxiliary tass, iii) negative influence on primary task. Experimental results show improvements over previously identified meta learning methods on 2 natural language datasets and 3 image datasets.\n\nStrengths:\nThe authors present a general framework for an important problem. It has many applications in a wide variety of fields and contributes to thinking about meta learning and pretraining in a more general way.\nExplanations, illustrations and mathematical derivations are clear and easy to understand.\nThe authors show that with a careful choice of hyperparameters, their approach can improve performance, especially in settings with limited data on the primary task. The results on natural language datasets are also interesting, showing that they achieve a higher performance when the auxiliary task doesn’t exactly match the primary task data.\n\nWeaknesses:\nSome of the explanations and especially the proof fall apart, when considering the k-largest principal vector of J*. Since k<<D, the sum of all gradients in g_aux will clearly still have a big influence on the performance of the primary task.\nThe proposed algorithm introduces a lot of additional hyperparameters and not all of those hyper parameters are properly discussed. Eta_aux and eta_prim are properly discussed and the authors convincingly show that these parameters are implicitly set by other methods as well. Figure 2 shows an ablation study but does not help practitioners to set the discussed values, given the large variance over 5 runs.\nThe choice of the subspace for g_aux seems very critical and the provided experimental results and discussion are somewhat lacking. How can a random choice of subspace basis improve the results? Calculating the randomized_lowrank_approx is only done every n steps, but there is no mention of n later. Some experimental results should be provided to convince the reader that the basis does not change too much given 2 different batches from the primary task.\n\nOther remarks:\nThe result in Table 2 is much better than the best result in Table 4 on the Cat-vs-Dogs experiment. What is the difference? Can the experiments in Table 4 be repeated to be more comparable to Table 2?\nPCGrad most closely resembles this work. What type of subspace basis is used in that work? It would be interesting to see a direct comparison between PCGrad and the proposed method with eta_aux = (alpha_aux, alpha_aux,- alpha_aux) and using the same basis.\nEarly stopping after 10 epochs seems quite short and might explain some of the large variance in the results.\n\nMinor remarks:\nk is introduced without much explanation, which was a bit confusing on first reading. It should be clearly stated that it is a hyper parameter on first mention.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "9fXahovi5GL", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The work studies the auxiliary task selection in deep learning to resolve the burden of selecting relevant tasks for pre-training or the multitask learning. By decomposing the auxiliary updates, one can reweight separately the beneficial and harmful directions so that the net contribution to the update of the primary task is always positive. The efficient implementation is experimented in text classification, image classification, and medical imaging transfer tasks.\n\nThe first contribution is the decomposition algorithm and reweighting of the auxiliary updates. It is a simple idea with a nice insight of treating the primary task and the auxiliary tasks in different manners. The decomposition allows a reweighting on the updates to optimize the primary task as much as possible while keeping the auxiliary tasks providing improvable directions. The second contribution is an efficient mechanism to approximate and calculate the SVD of the Jacobian of the primary task. The mechanism is implemented from an existing randomized approximation method. The third contribution is a set of experiments verifying the proposed method. The experiments include text classification, image classification, and medical imaging transfer tasks. The most salient result is the 99% data efficiency to achieve improvable performance in the medical imaging transfer task.\n\nConcerns\n\nBesides the above positive contributions, following are some concerns from the observations:\n\n1. The relative improvements comparing to the baselines in Table 1 and Table 2 do not seem as much as that in (Gururangan et al. 2020) and (Yu et al., 2020), respectively. \n\n2. The weights reported in the experiments are 1 or -1 in the experiments. For instance, \\eta_aux = (1, 1, -1) is reported in the image classification task.\n\nThe reader would expect much better improvements when given the freedom to reassign the weights on the decomposed directions, especially when the harmful part has a negated weight. Moreover, why are the values chosen in \\eta 1 or -1? Would there be a nicer balance between, say, the beneficial and the harmful parts? For instance, would \\eta = (1, 0.8, -0.9) be a better choice? It would be crucial that the authors can explain furthermore or support further experiments to confirm whether the potential of this decomposition algorithm is fully demonstrated or not.\n\n\n=====================\n\nPost Rebuttal\n\nI have read the authors' response. All my concerns are addressed properly. However, I still doubt that even the corner cases of \\eta have a better performance, would there be a systematic way to find the optimal parameters reflecting the true potential of this method. Thus, I will keep my score unchanged.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: After engaging in some good interactive discussions all but one reviewer settled on a rating of marginal accept. The most negative reviewer didn't really provide a clear enough explanation of what was lacking in the work. The other reviewers felt that the observed gains for this multi-task learning framework were clear enough that the work is worthy of some attention by the community. The AC recommends acceptance, but one may consider this recommendation as a just past the line for acceptance recommendation.
Decision: Accept (Poster)