$\alpha$VIL: Learning to Leverage Auxiliary Tasks for Multitask Learning

1 INTRODUCTION . In Machine Learning , we often encounter tasks that are at least similar , if not even almost identical . For example , in Computer Vision , multiple datasets might require object segmentation or recognition ( Deng et al. , 2009 ; LeCun et al. , 1998 ; Lin et al. , 2014 ) whereas in Natural Language Processing , tasks can deal with sentence entailment ( De Marneffe et al. , 2019 ) or paraphrase recognition ( Quirk et al. , 2004 ) , both of which share similarities and fall under the category of Natural Language Understanding . Given that many such datasets are accessible to researchers , a naturally emerging question is whether we can leverage their commonalities in training setups . Multitask Learning ( Caruana , 1993 ) is a Machine Learning paradigm that aims to address the above by training a group of sufficiently similar tasks together . Instead of optimizing each individual task ’ s objective , a shared underlying model is fit so as to maximize a global performance measure , for example a LeNet-like architecture ( LeCun et al. , 1998 ) for Computer Vision , or a Transformer-based encoder ( Vaswani et al. , 2017 ) for Natural Language Processing problems . For a broader perspective of Multitask Learning approaches , we refer the reader to the overviews of Ruder ( 2017 ) ; Vandenhende et al . ( 2020 ) . In this paper we introduce αVIL , an approach to Multitask Learning that estimates individual task weights through direct , gradient-based metaoptimization on a weighted accumulation of taskspecific model updates . To our knowledge , this is the first attempt to leverage task-specific model deltas , that is , realized differences of model parameters before and after a task ’ s training steps , to directly optimize task weights for target task-oriented multitask learning . We perform initial experiments on multitask setups in two domains , Computer Vision and Natural Language Understanding , and show that our method is able to successfully learn a good weighting of classification tasks . 2 RELATED WORK . Multitask Learning ( MTL ) can be divided into techniques which aim to improve a joint performance metric for a group of tasks ( Caruana , 1993 ) , and methods which use auxiliary tasks to boost the performance of a single target task ( Caruana , 1998 ; Bingel & Søgaard , 2017 ) . Some combinations of tasks suffer when their model parameters are shared , a phenomenon that has been termed negative transfer . There have been efforts to identify the cause of negative transfer . Du et al . ( 2018 ) use negative cosine similarity between gradients as a heuristic for determining negative transfer between target and auxiliary tasks . Yu et al . ( 2020 ) suggest that these conflicting gradients are detrimental to training when the joint optimization landscape has high positive curvature and there is a large difference in gradient magnitudes between tasks . They address this by projecting task gradients onto the normal plane if they conflict with each other . Wu et al . ( 2020 ) hypothesize that the degree of transfer between tasks is influenced by the alignment of their data samples , and propose an algorithm which adaptively aligns embedded inputs . Sener & Koltun ( 2018 ) avoid the issue of negative transfer due to competing objectives altogether , by casting MTL as a Multiobjective Optimization problem and searching for a Pareto optimal solution . In this work , we focus on the target task approach to Multitask Learning , tackling the problem of auxiliary task selection and weighting to avoid negative transfer and maximally utilize positively related tasks . Auxiliary tasks have been used to improve target task performance in Computer Vision , Reinforcement Learning ( Jaderberg et al. , 2016 ) , and Natural Language Processing ( Collobert et al. , 2011 ) . They are commonly selected based on knowledge about which tasks should be beneficial to each other through the insight that they utilize similar features to the target task ( Caruana , 1998 ) , or are grouped empirically ( Søgaard & Goldberg , 2016 ) . While this may often result in successful task selection , such approaches have some obvious drawbacks . Manual feature-based selection requires the researcher to have deep knowledge of the available data , an undertaking that becomes ever more difficult with the introduction of more datasets . Furthermore , this approach is prone to failure when it comes to Deep Learning , where model behaviour does not necessarily follow human intuition . Empirical task selection , e.g. , through trialling various task combinations , quickly becomes computationally infeasible when the number of tasks becomes large . Therefore , in both approaches to Multitask Learning ( optimizing either a target task using auxiliary data or a global performance metric ) , automatic task weighting during training can be beneficial for optimally exploiting relationships between tasks . To this end , Guo et al . ( 2019 ) use a two-staged approach ; first , a subset of auxiliary tasks which are most likely to improve the main task ’ s validation performance is selected , by utilizing a MultiArmed Bandit , the estimates of which are continuously updated during training . The second step makes use of a Gaussian Process to infer a mixing ratio for data points belonging to the selected tasks , which subsequently are used to train the model . A different approach by Wang et al . ( 2020 ) aims to directly differentiate at each training step the model ’ s validation loss with respect to the probability of selecting instances of the training data ( parametrised by a scorer network ) . This approach is used in multilingual translation by training the scorer to output probabilities for all of the tasks ’ training data . However , this method relies on noisy , per step estimates of the gradients of the scorer ’ s parameters as well as the analytical derivation of it depending on the optimizer used . Our method in comparison is agnostic to the optimization procedure used . Most similarly to our method , Sivasankaran et al . ( 2017 ) recently introduced Discriminative Importance Weighting for acoustic model training . In their work , the authors train a model on the CHiME-3 dataset ( Barker et al. , 2015 ) , adding 6 artificially perturbed datasets as auxiliary tasks . Their method relies on estimating model performances on the targeted validation data when training tasks in isolation , and subsequently using those estimates as a proxy to adjust individual task weights . Our method differs from this approach by directly optimizing the target validation loss with respect to the weights applied to the model updates originating from training each task . 3 α-VARIABLE IMPORTANCE LEARNING The target task-oriented approach to Multitask Learning can be defined as follows . A set of classification tasks T = { t1 , t2 , . . . , tn } are given , each associated with training and validation datasets , Dtrainti and D dev ti , as well as a target task t ∗∈T . We want to find weights W= { ω1 , ω2 , . . . , ωn } capturing the importance of each task such that training the parameters θ of a Deep Neural Network on the weighted sum of losses for each task maximizes the model ’ s performance on the target task ’ s development set : θ∗ = argmin θ |T |∑ i=0 wi∑ w · Lti ( Dtrainti , θ ) s.t . L t∗ ( Ddevt∗ , θ∗ ) ≈ min θ Lt ∗ ( Ddevt∗ , θ ) ( 1 ) where Lti ( Dti , θ ) is defined as the average loss over all data points in the dataset Dti for network parameters θ , computed using the appropriate loss function for task ti ( in our case the standard cross entropy loss ) : Lti ( Dti , θ ) = 1 |Dti | ∑ k L ( xk , yk ; θ ) , ( xk , yk ) ∈ Dti ( 2 ) The introduction of task weights w in Equation 1 aims at scaling the tasks ’ model updates in a way that positive transfer towards the target is exploited and negative transfer avoided . It is crucial therefore to have an efficient and reliable way of estimating the influence of tasks on the target ’ s performance , and of adjusting the weights accordingly during model training . To this end , we introduce α-Variable Importance Learning ( αVIL ) , a novel method for target taskoriented Multitask training , outlined in Algorithm 1. αVIL introduces a number of additional parameters – α-variables – into the model , which are associated with the actually realized task-specific model updates . Algorithm 1 : The α-Variable Importance Learning algorithm . Data : Model parameters θ ; a set of tasks T = { t1 , . . . , tn } ; a target task t∗ ; training data Dtrainti for each task ti ; development data D dev t∗ for the target task ; maximum number of training epochs E ; ratio ρ of tasks ’ training data to sample per epoch ; number of α tuning steps s Result : updated parameters θ , optimized for performance on t∗ 1 W← { wti = 1 | ti ∈ T } // initialize all task weights to 1 2 for = 1 . . . E do 3 for ti ∈ T do 4 D ti ρ∼ Dtrainti // sample task-specific data 5 θti ← argminθ′ wti∑ wL ti ( D ti , θ ′ ) // task ’ s model update starting at θ 6 δti ← θti − θ 7 // task-specific weight update , optimizing wrt . α parameters on δ 8 for s ∈ 1 . . . s do 9 { α1 , α2 , . . . , α|T | } ← argmin { α1 , α2 , ... , α|T | } L ( D dev t∗ , θ + α1δ1 + . . .+ α|T |δ|T | ) 10 θ ← θ + α1δ1 + . . .+ α|T |δ|T | 11 W← { wti + ( αti − 1 ) | ti ∈ T } During training , our approach first performs weighted task-specific model updates on a proportion of the available training data for each individual task , starting from the current model parameters . It collects the resulting model deltas , i.e. , the differences between the model ’ s parameters before and after the singletask update and resets the model . After this delta collection phase , the optimal mixing factors , that is , { α1 , α2 , . . . , α|T | } of the model updates are found , such that the parameters resulting from the interpolation of scaled task-specific δ ’ s minimize the loss on the target task ’ s development data . The α–parameters can be optimized through any type of optimization method however , since our models are end to end differentiable , we can backpropagate directly and use gradient descent . Once we have found the optimal mixing ratio of task updates , we write the new state back to the model , and update the task weights subject to the optimized α parameters . The task weight update rule ( line 11 in Algorithm 1 ) combined with the weighted task-specific model updates ( line 5 ) tries to capture the intuition that if a task update was up- or down-scaled in the α-tuning stage , we likely want to update the parameters more/less for this task , during the next delta collection phase .

Summary: This paper proposes a novel multi-task learning method which adjusts task weights dynamically during training, by exploiting task-specific updates of the model parameters between training epochs. Specifically, the proposed model takes the differences between the model’s parameters before and after the singletask update, after that the mixing factors of the model updates are found based on the differences to minimize the loss on the target task’s development data. Empirical studies are performed on tasks of computer vision and natural language understanding.