Multi-task Learning with Gradient Communication

1 INTRODUCTION A1 A2 A3 ∇LA1→A2 ∇LA2→A3 ( a ) C A B ∇LA↔C ∇LC↔B ∇LA↔B ( b ) Figure 1 : A , B , C represent different tasks.∇θL denotes the gradients with respect to θ . ( a ) . Gradient Passing within a task . ( b ) . Gradient Passing across tasks . Multi-task learning has been established as an effective learning method in deep learning , driving state-of-theart results to new levels in a number of tasks , ranging from computer vision ( Misra et al. , 2016 ; Zamir et al. , 2018 ) to natural language processing ( Collobert & Weston , 2008 ; Luong et al. , 2015 ) . While various kinds of neural multi-task learning approaches have been proposed , a commonality is the two types of feature spaces they defined to store taskspecific features and task-invariant features . And then interaction mechanisms among different feature spaces will be explored in diverse ways , such as hard sharing ( Collobert & Weston , 2008 ) or soft sharing ( Misra et al. , 2016 ; Ruder12 et al. , 2017 ) schemes , flat ( Yang et al. , 2016 ; Liu et al. , 2016 ) or structural sharing schemes ( Hashimoto et al. , 2016 ; Søgaard & Goldberg , 2016 ) . Despite their great success , a potential limitation of most existing works is that shared space is entangled with private features since it is not enforced through the use of explicit loss functions ( Bousmalis et al. , 2016 ) . Specifically , we would have extracted shared features and disentangled them into separated spaces . However , in reality , methods in use just collect all the features together into a common space , instead of learning shared rules across different tasks . This problem , which we define as “ pretend-to-share ” , clearly suffers from the following issues : 1 ) IN-TASK SETTING : shared space is redundant and contaminated by task-specific features , leaving potential hazards for other training tasks . 2 ) OUT-OF-TASK SETTING : the learned model can not effectively adapt to new tasks since a real common structure is not discovered . A few works explore this problem with adversarial training and shows that explicitly disentangling representations into private and shared spaces will improve model ’ s performance ( Bousmalis et al. , 2016 ; Liu et al. , 2017 ) . In this paper , we propose an orthogonal technique to alleviate above problem . Specifically , we first describe a general framework for multi-task learning called Parameters ReadWrite Networks ( PRaWNs ) that can simply abstract the commonalities between existing typical neural models for multi-task learning . Then , we explain the reason for pretend-to-share problem within PRaWN that each task has the permission to modify shared parameters without any communication protocol , which makes it easier to update parameters along diverse directions . Therefore , the problem is translated into this question : how to address the conflict among different tasks happening in the process of parameter updating . We resolve this conflict by introducing an inductive bias for multi-task learning . Specifically , different tasks can communicate with each other by sending ( or receiving ) both hidden variables and gradients . As shown in Fig.1 ( b ) , during the forward phase , tasks A , B , C are capable of sending ( and receiving ) the gradients to ( and from ) each other . Then during the backward phase , each task is aware of how other tasks are modifying the parameters , making it possible to modify in a more consistent way . Concretely , by communicating with gradients , we can apply consistency constraint to each task during the process of parameter updating , which can prevent private features from slipping into shared space thereby alleviating the pretend-to-share problem . Technically , we explore two kinds of communication mechanisms for gradient passing : pairwise and list-wise communication , where the latter one can take task relatedness into consideration . We evaluate our models on three groups of multi-task learning datasets , range from natural language processing tasks to computer vision . The results show that our models are more expressive since we can learn a shared space which are more pure , driving task-dependent features away . To summarize , we make the following contributions : • We describe a general framework , which can not only help us analyze current models in an unified way [ Section 3 ] , but find potential limitations , and move forward with suitable approaches [ Section 4 ] . • We study a task-agnostic problem existing in many multi-task learning models , and propose to alleviate it by allowing different tasks to communicate with gradients . • We present an elaborate qualitative analysis , giving some insight into the internal mechanisms of proposed gradient passing method . 2 RELATED WORK . Multi-task Learning There is a large body of works relating to multi-task learning , and we will give systematical analysis in next section . This paper studies a task-agnostic problem existing in many multi-task learning models . Existing models expect to disentangle features into different spaces while features learned in practice are entangled in shared space . Several papers ( Liu et al. , 2017 ; Chen et al. , 2017 ) also observe this problem more recently and they propose address it with adversarial training . Despite its effectiveness , it suffers from difficult training process to achieve the balance between generator and discriminator . Besides , adversarial multi-task training methods ignore the relationship between different tasks , leading to low-capacity of learned shared space . Gradient Passing Between Samples Recently , some works begin to model the dependencies of training samples . Specifically , one sample xi can pass gradients to another one xj , in which an intermediate function is used to transform gradients into new weights ( called fast weights ) for xj . This function usually takes different forms , ranging from recurrent neural networks ( Andrychowicz et al. , 2016 ; Ravi & Larochelle , 2016 ) , temporal convolutions with attention ( Mishra et al. , 2018 ) , or memory networks ( Santoro et al. , 2016 ) . Meanwhile , Finn et al . ( 2017 ) explores a special case where the function is an updating function of SGD . However , the samples ’ dependencies studied by these works are within a task as shown in Fig.1 ( b ) , whose motivations are to learn a general updating rule , which can achieve fast adaptation to unseen samples . Different with these works ( Finn et al. , 2017 ; Gu et al. , 2018 ) , we model the samples ’ dependencies cross different tasks , aiming to disentangle private and shared features into different spaces . Domain Adaptation Domain adaptation ( DA ) alone is a generally well studied area both in natural language processing ( Li & Zong , 2008 ; Blitzer et al. , 2007 ) and computer vision ( Saenko et al. , 2010 ) communities . And there are several differences between domain adaptation and multi-task learning ( MTL ) . 1 ) For domain adaption , there is an explicit pair of source and target domains . Addtionally , the purpose is to adapt the model learned on source domains to target domains . Therefore , DA cares much about the performance on target domains while MTL focuses on performance both in IN-TASK setting and OUT-OF-TASK setting . 2 ) For domain adaptation , different domains usually have the same label space , and all parameters for different domains are shared , with no need for disentangling shared and private spaces . More specifically , some recent works introduce meta learning methods for fast domain adaptation ( Li et al. , 2017 ; Kang & Feng ; Yu et al. , 2018 ) . Apart from above two points , the difference with this paper is that our motivation starts from the pretend-to-share problem in multi-task learning , then we seek an approach to learn a shared space , which is not contaminated by private features . 3 PARAMETERS READ-WRITE NETWORKS . Recent years have seen remarkable success in the use of various neural multi-task learning models to natural languange processing and computer vision . The time is ripe to propose a more general framework , in which we can systematically analyze current models , find potential limitations , and move forward with suitable approaches . With above in mind , in this paper , we propose such a framework , called Parameters Read-Write Networks ( PRaWNs ) , which allow us to dissect existing models and explore additional novel variations within this framework . We will first describe details of PRaWNs . 3.1 CONCEPT DEFINITIONS IN PRAWN . Task agents and parameters are two concepts in PRaWN . And the high-level idea of PRaWN to describe multi-task learning process can be expressed as follows : different task agents read information from different types of parameters , and then they write new information to parameters to satisfy some constraints . Task Agents In PRaWN , each task is regarded as an agent with three elements ( D , READ-OP , WRITE-OP ) , in which D denotes the data belonging to the current task . READ-OP and WRITE-OP represent two operations that the current task can execute . The forms of the tasks can differ in various ways , such as the domain ( multi-domain learning ) or the language ( cross-lingual learning ) . Parameters Parameters are the objects that operations ( READ-OPs and WRITE-OPs ) of task agents could execute to . Generally , parameters can be classified into different types based on the following aspects . • Sharable or Private : Sharable parameters can be read and modified by multiple tasks . • Readable or Non-readable : If θ for the task agent k is readable , it means that READ-OP of task k can be executed . • Writable or Non-writable : If θ for the task agent k is readable , it means that WRITE-OP of task k can be executed . According to above three aspects of parameters , we can enumerate four effective modes for general multi-task learning as shown Fig . 2 . We use swr to represent them and the r̄ denotes non-readable . For each task , we can understand its sharing schema based on its constituents of parameters . Detailed examples can be found in Appendix . 3.2 READ OPERATION . The purpose of READ-OP is to gather useful information rk based on current input samples X k of task k from all readable parameters . rk = READ-OP ( Θ∗rk , X k ) ( 1 ) = OUT ( θs̄wrk , ENC ( X k , { Θ∗rk − θs̄wrk } ) ) ( 2 ) where we refer to Θ as a set of parameters θ. Θ∗rk = { Θswrk , Θs̄wrk , Θs̄w̄rk } represents all readable parameters with respect to task k. OUT denotes output layer while ENC denotes feature encoding layer , ENC can be instantiated as CNN or RNN layer . For this operation , the key step is how to effectively define readable parameters Θ∗rk , organizing their dependency relationship and then aggregating useful information from them . Specifically , the process to seek for desirable READ-OPs can be abstracted to a architecture searching problem ( Chen et al. , 2018a ) , which has raised a large body of existing works on this as shown in Tab.1 . Here , we show some typical READ-OPs . Flat READ-OPs : Θ∗rk = { Θswrk , Θs̄wrk } and { Θs̄wrk − θs̄wrk } = ∅ Models with flat READ-OP usually assume that all parts of neural network are sharable except output layer , suggesting that different tasks have no private space to encode input samples . Star-network READ-OPs : Θ∗rk = { Θswrk , Θs̄wrk } { Θs̄wrk − θs̄wrk } is used for learning task-dependent representations . Different with above models , star-network operation introduces an extra private encoder for each task , aiming to disentangle task-specific and task-invariant features into different spaces . Structural READ-OPs : Θ∗rk = { Θswrk , Θs̄wrk , Θs̄w̄rk } Clearly , for above two operations , there is no definitions of parameters θs̄w̄rk for any k , which means that any two tasks can not directly communicate with each other . To address this problem , structural READ-OPs are proposed , where each task can read information from the parameters of other tasks .

Summary: In this paper, the authors propose a general framework for multi-task learning (MTL) in neural models. The framework is general for including some of the current neural models for MTL. Under the framework, the author propose a new method that could allow tasks to communicate each other with explicit gradients. Based on the gradients being communicated, the system could adjust the updates of one task based on the gradient information of the other task. Also, prior task relatedness information could be incorporated to the system.