Title: MULTI-TASK LEARNING WITH GRADIENT COMMUNI-CATION

Abstract: In this paper, we describe a general framework to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks. We propose to alleviate this problem by incorporating a new inductive bias into the process of multi-task learning, that different tasks can communicate with each other not only by passing hidden variables but gradients explicitly. Experimentally, we evaluate proposed methods on three groups of tasks and two types of settings (IN-TASK and OUT-OF-TASK). Quantitative and qualitative results show their effectiveness.

:

Multi-task learning has been established as an effective learning method in deep learning, driving state-of-theart results to new levels in a number of tasks, ranging from computer vision (Misra et al., 2016; Zamir et al., 2018) to natural language processing (Collobert & Weston, 2008; Luong et al., 2015) .

While various kinds of neural multi-task learning approaches have been proposed, a commonality is the two types of feature spaces they defined to store taskspecific features and task-invariant features. And then interaction mechanisms among different feature spaces will be explored in diverse ways, such as hard sharing (Collobert & Weston, 2008) or soft sharing (Misra et al., 2016; Ruder12 et al., 2017) schemes , flat (Yang et al., 2016; Liu et al., 2016) or structural sharing schemes (Hashimoto et al., 2016; Søgaard & Goldberg, 2016) .

Despite their great success, a potential limitation of most existing works is that shared space is entangled with private features since it is not enforced through the use of explicit loss functions (Bousmalis et al., 2016) . Specifically, we would have extracted shared features and disentangled them into separated spaces. However, in reality, methods in use just collect all the features together into a common space, instead of learning shared rules across different tasks. This problem, which we define as "pretend-to-share", clearly suffers from the following issues: 1) IN-TASK SETTING: shared space is redundant and contaminated by task-specific features, leaving potential hazards for other training tasks. 2) OUT-OF-TASK SETTING: the learned model can not effectively adapt to new tasks since a real common structure is not discovered.

A few works explore this problem with adversarial training and shows that explicitly disentangling representations into private and shared spaces will improve model's performance (Bousmalis et al., 2016; Liu et al., 2017) . In this paper, we propose an orthogonal technique to alleviate above problem. Specifically, we first describe a general framework for multi-task learning called Parameters Read-Write Networks (PRaWNs) that can simply abstract the commonalities between existing typical neural models for multi-task learning. Then, we explain the reason for pretend-to-share problem within PRaWN that each task has the permission to modify shared parameters without any communication protocol, which makes it easier to update parameters along diverse directions.

Therefore, the problem is translated into this question: how to address the conflict among different tasks happening in the process of parameter updating. We resolve this conflict by introducing an inductive bias for multi-task learning. Specifically, different tasks can communicate with each other by sending (or receiving) both hidden variables and gradients. As shown in Fig. 1 (b), during the forward phase, tasks A, B, C are capable of sending (and receiving) the gradients to (and from) each other. Then during the backward phase, each task is aware of how other tasks are modifying the parameters, making it possible to modify in a more consistent way. Concretely, by communicating with gradients, we can apply consistency constraint to each task during the process of parameter updating, which can prevent private features from slipping into shared space thereby alleviating the pretend-to-share problem. Technically, we explore two kinds of communication mechanisms for gradient passing: pairwise and list-wise communication, where the latter one can take task relatedness into consideration.

We evaluate our models on three groups of multi-task learning datasets, range from natural language processing tasks to computer vision. The results show that our models are more expressive since we can learn a shared space which are more pure, driving task-dependent features away.

To summarize, we make the following contributions:

• We describe a general framework, which can not only help us analyze current models in an unified way [Section 3], but find potential limitations, and move forward with suitable approaches [Section 4].

• We study a task-agnostic problem existing in many multi-task learning models, and propose to alleviate it by allowing different tasks to communicate with gradients.

• We present an elaborate qualitative analysis, giving some insight into the internal mechanisms of proposed gradient passing method.

2. RELATED WORK:

Multi-task Learning There is a large body of works relating to multi-task learning, and we will give systematical analysis in next section. This paper studies a task-agnostic problem existing in many multi-task learning models. Existing models expect to disentangle features into different spaces while features learned in practice are entangled in shared space. Several papers (Liu et al., 2017; Chen et al., 2017) also observe this problem more recently and they propose address it with adversarial training . Despite its effectiveness, it suffers from difficult training process to achieve the balance between generator and discriminator. Besides, adversarial multi-task training methods ignore the relationship between different tasks, leading to low-capacity of learned shared space.

Gradient Passing Between Samples Recently, some works begin to model the dependencies of training samples. Specifically, one sample x i can pass gradients to another one x j , in which an intermediate function is used to transform gradients into new weights (called fast weights) for x j . This function usually takes different forms, ranging from recurrent neural networks (Andrychowicz et al., 2016; Ravi & Larochelle, 2016) , temporal convolutions with attention (Mishra et al., 2018) , or memory networks (Santoro et al., 2016) . Meanwhile, Finn et al. (2017) explores a special case where the function is an updating function of SGD. However, the samples' dependencies studied by these works are within a task as shown in Fig. 1 (b), whose motivations are to learn a general updating rule, which can achieve fast adaptation to unseen samples. Different with these works (Finn et al., 2017; Gu et al., 2018) , we model the samples' dependencies cross different tasks, aiming to disentangle private and shared features into different spaces.

Domain Adaptation Domain adaptation (DA) alone is a generally well studied area both in natural language processing (Li & Zong, 2008; Blitzer et al., 2007) and computer vision (Saenko et al., 2010) communities. And there are several differences between domain adaptation and multi-task learning (MTL). 1) For domain adaption, there is an explicit pair of source and target domains. Addtionally, the purpose is to adapt the model learned on source domains to target domains. Therefore, DA cares much about the performance on target domains while MTL focuses on performance both in IN-TASK setting and OUT-OF-TASK setting. 2) For domain adaptation, different domains usually have the same label space, and all parameters for different domains are shared, with no need for disentangling shared and private spaces. More specifically, some recent works introduce meta learning methods for fast domain adaptation (Li et al., 2017; Kang & Feng; Yu et al., 2018) . Apart from above two points, the difference with this paper is that our motivation starts from the pretend-to-share problem in multi-task learning, then we seek an approach to learn a shared space, which is not contaminated by private features.

3. PARAMETERS READ-WRITE NETWORKS:

Recent years have seen remarkable success in the use of various neural multi-task learning models to natural languange processing and computer vision. The time is ripe to propose a more general framework, in which we can systematically analyze current models, find potential limitations, and move forward with suitable approaches. With above in mind, in this paper, we propose such a framework, called Parameters Read-Write Networks (PRaWNs), which allow us to dissect existing models and explore additional novel variations within this framework. We will first describe details of PRaWNs.

3.1. CONCEPT DEFINITIONS IN PRAWN:

Task agents and parameters are two concepts in PRaWN. And the high-level idea of PRaWN to describe multi-task learning process can be expressed as follows: different task agents read information from different types of parameters, and then they write new information to parameters to satisfy some constraints.

Task Agents In PRaWN, each task is regarded as an agent with three elements (D,READ-OP, WRITE-OP) , in which D denotes the data belonging to the current task. READ-OP and WRITE-OP represent two operations that the current task can execute. The forms of the tasks can differ in various ways, such as the domain (multi-domain learning) or the language (cross-lingual learning).

Modes Properties:

Sharable Writable Readable Parameters Parameters are the objects that operations (READ-OPs and WRITE-OPs) of task agents could execute to. Generally, parameters can be classified into different types based on the following aspects.

swr swr × s wr × × s wr × × ×

• Sharable or Private: Sharable parameters can be read and modified by multiple tasks.

• Readable or Non-readable: If θ for the task agent k is readable, it means that READ-OP of task k can be executed.

• Writable or Non-writable: If θ for the task agent k is readable, it means that WRITE-OP of task k can be executed.

According to above three aspects of parameters, we can enumerate four effective modes for general multi-task learning as shown Fig. 2 . We use swr to represent them and the r denotes non-readable.

For each task, we can understand its sharing schema based on its constituents of parameters. Detailed examples can be found in Appendix.

3.2. READ OPERATION:

The purpose of READ-OP is to gather useful information r k based on current input samples X k of task k from all readable parameters. where we refer to Θ as a set of parameters θ. Θ * r k = {Θ swr k , Θ swr k , Θ s wr k } represents all readable parameters with respect to task k. OUT denotes output layer while ENC denotes feature encoding layer, ENC can be instantiated as CNN or RNN layer.

EQUATION

For this operation, the key step is how to effectively define readable parameters Θ * r k , organizing their dependency relationship and then aggregating useful information from them. Specifically, the process to seek for desirable READ-OPs can be abstracted to a architecture searching problem (Chen et al., 2018a) , which has raised a large body of existing works on this as shown in Tab.1. Here, we show some typical READ-OPs.

Flat READ-OPs: Θ * r k = {Θ swr k , Θ swr k } and {Θ swr k -θ swr k } = ∅ Models with flat READ-OP usually assume that all parts of neural network are sharable except output layer , suggesting that different tasks have no private space to encode input samples.

Star-network READ-OPs:

: Θ * r k = {Θ swr k , Θ swr k } {Θ swr k -θ swr

k } is used for learning task-dependent representations. Different with above models, star-network operation introduces an extra private encoder for each task, aiming to disentangle task-specific and task-invariant features into different spaces.

Structural READ-OPs: Θ * r k = {Θ swr k , Θ swr k , Θ s wr k } Clearly, for above two operations, there is no definitions of parameters θ s wr k for any k, which means that any two tasks can not directly communicate with each other. To address this problem, structural READ-OPs are proposed, where each task can read information from the parameters of other tasks.

3.3. WRITE OPERATION:

For task agent k, the write operation will make modifications to its writable parameters to satisfy some constraints.

EQUATION

where Θ * w * k = {Θ swr k , Θ swr k } represents the parameter sets, each of which is writable with respect to task k.

During the writing phase, a key question is how to introduce desirable constraints to modify writable parameters Θ * w * k for each tasks. Many existing works try to incorporate different kinds of losses to achieve that, where the task-oriented constraint is commonly used to both update Θ swr k and Θ swr k .

Task-Oriented Constraint This constraint usually depends on specific task. For example , for a classification task, L task can be defined as:

EQUATION

where L represents the cross-entropy loss.

To incorporate more inductive bias, some extra constraints are introduced. For example, adversarial constraint L Adv is used to encourage shared space more pure while orthogonal constraint L Dif f introduce an inductive bias that shared and private feature space should be more orthogonal. More descriptions of these two constraints can be found in Appendix.

4. EXPLORING NEW COMMUNICATION PROTOCOLS FOR PRAWN:

Given how many specific multi-task learning models have appeared in the literature, an general framework can not only help us deeply understand existing work in an unified way, but can guide us to find potential limitation and motivate us to propose new variations for a specific application.

4.1. PROBLEMS AND MOVING FORWARD:

In PRaWN framework, a lot of works focus on designing READ-OPs with different forms, aiming to obtain the most suitable sharing schema. However, the pretend-to-share problem is less studied in multi-task learning. By dissecting multi-task learning models into different phases within PRaWN, we can get more intuitive understanding towards this problem.

Parameters Corruption:

We attribute the reason for pretend-to-share to the inconsistent WRITE-OPs executed by different tasks. Specifically, each task has the permission to modify shared parameters Θ swr k without any communication protocol, which makes it easier to update parameters along diverse directions.

Formally, given two tasks A and B, they will both execute WRITE-OP towards their shared parameters Θ swr k based on their data D A and D B as follows:

EQUATION

The independency of WRITE-OPs among different tasks make it hard to let multiple tasks update parameters consistently. Therefore, the shared space we finally learned is redundant and contaminated by task-specific features. To achieve the above idea, in this paper, we propose two types of approaches as a primary exploration, and hopefully, more effective methods can be put as future work.

Pair-wise Communication:

In this architecture, we only consider the communication of any two tasks. Specifically, for tasks A and B, they can explicitly send their gradient to each other, telling how they modified shared parameters. Simultaneously, the receiver should update parameters with the consideration of the other task's modifications.

Formally, the model will first computed the gradient ∇ Θ swr L A based on task A, and then the gradient will be passed to task B.

READ-OP = OUT(θ swr B , ENC(X B , {Θ * r B -θ swr B }, ∇ Θ swr L A )) (7) In practice, the highlighted term can be used in different ways. Here, we utilize them to generate a fast weight φ s based on Θ swr : φ s = g(Θ swr , ∇ Θ swr L A ) (8) where g can be a optimizer or parameterized function, which can be defined similar to many existing works relating to meta learning (Andrychowicz et al., 2016; Ravi & Larochelle, 2016; Finn et al., 2017) .

Then, our optimized goal is to produce maximal performance on sample (X B , Y B ) using the fast weight φ s

EQUATION

Require: A set of training tasks {D T i } K i=1 , where Ti is drawn from p(T )

1: Initialize Θ := {θ s , θ p 1 , • • • , θ p K } 2: while not done do 3: for T k ∼ p(T ) do 4: Sample a batch of samples B T k = {X, Y } ∈ D T k 5: Evaluate gradients ∇ θ s ,θ p k L task (θ s , θ p k , B T k ) 6: Update parameters: θ p k ← α∇ θ p k L task 7: if Pairwise-Communication then 8: Sample a batch B T j = {X, Y } ∈ D T j T k = Tj 9: LGP = min θ s L(B T j , ∇ θ s L k , θ s ) 10:

else if Listwise-Communication then 11:

Compute a weight list β k 12:

for Tj ∼ p(T ) do T k = Tj

13::

L k←j GP = min θ s L(B T j , ∇ θ s L k , θ s ) 14:

end for 15:

LGP = j∼T min θ s β k←j L k←j GP 16: end if 17:

Update parameters: θ s ← α∇ θ s LGP 18:

end for 19: end while 

List-wise:

Communication In this approach, each task T k will be appended with a task list

T = {T 1 , • • • , T j , • • • , T K-1 } (T j = T k ) and a weight list β k = {β 1 , • • • , β K-1 }

, where β kj measures the relatedness between task k and j. Moreover, task T k can communicate with any task from T . Formally, for task k, we pass its gradients to the other tasks and achieve maximal performance on corresponding samples D j .

EQUATION

where D j denotes training samples from task j, β k←j describes the task relationship between k and j, which can be pre-defined empirically or learned dynamically.

Empirical Methods for Obtaining Weight Lists:

We provide a simple way to compute β for these tasks which have similar output spaces, such as multiple domains or languages. we take turns choosing one task and use its training data to train a basic model (will be described as follows.). Then, we use the testing data of the remaining tasks to test the basic model.

In this way, we can obtain a matrix V ∈ R K×K , in which each cell V i,j stores the accuracy of testing task j, which is evaluated on model learned from training task i. We then update matrix V as following operation: for each column j:

EQUATION

)

where v = min{V i,j /V j,j , 1} and q is a hyper-parameter. Then Ṽi,j can be used as a weight β kj .

4.3. PUT IT ALL TOGETHER:

We put task-oriented constraint L task and gradient passing L GP together, which can both learn private features and learn more universal shared rules, since explicitly gradient passing allow shared parameters to be updated in a more consistent way.

5. EXPERIMENT:

In this section, we investigate the empirical performance of our proposed models on three groups of multi-task learning datasets: text classification (Multi-Sent) (Liu et al., 2017) , image aesthetic assessment (Murray et al., 2012) (Multi-AVA), and sequence tagging. Each dataset contains several related tasks. The reason that we utilize these datasets is they can provide a sufficient testing ground (i.e. IN-TASK setting and OUT-OF-TASK setting).

5.1. TASKS AND DATASETS:

In this section, we will give a brief description about evaluated tasks and datasets.

Text Classification This dataset contains different text classification tasks involving different popular review corpora, such as "books" and "apparel" Liu et al. (2017) Image Aesthetic Assessment The image aesthetic assessment can be formulated as a classification problem of predicting an image high-or low-aesthetic categories within the supervised learning framework (Murray et al., 2012; Deng et al., 2017) . The multi-domain AVA dataset contains 90,000 images covering 10 different domains For each domain, the data is split into training (6,000), validation (1,000), and testing (2,000) sets.

5.2. EXPERIMENTAL SETTINGS AND IMPLEMENTATION DETAILS:

For Multi-Sent and Multi-AVA datasets, we randomly divide 10 tasks into two groups for IN-TASK setting and OUT-OF-TASK setting. As a contrast, each task will first be evaluated separately by corresponding base models. For text classification and sequence labelling tasks, the base models are implemented as (Liu et al., 2017 ) (LSTM encoder with MLP layer) and (Huang et al., 2015) (LSTM encoder with CRF layer). For image aesthetic assessment task, we adopt a 50-layer Residual Network (He et al., 2016) as the base model. Specifically, for an image I, it will be first vectorized into a feature vector x by the penultimate layer of residual network, which has been pre-trained on Imagenet. Then the obtained feature vector x will be passed into two-layer fully-connected layers, which followed by a softmax function.

Model Settings We evaluate models under two specific evaluation settings, including IN-TASK, OUT-OF-TASK. For OUT-OF-TASK setting, learned shared parameters will be combined with new parameters for a task, which has not be seen during training phase in IN-TASK setting. Then all the parameters will be learned with a few training samples of new task.

Besides, we select different combinations of READ-OPs and constraints of WRITE-OPs as basic multi-task learning frameworks, then investigate their effectiveness when gradient passing mechanism is introduced or not, which results in the following evaluated models:

• FR: Flat READ-OP with task-specific constraint.

• SR: Star-network READ-OP with task-specific constraint.

• ASR: SR with adversarial training and orthogonal constraints.

Our models are built upon FR and SR, and we use the prefixes "PGP" and "LGP" to denote pair-wise gradient passing and list-wise gradient passing respectively.

5.3. MAIN RESULTS:

In-task Setting Tab.2 shows performances of our models on three groups of datasets. For in-task setting, we can obtain the following observations: 1) For different types of tasks, READ-OPs perform Table 2 : Performances of our models on three groups of multi-task learning datasets against typical baselines. ∆ denotes the improvement value compared with the base model under single task setting.

In OUT-OF-TASK setting, we transfer shared layers learned during in-task setting to new tasks, which are not seen during in-task setting. For Multi-AVA dataset, there is no model with star-network READ-OP, since here the feature extractor (ResNet) is used as a pre-trained encoder, whose parameters are fixed and we don't learn a separate ResNet from scratch.

diversely. For example, on Multi-Sent dataset, both flat and star-network READ-OP can achieve significant improvements compared with single task model. However, for tagging task, FR model (flat READ-OP ) works worse than single task model while the performance will be improved when SR is utilized. We attribute the reason to the higher discrepancy existing in tagging tasks. (The output space of each task in tagging is totally different.) 2) The performances of FR and SR can be significantly improved when gradient passing mechanism is introduced. Specifically, LGP-SR outperforms other models on Multi-Sent dataset, which not only shows the effectiveness of propose methods but indicates the importance of taking relatedness of tasks into consideration.

3) PGP-SR and LGP-SR can surpass ASP both on Multi-Sent and tagging datasets, which suggests that it's more effective for multi-task learning to allow different tasks to communicate by gradients. with gradient passing mechanisms show better generalization to unseen tasks, which further suggests that our models can capture more universal features during the training phase in the IN-TASK setting.

Out-of-:



5.4. ANALYSIS:



5.4.1. IMPACT OF THE DIFFERENT NUMBERS OF TRAINING SAMPLES:

For OUT-OF-TASK setting, we additionally investigate how the performances changes for different models as the number of training samples (#samples) changes. Specifically, here we plotted the accuracy curves of our model (LGP-SR) against two baselines SR and ASR on five text classification tasks. As shown in Fig. 4 , we can get following observations: 1) When #samples is less than 500, LGP-SR can achieve the fastest adaptation on last four tasks. 2) For each task, the highest performances are achieved by LGP-SR. Additionally, for "Camera,Health,Music,Video" tasks, the performance of MPS can surpass the model in single-task setting, where 1600 training samples are used. Notably, for "Camera, Music, Video" tasks, performances of LGP-SR can reach the grey horizontal line using half the training data.

5.4.2. EVOLUTION OF PARAMETERS:

To get more intuitive understanding of how gradient passing mechanism influences the shared spaces, we visualize how the shared and private parameters' values evolve under our model PGP-SR compared to the typical baseline SR. Specifically, we randomly chose two tasks "electronics" and "books" to train models SR and PGP-SR. Then, during the training phase, we projected the parameter space using PCA and plot their curves as the epoch increases. As shown in Fig. 5 , we can observe that: for SR model, when the updating directions of two private parameters are inconsistent, shared parameters will be updated along one of them. However, for PGP-SR as shown in Fig. 5-(b) , the updating direction of shared parameters is an integration of two private updating directions, which can prevent shared parameters from fitting more task-specific features.

5.4.3. BEHAVIOURS OF NEURONS IN SHARED LAYER:

We can gain some insight into the internal mechanisms of gradient passing method by studying the neuron activations in shared layer h ∈ R d as they process test set data of different tasks (d is the number of neurons). We were particularly interested in looking at which parts of words can be regraded as useful features by neurons in shared layer. To achieve this, we introduce a variable q for each word w, which is used to describe what the possibility is that the word can be regarded as a useful feature by neurons in h. Formally, we refer to n w as the times that w appears in test data and d is the number of neurons. Then, we define q = Nmax (nw×d) , where N max denotes how many times neurons achieve the highest activation on the word w. We compute q for each word in V with SR and LGP-SR models, in which V = {V 1 , • • • , V 5 } denotes the vocabulary of five test sets (in-task). We then divide V into two parts: the intersection of five sub-vocabularies Figure 5 : Evolution of parameters during training phase projected into 3D space using PCA for SR and PGP-SR models. z-axis describes different epochs. Cyan and green lines represent the evolution of private parameters in task "electronics" and "books", while the red line denotes the evolution of parameters shared by these two tasks. Tab.3 shows the top 10 word features with high values of q on two parts of vocabularies and we can obtain the following observations: 1) Both SR and LGP-SR can easily pick up useful and sharable words in 5 i V i . According to our statistics, there are 1073 words in 5 i V i , and we find the words with high values of p do make sense. For examples, the words with higher ranking "excellent, disappointed, doesnt" are key patterns for all tasks to make prediction of one sentence's sentiment. 2) For words in 7 APPENDIX 

where Y k represent the true labels for the types of current tasks.

Orthogonal Constraint This constraint introduce an inductive bias that shared and private feature space should be more orthogonal.

EQUATION

where • 2 F is the squared Frobenius norm. S k and P k are two matrics, whose rows are the output of shared extractor OUT(X k , Θ swr k ) and task-specific extractor OUT(X k , Θ swr k ) of an input sentence.

i V i and its

i V i , the SR model mistakenly regards more words involving task-specific information as sharable features, such as the name "Brosnan" and entity "sauce". By contrast, LGP-SR still can pick up useful words such as "dissatisfied, havent, overlong".6 CONCLUSIONIn this paper, we introduce an inductive bias for multi-task learning, allowing different tasks to pass gradients explicitly, which makes it available to restrict different tasks make consistent parameter updating. In this way, we can avoid the leakage of private information into the shared space The empirical results not only show the effectiveness of our proposed method, but give some insight into its internal mechanisms.

In this way, we additionally find that for those words in 5 i Vi, it's difficult for baseline model to pick up sharable features.

Figures and Tables:

Figure 1: A, B, C represent different tasks.∇ θ L denotes the gradients with respect to θ. (a). Gradient Passing within a task. (b). Gradient Passing across tasks.

Figure 2: Effective modes of different types parameters for general multi-task learning.

. (2018b);Misra et al. (2016) Table1: Typical READ-OPTs with their basic model frameworks, constituent parameters, and related instantiations. The second column illustrates the sketch of each READ-OP, which takes as input two tasks A and B. Solid lines show the information flowing of tasks A. Cyan boxes denote private encoders while red boxes represent shared encoders. Blue circles denote output layers. The third column shows the different types of parameters each READ-OP consists of with respect to the task A.

GRADIENT PASSING FOR MULTI-TASK COMMUNICATION With above problems in mind, we propose to introduce a new inductive bias for inter-task communication, in which different tasks should tell others what modifications they have made to shared parameters. Additionally, their communication should support a new protocol that modifications induced by different tasks should be as consistent as possible.

Figure 3: Gradient Passing for Multi-task Communication. The superscript s and p represent parameters which can be shared or not (private).

Figure 4: Performances of SR (Star-network), ASR (Adversarial Star-network) and LGP-SR (Starnetwork with list-wise gradient passing) models with different numbers (from 100 to 1000) of training samples on five tasks. The darker grey horizontal line shows the performances in single-task setting (with 1600 training samples).

PGP-SR (Pair-wise Gradient Passing)

i denotes the intersection of five sub-vocabularies and 5 i V i represents its complement. complement 5 i V i , which allows us to easily analyze the characteristics of learned features in shared space between two models 1 .

EXTRA CONSTRAINTS FOR WRITE-OPSAdversarial Constraint This loss is used to encourage shared space more pure,L Adv = min {Θ swr -θ swr } max θ swr L(OUT(θ swr , ENC(Y k , {Θ swrθ swr })))

. Each sub-task aims at predicting a correct sentiment label (positive or negative) for a given sentence. All the datasets in each task are partitioned into a training set, development set, and test set with the proportions of 1400, 200, and 400 samples respectively. Sequence Labelling We choose POS, Chunking, and NER as evaluation tasks on Penn Treebank, CoNLL 2000, and CoNLL 2003 respectively.

Table content: <table/>

Task Setting This setting is designed to test the transferability of shared knowledge for different models. As shown in Tab.2, on Multi-Sent dataset, our model PGP-SR and LGP-SR can achieve comparable results with single task model, which is trained on 1600 samples, while we just use 200 samples. Additionally, on both Multi-AVA and Multi-Sent datasets, READ-OPs

Table content: <table><tr><td>0.85</td><td/><td>0.84 0.86</td><td/><td>0.78 0.8</td><td/><td>0.82</td><td/><td>0.82 0.84 0.86</td><td/></tr><tr><td>0.75 0.8</td><td>SR ASR LGP-SR</td><td>0.76 0.78 0.8 0.82</td><td>SR ASR LGP-SR</td><td>0.72 0.74 0.76</td><td>SR ASR LGP-SR</td><td>0.78 0.8</td><td>SR ASR LGP-SR</td><td>0.8 0.74 0.76 0.78</td><td>SR ASR LGP-SR</td></tr><tr><td/><td>100 200 300 400 500 600 700 800 9001000</td><td/><td>100 200 300 400 500 600 700 800 9001000</td><td colspan="2">100 200 300 400 500 600 700 800 9001000</td><td colspan="2">100 200 300 400 500 600 700 800 9001000</td><td colspan="2">100 200 300 400 500 600 700 800 9001000</td></tr><tr><td/><td>(a) Camera</td><td/><td>(b) Health</td><td>(c) Music</td><td/><td>(d) Toys</td><td/><td>(e) Video</td><td/></tr></table>

, gift, doesnt, fantastic, wonderfully, awesome, pleased, perfectly,alright   LGP-SR disappointed, doesnt, reliable, sucks, excellent, pleased, regret, fantastic, perfectly, amazing5 i ViSRBrosnan, boils, mixes, Lasko, sauce, straightened, breathable, distinctly, havent, sweatpants LGP-SR richness, dissatisfied, havent, roaster, overlong, doesnt, meaty, easiest, orginally, buzzes

Table content: <table><tr><td colspan="2">Settings</td><td>Features</td></tr><tr><td>5 i Vi</td><td>SR</td><td>definitely, excellent</td></tr></table>

Top-10 word features in share space with high values of q on two parts of vocabularies. The shared spaces are constructed by SR and LGP-SR models respectively.

Table content: <table/>

Peer review (iclr2019):
 This paper proposes that models for different tasks in multi-task learning cannot only share hidden variables but also gradients. Pros: - The overall framework is theoretically motivated and intuitive. The idea of passing gradients for multi-task learning is interesting and the execution using fast weights is plausible. - The experiments are extensive and cover three different task combinations in different domains. - The results are convincing and the additional analyses are compelling. Cons: - I would have liked to see a toy example or at least a bit more justification for the "pretend-to-share" problem that models "collect all the features together into a common space, instead of learning shared rules across different tasks". As it is, evidence for this seems to be mostly anecdotal, even though this forms the central thesis of the paper. - I found the use of Read and Write ops confusing, as similar terminology is widely used in memory-based networks (e.g. [1]). I would have preferred something that makes it clearer that updates are constrained in some way as "writing" implies that the location is constrained, rather than the update minimizing a loss. Questions: - How is the weight list of task similarities .beta learned when the tasks don*t share the same output space? How useful is the .beta? - Could you elaborate on what is the difference between pair-wise gradient passing (PGP) and list-wise gradient passing (LGP) [1] Graves, A., Wayne, G., & Danihelka, I. (2014). Neural turing machines. arXiv preprint arXiv:1410.5401.