C ONDITIONALLY A DAPTIVE M ULTI -T ASK L EARNING : I MPROVING T RANSFER L EARNING IN NLP U SING F EWER P ARAMETERS & L ESS D ATA
Anonymous authors
Paper under double-blind review
A BSTRACT
Multi-Task Learning (MTL) networks have emerged as a promising method for 1 transferring learned knowledge across different tasks.
However, MTL must deal 2 with challenges such as: overﬁtting to low resource tasks, catastrophic forgetting, 3 and negative task transfer, or learning interference.
Often, in Natural Language 4 Processing (NLP), a separate model per task is needed to obtain the best perfor5 mance.
However, many ﬁne-tuning approaches are both parameter inefﬁcient, i.e., 6 potentially involving one new model per task, and highly susceptible to losing 7 knowledge acquired during pretraining.
We propose a novel Transformer based 8 Adapter consisting of a new conditional attention mechanism as well as a set of 9 task-conditioned modules that facilitate weight sharing.
Through this construction, 10 we achieve more efﬁcient parameter sharing and mitigate forgetting by keeping 11 half of the weights of a pretrained model ﬁxed.
We also use a new multi-task data 12 sampling strategy to mitigate the negative effects of data imbalance across tasks.
13 Using this approach, we are able to surpass single task ﬁne-tuning methods while 14 being parameter and data efﬁcient (using around 66% of the data).
Compared to 15 other BERT Large methods on GLUE, our 8-task model surpasses other Adapter 16 methods by 2.8% and our 24-task model outperforms by 0.7-1.0% models that 17 use MTL and single task ﬁne-tuning.
We show that a larger variant of our single 18 multi-task model approach performs competitively across 26 NLP tasks and yields 19 state-of-the-art results on a number of test and development sets.
Our code is 20 publicly available 1 .
I NTRODUCTION
The introduction of deep, contextualized Masked Language Models (MLM) 2 trained on massive 23 amounts of unlabeled data has led to signiﬁcant advances across many different Natural Language 24 Processing (NLP) tasks (Peters et al., 2018;
Liu et al., 2019a).
Much of these recent advances can be 25 attributed to the now well-known BERT approach (Devlin et al., 2018).
Substantial improvements over 26 previous state-of-the-art results on the GLUE benchmark 3 (Wang et al., 2018) have been obtained by 27 multiple groups using BERT models with task speciﬁc ﬁne-tuning.
The “BERT-variant + ﬁne-tuning” 28 formula has continued to improve over time with newer work constantly pushing the state-of-the-art 29 forward on the GLUE benchmark.
The use of a single neural architecture for multiple NLP tasks has 30 shown promise long before the current wave of BERT inspired methods (Collobert & Weston, 2008) 31 and recent work has argued that autoregressive language models (ARLMs) trained on large-scale 32 datasets – such as the GPT family of models (Radford et al., 2018), are in practice multi-task learners 33 (Brown et al., 2020).
However, even with MLMs and ARLMs trained for multi-tasking, single task 34 ﬁne-tuning is usually also employed to achieve state-of-the-art performance on speciﬁc tasks of 35 interest.
Typically this ﬁne-tuning process may entail: creating a task-speciﬁc ﬁne-tuned model 36 (Devlin et al., 2018), training specialized model components for task-speciﬁc predictions (Houlsby 37 et al., 2019) or ﬁne-tuning a single multi-task architecture (Liu et al., 2019b). paper
Appendix.
parameters may have other issues.
Recent analyses of such MLM have shed light on the linguisticknowledge that is captured in the hidden states andattention maps (Clark et al., 2019b; Tenney et al.
,2019a; Merchant et al., 2020).
Particularly, BERThas middle Transformer (Vaswani et al., 2017) layers that are typically the most transferable to adownstream task (Liu et al., 2019a).
The modelproxies the steps of the traditional NLP pipeline ina localizable way (Tenney et al., 2019a) — withbasic syntactic information appearing earlier in thenetwork, while high-level semantic information appearing in higher-level layers.
Since pretraining isusually done on large-scale datasets, it may be useful, for a variety of downstream tasks, to conservethat knowledge.
However, single task ﬁne-tuningcauses catastrophic forgetting of the knowledgelearned during MLM (Howard & Ruder, 2018).
To preserve knowledge, freezing part of a pretrainednetwork and using Adapters for new tasks have shown promising results (Houlsby et al., 2019).
Inspired by the human ability to transfer learned knowledge from one task to another new task,
Multi-Task Learning (MTL) in a general sense (Caruana, 1997; Rajpurkar et al., 2016b; Ruder, 2017)has been applied in many ﬁelds outside of NLP.
Caruana (1993) showed that a model trained in amulti-task manner can take advantage of the inductive transfer between tasks, achieving a bettergeneralization performance.
MTL has the advantage of computational/storage efﬁciency (Zhang &
Yang, 2017), but training models in a multi-task setting is a balancing act; particularly with datasetsthat have different: (a) dataset sizes, (b) task difﬁculty levels, and (c) different types of loss functions.
In practice, learning multiple tasks at once is challenging since negative transfer (Wang et al.
, 2019a),task interference (Wu et al., 2020; Yu et al., 2020) and catastrophic forgetting (Serrà et al., 2018) canlead to worse data efﬁciency, training stability and test performance across tasks compared to singletask ﬁne-tuning.
One of our objectives here is to understand if it is possible to outperform individually ﬁne-tuned
BERT-based models using only MTL.
Towards that end, we seek to improve pretraining knowledgeretention and multi-task inductive knowledge transfer.
Our contributions consist of the following:
A new Transformer Attention Module using block-diagonal Conditional Attention (section 2.1) that allows the original query-key based attention to account for task-speciﬁc biases.2.
A new set of modules that adapt a pretrained MLM Transformer to new tasks, facilitate weightsharing in MTL, using:• A Conditional Alignment method that aligns the data of diverse tasks and that performs betterthan its unconditioned and higher capacity predecessor (section 2.2).•
A Conditional Layer Normalization module that adapts layer normalization statistics tospeciﬁc tasks (section 2.3) .•
A Conditional Adapter that facilitates weight sharing and task-speciﬁc information ﬂow fromlower layers (Section 2.4).3.
A novel way to prioritize tasks with an uncertainty based multi-task data sampling method thathelps balance the sampling of tasks during MTL to avoid catastrophic forgetting (see Section 2.5).
Architectural elements, mentioned in points 1 and 2 above, comprise a single adapter that is describedby module in the methodology section.
Our Conditional Adaptive Multi-Task Learning (CA-MTL)approach is illustrated in Figure 1.
To the best of our knowledge, our work is the ﬁrst to explore theuse of a latent representation of tasks to modularize and adapt pretrained architectures.
Further, webelieve our work is also the ﬁrst to examine uncertainty sampling for large-scale multi-task learningin NLP.
We show the efﬁcacy of CA-MTL by: (a) testing on 26 different tasks and (b) presentingstate-of-the-art results on a number of test sets as well as superior performance against both single-taskand MTL baselines.
Moreover, we further demonstrate that our method has advantages over (c) otheradapter networks, and (d) other MTL sampling methods.
Finally, we provide ablations and separateanalysis of the MT-Uncertainty Sampling technique in section 4.1 and of each component of theadapter in 4.2.
M ETHODOLOGY
This section is organized according to the two main MTL problems that we will tackle: (1) How to 97 modularize a pretrained network with task latent representations?
(2) How to balance different tasks in MTL?
We deﬁne each task as: T i (cid:44) { p i ( y i | x i , z i ) , L i , ˜ p
i ( x i ) } , where z i is task i ’s embedding, 99 L i is the task loss, and ˜ p
i ( x i ) is the empirical distribution of the training data pair { x i , y i } , fori ∈ { 1 , . .
, T } and T the number of supervised tasks.
The MTL objective is:
[Figure]
[Equation]
where f is the predictor function (includes encoder model and decoder heads), φ ( z ) are learnablegenerated weights conditioned on z , and θ i are task-speciﬁc parameters for the output decoderheads.
We now present ﬁve different modiﬁcations and extensions that we have made to the generic
Transformer architecture.
In our ablation study of Table 1, we outline the effects of each componentby reporting the average GLUE score for various conﬁgurations.
2. C ONDITIONAL A TTENTION
Given d , the input dimensions, the query Q , the key K , and the value V as deﬁned in Vaswani et al.
(2017), we redeﬁne the attention operation:
[Equation]
where L is the input sequence, N the number of block matrices A n ∈ R ( L/N )
× ( L/N ) along the diagonal of the 109 attention matrix, and M ( z i ) =
diag ( A (cid:48) 1 , . . .
, A (cid:48) N ) a block 110 diagonal conditional matrix.
While the original attention 111 matrix depends on the hidden states h , M ( z i ) is a learnable weight matrix that only depends on the task embedding z i ∈ R d .
γ i , β i : R d (cid:55)→ R L 2 /N 2 are Feature
Wise Linear Modulation (Perez et al., 2018) functions.
We also experimented with full-block
Conditional Attention ∈ R L × L .
Not only did it have N 2 more parameters compared to the blockdiagonal variant, but it also performed signiﬁcantly worse on the GLUE development set (see FBAvariant in table 15).
It is possible that GLUE tasks derive a certain beneﬁt from localized attention thatis a consequence of M ( z i ) .
With M ( z i ) , each element in a sequence can only attend to other elementsin its subsequence of length L/N .
In our experiments we used N = d/L .
The full Conditional Attention mechanism used in our experiments is illustrated in Figure 2.
2. C ONDITIONAL A LIGNMENT
Wu et al.
(2020) showed that in MTL having T separate alignment modules R 1 , . .
, R T increases
BERT LARGE avg. scores on ﬁve GLUE tasks (CoLA, MRPC, QNLI, RTE, SST-2) by 2.35%.
Inspired by this work, we found that adding a task conditioned alignment layer between the input embedding layer and the ﬁrst BERT Transformer layer improved multi-task model performance.
However, instead of having T separate alignment matrices R i for each T task, one alignment matrix ˆ R is generated as a function of the task embedding z i .
As in Wu et al. (2020), we tested this module on the same ﬁve GLUE tasks and with BERT LARGE . Enabling task conditioned weight sharing across covariance alignment modules allows us to outperforms BERT LARGE by 3.61%. This is 1.26 % higher than having T separate alignment matrices.
Inserting ˆ R into BERT, yields the following encoder function ˆ f :
[Equation]
where x i ∈ R
d is the layer input, g θ i is the decoder head function for task i with weights θ
i , E thefrozen BERT embedding layer, B the BERT Transformer layers and R the linear weight matrix ofa single task conditioned alignment matrix.
γ i , β i : R d (cid:55)→ R d are Feature Wise Linear Modulationfunctions.
2. C ONDITIONAL L AYER N ORMALIZATION (CLN)
We extend the Conditional Batch Normalization idea from de Vries et al.
(2017) to Layer Normalization (Ba et al., 2016).
For task T i , i ∈ { 1 , . . .
, T } :
[Equation]
where h i is the CLN output vector, a i are the preceding layer activations associated with task i , µand σ are the mean and the variance of the summed inputs within each layer as deﬁned in Ba et al.(2016).
Conditional Layer Normalization is initialized with BERT’s Layer Normalization afﬁnetransformation weights and bias γ (cid:48) and β (cid:48) from the original formulation: h
= 1 σ (cid:12) ( a − µ ) ∗ γ (cid:48)
+ β (cid:48) .
During training, the weight and bias functions of γ i ( ∗ ) and β i ( ∗ ) are always trained, while theoriginal Layer Normalization weight may be kept ﬁxed.
This module was added to account for taskspeciﬁc rescaling of individual training cases.
Layer Normalization normalizes the inputs acrossfeatures.
The conditioning introduced in equation 2.3 allows us to modulate the normalization’soutput based on a task’s latent representation.
2. C ONDITIONAL A DAPTERS
[Figure]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table]
We created a task conditioned two layer feed-forwardneural network (called a Conditional Feed Forward or
CFF in Figure 3) with a bottleneck.
The conditionalbottleneck layer follows the same transformation asin equation 2.
The adapter in Figure 3a is placed inside a Transformer layer.
The conditional bottlenecklayer is also the main building block of the skip connection seen in Figure 3b.
This Conditional Adapterallows lower layer information to ﬂow upwards depending on the task.
Our intuition for introducing thiscomponent is related to recent studies (Tenney et al.
,2019a) that showed that the “most important layersfor a given task appear at speciﬁc positions”.
As withthe other modules described so far, each task adaptation is created from the weights of a single sharedadapter that is modulated by the task embedding.
2. M ULTI -T ASK U NCERTAINTY S AMPLING
MT-Uncertainty Sampling is a task selection strategy that is inspired by Active Learning techniques.
Our algorithm 1 in the Appendix, Section A.2.
Similar to Active Learning, our algorithm ﬁrst evaluates the model
uncertain MT-Uncertainty Sampling uses Shannon Entropy, an uncertainty measure, to choose training examples by ﬁrst doing forward pass through the model with b × T input samples.
For an output classiﬁcation prediction with C i possible classes and probabilities ( p i, 1 , . . .
, p i,C i ) , the Shannon Entropy H i , for task T
i and i ∈ { 1 , . .
, T } , our uncertainty measure U ( x ) are given by: [Equation] max i
[Equation]
max [Equation] [Equation] where ¯ H i is the average Shannon Entropy across b samples of task t , H (cid:48)
i , the Shannon entropy ofchoosing classes with uniform distribution and ˆ H , the maximum of each task’s average entropy overb samples.
H (cid:48)
i is normalizing factor that accounts for differing number of prediction classes (withoutthe normalizing factor H (cid:48)
i , tasks with a binary classiﬁcation C i = 1 were rarely chosen).
Further, tolimit high entropy outliers and to favor tasks with highest uncertainty, we normalize with ˆ H .
Themeasure in eq.
4 allows Algorithm 1 to choose b samples from b × T candidates to train the model.
R ELATED W ORK
Multi-Tasking in NLP and other ﬁelds To take advantage of the potential positive transfer ofknowledge from one task to another, several works have proposed carefully choosing which tasks totrain as an intermediate step in NLP before single task ﬁne-tuning (Bingel & Søgaard, 2017; Kerinecet al., 2018;
Wang et al., 2019a; Standley et al., 2019; Pruksachatkun et al., 2020; Phang et al., 2018).
The intermediate tasks are not required to perform well and are not typically evaluated jointly.
In thiswork, all tasks are trained jointly and all tasks used are evaluated from a single model .
In Natural Language Understanding (NLU), it is still the case that to get the best task performance one oftenneeds a separate model per task (Clark et al., 2019c; McCann et al., 2018). At scale, Multilingual
NMT systems (Aharoni et al., 2019) have also found that MTL model performance degrades as thenumber of tasks increases.
We notice a similar trend in NLU with our baseline MTL model. Recently,approaches in MTL have tackled the problem by designing task speciﬁc decoders on top of a sharedmodel (Liu et al., 2019b) or distilling multiple single-task models into one (Clark et al., 2019c).
Nonetheless, such MTL approaches still involves single task ﬁne-tuning.
In this paper, we showthat it is possible to achieve high performance in NLU without single task ﬁne-tuning.
MTL weightsharing algorithms such as Mixture-of-Experts (MoE) have found success in NLP (Lepikhin et al.,2020). CA-MTL can complement MoE since the Transformers multi-headed attention can be seen asa form of MoE (Peng et al., 2020). In Vision, MTL can also improve with optimization (Sener &
Koltun, 2018) or gradient-based approaches (Chen et al., 2017; Yu et al., 2020). Adapters.
With single task ﬁne-tuning, we have one model per task.
For T tasks, we would need T 179 models, multiplying system memory requirements by T . Adapter networks provide another promisingavenue to limit the number of parameters needed when confronted with a large number of tasks.
Adapters are trainable modules that are attached in speciﬁc locations of a pretrained network.
Thisapproach is useful with pretrained MLM models that have rich linguistic information (Tenney et al.
,2019b; Clark et al., 2019b; Liu et al., 2019a; Tenney et al., 2019a).
Recently, both Houlsby et al.(2019) added an adapter to a pretrained BERT model by ﬁne-tuning the layer norms and adding feedforward bottlenecks in every Transformer layer.
However, such methods adapt each task individuallyduring the ﬁne-tuning process.
Unlike prior work, our method harnesses the vectorized representationsof tasks to modularize a single pretrained model across all tasks. Stickland et al.
(2019) and Tay et al.(2020) also mix both MTL and adapters with BERT and T5 encoder-decoder (Raffel et al., 2019b)respectively by creating local task modules that are controlled by a global task agnostic module. The main drawback is that a new set of non-shared parameters must be added when a new task isintroduced. CA-MTL shares all parameters and is able to re-modulate existing weights with a newtask embedding vector.
Active Learning, Task Selection and
Sampling Our sampling technique is similar to the onesfound in several active learning algorithms (Chen et al., 2006) that are based on Shannon entropyestimations.
Reichart et al.
(2008) and Ikhwantri et al.
(2018) examined Multi-Task Active Learning(MTAL) using a two task annotation scenario and showed performance gains while needing lesslabeled data.
Our approach is a substantially different variant of MTAL since it was developed fortask selection.
Instead of choosing one informative sample for T different learners (or models) foreach T tasks, we choose T tasks samples for one model to learn all tasks.
Our algorithm differs inthree ways: a) we use uncertainty sampling to maximize large scale MTL ( (cid:29) 2 tasks) performancevia the modularization of a shared neural architecture; b) the algorithm weights each sample by thecorresponding task score; c) the Shannon entropy is normalized to account for various losses (seeequation 5).
Recently, Glover & Hokamp (2019) explored task selection in MTL using learningpolicies based on counterfactual estimations (Charles et al., 2013).
However, such method considersonly ﬁxed stochastic parameterized policies while our method adapts its selection criterion based onmodel uncertainty throughout the training process.
Other than MTAL, Kendall et al. (2017) leveragedmodel uncertainty to balance MTL losses but not to select tasks as is proposed here.
E XPERIMENTS AND R ESULTS
We show that our adapter of section 2 achieve parameter efﬁcient transfer for 26 NLP tasks. We haveorganized our experiments and discussion of results in the following way for each section:
we study our MT Uncertainty Sampling vs other task sampling methods on a baseline model(without adapters).
We also show how MT Uncertainty helps avoid catastrophic forgetting. • 4.2- we analyze covariate shift and study ablations of CA-MTL modules. We observe higher average scores and lower score variance, revealing that CA-MTL helps mitigate negative task transfers.
Input embeddings after Conditional Alignment exhibit improved task covariance similarity. we test CA-MTL on 8 tasks, and observe improved performance compared to other Adapters. • 4.4- we provide a simple method to “reconﬁgure” CA-MTL’s weights on a new task using task embeddings which facilitates more efﬁcient knowledge transfer.
Speciﬁcally, CA-MTL delivers state-of-the-art results for the SciTail and SNLI evaluations in the low data regime.• 4.5-
we investigate large scale MTL on 24 tasks. CA-MTL exhibits higher performance withincreased task count, demonstrating its ability to better balance model capacity.
We compare with strong, BERT/RoBERTa-based, techniques that use both MTL and single task ﬁne-tuning in
Table 5. We ﬁnd our approach again yields state-of-the-art results (see tables 6a, 6b, and 6c).Our implementation of
CA
-MTL is based on HuggingFace (
Wolf et al
Hyperparameters
and our experimental set up are outlined in A. To preserve the weights of the pretrained model,
CA
-MTL’s bottom half Transformer layers are frozen in all experiments (except in section 4.4). Wealso tested different layer freezing conﬁgurations
and found that freezing half the layers worked best on average (see
Section A M ULTI -T ASK U NCERTAINTY S AMPLING
[Figure]
Our MT-Uncertainty sampling strategy, from section 2.5, is compared to 3 other task selection schemes: a)
Counterfactual b) Task size c) Random.
We used a BERT BASE (no adapters) on 200k iterations and with the same hyperparameters as in Glover & Hokamp (2019).
For more information on Counterfactual task selection, we invite the reader to consult the full explanation in Glover & Hokamp (2019).
For T tasks and the dataset D i for tasks i ∈ { 1 , . . .
, T } , we rewrite the deﬁnitions of Random π rand and Task size π | task | sampling:
[Equation]
[Figure] In Figure 4, we see from the results that MTUncertainty converges faster by reaching the 80%average GLUE score line before other task samplingmethods.
Further, MT-Uncertainty maximum scoreon 200k iterations is at 82.2, which is 1.7% higherthan Counterfactual sampling. The datasets in the
GLUE benchmark offers a wide range of dataset sizes. This is useful to test how MT-Uncertainty managesa jointly trained low resource task (CoLA) and highresource task (MNLI). Figure 5 explains how catastrophic forgetting is curtailed by sampling tasks before performance drops.
With π rand , all of CoLA’stasks are sampled by iteration 500, at which point the larger MNLI dataset overtakes the learningprocess and CoLA’s dev set performance starts to diminish. On the other hand, with MT-Uncertaintysampling, CoLA is sampled whenever Shannon entropy is higher than MNLI’s. The model ﬁrstassesses uncertain samples using Shannon Entropy then decides what data is necessary to train on. This process allows lower resource tasks to keep performance steady.
We provide evidence in Figure 8 of A.2 that MT-Uncertainty is able to manage task difﬁculty — by choosing the most difﬁcult tasksﬁrst.
4. A BLATION AND M ODULE A NALYSIS
[Table]
In Table 1, we present the results of an ablation study
[Table]
to {+determine which elements of CA-MTL [Table] had the largest positive gain on average GLUE scores. 253 Starting from a MTL BERT BASE baseline trained us
+}
ing random task sampling ( π rand ). Apart for the 255 Conditional Adapter, each module as well as MT- [Table] Uncertainty lift overall performance and reduce vari-
[Table]
ance across tasks.
Please note that we also included 258 accuracy/F1 scores for QQP, MRPC and Pearson/ Spearman correlation for STS-B to calculate scorestandard deviation σ .
Intuitively, when negative task transfer occurs between two tasks, either (1) taskinterference is bidirectional and scores are both impacted, or (2) interference is unidirectional and onlyone score is impacted.
We calculate σ to get a complete picture of how task performance moves acrossthe board.
As we can see from Table 1, Conditional Attention, Conditional Alignment, Conditional
Layer Normalization, MT-Uncertainty play roles in reducing σ and increasing performance acrosstasks.
This provides partial evidence of CA-MTL’s ability to mitigating negative task transfer.
We show that Conditional Alignment can learn to capture covariate distribution differences with task embeddings colearned from other adapter components of CA-MTL.
In Figure 6, we arrive at similar conclusions as Wu et al.
(2020), who proved that negative task transfer is reduced when task covariances are aligned.
The authors provided a “covariance similarity score” to gauge covariance alignment.
For task i and j with m i and m j data samples respectively, and given d dimensional inputs to the ﬁrst Transformer layer X i ∈
R
m i × d and X j ∈ R m j × d , we rewrite the steps to calculate the covariance similarity score between task i and j : (a) Take the covariance matrix X (cid:62)
i X i , (b) Find its best rank- r i approximation
U
i,r i D i,r i U (cid:62) i,r i , where r i is chosen to contain 99% of the singular values.
(c) Apply steps (a), (b) to X j , and compute the covariance similarity score CovSim i,j :
[Equation]
Since we are training models with T tasks, we take the average covariance similarity score CovSim i 266 between task i and all other tasks.
We measure CovSim i using equation 7 between 9 single-taskmodels trained on individual GLUE tasks.
For each task in Figure 6, we measure the similarityscore on the MTL trained BERT BASE baseline, e.g., CoLA (MTL), or CA-MTL BERT-BASE model, e.g.,
MNLI (CA-MTL).
Our score improvement measure is the % difference between a single task modeland MTL or CA-MTL on the particular task.
We ﬁnd that covariance similarity increases for 9 tasksand that performance increases for 7 out 9 tasks.
These measurements conﬁrm that the Conditional
Alignment is able to align task covariance, thereby helping alleviate task interference.
4. J OINTLY TRAINING ON 8 TASKS : GLUE
In Table 2, we evaluate the performance of CA-MTL against single task ﬁne-tuned models, MTL aswell as the other BERT-based adapters on GLUE.
As in Houlsby et al. (2019), MNLI m and MNLI mm 276 are treated as separate tasks. Our results indicate that CA-MTL outperforms both the BASE adapter,
PALS+Anneal
Sampling (Stickland et al., 2019), and the LARGE adapter, Adapters-256 (Houlsbyet al., 2019).
Against single task (ST) models, CA-MTL is 1.3% higher than BERT BASE , with 5 out 9tasks equal or greater performance, and 0.7% higher than BERT LARGE , with 3 out 9 tasks equal orgreater performance.
ST models, however, need 9 models or close to 9 × more parameters for all 9tasks.
We noted that CA-MTL BERT-LARGE ’s average score is driven by strong RTE scores.
While RTE beneﬁts from MTL, this behavior may also be a side effect of layer freezing.
In Table 15, we see that
CA-MTL has gains over ST on more and more tasks as we gradually unfreeze layers
[Table]
[Table]
[Table]
[Table]
In Table 3 we examine the ability of ourmethod to quickly adapt to new tasks.
Weperformed domain adaptation on SciTail (Khotet al., 2018) and SNLI (Bowman et al., 2015)datasets using a CA-MTL BASE model trained on GLUE and a new linear decoder head.
We testedseveral pretrained and randomly initialized task embeddings in a zero-shot setting. The completeset of experiments with all task embeddings can be found in the Appendix, Section A.4.
We thenselected the best task embedding for our results in Table 3.
STS-B and MRPC MTL-trained taskembeddings performed best on SciTail and SNLI respectively.
CA-MTL BERT-BASE has faster adaptation than MT-DNN SMART (Jiang et al., 2020) as evidenced by higher performances in low-resourceregimes (0.1% and 1% of the data).
When trained on the complete dataset, CA-MTL BERT-BASE is onpar with MT-DNN SMART .
Unlike MT-DNN SMART however, we do not add context from a semanticsimilarity model – MT-DNN SMART is built off HNN (He et al., 2019).
Nonetheless, with a largermodel, CA-MTL surpasses MT-DNN SMART on the full SNLI and SciTail datasets in Table 6.
4. J OINTLY TRAINING ON 24 TASKS : GLUE/S UPER -GLUE, MRQA AND WNUT
Effects of Scaling Task Count.
In Figure 7 we continue to test if CA-MTL mitigates task interferenceby measuring GLUE average scores when progressively adding 9 GLUE tasks, 8 Super-GLUE tasks(Wang et al., 2019b), 6 MRQA tasks (Fisch et al.,2019).
Tasks are described in Appendix section A.3.
The results show that adding 23 tasks drops the performance of our baseline MTL BERT BASE ( π rand ).
MTL BERT increases by 4.3% when adding MRQAbut, with 23 tasks, the model performance drops by1.8%.
The opposite is true when CA-MTL modulesare integrated into the model.
CA-MTL continues toshow gains with a large number of tasks and surpasses the baseline MTL model by close to 4% whentrained on 23 tasks.24-task CA-MTL.
We jointly trained large MTL baselines and CA-MTL models on GLUE/SuperGLUE/MRQA and Named Entity Recognition (NER)
WNUT2017 (Derczynski et al., 2017).
Sincesome dev.
set scores are not provided and since RoBERTa results were reported with a median scoreover 5 random seeds, we run our own single seed ST/MTL baselines (marked “ReImp”) for a faircomparison.
The dev. set numbers reported in Liu et al. (2019c) are displayed with our baselines in Table 13.
Results are presented in Table 4. We notice in Table 4 that even for large models, CA-MTLprovides large gains in performance on average over both ST and MTL models. For the BERT basedmodels, CA-MTL provides 2.3% gain over ST and higher scores on 17 out 24 tasks. For RoBERTa based models, CA-MTL provides 1.2% gain over ST and higher scores on 15 out 24 tasks.
We remindthe reader that this is achieved with a single model.
Even when trained with 16 other tasks, it isinteresting to note that the MTL baseline perform better than the ST baseline on Super GLUE wheremost tasks have a small number of samples.
Also, we used NER to test if we could still outperformthe ST baseline on a token-level task, signiﬁcantly different from other tasks.
Unfortunately, while CA-MTL performs signiﬁcantly better than the MTL baseline model, CA-MTL had not yet overﬁt onthis particular task and could have closed the gap with the ST baselines with more training cycles. [Table] [Table]
Comparisons with other methods.
In Table
[Table]
5, CA-MTL BERT is compared to other Large
[Table]
BERT based methods that either use MTL +
[Table]
ST, such as MT-DNN (Liu et al., 2019b), in-
[Table]
termediate tasks + ST, such as STILTS (Phang 334 et al., 2018) or MTL model distillation + ST, 335 such as BAM!
(Clark et al., 2019c).
Our 336 method scores higher than MT-DNN on 5 of
[Table]
9 tasks and by 1.0 % on avg.
Against STITLS, 338 CA-MTL realizes a 0.7 % avg.
score gain, sur339 passing scores on 6 of 9 tasks.
We also show 340 that CA-MTL RoBERTa is within only 1.6 % of a 341 RoBERTa ensemble of 5 to 7 models per task and that uses intermediate tasks.
[Table]
[Table] [Table] [Table] Using our 24-task CA-MTL large
[Table]
RoBERTa-based model, we report 344 NER F1 scores on the WNUT 345 test set in Table 6a.
We com346 pare our result with RoBERTa LARGE 347 and XLM-R LARGE (Nguyen et al., 348 2020) the current state-of-the-art 349 (SOTA).
Our model outperforms 350 XLM-R LARGE by 1.6%, reaching a 351 new state-of-the-art.
Using domain 352 adaptation as described in Section 353 4.4, we report results on the SciTail 354 test set in Table 6b and SNLI test set 355 in Table 6b.
For SciTail, our model matches the current SOTA 4 ALUM (Liu et al., 2020), a RoBERTalarge based model that additionally uses the SMART (Jiang et al., 2020) ﬁne-tuning method.
For
SNLI, our model outperforms SemBert, the current SOTA 5 .
C ONCLUSION
[Table] [Table] Multi-task Learning (MTL) is promising for two main reasons. First, 360 we can harness knowledge learned from other tasks to improve per361 formance.
Second, only one model is needed to solve multiple tasks, 362 reducing the disk space requirements for downstream devices. In a 363 large-scale 24-task NLP experiment, CA-MTL outperforms fully tuned 364 single task models by 2.3% for BERT Large and by 1.2% for RoBERTa 365 Large. When a BERT vanilla MTL model sees its performance drop as we increase the number of tasks, CA-MTL scores continue to climb Each CA-MTL module that adapts a Transformer model is able to reduce performance variances between tasks, increase average scores and align covariances between tasks.
This evidence shows that CA- MTL is able to mitigate task interference and promote more efﬁcient parameter sharing. We showed that MT-Uncertainty is able to avoid degrading performances of low resource tasks.
Tasks are sampled whenever the model sees entropy increase, helping avoid catastrophic forgetting.
We think that improving the efﬁciency of our proposed
MT-Uncertainty algorithm is a good objective for future work
A A PPENDIX
A. S UMMARY OF A CRONYMS
Acronyms of datasets and descriptions can be found below in section A.3.
[Table]
A. U NCERTAINTY S AMPLING : A LGORITHM AND A DDITIONAL R ESULTS
Input: Training data D t for task t ∈ [1 , . .
, T ] ; batch size b ; C t possible output classes for task t ; f := f φ ( z i ) ,θ i our model with weights φ, θ i ;
Output: B (cid:48) - multi-task batch of size b 1 B ← ∅ 2 for t ← 1 to T do
Generate x t := { x t, 1 , . . .
, x t,b } i.i.d.
∼ D t [Table] for i ← 1 to b do
H t,i ← −
C i c
=1 p c f t i )) log p c
end
Compute ¯ H t ← 1 b (cid:80) x ∈ x i H t i
Compute H (cid:48)
t ← − (cid:80) C t c =1 1 C t log (cid:20)
C t (cid:21) (cid:46)
B ← B ∪ x t and D t ← D t \ x t if D t = ∅ then
Reload D t end for i ← 1 to b do
Compute: U t,i ← H t,i /H (cid:48)
t
end 19 end 20 Compute ˆ H ← max i ∈{ 1 ,...,T } [ ¯ H t ] (cid:46) of task with highest average entropy 21 Update U t i ← U t i / ˆ H (cid:46) Normalize each sample’s uncertainty measure 22 B (cid:48)
← top uncertainty
An advantage of our MT-Uncertainty Sampling approach is its ability to manage task difﬁculty.
Thisis highlighted in Figure 8.
In this experiment, we estimated task difﬁculty using the Evolutionary
Data Measures (EDM) 6 proposed by Collins et al.
The task difﬁculty estimate relies onmultiple dataset statistics such as the data size, class diversity, class balance and class interference.
Interestingly, estimated task difﬁculty correlates with the ﬁrst instance that the selection of a speciﬁctask occurs.
Supposing that QNLI is an outlier, we notice that peaks in the data occur whenever tasksare ﬁrst selected by MT Uncertainty sampling.
This process follows the following order: 1.
MNLI 2.
CoLA 3.
RTE 4.
QQP 5.
MRPC 6.SST-2, which is the order from highest task difﬁculty to lowesttask difﬁculty using EDM.
As opposed to Curriculum Learning (Bengio et al., 2009), MT-Uncertaintydynamically prioritizes the most difﬁcult tasks.
As also discovered in MTL vision work (Guo et al.,2018), this type of prioritization on more difﬁcult tasks may explain MT-Uncertainty’s improvedperformance over other task selection methods. In MTL, heuristics to balance tasks during training istypically done by weighting each task’s loss differently. We see here how MT-Uncertainty is able toprioritize task difﬁculty.
While the EDM difﬁculty measure is shown to correlate well with model performance, it lacksprecision.
As reported in Collins et al.
, the average score achieved on the Yahoo Answersdataset is 69.9% and its difﬁculty is 4.51.
The average score achieved on Yelp Full is 56.8%, 13.1%less than Yahoo Answers and its difﬁculty is 4.42.
The authors mention that “This indicates that thedifﬁculty measure in its current incarnation may be more effective at assigning a class of difﬁculty todatasets, rather than a regression-like value”.
[Figure]
N u m b e r
o f s a m p l e s
Task Difﬁculty
MNLI 4. QNLI 3. CoLA 3. RTE 3.
MRPC 3.
QQP.
SST-2 3.
A. D ATASET D ESCRIPTION
The datasets that were used for the domain adaptation experiments were SciTail 7 and SNLI 8 .
Wejointly trained a CA-MTL RoBERTa-LARGE model on 9 GLUE tasks, 8 Super-GLUE 9 tasks, 6 MRQA 10tasks, and on WNUT2017 11 (Derczynski et al., 2017).
All GLUE tasks are binary classiﬁcation, except STS-B (regression) and MNLI (three classes).
Weused the same GLUE data preprocessing as in Devlin et al.
[Table]
et al
[Table] [Table]
et al.
[Table] [Table]
Dolan &
[Table] [Table]
Cer et al
[Table] [Table]
et al
[Table] [Table]
Wang et al
[Table] [Table]
[Table] [Table]
SuperGLUE has a more diverse task format than GLUE, which is mostly limited to sentence andsentence-pair classiﬁcation.
We follow the same preprocessing procedure as in Wang et al.
(2019b)
[Table]
All tasks are binary classiﬁcation tasks, except CB (three classes).
Also, WiC and WSC are spanbased classiﬁcation tasks.
We used the same modiﬁed MRQA dataset and preprocessing steps thatwere used in Joshi et al.
All MRQA tasks are span prediction tasks which seeks to identifystart and end tokens of an answer span in the input text.
[Table]
SNLI is a natural inference task where we predict three classes.
Examples of three target labelsare: Entailment, Contradiction, and Neutral (irrelevant).
SciTail is a textual entailment dataset.
Thehypotheses in SciTail are created from multiple-choice science exams and the answer candidates(premise) are extracted from the web using information retrieval tools.
SciTail is a binary true/falseclassiﬁcation tasks that seeks to predict whether the premise entails the hypothesis.
The two datasetsare used only for domain adaptation in this study (see section A.4 for the details of our approach).
[Table]
Before testing models on domain adaptation in section 4.
[Table]
we ran zero-shot evaluations on thedevelopment set of SciTail and SNLI.
Table 12 outlines 8-task CA-MTL BERT-BASE ’s zero-shot transferabilities when pretrained on GLUE with our MTL approach.
We expand the task embedding layerto accommodate an extra task and explore various embedding initialization.
We found that reusing
STS-B and MRPC task embeddings worked best for SciTail and SNLI respectively.
A. M ORE E XPERIMENTAL D ETAILS
We used a batch size of 32 and a seed of 12 in all experiments. We used Adam (Kingma & Ba, 2015)as the optimizer with a learning rate of 2e-5. We applied a learning rate decay with warm up overthe ﬁrst 10% of the training steps.
Unless otherwise speciﬁed, we used 5 epochs, a seed of 12 anda sequence length of 128. Additional details are outlined in section . Our data prepossessing andlinear decoder heads are the same as in Devlin et al. We used the same dropout rate of 0.1in all layers.
To run our experiments, we used either four NVIDIA P100 GPU for base models orfour NVIDIA V100 GPU for larger ones. We {+did not perform parameter search. We do not useensemble of models or task-speciﬁc tricks (Devlin et al., 
+}
; Liu et al., 2019b; Clark et al., 2019c). All models are either 12 Transformer layers for BASE and 24 Transformer layers for LARGE. Apartfrom CA-MTL, models trained in multi-task learning (BERT or RoBERTa without adapters) usedrandom task sampling. For Table 1 and Figure 7, all BERT-based model have half their layers frozen(untrained) for a fair comparison of ablation results.
For the 24-task MTL and CA-MTL models in Tables 4 and 5, we increased the input sequence length to 256 and used 8 epochs.
In this section, we present our baseline results for BERT, RoBERTa, CA-MTL as well as other models.
Our single task results (ST) that we ran ourselves surpass other paper’s reported scores in table 13. Liu et al. (2019c) reports random seed median scores for RoBERTa. However, our RoBERTa STbaseline matches or surpasses the original paper’s scores 4 out 7 times on the development setwhen scores are comparable (QQP F1 and STS-B spearman are not reported). A. S OME R ESULTS ON LAYER F REEZING AND WITH F ULL B LOCK A TTENTION .
All experiments in this section were run for only 5 epochs, exclusively on the GLUE dataset for thelarge BERT-based 8-task CA-MTL model. Results in Figure 15 reveal that...
[Table]
[Table] [Table]