C ONDITIONALLY A DAPTIVE M ULTI -T ASK L EARNING : I MPROVING T RANSFER L EARNING IN NLP U SING F EWER P ARAMETERS & L ESS D ATA
A BSTRACT
Multi-Task Learning (MTL) networks have emerged as a promising method for transferring learned knowledge across different tasks.
However, MTL must deal with challenges such as: overﬁtting to low resource tasks, catastrophic forgetting, and negative task transfer, or learning interference.
Often, in Natural Language Processing (NLP), a separate model per task is needed to obtain the best performance.
However, many ﬁne-tuning approaches are both parameter inefﬁcient, i.e., potentially involving one new model per task, and highly susceptible to losing knowledge acquired during pretraining.
We propose a novel Transformer based Adapter consisting of a new conditional attention mechanism as well as a set of task-conditioned modules that facilitate weight sharing.
Through this construction, we achieve more efﬁcient parameter sharing and mitigate forgetting by keeping half of the weights of a pretrained model ﬁxed.
We also use a new multi-task data sampling strategy to mitigate the negative effects of data imbalance across tasks.
Using this approach, we are able to surpass single task ﬁne-tuning methods while being parameter and data efﬁcient (using around 66% of the data for weight updates).
Compared to other BERT Large methods on GLUE, our 8-task model surpasses other Adapter methods by 2.8% and our 24-task model outperforms by 0.7-1.0% models that use MTL and single task ﬁne-tuning.
We show that a larger variant of our single multi-task model approach performs competitively across 26 NLP tasks and yields state-of-the-art results on a number of test and development sets.
Our code is publicly available at https://github.com/CAMTL/CA-MTL .
I NTRODUCTION
The introduction of deep, contextualized Masked Language Models (MLM) 1 trained on massive amounts of unlabeled data has led to signiﬁcant advances across many different Natural Language Processing (NLP) tasks (Peters et al., 2018;
Liu et al., 2019a).
Much of these recent advances can be attributed to the now well-known BERT approach (Devlin et al., 2018).
Substantial improvements over previous state-of-the-art results on the GLUE benchmark (Wang et al., 2018) have been obtained by multiple groups using BERT models with task speciﬁc ﬁne-tuning.
The “BERT-variant + ﬁne-tuning” formula has continued to improve over time with newer work constantly pushing the state-of-the-art forward on the GLUE benchmark.
The use of a single neural architecture for multiple NLP tasks has shown promise long before the current wave of BERT inspired methods (Collobert & Weston, 2008) and recent work has argued that autoregressive language models (ARLMs) trained on large-scale datasets – such as the GPT family of models (Radford et al., 2018), are in practice multi-task learners (Brown et al., 2020).
However, even with MLMs and ARLMs trained for multi-tasking, single task ﬁne-tuning is usually also employed to achieve state-of-the-art performance on speciﬁc tasks of interest.
Typically this ﬁne-tuning process may entail: creating a task-speciﬁc ﬁne-tuned model (Devlin et al., 2018), training specialized model components for task-speciﬁc predictions (Houlsby et al., 2019) or ﬁne-tuning a single multi-task architecture (Liu et al., 2019b).
[Figure]
parameters may have other issues.
Recent analyses of such MLM have shed light on the linguistic knowledge that is captured in the hidden states and attention maps (Clark et al., 2019b; Tenney et al., 2019a;
Merchant et al., 2020).
Particularly, BERT has middle Transformer (Vaswani et al., 2017) layers that are typically the most transferable to a downstream task (Liu et al., 2019a).
The model proxies the steps of the traditional NLP pipeline in a localizable way (Tenney et al., 2019a) — with basic syntactic information appearing earlier in the network, while high-level semantic information appearing in higher-level layers.
Since pretraining is usually done on large-scale datasets, it may be useful, for a variety of downstream tasks, to conserve that knowledge.
However, single task ﬁne-tuning causes catastrophic forgetting of the knowledge learned during MLM (Howard & Ruder, 2018).
To preserve knowledge, freezing part of a pretrained network and using Adapters for new tasks have shown promising results (Houlsby et al., 2019).
Inspired by the human ability to transfer learned knowledge from one task to another new task,
Multi-Task Learning (MTL) in a general sense (Caruana, 1997; Rajpurkar et al., 2016b; Ruder, 2017) has been applied in many ﬁelds outside of NLP.
Caruana (1993) showed that a model trained in a multi-task manner can take advantage of the inductive transfer between tasks, achieving a better generalization performance.
MTL has the advantage of computational/storage efﬁciency (Zhang &
Yang, 2017), but training models in a multi-task setting is a balancing act; particularly with datasets that have different: (a) dataset sizes, (b) task difﬁculty levels, and (c) different types of loss functions.
In practice, learning multiple tasks at once is challenging since negative transfer (Wang et al., 2019a), task interference (Wu et al., 2020; Yu et al., 2020) and catastrophic forgetting (Serrà et al., 2018) can lead to worse data efﬁciency, training stability and generalization compared to single task ﬁne-tuning.
Using Conditionally Adaptive Learning, we seek to improve pretraining knowledge retention and multi-task inductive knowledge transfer.
Our contributions are the following:
• A new task conditioned Transformer that adapts and modulates pretrained weights (Section 2.1) .
• A novel way to prioritize tasks with an uncertainty based multi-task data sampling method that helps balance the sampling of tasks to avoid catastrophic forgetting (Section 2.2) .
Our Conditionally Adaptive Multi-Task Learning (CA-MTL) approach is illustrated in Figure 1.
To the best of our knowledge, our work is the ﬁrst to explore the use of a latent representation of tasks to modularize and adapt pretrained architectures.
Further, we believe our work is also the ﬁrst to examine uncertainty sampling for large-scale multi-task learning in NLP.
We show the efﬁcacy of CA-MTL by: (a) testing on 26 different tasks and (b) presenting state-of-the-art results on a number of test sets as well as superior performance against both single-task and MTL baselines.
Moreover, we further demonstrate that our method has advantages over (c) other adapter networks, and (d) other MTL sampling methods.
Finally, we provide ablations and separate analysis of the MT-Uncertainty Sampling technique in section 4.1 and of each component of the adapter in 4.2.
M ETHODOLOGY
This section is organized according to the two main MTL problems that we will tackle: (1) How to modularize a pretrained network with latent task representations?
(2) How to balance different tasks in MTL?
We deﬁne each task as: T i (cid:44) { p i ( y i | x i , z i ) , L i , ˜ p
i ( x i ) } , where z i is task i ’s learnable shallow embedding, L i is the task loss, and ˜ p
i ( x i ) is the empirical distribution of the training data pair { x i , y i } , for i ∈ { 1 , . .
, T } and T the number of supervised tasks.
The MTL objective is:
[Equation]
where f is the predictor function (includes encoder model and decoder heads), φ ( z ) are learnable generated weights conditioned on z , and θ i are task-speciﬁc parameters for the output decoder heads. z is constructed using an embedding lookup table.
Our task conditioned Transformer architecture is based on one simple concept. We either add conditional layers or modulate existing pretrained weights using a task representation by extending Feature Wise Linear Modulation (Perez et al., 2018) functions in several ways depending on the
Transformer layer.
We deﬁne our framework below. Deﬁnition 1 (Conditional Weight Transformations) . Given a neural network weight matrix W , we compute transformations of the form φ ( W | z i ) = γ i ( z i ) W
+ β i ( z i ) , where γ i and β i are learned functions that transform the weights based on a learned vector embedding z i , for task i .
Deﬁnition 2 (Conditionally Adaptive Learning) .
In our setting, Conditionally Adaptive Learning is the process of learning a set of φ s for the conditionally adaptive modules presented below along with a set of task embedding vectors
z i for T tasks, using a multi-task loss (see equation 1).
In the subsections that follow: We introduce a new Transformer Attention Module using blockdiagonal Conditional Attention that allows the original query-key based attention to account for task-speciﬁc biases (section 2.1.1 ).
We propose a new Conditional Alignment method that aligns the data of diverse tasks and that performs better than its unconditioned and higher capacity predecessor (section 2.1.2 ).
We adapt layer normalization statistics to speciﬁc tasks using a new Conditional
Layer Normalization module (section 2.1.3 ).
We add a Conditional Bottleneck that facilitates weight sharing and task-speciﬁc information ﬂow from lower layers (section 2.1.4 ).
In our experiments we provide an ablation study of these components (Table 1) examining performance in terms of GLUE scores.
2.1. C ONDITIONAL A TTENTION
Given d , the input dimensions, the query Q , the key K , and the value V as deﬁned in Vaswani et al.
(2017), we redeﬁne the attention operation:
[Equation]
, z [Equation] where (cid:76) is the direct sum operator (see section A.6), N is the number of block matrices A n ∈ R ( L/N ) ×
( L/N ) along the diagonal of the attention matrix, L is the input sequence, M ( z i ) =
diag ( A (cid:48) 1 , . . .
, A (cid:48) N ) is a block diagonal conditional matrix.
Note that A n is constructed using L/N trainable and randomly initialized L/N dimensional vectors.
While the original attention matrix depends on the hidden states h , M ( z i ) is a learnable weight matrix that only depends on the task embedding z i ∈ R d .
γ i , β i : R d (cid:55)→ R L 2 /N 2 are Feature Wise
Linear Modulation (Perez et al., 2018) functions.
We also experimented with full-block Conditional
Attention ∈ R L × L .
Not only did it have N 2 more parameters compared to the block-diagonal variant, but it also performed signiﬁcantly worse on the GLUE development set (see FBA variant in Table 10).
It is possible that GLUE tasks derive a certain beneﬁt from localized attention that is a consequence of M ( z i ) .
With M ( z i ) , each element in a sequence can only attend to other elements in its subsequence of length L/N .
In our experiments we used N = d/L .
The full Conditional Attention mechanism used in our experiments is illustrated in Figure 2.
2.1. C ONDITIONAL A LIGNMENT
Wu et al.
(2020) showed that in MTL having T separate alignment modules R 1 , . .
, R T increases
BERT LARGE avg.
scores on ﬁve GLUE tasks (CoLA, MRPC, QNLI, RTE, SST-2) by 2.35%.
Inspired by this work, we found that adding a task conditioned alignment layer between the input embedding layer and the ﬁrst BERT Transformer layer improved multi-task model performance.
However, instead of having T separate alignment matrices R i for each T task, one alignment matrix ˆ R is generated as a function of the task embedding z i .
As in Wu et al.
(2020), we tested this module on the same ﬁve GLUE tasks and with BERT LARGE .
Enabling task conditioned weight sharing across covariance alignment modules allows us to outperforms BERT LARGE by 3.61%.
This is 1.26 % higher than having T separate alignment matrices.
Inserting ˆ R into BERT, yields the following encoder function ˆ f :
[Equation]
where x i ∈ R
d is the layer input, g θ i is the decoder head function for task i with weights θ
i , E the frozen BERT embedding layer, B the BERT Transformer layers and R the linear weight matrix of a single task conditioned alignment matrix.
γ i , β i : R d (cid:55)→ R d are Feature Wise Linear Modulation functions.
2.1. C ONDITIONAL L AYER N ORMALIZATION (CLN)
We extend the Conditional Batch Normalization idea from de Vries et al.
(2017) to Layer Normalization (Ba et al., 2016).
For task T i , i ∈ { 1 , . . .
, T } :
[Equation]
where h i is the CLN output vector, a i are the preceding layer activations associated with task i , µ and σ are the mean and the variance of the summed inputs within each layer as deﬁned in Ba et al.
Conditional Layer Normalization is initialized with BERT’s Layer Normalization afﬁne transformation weights and bias γ (cid:48) and β (cid:48) from the original formulation: h = 1 σ (cid:12) ( a − µ ) ∗ γ (cid:48)
+ β (cid:48) .
During training, the weight and bias functions of γ i ( ∗ ) and β i ( ∗ ) are always trained, while the original Layer Normalization weight may be kept ﬁxed.
This module was added to account for task speciﬁc rescaling of individual training cases.
Layer Normalization normalizes the inputs across features.
The conditioning introduced in equation 2.1.3 allows us to modulate the normalization’s output based on a task’s latent representation.
2.1. C ONDITIONAL B OTTLENECK
We created a task conditioned two layer feed-forward bottleneck layer (CFF up/down in Figure 3).
The conditional bottleneck layer follows the same transformation as in equation 2.
The module in Figure 3a is added to the top most Transformer layers of CA-MTL BASE and uses a CLN.
For CA-MTL LARGE this module is the main building block of the skip connection added alongside all Transformer layers seen in Figure 3b. The connection at layer j takes in the matrix sum of the Transformer layer output at j and the previous connection’s output at j − 1 .
The Conditional bottleneck allows lower layer information to ﬂow upwards depending on the task.
Our intuition for introducing this component is related to recent studies (Tenney et al., 2019a) that showed that the “most important layers for a given task appear at speciﬁc positions”.
As with the other modules described so far, each task adaptation is created from the weights of a single shared adapter that is modulated by the task embedding.
2. M ULTI -T ASK U NCERTAINTY S AMPLING
MT-Uncertainty Sampling is a task selection strategy that is inspired by Active Learning techniques.
Our algorithm 1 is outlined in the Appendix, Section A.2.
Similar to Active Learning, our algorithm ﬁrst evaluates the model uncertainty.
MT-Uncertainty Sampling uses Shannon Entropy, an uncertainty measure, to choose training examples by ﬁrst doing forward pass through the model with b × T input samples.
For an output classiﬁcation prediction with C i possible classes and probabilities ( p i, 1 , . . .
, p i,C i ) , the Shannon Entropy H i , for task T
i and i ∈ { 1 , . .
, T } , our uncertainty measure U ( x ) are given by:
[Equation]
c
[Equation] [Equation]
where ¯ H i is the average Shannon Entropy across b samples of task t , H (cid:48)
i , the Shannon entropy of choosing classes with uniform distribution and ˆ H , the maximum of each task’s average entropy over b samples.
H (cid:48)
i is normalizing factor that accounts for differing number of prediction classes (without the normalizing factor H (cid:48)
i , tasks with a binary classiﬁcation C i = 1 were rarely chosen).
Further, to limit high entropy outliers and to favor tasks with highest uncertainty, we normalize with ˆ H .
The measure in eq.
4 allows Algorithm 1 to choose b samples from b × T candidates to train the model.
R ELATED W ORK
Multi-Tasking in NLP. To take advantage of the potential positive transfer of knowledge from one task to another, several works have proposed carefully choosing which tasks to train as an intermediate step in NLP before single task ﬁne-tuning (Bingel & Søgaard, 2017; Kerinec et al., 2018;
Wang et al., 2019a; Standley et al., 2019; Pruksachatkun et al., 2020;
Phang et al., 2018).
The intermediate tasks are not required to perform well and are not typically evaluated jointly.
In this work, all tasks are trained jointly and all tasks used are evaluated from a single model .
In Natural
Language Understanding (NLU), it is still the case that to get the best task performance one often needs a separate model per task (Clark et al., 2019c; McCann et al., 2018).
At scale, Multilingual
NMT systems (Aharoni et al., 2019) have also found that MTL model performance degrades as the number of tasks increases.
We notice a similar trend in NLU with our baseline MTL model.
Recently, approaches in MTL have tackled the problem by designing task speciﬁc decoders on top of a shared model (Liu et al., 2019b) or distilling multiple single-task models into one (Clark et al., 2019c).
Nonetheless, such MTL approaches still involves single task ﬁne-tuning.
In this paper, we show that it is possible to achieve high performance in NLU without single task ﬁne-tuning.
Adapters.
Adapters are trainable modules that are attached in speciﬁc locations of a pretrained network. They provide another promising avenue to limit the number of parameters needed when confronted with a large number of tasks.
This approach is useful with pretrained MLM models that have rich linguistic information (Tenney et al., 2019b;
Clark et al., 2019b; Liu et al., 2019a; Tenney et al., 2019a).
Recently, Houlsby et al. (2019) added an adapter to a pretrained BERT model by ﬁne-tuning the layer norms and adding feed forward bottlenecks in every Transformer layer.
However, such methods adapt each task individually during the ﬁne-tuning process.
Unlike prior work, our method harnesses the vectorized representations of tasks to modularize a single pretrained model across all tasks.
Stickland et al.
(2019) and Tay et al. (2020) also mix both MTL and adapters with BERT and T5 encoder-decoder (Raffel et al., 2019) respectively by creating local task modules that are controlled by a global task agnostic module.
The main drawback is that a new set of non-shared parameters must be added when a new task is introduced.
CA-MTL shares all parameters and is able to re-modulate existing weights with a new task embedding vector.
Active Learning, Task Selection and Sampling.
Our sampling technique is similar to the ones found in several active learning algorithms (Chen et al., 2006) that are based on Shannon entropy estimations.
Reichart et al.
(2008) and Ikhwantri et al.
(2018) examined Multi-Task Active Learning (MTAL), a technique that chooses one informative sample for T different learners (or models) for each T tasks.
Instead we choose T tasks samples for one model .
Moreover, the algorithm weights each sample by the corresponding task score, and the Shannon entropy is normalized to account for various losses (see equation 5). Also, our algorithm is used in a large scale MTL setup ( (cid:29) 2 tasks).
Recently, Glover & Hokamp (2019) explored task selection in MTL using learning policies based on counterfactual estimations (Charles et al., 2013).
However, such method considers only ﬁxed stochastic parameterized policies while our method adapts its selection criterion based on model uncertainty throughout the training process.
E XPERIMENTS AND R ESULTS
We show that our adapter of section 2 achieve parameter efﬁcient transfer for 26 NLP tasks.
Our implementation of CA-MTL is based on HuggingFace (Wolf et al., 2019).
Hyperparameters and our experimental set-up are outlined in A.5.
To preserve the weights of the pretrained model, CA-MTL’s bottom half Transformer layers are frozen in all experiments (except in section 4.4).
We also tested different layer freezing conﬁgurations and found that freezing half the layers worked best on average (see Section A.8).
4. M ULTI -T ASK U NCERTAINTY S AMPLING
[Figure]
Our MT-Uncertainty sampling strategy, from section 2.2, is compared to 3 other task selection schemes: a)
Counterfactual b) Task size c) Random.
We used a
BERT BASE (no adapters) on 200k iterations and with the same hyperparameters as in Glover & Hokamp (2019).
For more information on Counterfactual task selection, we invite the reader to consult the full explanation in Glover & Hokamp (2019).
For T tasks and the dataset D i for tasks i ∈ { 1 , . . .
, T } , we rewrite the deﬁnitions of Random π rand and Task size π | task | sampling:
[Equation]
[Figure]
In Figure 4, we see from the results that MTUncertainty converges faster by reaching the 80% average GLUE score line before other task sampling methods.
Further, MT-Uncertainty maximum score on 200k iterations is at 82.2, which is 1.7% higher than Counterfactual sampling.
The datasets in the GLUE benchmark offers a wide range of dataset sizes.
This is useful to test how MT-Uncertainty manages a jointly trained low resource task (CoLA) and high resource task (MNLI).
Figure 5 explains how catastrophic forgetting is curtailed by sampling tasks before performance drops.
With π rand , all of CoLA’s tasks are sampled by iteration 500, at which point the larger MNLI dataset overtakes the learning process and CoLA’s dev set performance starts to diminish.
On the other hand, with MT-Uncertainty sampling, CoLA is sampled whenever Shannon entropy is higher than MNLI’s.
The model ﬁrst assesses uncertain samples using Shannon Entropy then decides what data is necessary to train on.
This process allows lower resource tasks to keep performance steady.
We provide evidence in Figure 8 of A.2 that MT-Uncertainty is able to manage task difﬁculty — by choosing the most difﬁcult tasks ﬁrst.
4. A BLATION AND M ODULE A NALYSIS
[Table]
In Table 1, we present the results of an ablation study to determine which elements of CA-MTL BERT-BASE had the largest positive gain on average GLUE scores.
Starting from a MTL BERT BASE baseline trained [-us
 ing-]using random task sampling ( π rand ).
Apart for the Conditional Adapter, each module as well as MTUncertainty lift overall performance and reduce variance across tasks.
Please note that we also included accuracy/F1 scores for QQP, MRPC and Pearson/ Spearman correlation for STS-B to calculate score standard deviation Task σ .
Intuitively, when negative task transfer occurs between two tasks, either (1) task interference is bidirectional and scores are both impacted, or (2) interference is unidirectional and only one score is impacted.
We calculate Task σ to characterize changes in the dynamic range of performance across multiple tasks.
We do this to asses the degree to which performance improvements are distributed across all tasks or only subsets of tasks.
As we can see from Table 1, Conditional Attention, Conditional Alignment, Conditional Layer Normalization, MT-Uncertainty play roles in reducing Task σ and increasing performance across tasks.
This provides partial evidence of CA-MTL’s ability to mitigating negative task transfer.
We show that Conditional Alignment can learn to capture covariate distribution differences with task embeddings co-learned from other adapter components of CA-MTL.
In Figure 6, we arrive at similar conclusions as Wu et al.
(2020), who proved that negative task transfer is reduced when task covariances are aligned.
The authors provided a “covariance similarity score” to gauge covariance alignment.
For task i and j with m i and m j data samples respectively, and given d dimensional inputs to the ﬁrst Transformer layer X i ∈
R
m i × d and X j ∈ R m j × d , we rewrite the steps to calculate the covariance similarity score between task i and j : (a) Take the covariance matrix X (cid:62)
i X i , (b) Find its best rank- r i approximation
U
i,r i D i,r i U (cid:62) i,r i , where r i is chosen to contain 99% of the singular values.
(c) Apply steps (a), (b) to X j , and compute the covariance similarity score CovSim i,j :
CovSim i,j := (cid:107) ( U i,r i D 1 / 2 i,r i ) (cid:62) U j,r j D 1 / 2 j,r j (cid:107) F (cid:107) U i,r i D 1 / 2 i,r i (cid:107) F · (cid:107) U j,r j D 1 / 2 j,r j (cid:107) F . CovSim i = T −(cid:88) j (cid:54) =
i CovSim i,j (7) Since we are training models with T tasks, we take the average covariance similarity score CovSim i between task i and all other tasks.
We measure CovSim i using equation 7 between 9 single-task models trained on individual GLUE tasks.
For each task in Figure 6, we measure the similarity score on the MTL trained BERT BASE baseline, e.g., CoLA (MTL), or CA-MTL BERT-BASE model, e.g.,
MNLI (CA-MTL).
Our score improvement measure is the % difference between a single task model and MTL or CA-MTL on the particular task.
We ﬁnd that covariance similarity increases for 9 tasks and that performance increases for 7 out 9 tasks.
These measurements conﬁrm that the Conditional
Alignment is able to align task covariance, thereby helping alleviate task interference.
4. J OINTLY TRAINING ON 8 TASKS : GLUE
In Table 2, we evaluate the performance of CA-MTL against single task ﬁne-tuned models, MTL as well as the other BERT-based adapters on GLUE.
As in Houlsby et al.
(2019), MNLI m and MNLI mm are treated as separate tasks.
Our results indicate that CA-MTL outperforms both the BASE adapter, or equal to. Results from: 1 Devlin et al. 2 Stickland et al. [Table]
Houlsby et al.
[Table]
PALS+Anneal
Sampling (Stickland et al., 2019), and the LARGE adapter, Adapters-256 (Houlsby et al., 2019).
Against single task (ST) models, CA-MTL is 1.3% higher than BERT BASE , with 5 outtasks equal or greater performance, and 0.7% higher than BERT LARGE , with 3 out 9 tasks equal or greater performance.
ST models, however, need 9 models or close to 9 × more parameters for alltasks.
We noted that CA-MTL BERT-LARGE ’s average score is driven by strong RTE scores.
While RTE beneﬁts from MTL, this behavior may also be a side effect of layer freezing.
In Table 10, we see that
CA-MTL has gains over ST on more and more tasks as we gradually unfreeze layers.
4. T RANSFER TO N EW T ASKS
[Table]
In Table 3 we examine the ability of our method to quickly adapt to new tasks.
We performed domain adaptation on SciTail (Khot et al., 2018) and SNLI (Bowman et al., 2015) datasets, using a CA-MTL BASE model trained on GLUE and a new linear decoder head.
We tested several pretrained and randomly initialized task embeddings in a zero-shot setting.
The complete set of experiments with all task embeddings can be found in the Appendix, Section A.4.
We then selected the best task embedding for our results in Table 3.
STS-B and MRPC MTL-trained task embeddings performed best on SciTail and SNLI respectively.
CA-MTL BERT-BASE has faster adaptation than MT-DNN SMART (Jiang et al., 2020) as evidenced by higher performances in low-resource regimes (0.1% and 1% of the data).
When trained on the complete dataset, CA-MTL BERT-BASE is on par with MT-DNN SMART .
Unlike MT-DNN SMART however, we do not add context from a semantic similarity model – MT-DNN SMART is built off HNN (He et al., 2019).
Nonetheless, with a larger model, CA-MTL surpasses MT-DNN SMART on the full SNLI and SciTail datasets in Table 6.
4. J OINTLY TRAINING ON 24 TASKS : GLUE/S UPER -GLUE, MRQA AND WNUT
When 23 tasks are trained jointly, performance of CA-MTL BERT-BASE continues to improve.
Effects of Scaling Task Count.
In Figure 7 we continue to test if CA-MTL mitigates task interference by measuring GLUE average scores when progressively adding 9 GLUE tasks, Super-GLUE tasks (Wang et al., 2019b), MRQA tasks (Fisch et al., 2019).
Tasks are described in Appendix section A.9.
The results show that adding 23 tasks drops the performance of our baseline MTL BERT BASE ( π rand ). MTL
BERT increases by 4.3% when adding MRQA but, with 23 tasks, the model performance drops by 1.8%.
The opposite is true when CA-MTL modules are integrated into the model.
CA-MTL continues to show gains with a large number of tasks and surpasses the baseline MTL model by close to 4% when trained on 23 tasks. [Table] [Table] [Table]
[Table]
[Table]
[Table] [Table] 24-task CA-MTL. We jointly trained large
MTL baselines and CA-MTL models on
GLUE/Super-GLUE/MRQA and Named Entity Recognition (NER)
WNUT2017 (Derczynski et al., 2017).
Since some dev.
set scores are not provided and since RoBERTa results were reported with a median score over 5 random seeds, we ran our own single seed ST/MTL baselines (marked “ReImp”) for a fair comparison.
The dev.
set numbers reported in Liu et al. (2019c) are displayed with our baselines in Table 9.
Results are presented in Table [Table]
We notice in Table 4 that even for large models, CA-MTL provides large gains in performance on average over both ST and MTL models.
For the BERT based models, CA-MTL provides 2.3% gain over ST and higher scores on 17 out 24 tasks.
For RoBERTa based models, CA-MTL provides 1.2% gain over ST and higher scores on 15 out 24 tasks.
We remind the reader that this is achieved with a single model.
Even when trained with 16 other tasks, it is interesting to note that the MTL baseline perform better than the ST baseline on Super GLUE where most tasks have a small number of samples.
Also, we used NER to test if we could still outperform the ST baseline on a token-level task, signiﬁcantly different from other tasks.
Unfortunately, while CA-MTL performs signiﬁcantly better than the MTL baseline model, CA-MTL had not yet overﬁt on this particular task and could have closed the gap with the ST baselines with more training cycles.
[Table]
Comparisons with other methods.
In Table 5, CA-MTL BERT is compared to other Large BERT based methods that either use MTL + ST, such as MT-DNN (Liu et al., 2019b), intermediate tasks + ST, such as
STILTS (Phang et al., 2018) or MTL model distillation + ST, such as
BAM!
(Clark et al., 2019c).
Our method scores higher than MT-DNN on 5 of 9 tasks and by 1.0 % on avg.
Against STILTS, CA-MTL realizes a 0.7 % avg.
score gain, surpassing scores on 6 of 9 tasks.
We also show that CA-MTL RoBERTa is within only 1.6 % of a RoBERTa ensemble of 5 to 7 models per task and that uses intermediate tasks.
Using our 24-task CA-MTL large RoBERTa-based model, we report
NER F1 scores on the WNUT2017 test set in Table 6a.
We compare our result with RoBERTa LARGE and XLM-R LARGE (Nguyen et al., 2020) the current state-of-the-art (SOTA).
Our model outperforms XLM-R LARGE by 1.6%, reaching a new state-of-the-art.
Using domain adaptation as described in Section 4.4, we report results on the SciTail test set in Table 6b and SNLI test set in Table 6b.
For
SciTail, our model matches the current SOTA 2 ALUM (Liu et al., 2020), a RoBERTa large based model that additionally uses the SMART (Jiang et al., 2020) ﬁne-tuning method.
For SNLI, our model outperforms SemBert, the current SOTA 3 . [Table]
C ONCLUSION
We believe that our experiments here have helped demonstrate the potential of task conditioned adaptive learning within a single model that performs multiple tasks.
In a large-scale 24-task NLP experiment, CA-MTL outperforms fully tuned single task models by 2.3% for BERT Large and by 1.2% for RoBERTa Large using 1.12 times the number of parameters, while single task ﬁne-tuning approach requires 24 separately tuned single task models or 24 times the number of parameters.
When a BERT vanilla MTL model sees its performance drop as the number of tasks increases, CA-MTL scores continue to climb.
Performance gains are not driven by a single task as it is often the case in MTL.
Each CA-MTL module that adapts a Transformer model is able to reduce performance variances between tasks, increasing average scores and aligning task covariances.
This evidence shows that CA-MTL is able to mitigate task interference and promote more efﬁcient parameter sharing.
We showed that MT-Uncertainty is able to avoid degrading performances of low resource tasks.
Tasks are sampled whenever the model sees entropy increase, helping avoid catastrophic forgetting.
Overall, CA-MTL offers a promising avenue to dynamically adapt and modularize knowledge embedded in large monolithic pretrained models. Extending such ideas will an objective for future work. A CKNOWLEDGMENTS
This research was supported by the Canada CIFAR AI Chairs Program, NSERC and PROMPT. Experiments in this article were conducted with Compute Canada and MILA computational infrastructure and we thank them for their support. We would like to thank Colin Raffel, Sandeep Subramanian, and Nicolas Gontier for their useful feedback and the anonymous reviewers for helpful comments, discussions and suggestions
A A PPENDIX A. S UMMARY OF A CRONYMS
Acronyms of datasets and descriptions can be found below in section A.9.
[Table]
A. U NCERTAINTY S AMPLING : A LGORITHM AND A DDITIONAL R ESULTS
[Table]
C i c
=1 p c ( f ( x t,i )) log p c ( f ( x t,i ))
end
Compute ¯ H t ← 1 b (cid:80) x ∈ x i H t,i (cid:46) Average entropy for task t
Compute H (cid:48)
t ← − (cid:80) C t c =1 1 C t log (cid:20)
C t (cid:21) (cid:46)
Max entropy (uniform distribution)
B ← B ∪ x t and D t ← D t \ x t if D t = ∅ then
Reload D t end for i ← 1 to b do
Compute: U t,i ← H t,i /H (cid:48)
t {+(cid:46) Uncertainty normalized with max entropy end 19 end 
+}
Compute ˆ H ← max i ∈{ 1 ,...,T } [ ¯ H t ] (cid:46) Entropy of task with highest average entropy 21 Update U t,i ← U t,i / ˆ H (cid:46) Normalize each sample’s uncertainty measure 22 B (cid:48)
← top _ b( {U t,i | t ∈ [1 , . . , T ] , i ∈ , b ] } ) (cid:46) b samples w/ highest uncertainty Return: With B solve model f
An advantage of our MT-Uncertainty Sampling approach is its ability to manage task difﬁculty.
This is highlighted in Figure 8.
In this experiment, we estimated task difﬁculty using the Evolutionary
Data Measures (EDM) 4 proposed by Collins et al.
The task difﬁculty estimate relies on multiple dataset statistics such as the data size, class diversity, class balance and class interference.
Interestingly, estimated task difﬁculty correlates with the ﬁrst instance that the selection of a speciﬁc task occurs.
Supposing that QNLI is an outlier, we notice that peaks in the data occur whenever tasks are ﬁrst selected by MT Uncertainty sampling.
This process follows the following order: 1.
MNLI 2.
CoLA 3.
RTE 4.
QQP 5.
MRPC 6.SST-2, which is the order from highest task difﬁculty to lowest task difﬁculty using EDM.
As opposed to Curriculum Learning (Bengio et al., 2009), MT-Uncertainty dynamically prioritizes the most difﬁcult tasks.
As also discovered in MTL vision work (Guo et al., 2018), this type of prioritization on more difﬁcult tasks may explain MT-Uncertainty’s improved performance over other task selection methods.
In MTL, heuristics to balance tasks during training is typically done by weighting each task’s loss differently.
We see here how MT-Uncertainty is able to prioritize task difﬁculty. [Figure]
[Table]
While the EDM difﬁculty measure is shown to correlate well with model performance, it lacks precision.
As reported in Collins et al.
, the average score achieved on the Yahoo Answers dataset is 69.9% and its difﬁculty is 4.51.
The average score achieved on Yelp Full is 56.8%, 13.1% less than Yahoo Answers and its difﬁculty is 4.42.
The authors mention that “This indicates that the difﬁculty measure in its current incarnation may be more effective at assigning a class of difﬁculty to datasets, rather than a regression-like value”.
A. O THER R ELATED W ORK
Multi-Tasking in NLP and other ﬁelds.
MTL weight sharing algorithms such as Mixture-of-Experts (MoE) have found success in NLP (Lepikhin et al., 2020).
CA-MTL can complement MoE since the
Transformers multi-headed attention can be seen as a form of MoE (Peng et al., 2020).
In Vision,
MTL can also improve with optimization (Sener & Koltun, 2018) or gradient-based approaches (Chen et al., 2017; Yu et al., 2020).
Active Learning, Task Selection and Sampling.
Ikhwantri et al.
(2018) examined multi-task active learning for neural semantic role labeling in a low resource setting, using entity recognition as the sole auxiliary task. They used uncertainty sampling for active learning and found that 12% less data could be used compared to passive learning.
Reichart et al.
(2008) has examined different active learning techniques for the two task annotation scenario, focusing on named entity recognition and syntactic parse tree annotations.
In contrast, here we examine the larger scale data regime, the modularization of a multi-task neural architecture, and the many task ( (cid:29) 2) setting among other differences.
Other than MTAL (Reichart et al., 2008; Ikhwantri et al., 2018), Kendall et al.
(2017) leveraged model uncertainty to balance MTL losses but to tasks as is proposed
A. Z ERO -S HOT R ESULTS ON S CI T AIL AND SNLI
Before testing models on domain adaptation in section 4.
[Table]
we ran zero-shot evaluations on the development set of SciTail and SNLI.
Table 8 outlines 8-task CA-MTL BERT-BASE ’s zero-shot transfer abilities when pretrained on GLUE with our MTL approach.
We expand the task embedding layer to accommodate an extra task and explore various embedding initialization.
We found that reusing
STS-B and MRPC task embeddings worked best for SciTail and SNLI respectively. [Table]
M ORE E XPERIMENTAL D ETAILS
We used a batch size of 32 and a seed of 12 in all experiments.
We used Adam (Kingma & Ba, 2015) as the optimizer with a learning rate of 2e-5.
We applied a learning rate decay with warm up over the ﬁrst 10% of the training steps.
Unless otherwise speciﬁed, we used 5 epochs, a seed of 12 and a sequence length of 128.
Additional details are outlined in section .
Our data prepossessing and linear decoder heads are the same as in Devlin et al.
We used the same dropout rate of 0. in all layers.
To run our experiments, we used either four NVIDIA P100 GPU for base models or four NVIDIA V100 GPU for larger ones.
We did not perform parameter search.
We do not use ensemble of models or task-speciﬁc tricks (Devlin et al., [-
-]2018;
Liu et al., 2019b; Clark et al., 2019c).
All models are either 12 Transformer layers for BASE and 24 Transformer layers for LARGE.
Apart from CA-MTL, models trained in multi-task learning (BERT or RoBERTa without adapters) used random task sampling.
For Table 1 and Figure 7, all BERT-based model have half their layers frozen (untrained) for a fair comparison of ablation results.
For the 24-task MTL and CA-MTL models in
Tables 4 and 5, we increased the input sequence length to 256 and used 8 epochs.
A. T HE D IRECT S UM O PERATOR
In section 2.1.1, we used the direct sum operator ⊕ .
This operation allows us to create a block diagonal matrix.
The direct sum of a matrix A ∈ R n × m and B ∈ R p × q results in a matrix of size ( m + p ) × ( n + q ) , deﬁned as:
A ⊕ B = (cid:20)
A B (cid:21) =    a· · · a 1 n · · · ... a m· · · a mn · · ·  · · · b· · · b 1 q . · · · b p· · · b pq 
 
A. B ASELINES AND O THER E XPERIMENTAL R ESULTS
In this section, we present our baseline results for BERT, RoBERTa, CA-MTL as well as other models.
Our single task results (ST) that we ran ourselves surpass other paper’s reported scores in Table 9.
Liu et al. (2019c) reports random seed median scores for RoBERTa.
However, our RoBERTa ST baseline matches or surpasses the original paper’s scores 4 out 7 times on the development set when scores are comparable (QQP F1 and STS-B spearman are not reported) [Table] [Table]
A. S OME R ESULTS ON LAYER F REEZING AND WITH F ULL B LOCK A TTENTION .
All experiments in this section were run for only 5 epochs, exclusively on the GLUE dataset for the large BERT-based 8-task CA-MTL model.
Results in Table 10 reveal that as we freeze more layers, performance tends to decrease. However, since we wanted to preserve as much pretrained knowledge as possible [Table] we chose to keep at least 50% of layers frozen.
While this has slightly lowered our performance on 9 GLUE tasks, we believe that keeping as much of the original pretrained weights is beneﬁcial when increasing the total number of tasks in MTL to 24 or more tasks.
However, we did not explore this hypothesis more.
[Table]
A. D ATASET D ESCRIPTION
The datasets that were used for the domain adaptation experiments were SciTail 5 and SNLI 6 .
We jointly trained a CA-MTL RoBERTa-LARGE model on 9 GLUE tasks, 8 Super-GLUE 7 tasks, 6 MRQAtasks, and on WNUT2017 9 (Derczynski et al., 2017).
All GLUE tasks are binary classiﬁcation, except STS-B (regression) and MNLI (three classes).
We used the same GLUE data preprocessing as in Devlin et al.
[Table]
[Table]
[Table]
SuperGLUE has a more diverse task format than GLUE, which is mostly limited to sentence and sentence-pair classiﬁcation.
We follow the same preprocessing procedure as in Wang et al.
(2019b)
[Table]
All tasks are binary classiﬁcation tasks, except CB (three classes).
Also, WiC and WSC are span based classiﬁcation tasks.
We used the same modiﬁed MRQA dataset and preprocessing steps that were used in Joshi et al.
All MRQA tasks are span prediction tasks which seeks to identify start and end tokens of an answer span in the input text.
[Table]
SNLI is a natural inference task where we predict three classes.
Examples of three target labels are: Entailment, Contradiction, and Neutral (irrelevant).
SciTail is a textual entailment dataset.
The hypotheses in SciTail are created from multiple-choice science exams and the answer candidates (premise) are extracted from the web using information retrieval tools.
SciTail is a binary true/false classiﬁcation tasks that seeks to predict whether the premise entails the hypothesis.
The two datasets are used only for domain adaptation in this study (see section A.4 for the details of our approach).