Original: BAM! Born-Again Multi-Task Networks for Natural Language Understanding
Revised: BAM! Born-Again Multi-Task Networks for Natural Language Understanding

Original: First Author Afﬁliation / Address line Afﬁliation / Address line Afﬁliation / Address lineemail@domain
Revised: { kevclark,urvashik,manning } @cs.stanford.edu { thangluong,qvl } @google.com

Original: Second Author

Original: Afﬁliation / Address line

Original: Afﬁliation / Address line

Original: Afﬁliation / Address lineemail@domain

Original: Abstract
Revised: Abstract

Original: It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.
Revised: It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.

Original: To help address this, we propose using knowledge distillation where single-task models teach a multi-task model.
Revised: To help address this, we propose using knowledge distillation where single-task models teach a multi-task model.

Original: We enhance this training with teacher annealing, a novel method that gradually transitions the model from distillation to supervised learning, helping the multi-task model surpass its single-task teachers.
Revised: We enhance this training with teacher annealing, a novel method that gradually transitions the model from distillation to supervised learning, helping the multi-task model surpass its single-task teachers.

Original: We evaluate our approach by multi-task ﬁne-tuning BERT on the GLUE benchmark.
Revised: We evaluate our approach by multi-task ﬁne-tuning BERT on the GLUE benchmark.

Original: Our method signiﬁcantly improves over standard single-task and multi-task training, resulting in state-of-the art accuracies.
Revised: Our method consistently improves over standard single-task and multi-task training.

Original:  Introduction
Revised:  Introduction

Original: Building a single model that jointly learns to perform many tasks effectively has been a longstanding challenge in NLP.
Revised: Building a single model that jointly learns to perform many tasks effectively has been a longstanding challenge in Natural Language Processing (NLP).

Original: However, multitask NLP remains ineffective for many applications, with multi-task models often performing worse than their single-task counterparts (Plank and Alonso, 2017; Bingel and Søgaard, 2017; McCann et al., 2018).
Revised: However, applying multi-task NLP remains difﬁcult for many applications, with multitask models often performing worse than their single-task counterparts (Plank and Alonso, 2017; Bingel and Søgaard, 2017; McCann et al., 2018).

Original: Motivated by these results, we propose a way of applying knowledge distillation (Hinton et al., 2015) so that single-task models effectively teach a multi-task model.
Revised: Motivated by these results, we propose a way of applying knowledge distillation (Bucilu et al., 2006; Ba and Caruana, 2014; Hinton et al., 2015) so that single-task models effectively teach a multi-task model.

Original: Knowledge distillation transfers knowledge from a “teacher” model to a “student” model by training the student to imitate the teacher’s predictions.
Revised: Knowledge distillation transfers knowledge from a “teacher” model to a “student” model by training the student to imitate the teacher’s outputs.

Original: In “born-again networks” (Furlanello et al., 2018), the teacher and studentmodels have the same neural architecture and model size, but surprisingly the student is able to surpass the teacher’s accuracy.
Revised: In “born-again networks” (Furlanello et al., 2018), the teacher and student have the same neural architecture and model size, but surprisingly the student is able to surpass the teacher’s accuracy.

Original: Our work extends these ideas to the multi-task setting. We compare Single → Multi 1 born-
Revised: Intuitively, distillation is effective because the teacher’s output distribution over classes provides more training signal than a one-hot label; Hinton et al. (2015) suggest that teacher outputs

Original: [Figure]
Revised: [Figure]

Revised: contain “dark knowledge” capturing additional information about training examples.

Revised: Our work extends born-again networks to the multi-task setting.

Original: again distillation with several other variants (Single → Single and Multi → Multi), and also explore performing multiple rounds of distillation (Single → Multi → Single → Multi).
Revised: We compare Single → Multiborn-again distillation with several other variants (Single → Single and Multi → Multi), and also explore performing multiple rounds of distillation (Single → Multi → Single → Multi).

Original: Furthermore, we propose a simple teacher annealing method that helps the student model outperform its teachers, substantially improving results.
Revised: Furthermore, we propose a simple teacher annealing method that helps the student model outperform its teachers. Teacher annealing gradually transitions the student from learning from the teacher to learning from the gold labels.

Original:  Our experiments build upon recent success in unsupervised pre-training (Dai and Le, 2015; Peters et al., 2018) and multi-task ﬁne-tune BERT (Devlin et al., 2018) to perform the tasks from the GLUE natural language understanding benchmark (Wang et al., 2019).
Revised: This method ensures the student gets a rich training signal early in training but is not limited to only imitating the teacher. Our experiments build upon recent success in self-supervised pre-training (Dai and Le, 2015; Peters et al., 2018) and multi-task ﬁne-tune BERT (Devlin et al., 2019) to perform the tasks from the GLUE natural language understanding benchmark (Wang et al., 2019).

Original: Our training method, which we call Born-Again Multi-tasking (BAM) 2 , consistently outperforms standard single-task and multi-task training, resulting in state-of-the-art accuracies on GLUE.
Revised: Our training method, which we call Born-Again Multi-tasking (BAM) 2 , consistently outperforms standard single-task and multi-task training.

Original: Further analysis shows the multi-task models beneﬁt from both better regularization and transfer between related tasks.
Revised: Further analysis shows the multi-task models beneﬁt from both better regu- larization and transfer between related tasks.

Original:  Related Work
Revised:  Related Work

Original: Multi-task learning for neural networks in general (Caruana, 1997) and within NLP speciﬁcally (Collobert and Weston, 2008; Luong et al., 2016) has been widely studied.
Revised: Multi-task learning for neural networks in general (Caruana, 1997) and within NLP speciﬁcally (Collobert and Weston, 2008; Luong et al., 2016) has been widely studied.

Original: Much of the recent work for
Revised: Much of the recent work for

Original: NLP has centered on neural architecture design: e.g., ensuring only beneﬁcial information is shared across tasks (Liu et al., 2017; Ruder et al., 2019) or arranging tasks in linguistically-motivated hierarchies (Søgaard and Goldberg, 2016;
Revised: NLP has centered on neural architecture design: e.g., ensuring only beneﬁcial information is shared across tasks (Liu et al., 2017; Ruder et al., 2019) or arranging tasks in linguistically-motivated hierarchies (Søgaard and Goldberg, 2016;

Original: Hashimoto et al., 2017; Sanh et al., 2019).
Revised: Hashimoto et al., 2017; Sanh et al., 2019).

Original: These contributions are orthogonal to ours because we instead focus on the multi-task training algorithm.
Revised: These contributions are orthogonal to ours because we instead focus on the multi-task training algorithm.

Revised: Distilling large models into small models (Kim and Rush, 2016; Mou et al., 2016) or ensembles of models into single models (Kuncoro et al., 2016;

Revised: Liu et al., 2019a) has been shown to improve results for many NLP tasks.

Revised: There has also been some work on using knowledge distillation to aide in multi-task learning.

Original: Knowledge distillation has been used to regularize multi-taskreinforcement learning agents (Parisotto et al., 2016;
Revised: In reinforcement learning, knowledge distillation has been used to regularize multi-task agents (Parisotto et al., 2016;

Original: Teh et al., 2017).
Revised: Teh et al., 2017).

Original: More relevant to this work, Tan et al.
Revised: In NLP, Tan et al.

Original: (2019) distill singlelanguage-pair machine translation systems into a many-language system.
Revised: (2019) distill singlelanguage-pair machine translation systems into a many-language system.

Original: However, they focus on multilingual rather than multi-task learning, use a more complex training procedure, and only experiment with Single → Multi distillation.
Revised: However, they focus on multilingual rather than multi-task learning, use a more complex training procedure, and only experiment with Single → Multi distillation.

Original: Concurrently with our work, Phang et al. (2018) and Liu et al. (2019) also explore ﬁne-tuning BERT using multiple tasks.
Revised: Concurrently with our work, several other recent works also explore ﬁne-tuning BERT using multiple tasks (Phang et al., 2018; Liu et al., 2019b; Keskar et al., 2019).

Original: However, they use only standard transfer or multi-task learning, instead focusing on ﬁnding beneﬁcial task pairs or augmenting the model with more sophisticated task-speciﬁc components.
Revised: However, they use only standard transfer or multi-task learning, instead focusing on ﬁnding beneﬁcial task pairs or designing improved task-speciﬁc components on top of BERT.

Original:  Methods 3. Multi-Task Setup
Revised:  Methods 3. Multi-Task Setup

Original: Model.
Revised: Model.

Original: All of our models are built on top of BERT (Devlin et al., 2018).
Revised: All of our models are built on top of BERT (Devlin et al., 2019).

Original: This model passes byte-pairtokenized (Sennrich et al., 2016) input sentences through a Transformer network (Vaswani et al., 2017), producing a contextualized representation for each token.
Revised: This model passes byte-pairtokenized (Sennrich et al., 2016) input sentences through a Transformer network (Vaswani et al., 2017), producing a contextualized representation for each token.

Original: The vector corresponding to the ﬁrst input token 3 c is passed into a task-speciﬁc classiﬁer.
Revised: The vector corresponding to the ﬁrst input token 3 c is passed into a task-speciﬁc classiﬁer.

Original: For classiﬁcation tasks, we use a standard softmax layer: softmax ( W c ) .
Revised: For classiﬁcation tasks, we use a standard softmax layer: softmax ( W c ) .

Original: For regression tasks, we  use a size-1 NN layer with a sigmoid activation: sigmoid ( w T c ) .
Revised: For regression tasks, we normalize the labels so they are between 0 and 1 and then use a size-1 NN layer with a sigmoid activation: sigmoid ( w T c ) .

Original: In our multi-task models, all of the model parameters are shared across tasks except for these classiﬁers on top of BERT, which means less than 0.01% of the parameters are task-speciﬁc.
Revised: In our multi-task models, all of the model parameters are shared across tasks except for these classiﬁers on top of BERT, which means less than 0.01% of the parameters are task-speciﬁc.

Original: Following BERT, the token embeddings and Transformer are initialized with weights from a “masked LM” pre-training phase 5 .
Revised: Following BERT, the token embeddings and Transformer are initialized with weights from a self-supervised pre-training phase.

Original: Training.
Revised: Training.

Original: Single-task training is performed as in Devlin et al.
Revised: Single-task training is performed as in Devlin et al.

Original: For multi-task training, examples of different tasks are shufﬂed together, even within minibatches.
Revised: For multi-task training, examples of different tasks are shufﬂed together, even within minibatches.

Original: The(unweighted) summed loss across all tasks is minimized.
Revised: The summed loss across all tasks is minimized.

Original: 3. Knowledge Distillation
Revised: 3. Knowledge Distillation

Original: We use D τ =  ( x 1 τ , y 1 τ ) , ..., ( x Nτ , y Nτ )  to denote the training set for a task τ and p τ (y | x, θ ) to denote the probability distribution over classes for task τ produced by a neural network with parameters θ on the input x .
Revised: We use D τ = { ( x 1 τ , y 1 τ ) , ..., ( x Nτ , y Nτ ) } to denote the training set for a task τ and f τ ( x, θ ) to denote the outputs for task τ produced by a neural network with parameters θ on the input x (for classiﬁcation tasks this is a distribution over classes).

Original: Standard supervised learning trains θ to minimize thecross-entropy loss on the training set:
Revised: Standard supervised learning trains θ to minimize the loss on the training set:

Original: [Equation]
Revised: [Equation]

Revised: ) where for classiﬁcation tasks (cid:96) is usually crossentropy.

Original: Knowledge distillation trains the model to instead match the predictions of a teacher model with parameters θ (cid:48) :
Revised: Knowledge distillation trains the model to instead match the predictions of a teacher model with parameters θ (cid:48) :

Original: [Equation]
Revised: [Equation]

Original: Note that our distilled networks are “born-again” in that the student has the same model architecture as the teacher, i.e., all of our models have the same prediction function p τ for each task.
Revised: Note that our distilled networks are “born-again” in that the student has the same model architecture as the teacher, i.e., all of our models have the same prediction function f τ for each task.

Original: For regression tasks, we train the student to minimize the L distance between its prediction and the teacher’s instead of using cross-entropy loss.
Revised: For regression tasks, we train the student to minimize the L distance between its prediction and the teacher’s instead of using cross-entropy loss.

Original: Intuitively, knowledge distillation improves training because the full distribution over labels provided by the teacher provides a richer training signal than a one-hot label.
Revised: Intuitively, knowledge distillation improves training because the full distribution over labels provided by the teacher provides a richer training signal than a one-hot label.

Original: See Furlanello et al.
Revised: See Furlanello et al.

Original: for a more thorough discussion.
Revised: for a more thorough discussion.

Original: Multi-Task Distillation.
Revised: Multi-Task Distillation.

Original: Given a set of tasks T , we train a single-task model with parameters θ τ on each task τ .
Revised: Given a set of tasks T , we train a single-task model with parameters θ τ on each task τ .

Original: For most experiments, we use the single-task models to teach a multi-task model:
Revised: For most experiments, we use the single-task models to teach a multi-task model with parameters θ :

Original: [Equation]
Revised: [Equation]

Original: However, we experiment with other distillation strategies as well.
Revised: However, we experiment with other distillation strategies as well.

Original: Teacher Annealing.
Revised: Teacher Annealing.

Original: In knowledge distillation, the student is trained to imitate the teacher.
Revised: In knowledge distillation, the student is trained to imitate the teacher.

Original: This raises the concern that the student may be limited by the teacher’s performance and not be able to substantially outperform the teacher.
Revised: This raises the concern that the student may be limited by the teacher’s performance and not be able to substantially outperform the teacher.

Original: To address this, we propose teacher annealing , which mixes the teacher prediction with the gold label during training.
Revised: To address this, we propose teacher annealing , which mixes the teacher prediction with the gold label during training.

Original: Speciﬁcally, thecross-entropy term in the summation becomes
Revised: Speciﬁcally, the term in the summation becomes

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: where λ is linearly increased from 0 to 1 throughout training.
Revised: where λ is linearly increased from 0 to 1 throughout training.

Original: Early in training, the model is mostly distilling to get as useful of a training signal as possible.
Revised: Early in training, the model is mostly distilling to get as useful of a training signal as possible.

Original: Towards the end of training, the model is mostly relying on the gold-standard labels so it can learn to surpass its teachers.
Revised: Towards the end of training, the model is mostly relying on the gold-standard labels so it can learn to surpass its teachers.

Original:  Experiments
Revised:  Experiments

Original: Data.
Revised: Data.

Original: We use the GLUE dataset (Wang et al., 2019), which consists of 9 natural language understanding tasks covering textual entailment (RTE and MNLI) question-answer entailment (QNLI), paraphrase (MRPC), question paraphrase (QQP), textual similarity (STS), sentiment (SST2), linguistic acceptability (CoLA), and Winograd Schema (WNLI).
Revised: We use the General Language Understanding Evaluation (GLUE) benchmark (Wang et al., 2019), which consists of 9 natural language understanding tasks on English data. Tasks cover textual entailment (RTE and MNLI) question-answer entailment (QNLI), paraphrase (MRPC), question paraphrase (QQP), textual similarity (STS), sentiment (SST-2), linguistic acceptability (CoLA), and Winograd Schema (WNLI).

Original: Training Details.
Revised: Training Details.

Original: Rather than simply shufﬂing the datasets for our multi-task models, we follow the task sampling procedure from Bowman et al.
Revised: Rather than simply shufﬂing the datasets for our multi-task models, we follow the task sampling procedure from Bowman et al.

Original: (2018), where the probability of training on an example for a particular task τ is proportional to |D τ | 0 .
Revised: (2018), where the probability of training on an example for a particular task τ is proportional to |D τ | 0 .

Original: This ensures that tasks with very large datasets don’t overly dominate the training.
Revised: This ensures that tasks with very large datasets don’t overly dominate the training.

Original: We also use the layerwise-learning-rate trick from Howard and Ruder (2018).
Revised: We also use the layerwise-learning-rate trick from Howard and Ruder (2018).

Original: If layer 0 is the
Revised: If layer 0 is the

Original: NN layer closest to the output, the learning rate for a particular layer d is set to BASE LR · α d (i.e., layers closest to the input get lower learning rates).
Revised: NN layer closest to the output, the learning rate for a particular layer d is set to BASE LR · α d (i.e., layers closest to the input get lower learning rates).

Original: The intuition is that pre-trained layers closer to the input learn more general features, so they shouldn’t be altered much during training.
Revised: The intuition is that pre-trained layers closer to the input learn more general features, so they shouldn’t be altered much during training.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: For single-task models, we use the same hyperparameters as in the original
Revised: For single-task models, we use the same hyperparameters as in the original

Original: BERT experiments except we pick a layerwiselearning-rate decay α of 1.0 or 0.9 on the dev set for each task.
Revised: BERT experiments except we pick a layerwiselearning-rate decay α of 1.0 or 0.9 on the dev set for each task.

Original: For multi-task models, we train the model for longer (6 epochs instead of 3) and with a larger batch size (128 instead of 32), using α = 0 .
Revised: For multi-task models, we train the model for longer (6 epochs instead of 3) and with a larger batch size (128 instead of 32), using α = 0 .

Original: 9 and a learning rate of 1e-4.
Revised: 9 and a learning rate of 1e-4.

Original: All models use the BERT-Large pre-trained weights.
Revised: All models use the BERT-Large pre-trained weights.

Original: Reporting Results.
Revised: Reporting Results.

Original: Dev set results report the average score 7 on all GLUE tasks except WNLI, for which methods can’t outperform a majority baseline.
Revised: Dev set results report the average score (Spearman correlation for STS, Matthews correlation for CoLA, and accuracy for the other tasks) on all GLUE tasks except WNLI, for which methods can’t outperform a majority baseline.

Original: All results show the median score of at least 20 trials with different randoms seeds.
Revised: Results show the median score of at least 20 trials with different random seeds.

Original: We ﬁnd using a large number of trials is essential because results can vary signiﬁcantly for different runs.
Revised: We ﬁnd using a large number of trials is essential because results can vary signiﬁcantly for different runs.

Original: For example, standard deviations in score are over ±for CoLA, RTE, and MRPC for multi-task models.
Revised: For example, standard deviations in score are over ±for CoLA, RTE, and MRPC for multi-task models.

Original: Single-task standard deviations are even larger.
Revised: Single-task standard deviations are even larger.

Original:  Results
Revised:  Results

Original: Main Results.
Revised: Main Results.

Original: We compare models trained with single-task learning, multi-task learning, and several varieties of distillation in Table 1.
Revised: We compare models trained with single-task learning, multi-task learning, and several varieties of distillation in Table 1.

Original: While standard multi-task training improves over single-task training for RTE (likely because it is closely related to MNLI), there is no improvement on the other tasks.
Revised: While standard multi-task training improves over single-task training for RTE (likely because it is closely related to MNLI), there is no improvement on the other tasks.

Original: In contrast, Single → Multi knowledge distillation improves or matches the performance of the other methods on all tasks except STS, the only regression task in GLUE.
Revised: In contrast, Single → Multi knowledge distillation improves or matches the performance of the other methods on all tasks except STS, the only regression task in GLUE.

Original: We believe distillation does not work well for regression tasks because there is no distribution over classes passed on by the teacher to aid learning.
Revised: We believe distillation does not work well for regression tasks because there is no distribution over classes passed on by the teacher to aid learning.

Original: The gain for Single → Multi over Multi is larger than the gain for Single → Single over Single, suggesting that distillation works particularly well in combination with multi-task learning.
Revised: The gain for Single → Multi over Multi is larger than the gain for Single → Single over Single, suggesting that distillation works particularly well in combination with multi-task learning.

Original: Interestingly, Single → Multi works substantially better than Multi → Multi distillation.
Revised: Interestingly, Single → Multi works substantially better than Multi → Multi distillation.

Original: We speculate it may help that the student is exposed to a diverse set of teachers in the same way ensembles bene-
Revised: We speculate it may help that the student is exposed to a diverse set of teachers in the same way ensembles beneﬁt from a diverse set of models, but future work is required to fully understand this phenomenon. In addition to the models reported in the table, we also trained Single → Multi → Single → Multi models. However, the difference with Single → Multi was not statistically signiﬁcant, suggesting there is little value in multiple rounds of distillation. [Table]

Revised: [Table]

Revised: Devlin

Original: [Table]
Revised: [Table]

Original: Liu
Revised: al., [Table] Large [Table]

Revised: Devlin et al.,

Revised: [Table]

Revised: STILTs

Original: [Table]
Revised: [Table]

Original: ﬁt from a diverse set of models, but future work is required to fully understand this phenomenon.
Revised: Phang et al. [Table] [Table] DNN (Liu et al., 2019b)

Original: In addition to the models reported in the table, we also trained Single → Multi → Single → Multi models.
Revised: [Table] Span-Extractive BERT on STILTs [Table] Snorkel

Original: However, the difference with Single → Multi was not statistically signiﬁcant, suggesting there is little value in multiple rounds of distillation.
Revised: [Table] BERT

Original:  Overall, a key beneﬁt of our method is robustness: while standard multi-task learning produces mixed results, Single → Multi distillation consistently outperforms standard single-task and multitask training.
Revised: [Table] Large [Table] Overall, a key beneﬁt of our method is robustness: while standard multi-task learning produces mixed results, Single → Multi distillation consistently outperforms standard single-task and multitask training.

Original: We also note that in some trials single-task training resulted in models that score quite poorly (e.g., less than 91 for QQP or less than 70 for MRPC), while the multi-task models have more dependable performance.
Revised: We also note that in some trials single-task training resulted in models that score quite poorly (e.g., less than 91 for QQP or less than 70 for MRPC), while the multi-task models have more dependable performance.

Original: Test Set Results.
Revised: Test Set Results.

Original: We compare against recent work by submitting to the GLUE leaderboard.
Revised: We compare against recent work by submitting to the GLUE leaderboard.

Original: We use
Revised: We use

Original: Single → Multi distillation.
Revised: Single → Multi distillation.

Original: [Table]
Revised: Following the procedure used by BERT, we train multiple models and submit the one with the highest average dev set score to the test set. BERT trained 10 models for each task (80 total); we trained 20 multi-task models. Results are shown in Table 2.

Original: model achieves a state-of-the-art accuracy of 82. compared to existing published results.
Revised: Our work outperforms or matches existing published results that do not rely on ensembling.

Original: However, due to the variance between trials discussed under “Reporting Results,” we think these  numbers should be taken with a grain of salt, as they only show the performance of a single trial.
Revised: However, due to the variance between trials dis- cussed under “Reporting Results,” we think these test set numbers should be taken with a grain of salt, as they only show the performance of individual training runs (which is further complicated by the use of tricks such as dev set model selection).

Original: We believe signiﬁcance testing over multiple trials would be needed to have a deﬁnitive comparison. 
Revised: We believe signiﬁcance testing over multiple trials would be needed to have a deﬁnitive comparison. Single-Task Fine-Tuning .

Revised: A crucial difference distinguishing our work from the STILTs, Snorkel

Revised: MeTaL, and MT-DNN KD methods in Table 2 is that we do not single-task ﬁne-tune our model.

Original: Ablation Study.
Revised: That is, we do not further train the model on individual tasks after the multi-task training ﬁnishes. While single-task ﬁne-tuning improves results, we think to some extent it defeats the purpose of multi-task learning: the result of training is one model for each task instead of a model that can perform all of the tasks. Compared to having many single-task models, a multi-task model is simpler to deploy, faster to run, and arguably more scientiﬁcally interesting from the perspective of building general language-processing systems. We evaluate the beneﬁts of single-task ﬁnetuning in Table 3.

Revised: Single-task ﬁne-tuning initializes models with multi-task-learned weights and then performs single-task training.

Revised: Hyperparameters are the same as for our single-task models except we use a smaller learning rate of 1e-5.

Revised: While single-task ﬁne-tuning unsurprisingly improves results, the gain on top of Single → Multi distillation is small, reinforcing the claim that distillation provides many of the beneﬁts of singletask training while producing a single uniﬁed model instead of many task-speciﬁc models.

Original:  We show the importance of teacher annealing and the other training tricks in Table 3.
Revised: Ablation Study. We show the importance of teacher annealing and the other training tricks in Table 4.

Original: We found them all to signiﬁcantly  improve scores.
Revised: We found them all to signiﬁcantly [Table] [Table] improve scores.

Original: Interestingly, using pure distillation without teacher annealing (i.e., ﬁxing λ = 0 ) performs no better than standard multi-task learning.
Revised: Using pure distillation without teacher annealing (i.e., ﬁxing λ = 0 ) performs no better than standard multi-task learning, demonstrating the importance of the proposed teacher annealing method.

Original: Comparing combinations of tasks.
Revised: Comparing Combinations of Tasks.

Original: Training on a large number of tasks is known to help regularize multi-task models (Ruder, 2017).
Revised: Training on a large number of tasks is known to help regularize multi-task models (Ruder, 2017).

Original: A related beneﬁt of multi-task learning is the transfer of learned “knowledge” between closely related tasks.
Revised: A related beneﬁt of multi-task learning is the transfer of learned “knowledge” between closely related tasks.

Original: We investigate these  by comparing several models on the RTE tasks, including one trained with a very[Table] [Table] closely related task (MNLI) and one trained with fairly unrelated tasks (QQP,CoLA, and SST).
Revised: We investigate these two beneﬁts by comparing several models on the RTE task, including one trained with a very closely related task (MNLI, a much large textual entailment dataset) and one trained with fairly unrelated tasks (QQP,

Original:  We use Single → Multi distillation (Single → Single in the case of the RTE-only model).
Revised: CoLA, and SST). We use Single → Multi distillation (Single → Single in the case of the RTE-only model).

Original: Both sets of auxilliary tasks improve RTE performance, suggesting that both beneﬁts are playing a role in improving multi-task models.
Revised: Results are shown in Table 5. We ﬁnd both sets of auxiliary tasks improve RTE performance, suggesting that both beneﬁts are playing a role in improving multi-task models.

Original: Interestingly, RTE + MNLI alone slightly outperforms the model performing all tasks, perhaps because training on MNLI, which has a very large dataset, is already enough to sufﬁciently regularize the model.
Revised: Interestingly, RTE + MNLI alone slightly outperforms the model performing all tasks, perhaps because training on MNLI, which has a very large dataset, is already enough to sufﬁciently regularize the model.

Original:  Discussion and Conclusion
Revised:  Discussion and Conclusion

Original: We have shown that Single → Multi distillation combined with teacher annealing produces results consistently better than standard single-task or multi-task training.
Revised: We have shown that Single → Multi distillation combined with teacher annealing produces results [Table] All GLUE

Original:  Achieving robust multi-task gains across many tasks has remained elusive in previous research, so we hope our work will make multi-task learning more broadly useful within
Revised: [Table] consistently better than standard single-task or multi-task training. Achieving robust multi-task gains across many tasks has remained elusive in previous research, so we hope our work will make multi-task learning more broadly useful within

Original: NLP.
Revised: NLP.

Original: However, with the exception of closely related tasks with small datasets (e.g., MNLI helping RTE), the overall size of the gains from our multitask method are small compared to the gains provided by transfer learning from unsupervised tasks (i.e., BERT).
Revised: However, with the exception of closely related tasks with small datasets (e.g., MNLI helping RTE), the overall size of the gains from our multi-task method are small compared to the gains provided by transfer learning from self-supervised tasks (i.e., BERT).

Original: It remains to be fully understood to what extent “unsupervised pre-training is all you need” and where transfer/multi-task learning from supervised tasks can provide the most value. 
Revised: It remains to be fully understood to what extent “self-supervised pre-training is all you need” and where transfer/multi-task learning from supervised tasks can provide the most value. Acknowledgements

Revised: We thank Robin Jia, John Hewitt, and the anonymous reviewers for their thoughtful comments and suggestions.

Revised: Kevin is supported by a Google PhD

Revised: Fellowship.