Let the Model Decide its Curriculum for Multitask Learning (2022)

Abstract: Curriculum learning strategies in prior multitask learning approaches arrange datasets in a difficulty hierarchy either based on human perception or by exhaustively searching the optimal arrangement. However, human perception of difficulty may not always correlate well with machine interpretation leading to poor performance and exhaustive search is computationally expensive. Addressing these concerns, we propose two classes of techniques to arrange training instances into a learning curriculum based on difficulty scores computed via model-based approaches. The two classes i.e Dataset-level and Instance-level differ in granularity of arrangement. Through comprehensive experiments with 12 datasets, we show that instance-level and dataset-level techniques result in strong representations as they lead to an average performance improvement of 4.17% and 3.15% over their respective baselines. Furthermore, we find that most of this improvement comes from correctly answering the difficult instances, implying a greater efficacy of our techniques on difficult tasks

## introduction in recent times, multi-task learning (mtl) @xcite i.e. developing one generalist model capable of handling multiple tasks has received significant attention from the nlp community @xcite @xcite @xcite . developing a single model in mtl has several advantages over multiple specialist models as it (i) can leverage knowledge gained while learning other tasks and perform better in limited-data scenarios @xcite , (ii) prevents overfitting to a single task, thus providing a regularization effect and increasing robustness @xcite , and (iii) provides storage and efficiency benefits because only one model needs to be maintained for all the tasks @xcite . prior work has shown that presenting training instances ordered by difficulty level benefits not only humans but also machines @xcite . arranging instances in a difficulty hierarchy i.e curriculum learning (easy to hard) and anti-curriculum learning (hard to easy) has been studied in mtl setup @xcite . these techniques arrange datasets either based on human perception of difficulty or by exhaustively searching the optimal arrangement. however, both these approaches have several limitations. firstly, human perception of difficulty may not always correlate well with machine interpretation, for instance, a dataset that is easy for humans could be difficult for machines to learn or vice-versa. secondly, exhaustive search is computationally expensive and becomes intractable as the number and size of datasets increase. in this work, we propose two classes of techniques that enable models to form their own learning curriculum in a difficulty hierarchy. the two classes i.e dataset-level and instance-level differ in the granularity of arrangement. in dataset-level techniques, we arrange datasets based on the average difficulty score of their instances and train the model sequentially such that all the instances of a dataset are learned together. in instance-level techniques, we relax the dataset boundaries and order instances solely based on their difficulty scores. we leverage two model-based approaches to compute the difficulty scores (section 2). we experiment with 12 datasets covering various nlp tasks and show that instance and dataset-level techniques result in stronger representations with an average performance gain of 4.17% and 3.15% over their respective baselines. furthermore, we analyze model predictions and find that difficult instances contribute most to this improvement implying greater effectiveness of our techniques on difficult tasks. we note that our techniques are generic and can be employed in any mtl setup. in summary, our contributions are as follows: 1. incorporating machine interpretation of difficulty in mtl: we introduce a novel framework for mtl that goes beyond human intuition of sample difficulty and provides model the flexibility to form its own curriculum at two granularities: instance-level and dataset-level. 2. performance improvement: we experiment with 12 nlp datasets and show that instance and dataset-level techniques lead to a considerable performance improvement of 4.17% and 3.15%. we note that our curriculum arrangement techniques can be used in conjunction with other multi-task learning methods such as dynamic sampling @xcite and prefinetuning @xcite to further improve their performance. 3. efficacy on difficult instances: our experiments in low-data regime reveal that the proposed techniques are most effective on difficult instances. this makes them more applicable for real-world tasks as they are often more difficult than abstract toy tasks and provide limited training instances. ## difficulty score computation in this section, we describe two model-based difficulty computation methods based on recent works. ## proposed techniques addressing the limitations of current approaches highlighted in section 1, we propose two classes of techniques to arrange training instances that allow models to form the learning curriculum based on their own difficulty interpretation. the technique classes i.e dataset-level and instance-level leverage difficulty scores computed using methods described in section 2 and follow the general training structure shown in algorithm 1. the training dataset d is divided into k splits (s 1 , ..., s k ) based on the difficulty score, and model m is trained sequentially on these ordered splits. furthermore, while training the model on split s i , a fraction (f rac) of instances from previous splits (s j (@xmath0.e forgetting the previous splits while learning a new split. note that d is a collection of multiple datasets in the mtl setup. the final step requires training on the entire dataset d as the evaluation sets often contain instances of all tasks and difficulty levels. dataset-level and instance level techniques vary in the way splits (s 1 , ..., s k ) are created as described below: dataset-level techniques: in this technique class, each dataset represents a split and is arranged based on the average difficulty score of its instances i.e score of a dataset d k is calculated as: instance-level techniques: here, we relax the dataset boundaries and arrange instances solely based on their difficulty scores. we study two approaches of dividing instances into splits (s 1 , ..., s k ): uniform and distribution-based splitting. in the former, we create k uniform splits from d, while in the latter, we divide based on the distribution of scores such that instances with similar scores are grouped in the same split 1 . the latter approach can result in unequal split sizes as we show in figure 1 that the number of instances varies greatly across difficulty scores. implementation details: we use three values of f rac: 0, 0.2, and 0.4 (refer algorithm 1), @xmath1, and @xmath2. for distribution-based splitting, we experiment by dividing d into 3 and 5 splits 1 . these hyper-parameters are selected based on development dataset performance. baseline methods: in mtl, heterogeneous batching where all the datasets are combined and a batch is randomly sampled has been shown to be much more effective than homogeneous and partitioned batching strategies @xcite . therefore, we use it as the baseline for instance-level techniques. for dataset-level techniques, we generate multiple dataset orders and take the average performance as the baseline. we average these baseline scores across 3 different runs. ## results and analysis performance improvement: instance and dataset-level techniques achieve an average improvement of 4.17% and 3.15% over their respective baseline methods. this improvement in consistent across all the datasets and also outperforms single-task performance in most cases. furthermore, we find that models leveraging average confidence method (2.2) outperform their counterparts using the cross review method (2.1) 1 rendering average confidence approach as more effective both in terms of performance and computation as cross review requires training multiple models (one for each meta-dataset). uniform vs distribution based splitting: in instance-level experiments, distribution-based splitting shows slight improvement over uniform splitting. we attribute this to the superior inductive bias resulting from the collation of instances with similar difficulty scores to the same split. effect of adding instances from previous splits: for dataset-level techniques, we find that it does not provide any improvement. this is because all the instances of a dataset are grouped in a single split therefore, adding instances from other splits doesn't contribute much to the inductive bias. furthermore, in the case of instance-level, it leads to a performance improvement because previous splits contain instances of the same dataset hence, providing the inductive bias. ## conclusion in this paper, we proposed two classes of techniques for mtl that allow models to form the learning curriculum based on their own interpretation of difficulty. comprehensive experiments with 12 datasets showed that our techniques lead to a performance improvement of 4.17% and 3.15%. furthermore, we found that difficult instances contribute most to this improvement, implying a greater efficacy of our techniques on difficult tasks. we hope that our techniques and findings will foster development of stronger multi-task learning models as our curriculum arrangement techniques can be used in conjunction with other multi-task learning methods such as dynamic sampling @xcite and pre-finetuning @xcite to further improve their performance. context -question datasets c: kyle doesn't wear leg warmers to bed, while logan almost always does. he is more likely to live in a colder climate. false, or true ? q: kyle is more likely to live in a colder climate. winogrande c: in order for an elevator to be legal to carry passengers in some jurisdictions it must have a solid inner door. false, or true ? q: what is another name for a freight elevator? does the context sentence contain answer to this question ? qnli c: what makes a great problem solver? false, or true? qqp, mrpc, q: how can i be a fast problem solver? are the two sentences semantically equivalent? paws c: i sell miscellaneous stuff in local fairs . contradiction, or neutral, or entailment ? q: i used to work a 9 5 job as a telemarketer . consistency of the dialogues ? dnli c: 205 total tajima' s are currently owned by the dealership. contradiction, or neutral, entailment ? q: less than 305 total tajima' s are currently owned by the dealership. equate c: two collies are barking as they play on the edge of the ocean contradiction, or neutral, or entailment ? snli, mnli, anli q: two dogs are playing together.