Multi-task Active Learning for Pre-trained Transformer-based Models (2022)

Abstract: Multi-task learning, in which several tasks are jointly learned by a single model, allows NLP models to share information from multiple annotations and may facilitate better predictions when the tasks are inter-related. This technique, however, requires annotating the same text with multiple annotation schemes, which may be costly and laborious. Active learning (AL) has been demonstrated to optimize annotation processes by iteratively selecting unlabeled examples whose annotation is most valuable for the NLP model. Yet, multi-task active learning (MT-AL) has not been applied to state-of-the-art pre-trained Transformer-based NLP models. This paper aims to close this gap. We explore various multi-task selection criteria in three realistic multi-task scenarios, reflecting different relations between the participating tasks, and demonstrate the effectiveness of multi-task compared to single-task selection. Our results suggest that MT-AL can be effectively used in order to minimize annotation efforts for multi-task NLP models.1

## introduction deep neural networks (dnns) have recently achieved state-of-the-art results for many natural language processing (nlp) tasks and applications. of particular importance are contextualized embedding models @xcite , most of which implement transformer-based architectures with the self-attention mechanism @xcite @xcite . nevertheless, dnns often require large labeled training sets in order to achieve good performance. while annotating such training sets is costly and laborious, the active learning (al) paradigm aims to minimize these costs by iteratively selecting valuable training examples for annotation. recently, al has been shown effective for dnns across various nlp tasks @xcite @xcite ). an appealing capability of dnns is performing multi-task learning (mtl): learning multiple tasks by a single model @xcite . this stems from their architectural flexibility-constructing increasingly deeper and wider architectures from basic building blocks-and in their gradient-based optimization, which allows them to jointly update parameters from multiple task-based objectives. indeed, mtl has become ubiquitous in nlp @xcite . mtl models for nlp can often benefit from using corpora annotated for multiple tasks, particularly when these tasks are closely related and can inform each other. prominent examples of multi-task corpora include ontonotes @xcite , the universal dependencies bank @xcite , and streusle @xcite . given the importance of multi-task corpora for many mtl setups, effective al frameworks that support mtl are becoming crucial. unfortunately, most al methods do not support annotations for more than one task. multi-task al (mt-al) was proposed by @xcite before the neural era, and adapted by @xcite to a neural architecture. recently, @xcite proposed an mt-al model for slot filling and intent detection, focusing mostly on lstms @xcite . in this paper, we are the first to systematically explore mt-al for large pre-trained transformer models. naturally, our focus is on closely related nlp tasks, for which multi-task annotation of the same corpus is likely to be of benefit. particularly, we consider three challenging real-life multi-task scenarios, reflecting different relations between the participating nlp tasks: 1. complementing tasks, where each task may provide valuable information to the other task: dependency parsing (dp) and named entity recognition (ner); 2. hierarchically related tasks, where one of the tasks depends on the output of the other: relation extraction (re) and ner; and 3. tasks with different annotation granularity: slot filling (sf, token level) and intent detection (id, sentence level). we propose various novel mt-al methods and tailor them to the specific properties of the scenarios in order to properly address the underlying relations between the participating tasks. our experimental results highlight a large number of patterns that can guide nlp researchers when annotating corpora with multiple annotation schemes using the al paradigm. ## previous work this paper addresses a previously unexplored problem: multi-task al (mt-al) for nlp with pre-trained transformer-based models. we hence start by covering al in nlp and then proceed with multi-task learning (mtl) in nlp. ## task definition -multi-task active learning in the mt-al setup, the al algorithm is provided with a textual corpus, where an initial (typically small) set of n 0 examples is labeled for t tasks. the al algorithm implements an iterative process, where at the i-th iteration the goal of the al algorithm is to select n i additional unlabeled examples that will be annotated on all t tasks, such that the performance of the base nlp model will be improved as much as possible with respect to all of them. while such a greedy strategy of gaining the most in the i-th iteration may not yield the best performance in subsequent iterations, most al algorithms are greedy, and we hence follow this strategy here as well. we focus on the standard setup of confidencebased al, where unlabeled examples with the lowest model confidence are selected for annotation. algorithm 1 presents a general sketch of such al algorithms, in the context of mtl. this framework, first introduced by @xcite , is a simple generalization of the single-task al (st-al) framework, which supports the annotation of data with respect to multiple tasks. ## confidence estimation in single-task and multi-task active learning we now introduce the confidence scores that we consider for single-task (st-al) and multi-task (mt-al) active learning. these confidence scores are essentially the core of confidence-based al algorithms (see steps 2-3 of algorithm 1). in ## mt-al for complementing tasks we start by investigating mt-al for two closely related, complementing, syntactic tasks: dependency parsing (dp) and named entity recognition (ner), which are often solved together by a joint multi-task model @xcite . ## mt-al for hierarchically related tasks until now, we have considered tasks (dp and ner) that are mutually informative but can be trained independently of each other. however, other multi-task learning scenarios involve a task that is dependent on the output of another task. a prominent example is the relation extraction (re) task that depends on the output of the ner task, since the goal of re is to classify and identify relations between named entities. importantly, if the ner part of the model does not perform well, this harms the re performance as well. sample selection in such a setup should hence reflect the hierarchical relation between the tasks. ## mt-al for tasks with different annotation granularity nlp tasks are defined on different textual units, with the most common examples of sentence-level and token-level tasks. our last investigation considers the scenario of two closely related tasks that are of different granularity: slot filling (sf, token-level) and intent detection (id, sentence-level). due to the different annotation nature of the two tasks, we have to define the cost of example annotation with respect to each. naturally, there is no correct way to quantify these costs, but we aim to propose a realistic model. we denote the cost of annotating a sample for sf with cost @xmath0, where m is the number of tokens in the sentence, tp is a fixed token annotation cost (we set @xmath1. the cost of annotating a sample for id is next denoted with cost @xmath2, where ts is a fixed sentence cost (we set @xmath3. our solution (see below) allows some examples to be annotated only with respect to one of the tasks. for examples that are annotated with respect to both tasks we consider an additive joint cost where the token-level term m is considered only once: @xmath4. in our experiments, we allow a fixed annotation budget of @xmath5. ## overall comparison as a final evaluation, we turn to compare the performance of our proposed joint-selection mt-al methods in all setups. in our first setup ( §5) we implemented all mt-al selection methods with greedy selection, considering uniform task importance. thereafter ( §6 and §7), we showed how these selection methods can be modified in the next two setups by either integrating non-uniform task weights or replacing the greedy selection with a blp loss function overconfidence scores. however, not all of our mt-al selection methods can be modified in such ways. 13 we hence report our final comparison given the conditions applied in the first setup: assuming uniform task weights and selecting samples in a greedy manner. each selection method has its cons and pros, which we outline in our final discussion. 13 see footnotes 7 and 11 for further details. mt-r, not surprisingly, is the worst performing method on average, as it makes no use of the model's predictions. nevertheless, the method performs quite well on the third setup (sf + id) when compared to the other methods that were trained with the greedy decision method, the least successful decision method of this setup. next, mt-avg performs well when the tasks are of equal importance (dp + ner), but achieves only moderate performance on the other setups. surprisingly, mt-max is highly effective despite its simplicity. it is mostly beneficial for the first two setups (dp + ner and ner + re), where the tasks are of the same annotation granularity. it is the third best method overall, and it does not lag substantially behind the best method, mt-par. interestingly, mt-min, which offers a complementary perspective to mt-max, is on average the worst mt-al method, excluding mt-r, and is mainly beneficial for the first setup (dp + ner). the next mt-al method, mt-par, seems to capture well the joint confidence space of the task pairs. it is the best method on average, achieving high average scores in all setups. however, when incorporating it with other training techniques, such as applying non-uniform weights (for the second setup), it is outperformed by the other mt-al methods. mt-rrf does not lag far behind mt-par, achieving similar results on most tasks, excluding the re and id tasks, which are the higher-level tasks of their setups. finally, mt-ind does not excel in three of the four tasks of the first two setups, while achieving the best average results on ner, when jointly trained with re. furthermore, the method demonstrates strong performance on the third setup, when the tasks are of different annotation granularity, justifying the independent annotation selection in this case. ## conclusions we considered the problem of multi-task active learning for pre-trained transformer-based models. we posed multiple research questions concerning the impact of multi-task modeling, multi-task selection criteria, overconfidence reduction, the relationships between the participating tasks, as well as budget constraints, and presented a systematic algorithmic and experimental investigation in order to answer these questions. our results demonstrate the importance of mt-al modeling in three challenging real-life scenarios, corresponding to diverse relations between the participating tasks. in future work, we plan to research setups with more than two tasks and consider language generation and multilingual modeling.