Few-shot initializing of Active Learner via Meta-Learning (2022)

Abstract: Despite the important evolutions in few-shot and zero-shot learning techniques, domain specific applications still require expert knowledge and significant effort in annotating and labeling a large volume of unstructured textual data. To mitigate this problem, active learning, and meta-learning attempt to reach a high performance with the least amount of labeled data. In this paper, we introduce a novel approach to combine both lines of work by initializing an active learner with meta-learned parameters obtained through meta-training on tasks similar to the target task during active learning. In this approach we use the pre-trained BERT as our text-encoder and meta-learn its parameters with LEOPARD, which extends the model-agnostic meta-learning method by generating task dependent softmax weights to enable learning across tasks with different number of classes. We demonstrate the effectiveness of our method by performing active learning on five natural language understanding tasks and six datasets with five different acquisition functions. We train two different meta-initializations, and we use the pre-trained BERT base initialization as baseline. We observe that our approach performs better than the baseline at low budget, especially when closely related tasks were present during meta-learning. Moreover, our results show that better performance in the initial phase, i.e., with fewer labeled samples, leads to better performance when larger acquisition batches are used. We also perform an ablation study of the proposed method, showing that active learning with only the meta-learned weights is beneficial and adding the meta-learned learning rates and generating the softmax have negative consequences for the performance.

## introduction in recent years, transformer-based models such as bert @xcite have been very successful in achieving high performance in natural language processing (nlp) tasks. these results are achieved by training with a significant amount of labeled data, which is often necessary to optimize a large number of weights in these types of models during the fine-tuning stage. this is a major obstacle because many machine learning applications lack widely available labeled data and the labeling task in high volumes can be tedious and expensive. two principal research areas to overcome this obstacle are active learning (al) and few-shot learning (fsl). few-shot learning was initially introduced to simulate the human ability to generalize quickly with only a few labeled examples @xcite . thus, the goal is to reach the highest possible performance with a small number of labelled data points (e.g., 4, 8, 16, . . . ). the field has made great progress after the introduction of optimization-based few-shot learning @xcite using the idea of meta-learning (ml) @xcite . the basic principle of meta-learning in this context is to allow the neural network to utilize the knowledge acquired from multiple tasks, represented in the network by its weights, for adaptation in new tasks. hence, initializing the network with weights learned from a variety of tasks can enable faster learning of similar tasks. the active learning field approaches this problem by only partially annotating the unlabeled data while attempting to achieve the highest performance. this is done by iteratively selecting a subset of unlabeled data points to be annotated by an "oracle" such that the selected points offer the highest learning benefit according to some metric, e.g., representativeness, diversity, uncertainty @xcite . in this work, we introduce a novel method to extend active learning with meta-learning to minimize the number of new data points that need to be annotated for achieving good performance. we do this by learning a favorable model initialization, via meta-learning, for the target task during active learning. we show a general approach on what to transfer to the active learning model and what to leave out. we demonstrate the effectiveness of our methodology by showing that it achieves higher performance than the baseline initialization on different natural language understanding tasks and datasets with the same number of annotation queries to the oracle; or eventually performs equally but by requiring fewer queries to the oracle. these results also show that our approach offers an advantage during active learning when larger annotation queries are used. importantly, we show that performance improvement is significantly enhanced when tasks based on similar principle are available, especially in a cold-start setting. ## related work active learning research focuses on developing novel acquisition functions for selecting data points to be annotated by the oracle. the primary metrics on which the acquisition functions act are uncertainty @xcite , diversity @xcite , and representativeness @xcite . furthermore, acquisition functions that incorporate a multitude of metrics achieve better performance @xcite . nevertheless, there is no capture function that consistently performs the best across different datasets or query sizes @xcite . which is why we implemented different acquisition functions for our methodology. active learning and meta-learning have been used in the same context before, but the role of meta-learning has been treated as an acquisition function @xcite . several of these works show that meta-learned acquisition functions are effective in a cold-start environment @xcite , which is mainly applied in computer vision. in this work, meta-learning is used to initialize the active learner and we show that it is effective in a cold-start environment and in a low-budget setting. there has been one similar work from @xcite , in the context of chemical peptide design. the mentioned work focuses on twelve different, but closely related tasks and uses metalearning to optimize the initial parameters for active learning of the twelve tasks. however, the methodology is not entirely explained. in the current work, we do give a clear methodology of combining meta-learning and active learning, which is broadly applicable in the machine learning domain. we show this within the nlp context using a wide range of tasks. a somewhat similar approach is using domain adaption to improve active learning @xcite . the difference with our proposed method is that meta-learning is able to learn an initialization for fast adaptation using multiple tasks and domains. on the other hand domain adaptation is only able to learn common features between two domains in the same task and is therefore not comparable. ## ablation study 6.1 generated softmax and learning rates to see the effects of using the generated softmax weights and the meta-learned learning rates α during active learning, we perform the active learning experiment with these elements included and initialized with θ * t , where @xmath0, @xmath1. as mentioned in section 3.3, α can become negative dur-ing meta-learning and figure 5 shows that there are a high density of negative learning rates after meta-learning on t . for these experiments, we set 0.010 0.005 0.000 0.005 0.010 learning rate 0 10 0 10 1 density any negative learning rate to α = 2e -5. the results are shown in figure 6 . here, we observe that after an initial gain in performance, the model seems to stop to learn. the average performance fluctuates around accuracy of 83%. we suspect this is caused by learning with learning rates above 1e -4 (see figure 5 ) with imbalanced batches during the start of training. for example, 0 100 200 300 400 500 600 700 800 900 1000 additional samples 80 82 84 86 88 accuracy (%) ## conclusion in this work, we have shown how to initialize the active-learner with meta-trained parameters on related tasks. this method provides a significant boost in performance in a low-budget setting. this effect is exaggerated during the early stages of active learning, due to fast adaptation capabilities by being able to few-shot learn. furthermore, the ability to few-shot learn provides better representations in the encoded space, making the scoring from implemented acquisition functions more accurate. which consequently, results in better active learning performance using larger acquisition batches.