F ree AL : Towards Human-Free Active Learning in the Era of Large Language Models (2023)

Abstract: Collecting high-quality labeled data for model training is notoriously time-consuming and labor-intensive for various NLP tasks. While copious solutions, such as active learning for small language models (SLMs) and prevalent in-context learning in the era of large language models (LLMs), have been proposed and alleviate the labeling burden to some extent, their performances are still subject to human intervention. It is still underexplored how to reduce the annotation cost in the LLMs era. To bridge this, we revolutionize traditional active learning and propose an innovative collaborative learning framework FreeAL to interactively distill and filter the task-specific knowledge from LLMs. During collaborative training, an LLM serves as an active annotator inculcating its coarse-grained knowledge, while a downstream SLM is incurred as a student to filter out high-quality in-context samples to feedback LLM for the subsequent label refinery. Extensive experiments on eight benchmark datasets demonstrate that FreeAL largely enhances the zero-shot performances for both SLM and LLM without any human supervision.

## introduction modern machine learning models typically require a huge collection of precisely labeled data, which can be a labor-intensive and time-consuming process. even worse, it can be unrealistic in some practical scenarios that demand much expertise, such as medical diagnosis and industrial applications. to this end, a plethora of approaches have been investigated to reduce the burden of annotation, including semi-supervised learning @xcite , learning with label noise @xcite , and so on. amongst them, active learning (ein-dor et al., 2020; @xcite is a prominent solution that interactively queries an external expert or oracle to mark the new data points that the model wants to learn from. these methods alleviate the labeling burden to some extent but still require human efforts in the annotation or construction of the oracle to start with. the recent prevalent large language models (llms) @xcite @xcite , such as chatgpt and palm @xcite , have exhibited strong zero-shot learning ability by proper prompt design, yet becoming a new remedy for data efficiency. even more inspiringly, llms emerge with the so-called in-context learning (icl) @xcite ability to learn from a few task-related labeled samples for boosted performance. despite the promise, some studies @xcite find that llms tend to underperform compared to finetuned small language models (slms) on challenging tasks, which is also verified in our empirical studies ( in this work, we present a novel collaborative learning paradigm freeal that revolutionizes traditional active learning by interactively distilling and filtering the task-related knowledge from the llms. our intuition is that, while llms are hard to fine-tune, they are competent zero-shot learners @xcite and can provide coarse-grained knowledge for downstream tasks. on the other hand, slms are effective weak learners @xcite ) that can distill valuable clean samples from noisy supervision. to integrate llms and slms synergistically as a whole, we design a collaborative training framework where llm operates as an active annotator infusing its knowledge and the slm acts as a student to filter out the high-quality input-label pairs to feed back the llm for subsequent label refinery. empirically, freeal iteratively boosts the unsupervised performance of both slms and llms during collaborative training for transductive and inductive settings. as depicted in figure 1 , freeal allows us to achieve an extraordinary annotation-performance trade-off by obtaining competitive results on par with the supervised counterparts while fully eliminating human annotation costs. our results prove the feasibility of human-free active labeling in the era of llms. ## background we consider unsupervised classification tasks without human annotations. given an unlabeled training dataset d @xmath0, where @xmath1, . . . , c} is inaccessible. our task is to predict the true label for both the training dataset d train and test dataset d test . our framework employs a pre-trained large language model (llm) p and a downstream small language model (slm) s. for the llm, we define a natural language template t (•) which contains additional task-related information and a verbalizer v (•) which maps each class label in {1, . . . , c} to a pre-defined token in the prompt. for the finetuning of slm s with parameters θ, we adopt the cross entropy loss l @xmath2, θ) for training, where s j (x i , θ) is the j-th entry of slm's output softmax probability for the input x i with the pseudo label ỹi j . few-shot in-context learning. when supervised data are available, we can directly employ the few-shot icl for inference. in concrete, given a demonstration supporting pool d @xmath3, ỹdemo i } m @xmath4, we construct a prompt including a test input x test and m-shot in-context examples {(x demo j , ỹdemo j )} m @xmath5. the final prompt steers the llm and the prediction is obtained via, despite the simplicity, the success of icl largely hinges on the demonstration pool d demo , which requires human efforts of careful annotation for every individual scenario and can be particularly annoying for challenging tasks. to bridge this gap, we resort to our proposed plug-and-play method freeal without involving any human supervision. ## freeal in this section, we introduce our proposed framework freeal which investigates the opportunity for human-free active learning in the llms era. in contrast to traditional active learning that requests human annotation in each training loop, freeal employs llms as weak annotators. in each training loop, we alternate the following steps: the overview of the freeal framework is displayed in figure 2 and its overall pipeline is also shown in algorithm 1. in what follows, we will elaborate on our freeal framework minutely. ## conclusion in this work, we overhaul the traditional active learning in the era of llms and propose a novel framework called freeal that merely relies on the knowledge of llms to enhance human-free generalization performance. the key idea of freeal is to distill and filter the task-related knowledge from llms with a collaborative framework, where the llm is employed as an active annotator and the slm is engaged as a weak learner to filter out valuable samples for label refinery. the empirical results indicate that freeal can largely improve unsupervised performance and reaches comparable performance with supervised rivals in some tasks. while our freeal framework operates autonomously without human supervision, it is flexible and can be easily boosted with additional limited human supervision, which we leave for our future work. we hope that our work can spark heightened interest in developing new active annotation algorithms in the era of llms.