Active Learning Principles for In-Context Learning with Large Language Models (2023)

Abstract: The remarkable advancements in large language models (LLMs) have significantly enhanced predictive performance in few-shot learning settings. By using only a small number of labeled examples, referred to as demonstrations, LLMs can effectively perform the task at hand through in-context learning. However, the process of selecting demonstrations for maximizing performance has received limited attention in prior work. This paper addresses the issue of identifying the most informative demonstrations for few-shot learning by approaching it as a pool-based Active Learning (AL) problem over a single iteration. We compare standard AL algorithms based on uncertainty, diversity, and similarity, and consistently observe that the latter outperforms all other methods, including random sampling. Our extensive experimentation involving a diverse range of GPT and OPT models across 24 classification and multi-choice tasks, coupled with thorough analysis, unambiguously demonstrates the importance of using demonstrations that are semantically similar to the domain of the test examples. In fact, we show higher average classification performance using “similar” demonstrations with GPT-2 (124M) than random demonstrations with GPT-Neox (20B). Notably, while diversity sampling shows promise, uncertainty sampling, despite its success in conventional supervised learning AL scenarios, performs poorly in in-context learning.

## introduction the field of natural language processing (nlp) has recently witnessed a remarkable paradigm shift with the emergence of in-context learning with large language models (llms), also referred to as few-shot learning @xcite . traditionally, nlp systems heavily relied on supervised learning approaches, where large amounts of labeled training data were necessary to achieve high * work done during an internship at fair, meta. predictive performance. however, in-context learning has changed this status-quo by enabling llms to learn from limited, context-specific examples and adapt to new tasks and domains with remarkable proficiency @xcite @xcite @xcite . unlike more traditional approaches, which require extensive retraining or fine-tuning for every new task, in-context learning empowers llms to generalize from a few examples that are fed to the model through prompting to learn a new task at hand, without any weight updates. the data efficiency of few-shot in-context learning of llms is indeed remarkable with only a small number of demonstrations. 1 still, such demonstrations constitute labeled data examples, raising two key questions: (1) when faced with tasks where there is only unlabeled data available, how can we select the most appropriate samples to label and then use as in-context demonstrations? (2) when we have labeled data for a given task, how can we efficiently identify the most informative combination of demonstrations for in-context learning? answering these questions is essential to ensure effective and efficient few-shot learning using llms. a growing line of work has investigated how incontext learning works @xcite @xcite , which demonstrations to use @xcite @xcite , how to form the prompt @xcite @xcite and whether ground truth labels matter @xcite @xcite @xcite . still, to the best of our knowledge, no prior work has explored the problem of in-context demonstration selection explicitly through the lens of active learning (al). based on the core principle that not all data points are equally useful, al @xcite aims to identify the most informative instances from a pool of unlabeled data for annotation. iterating through model training, data acquisition and human annotation, the goal is to achieve data efficiency. a data-efficient al algorithm ensures that a model achieves satisfactory performance on a withheld test set by selecting only a small fraction of the unlabeled data for annotation that typically is better than randomly selecting and annotating data of equal size. in this paper, our main aim is to redefine the concept of data efficiency within the framework of in-context learning inspired by conventional active learning settings. for this purpose, we assume that given a pool of labeled or unlabeled data, the objective is to identify a set of k examples that will serve as demonstrations to an llm, resulting in optimal performance on a held-out test set. given this formulation of data efficiency, we explore the effectiveness of the most prevalent al approaches based on uncertainty @xcite @xcite , diversity @xcite @xcite and similarity @xcite @xcite , as demonstration selection methods for in-context learning (figure 1 ). our key contributions are as follows: ## experimental setup models we evaluate 15 llms in total, 8 models from the gpt @xcite @xcite and 7 from the opt @xcite family. we choose our models to span from a few million to tens of billions parameters, as we want to study how the model size affects the effectiveness of in-context example selection methods. all models considered in this work are publicly available. tasks & datasets following @xcite , we evaluate all llms in 15 classification and 9 multi-choice tasks taken from the crossfit @xcite benchmark. we provide details for all tasks and datasets considered in the appendix a.1. in-context learning prompting unless specified otherwise, we sample @xmath0, i.e., labeled data, from the pool with each al method. after collecting the k input-label pairs, we concatenate them all together with the test example that we want to make a prediction for to form the llm prompt (figure 2 ). our implementation, including prompt verbalizers, is based on those by @xcite and @xcite . ## results figure 3 shows the results on few-shot incontext learning across all data acquisition methods (random, diversity, uncertainty and similarity), model families (gpt and opt) and tasks (classification and multi-choice question answering). 3 overall, we observe the anticipated trend of performance enhancement with increasing scale, particularly notable in the multi-choice tasks for both opt and gpt models. still, the most remarkable finding is the substantial performance improvement achieved by selecting similar in-context examples for few-shot learning, particularly in classification tasks. this observation aligns with the findings reported by @xcite , who demonstrated similar patterns in sentiment analysis tasks with gpt-3. our results indicate that the selection of appropriate demonstrations can hold greater significance than the number of model parameters, at least within the scope of the models evaluated in this study. in multi-choice tasks, similarity is also the top-performing acquisition method, while the other three approaches exhibit closely competitive performance. the data selection method based on diversity is consistently the second best approach after similarity (with very few exceptions in the multichoice tasks for opt models). even though it is not the top performing method, we can consider that consistently outperforming random sampling is a strong signal that diversity in the demonstrations is a characteristic of effective demonstrations. @xcite explore the setting of compositional generalization, where models are tested on outputs with structures that are absent from the training set and thus selecting similar demonstrations is insufficient. they show that combining diverse demonstrations with in-context learning substantially improves performance for the task of compositional generalization semantic parsing. remarkably, uncertainty sampling, typically regarded as one of the best approaches for traditional supervised al @xcite @xcite , exhibits the lowest performance. this finding contradicts the conventional al principles that suggest selecting a few highly uncertain labeled data points for data efficiency. similar to our findings, gonen et al. (2022) explore the performance variabilty of difsimilarity is consistently the best performing approach overall, followed by diversity and random. interestingly, we observe that uncertainty sampling underperforms in this setting of in-context learning. ## conclusion in this study, we have examined the selection of demonstrations, i.e., labeled data that provide examples of solving a task, for in-context learning with llms. we formulated the selection process as a single iteration active learning problem and evaluated four standard approaches: uncertainty, diversity, similarity, and random sampling. our evaluation involved 15 models of varying size from the gpt and opt families, encompassing 15 classification tasks and 9 multi-choice tasks. through extensive experimentation, we have demonstrated that selecting demonstrations that are semantically similar to the test input examples consistently outperforms all other methods by a significant margin across all model families, sizes, and tasks. this corroborates findings of several previous and concurrent studies that explore the properties of "good" in-context examples @xcite . interestingly, our findings reveal that uncertainty sampling, although effective in supervised learning, underperforms in the in-context learning paradigm. this highlights the importance of our work in exploring the principles of active learning in the context of few-shot learning.