Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point Process (2024)

Abstract: In-context learning (ICL) is a few-shot learning paradigm that involves learning mappings through input-output pairs and appropriately applying them to new instances. Despite the remarkable ICL capabilities demonstrated by Large Language Models (LLMs), existing works are highly dependent on large-scale labeled support sets, not always feasible in practical scenarios. To refine this approach, we focus primarily on an innovative selective annotation mechanism, which precedes the standard demonstration retrieval. We introduce the Language Model-based Determinant Point Process (LM-DPP) that simultaneously considers the uncertainty and diversity of unlabeled instances for optimal selection. Consequently, this yields a subset for annotation that strikes a trade-off between the two factors. We apply LM-DPP to various language models, including GPT-J, LlaMA, and GPT-3. Experimental results on 9 NLU and 2 Generation datasets demonstrate that LM-DPP can effectively select canonical examples. Further analysis reveals that LLMs benefit most significantly from subsets that are both low uncertainty and high diversity.

## introduction as large pre-trained language models (llms) @xcite @xcite @xcite grow in scale, they not only exhibit enhanced linguistic capabilities and expanded world knowledge but also demonstrate a novel ability for in-context learning. specifically, llms have shown proficiency in learning from a limited set of input-output examples (known as demonstrations @xcite ), and effectively applying these learned mappings to new, unseen instances. this novel few-shot learning paradigm, which avoids parameter updates, has become a popular and efficient method for utilizing llms @xcite @xcite . previous studies have investigated which instances can serve as effective prompts for icl @xcite @xcite . they have demonstrated that retrieving specific similar contexts for individual test queries can significantly improve performance (instance level) and ground truth matters for support examples. to assign appropriate demonstrations to all test queries, support sets necessitate diversity and broad coverage, usually achieved through large labeled data, following the principle that monte carlo estimation accuracy improves with larger samples. nonetheless, these extensive datasets are often impractical to obtain. we investigate the selection of demonstrations from the perspective of active learning (al) @xcite . based on the core principle that not all data points are of equal value, al aims to identify the most effective instances in an unlabeled data pool for annotation. @xcite elucidates that high semantic similarity, low uncertainty, and high diversity comprise an effective and efficient annotation strategy. similarly, @xcite demonstrates that lower prompt perplexity is closely associated with better performance. while @xcite 's vote-k framework adopts a data-centric perspective (i.e., selecting examples that balance diversity and representativeness), it neglects the assessment of uncertainty and the inter-relationship among context examples. in this paper, we pursue a more universally applicable yet straightforward solution, incorporating confidence signals of llms to select annotation instances that are maximally diverse and exhibit low uncertainty. to address this need, we introduce a generic approach, lm-dpp, which jointly models uncertainty and diversity within the support set through a conditional determinantal point process. specifically, we employ llms' perplexity to score each candidate instance in the support set, which serves as a measure of the llms' uncertainty. then a gram matrix is constructed to balance the uncertainty and diversity of candidate instances and polynomialtime maximum a posteriori (map) inference @xcite is applied to identify the most useful subset of instances to be annotated. from the perspective of selective annotation, we consider extremely low-resource icl scenarios as those in which the available annotated examples are limited to a few dozen instances. our focus centers on identifying which specific set of demonstrations can most effectively harness the capabilities of llms within this challenging context. we validate our method through extensive experiments on 9 nlu and 2 generation datasets. we also demonstrate the versatility of lm-dpp by adapting it to the large language model gpt-3 (175b). the experimental results illustrate that our approach can effectively balance two critical factors, uncertainty and diversity. in summary, our contributions are as follows. ## methodology in this section, we introduce technical details of lm-dpp for selecting annotation instances exhibiting both high diversity and low uncertainty. formally, given a set of unlabeled samples @xmath0, lm-dpp aims to select a subset l ⊂ x for annotation, where |l| = m is the annotation budget, such that the language models (llms) maintains high icl performance on the test set d test . as shown in figure 2 , given a pre-trained language model (plm) g, we first score candidate instances x i using the perplexity of the llms ( §2.1). we then compute vector representations for the candidate instances, utilizing a conditional kernel matrix to balance diversity and low uncertainty ( §2.2). subsequently, we perform a greedy map inference algorithm to filter the candidate annotation set ( §2.3). ## related work and background determinantal point process the determinantal point process (dpp) is an elegant probabilistic model that captures negative correlations and allows for efficient algorithms in sampling, marginalization, and conditioning @xcite . formally, a point process p is a probability measure on the power set of v, that is, the set of all discrete items 2 v . if y is a random subset drawn according to p, then for every s ⊆ y : for some kernel matrix @xmath1, real and positive semidefinite. l s denotes the submatrix of l obtained by restricting to the rows and columns indexed by s. the operator det(•) represents the determinant of a matrix. typically, the dpp kernel l can be written as a gram matrix, l @xmath2, a j ), where k(•, •) is the kernel associated with the determinantal point process, often expressed as ϕ(a i ) t ϕ(a j ), ϕ is the feature map of a reproducing kernel @xcite . under distribution p, our objective is maximum a posteriori (map) inference, which is to find the subset of items with the highest probability, corresponding to the most diverse subset of items. although finding the mode of a dpp is np-hard, pioneering works @xcite @xcite have largely relied on greedy algorithms or sampling methods, and have succeeded in performing greedy map inference within polynomial time. in-context learning the capacity for in-context learning has been observed in large-scale pretrained language models (plms) such as gpt-3, representing a few-shot learning paradigm that does not require any parameter updates. it involves pre-pending a small number of demonstrations as prompts before the test input, allowing llms to discern patterns and "learn" to predict. formally, let x be the test query to be addressed, and s(•, •) be the cosine similarity. standard icl prompts the language model g with a set of example input-output pairs {(x 1 , y 1 ) . . . (x m , y m )} and predicts the answer ŷ for the query. typically, the pairs (x i , y i ) are retrieved from a train set d within the same domain through similarity. recent works have aimed to enhance icl by selecting valuable demonstrations @xcite , optimizing the order of demonstrations @xcite , etc. @xcite utilize selective annotation to significantly reduce annotation costs while ensuring high icl performance. @xcite explore the corpus-level in-context learning via dpp and mention the need to use gold labels to score candidate samples. ceil @xcite train the demonstration retriever with a learnable conditional dpp. however, these existing works are highly dependent on large annotated support sets. ## conclusion and future work in this work, we focus primarily on an innovative selective annotation mechanism and introduce an efficient annotation practice, lm-dpp. it selects both diverse and low-uncertainty examples for annotation and demonstrates promising results in various lms. moreover, empirical results validate the generalizability of lm-dpp across model size and annotation budget scaling. in the future, we plan to apply lm-dpp to more nlp tasks and explore annotation-free selection methods.