Selective Annotation Makes Language Models Better Few-Shot Learners
Authors: Hongjin SU
Jungo Kasai
Chen Henry Wu
Weijia Shi
Tianlu Wang
Jiayi Xin
Rui Zhang
Mari Ostendorf
Luke Zettlemoyer
Noah A. Smith
Tao Yu
Venue: ICLR 2023 poster (2023)
Keywords: few-shot learning
language models
in-context learning
active learning

Abstract: Many recent approaches to natural language tasks are built on the remarkable abilities of large language models. Large language models can perform in-context learning, where they learn a new task from a few task demonstrations, without any parameter updates. This work examines the implications of in-context learning for the creation of datasets for new natural language tasks. Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time. Based on this framework, we propose an unsupervised, graph-based selective annotation method, voke-k, to select diverse, representative examples to annotate. Extensive experiments on 10 datasets (covering classification, commonsense reasoning, dialogue, and text/code generation) demonstrate that our selective annotation method improves the task performance by a large margin. On average, vote-k achieves a 12.9%/11.4% relative gain under an annotation budget of 18/100, as compared to randomly selecting examples to annotate. Compared to state-of-the-art supervised finetuning approaches, it yields similar performance with 10-100x less annotation cost across 10 tasks. We further analyze the effectiveness of our framework in various scenarios: language models with varying sizes, alternative selective annotation methods, and cases where there is a test data domain shift. We hope that our studies will serve as a basis for data annotations as large language models are increasingly applied to new tasks.

Published as a conference paper at ICLR 2023
SELECTIVE ANNOTATION MAKES LANGUAGE MODELS BETTER FEW-SHOT LEARNERS
Hongjin Su Jungo Kasai*
Chen Henry Wu
Weijia Shi
Tianlu Wang* Jiayi Xin
Rui Zhang* Mari Ostendorf
Luke Zettlemoyer
Noah A. Smith & Tao Yu *
The University of Hong Kong
University of Washington
Allen Institute for Al
Carmegie Mellon University
Penn State University
Meta Al
[hisu, tyueecs kku.kk, herrychenww@emu.cem edu, ostendor@uw.cem edu [jkasai, sw30419, 1sz, nasmith]@cs wwshington. edu
ABSTRACT
Many recent approaches to natural language tasks built on the remarkable abilities of large language models. Large language models can perform in-context learning, where they learn new task from a few task demonstrations, without any parameter updates. This work examines the implications of in-context Iearn- ing for the creation of datasets for new natural language tasks. Departing from recent in-context learning methods, we formulate an annotation-effcciet two-step framework: selective annotation that chooses pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task exam- ples from the annotated pool at test time. Based on this framework, we propose an unsupervised, graph-based selective annotation method, vote-k. to select diverse, representative examples to annotate. Extensive experiments on 10 datasets (cov- ering classification, commonsense reasoning, dialogue, and text/code generation) demonstrate that our selective annotation method improves the task performance by large margin. On average, vote-k achieves 12.9% relative gain under an annotation budget of 18/100, as compared to randomly selecting examples to supervised finetuning approaches, yields
annotate. Compared to state-of-the- similar performance with 10- x00 less annotation cost across 10 tasks. We further analyze the effectiveness of our framework in various scenarios: language models with varying sizes, alternative selective annotation methods, and cases where there is a test data domain shift. We hope that our studies will serve as a basis for data annotations as large language models are increasingly applied to new tasks
wookw Large labeed data
Step I (our focus) Selective Annotation before test time
Stop 2
Prompt Retrieval during test time
in-contect
ecamples
80
Large unlabeled data
Predict
Selected unabeled data
Test output
55
Small
Select
Annotate
Test irput
18
100
300
800
Annotation si2e
Figure I: Left: Our two-step framework for in-context learning. Instead of assuming access to large labeled data, we first select small number of (diverse and representative) unlabeled examples to annotate before test time. At test time, we retrieve in in-context examples from the small annotated pool. Right: In-context learning performance representative tasks (HellaSwag commonsense reasoning, MRPC paraphrase detection, and MWOZ. dialogue state tracking). Here we experiment with GPT-J and fodex-davinci--00 annotation methods are presented: random selection and our vote-k method. We observe that an appropriate selective annotation method largely improves the in-context learning performance with smaller variance over random selection under varying annotation budgets.
varying annotation budgets averaged over three
Two selective
Our code is available http://tthaacco////////////eeeee

---

Published as a conference paper at ICLR 2023
INTRODUCTION
Much recent work builds approaches to natural language tasks on the impressive abilities of large language models (e.g.. GPT-3; Brown et al. 2020). Large language models can perform downstream tasks by conditioning generation on a few task demonstrations, thereby avoiding the need for any parameter updates. This new, few-shot learning paradigm is called in-context learning and has become an attractive alternative to supervised finetuning (Liu et al. (021). In this work, we study the implications of this remarkable capability of large language models for dataset creation and annotation. We extensively examine how reduce the manual annotation cost while retaining high in-context learning performance.
Although in-context learning was originally proposed for few-shot learning, recent works show that retrieving prompts from a large set of annotated examples is necessary to achieve good performances (Liu et al. 2022; Rubin et al.) 2022). In partieular, they show that the performance substantially improves when similar examples (under some embedding function) are retrieved as in-context examples specifically for each test input (Liu al.) 2022). Each test sample only requires a few in-context examples in its prompt. Different test instances, however, require different in-context examples with their associated annotations. necessitating large set of annotated examples.
Distinct from these recent efforts. we establish a two-step framework to better understand and improve the annotation efficiency (Fig. [T: the first step is selective annotation that picks a small number of instances to get annotated before test time, followed by prompt retrieval that retrieves in-context examples for each test instance from the annotated data. The total annotation budget is the number of examples selected and annotated in the first step. The second step is bounded by the number of examples that can fit as input to language model. Based on this framework, we propose an unsupervised, graph-basee -ased selective annotation method, named vote-k, that selects diverse and representative instances to be annotated.
Our extensive experiments over 10 datasets across diverse tasks (covering classification, commonsense reasoning, dialogue, and text/code generation; see Tab. [ demonstrate that our graph- selective
annotation method, vote-k ( (.2 substantially improves the in-context learning performance by balancing the diversity and representativeness of annotated samples. For instance, vote-k, combined with similarity- prompt retrieval (Liu al. 2022 Rubin et al.) 2022), achieves 1 11.4% relative gain under a budget of 100 annotated examples and 12.9% relative gain when only 18 examples are annotated; 18 samples can fit into language models' input, meaning the prompt retrieval step is not needed. Moreover, the improvement is consistent across language models with varying sizes (2B-175B parameters) ($4.2). This finding is in contrast with finetuning, where we cannot see the effectiveness of selective annotation over random baseline, due to outliers (Karamcheti et al.) 2021) or training instability (D' Arcy & Downey 2022). We hypothesize that in-context learning with similarity-based prompt retrieval is more robust to small annotation sizes and outliers because only the most similar examples are retrieved for each test instance. Indeed, we observe that random prompt retrieval fails to benefit from selective annotation (4.44, providing support for hypothesis.
Besides performance comparisons within a fixed anmotation budget, we show that selective annotation provides better few-shot performance with 5-100x less annotation cost for new natural language tasks. In-context learning with 18 examples selected by vote-k achieves higher performance than 100 randomly selected examples on out of the 10 tasks. also outperforms strong finetuning methods by a large margin (Fig. 2) and requires 10-100x less annotations for similar performance ($4.I). We observe that in-context learning quickly (100 or 300 samples are annotated) converges decent performance when vote-k selective annotation is applied. These results suggest that large language models do not require large annotated datasets (e.g., 10K) due to their ability to adapt new tasks through simple prompting.
Selective annotation also makes in-context learning much more stable. In real-world scenarios, even collecting unlabeled data is non-trivial and introduces randomness. We simulate such randomness in our experimental setting by subsampling the original unlabeled data multiple times. Our results suggest that vote-k selective annotation largely reduces the variance of in-context learning even in this setting (Tab. 2. Further analysis shows larger improvements when there is a domain shift between training and test data (e.g.. text from different Amazon users; Koh al. 2021 $4.3) Finally, when compared with previous selective annotation methods designed for supervised trainingfinetuiing demonstrate that vote-k selective annotation consistently improves the performance ($4.5). As
2

---

Published as a conference paper at ICLR 2023
in-context learning has been applied to increasingly more natural language processing applications, we hope that our cmmotattionefffciien framework will provide useful guidance for both researchers and practitioners.
2 SELECTIVE ANNOTATION FOR IN-CONTEXT LEARNING
In-context learning only requires a few annotated examples per test instance (few-shot learning), while avoiding expensive finetuning the whole training data. It is, however, often assumed that all annotated training data are available for prompt retrieval (e.g.- et al., 2022 Rubin et al. (022). Yet the implied total annotation costs are hardly discussed in previous work. We develop better practice for few-shot learning with large language models by carefully studying the total annotation cost required for in-context learning. We also study how examples should be selected annotate, in order to make in-context learning perform better for new tasks. We formulate general framework (Fig. leftt that consists of two steps: selective annotation
1 and prompt retrieval ($2.2)
2.1
SELECTIVE ANNOTATION
The first step chooses examples to annotate before test time. This process thus determines the total annotation budget. This selective annotation process is largely ignored in the recent literature for in-context learning. We will demonstrate, however, that the annotation cost can be substantially reduced by choosing a small set of diverse, representative examples, while retaining the downstream performance ) Formally, given set of unlabeled samples X' {2²} ) selective annotation aims at selecting a subset C to be annotated, where [ = M is the annotation budget. We discuss vote-k selective annotation method and other selective annotation baselines below.
Vote-k The goal of selective annotation for in-context learning is to select diverse and representative examples; representativeness will help many test instances to find similar demonstrations, while diversity increases the total coverage. We develop vote-k, a graph- based method that promotes both diversity and representativenness. A detailed algorithm can be found in Appendix G We first compute a vector representation for each unlabeled training instance using Sentence-BERT (Reimers) Gurevych] 2019) by averaging the resulting vectors over the text input words P embedding vectors to create a directed graph G' = (V, E') where the vertices are the unlabeled instances X' as defined above. For each vertex v E V, we create an edge to its i nearest vertices in terms of the cosine similarity between the embeddings. Now let L and / denote the sets of already chosen (i.e., labeled) samples and remaining samples, respectively. Initially, = 0. Every vertex E U is scored by modified degree:
We then use the
score(u)=
Σ
((V), where = -=((EE((())))))) P p>l
vé(ll(e.)EE..ve))
where discounts v that is close to the already selected instances, thereby encouraging diversity. In every iteration, we take arg max score(u) and move it from L/ to L. We run M/10 these iterations; after this process, the current labeled L has M/10 samples (up to Line 7 in Algorithm Subsequently, we use L as the in-context learning examples for large language model, e.g..GP GPT (Wang & Komatsuzaki] 2021), and generate a prediction for every instance in U. We then compute the average log probability over the generation output as the model's confidence score (Line & to Line i0 Algorithm D}. We then partition U/ into M equal-sized buckets, based on their confidence scores (e.g., if M = 100, we group the unlabeled instances by percentile). We add to L the example with the maximum score from each of the first 9M/10 buckets (discarding the M //10 buckets with the most confident examples), resulting in C = M (Line 11 to Line 15 in Algorithm D. This further encourages diversity by selecting instances with varying confidence scores from in-context learning. We tuned & and in our preliminary experiments, and found that k= 150 and p= 10 perform well across many datasets Fl We will explore other selective annotation methods from prior work on active learning coreset selection ($4.5) and that vote-k outperforms these alternative methods.
Random and Other Selective Annotation Methods To quantify the effect of selective annota- tion, we also provide random and other baselines. For randomly-selected annotation, we conduct
hhttps:
co/ sent enceeeeeeeee seice-trane rann formers r /al -mpnet- sappnet-baee-- base-v2 We also explored x= 100, 150, 200, hut did not lead substantial performance changes.
3

---

Published as a conference paper at ICLR 2023
experiments three times and report the average score. We will show that these baselines substantially underperform the vote-k method ($3.3), demonstrating the importance of the selective annotation step to reduce the total annotation cost.
2.2 PROMPT RETRIEVAL
Once we have of annotated examples L from selective annotation, we retrieve few examples from the annotated set as in-context examples for cach test instance. Following recent work (Liu al. 2022). we will compute embeddings for all annotated samples using Sentence-BERG and find the most similar examples to each test instance in terms of cosine similarity.
EXPERIMENTS
We conduct extensive experiments over 10 diverse datasets, spanning 9 distinct tasks, and show better approach to few-shot learning than previously considered. In general, we find that the first step of selective annotation particularly crucial reduce the amount of required annotation.
3.1 DATASETS AND TASKS
We use 10 diverse NI.P datasets across tasks that are listed in Table [ These datasets involve different task formulations, thereby allowing for extensive evaluations in varying scenarios. Some of those are included in the widely-used GLUE benchmark (Wang et al. 2019). Appendix A illlssraatss details of the 10 datasets with examples.
For each dataset, we use the standard train/dev./tess split available from the Transformers library (Wolf et al. 2020). In the selective annotation step, remove all labels in the training data. For the datasets that have test data available publicly, we use the the test data for evaluation (SST-5, XSUM,
MWoZ., and DBpedia). For the others, follow prior work de.g..JJang
al.. 2020 Lan et al. 2020
Gao et al. 2021) and use the dev. data for evaluation [ We evaluate the methods by accuracy for all classification and multiple-choice selection datasets, joint accuracy (Budzianowski et al. (018) for MWoZ, test suite accuracy (Zhong et al. (2000) for GeoQuery, exact matching (Rajpurkar et al.
2016 for NQ, and ROUGE-L (Lin) 2004) for XSum.
Dataset
Task
In- In-Context Learning Models
MRPC Dolan et al. 2004
Paraphrase Detection
GPT-Neo, GPT-J, GPT-3
SST-5 (Socher el al. 20135
Sentiment Analysis
GPT-J
Classification
DBpedia Lehmann et al 2015
Topic Classification
GPT-J
MNLI Willianis ef al. 2013
Natural Language Inference
GPT-J
RTF Rentivogi
2009
Natural Language Inference
GPT-J
Multiple-Choice HellaSwag Zellers et 2019
Commonsense Reasoning
OPT. GPT-Nco, GPT-J, GPT-3
Dialogue
MWoZ 2.4 Budzianowski
2018
Dialogue State Tracking
Codex-fcushman, davinci-002)
GeoQuery (Zelle Mooney
1996
Semantic Parsing
Codex-davinci-00:
Generation
NQ (Kwiakkwwkii al. 2019
Open-Domain QA
Codexxdavinci-002
XSUM (Narayan el al. 2018
Summarization
GPT-J
Table 1: All the 10 datasets and the in-context learning models used in our experiments. GPT-J and -o0ex-davinci--022a. are used by default. Other in-context learning models are explored analysis.
Measuring Stability Given set of unlabeled data, vote-k selective annotation algorithm deterministic, without any randomness. However, we note that in real scenarios, even getting unlabeled samples is not trivial and getting unlabeled samples can be process with large variance. To simulate this real setting, we perform selective annotation from 3K instances that are randomly subsampled from the original training data for each task. For each experiment, subsampling three times, and results are averaged over the three trials. We will find that vote-k still substantially improves stability over alternative selective annotation methods.
repeat this
'The one exception is GeoQuery, where concatenated the dev. and test data to have reliable evaluations
larger data.
4

---

Published as a conference paper at ICLR 2023
3.2 IN-CONTEXT LEARNING MODELS
We mainly perform experiments using GPT-J with 6B parameters (Wang & Komatsuzaki 2021 due to our computational budget. The exceptions are the MWoZ. GeoQuery, and NQ datasets, where we use Nodex-davinci-002 (Chen al. 2021 variant of GPT-3 finetuned on code data from the web. Codex is particularly effective for structured prediction such as semantic parsing, and we found is indeed effective on three datasets (MWoZ, GeoQuery, and NQ) in our preliminary experiments. We will explore the effectiveness of selective annotation on the largest publically available language models, OPT-175B (Zhang al. 2022) for HellaSwag (Fig. 4 and 2odex-davincci0002 for MWoZ, over varying annotation budgets. We will also explore other language models with different sizes for three representative tasks (HellaSwag, MWoZ, and SST-5) in $4.2 GPT-3 with 175B (Brown) et al.] 2020) and GPT-Neo with 2.7B parameters (Black et al. 2021). Our later experiments will show the same patterns among selective annotation methods over these different language models. For the classification and multiple-choice tasks, we compute the average log score for each choice and choose the maximum one. For generation tasks, we simply perform beam-search decoding.
See Appendix B for our in-context learning prompt templates for all 10 datasets. For every test instance, we feed as much retrieved samples as possible into the language model until the maximum token length is reached. On average, the number of samples N fed into the language model is 13.4 across different experiments. The in-context examples are concatenated in the ascending order of the similarity so that more similar examples benefit from the recency bias (Lu et al. 2022).
3.3
MAIN RESULTS
Method
Classification
Multi-Choice Dialogue
Generation NQ
[C]
Selection
MRPC SST-5
MNLI DBpedia RTE
HSwag
MWoZ
GeoQ
XSum
100
Random
63.5
44.2
37.4
89.8
51.5
65.2
47.2
78.6
30.8
15.3
100
Vote-k
70.7
53.0
47.3
93.4
55.5
70.7
51.4
82.8
33.6
17.2
100
A Absolute gain
+7.2
+8.8
+9.9
+3.6
+4.0
+5.5
+4.2
+4.2
+2.8
+1.9
18
Random
59.6
39.8
36.7
77.6
50.4
62.5
33.6
62.4
29.8
13.6
18
Vote-k
64.2
47.6
41.0
87.1
54.3
67.4
42.8
72.5
32.3
15.2
18 A Absolute gain
+4.8
+7.8
+4.3
+9.5
+3,9
+4.9
+8.8
+9.9
+2.5
+1.6
Table 2: In-context learning results with randomlyy selected and vote-k selective annotation methods
on all 10 datasets, with an annotation budget of 100 or 18. There is no prompt retrieval step when only 18 samples are annotated since all annotated samples can fit into the in-context learning model's input. Across the board, selective annotation with vote-k substantially outperforms the randomly-selecte annotation baseline for in-context learning. Further, vote-k largely reduces the variance over three trials (see the min and max results in Appendix C. making in-context learning more stable.
Seen in Table 2 are results from all 10 diverse datasets with the annotation budgets of C {18, 100). 18 is chosen so that all annotated examples can be fit to the prompt for the language models without prompt retrieval. Over all datasets, vote-k selective annotation outperforms the random baseline by a large margin (5.2% absolute gain and 1 11.4% relative gain on average) when the annotation budget is 100. Even when only 18 examples are annotated and fixed as the in-context examples for all testing instances (no prompt retrieval step), in-context learning with vote-k still improves the randomly-selecter annotation baseline (5.8% absolute gain and 12.9% relative gain on average). Particularly noteworthy is that in-context learning with 18 examples selected by vote-k achieves higher performance than the one with 100 randomly selected examples on 6 out of 10 tasks. Moreover, vote-k is a deterministic selective annotation method, conditioned on a set of unlabeled samples. Therefore, the variance of vote-k comes solely from how the unlabeled samples are collected, largely improving the robustness of in-context learning. We therefore recommend that researchers and practitioners use selective annotation (e.g., our vote-k method) to better benefit from the few-shot learning capability of large language models with stability. Our later experiments will also illustrate that vote-k consistently outperforms alternative selective annotation methods ($4.5)
'The parameter size of Codex not officially confirmed, but is likely to he 175B.
5

---

Published as a conference paper at ICLR 2023
4 ANALYSIS
Our extensive experiments demonstrated that selective annotation is important for the success of in-context learning. Here we conduct detailed analysis to provide further guidance for researchers and practitioners of few-shot in-context learning. We analyze selective annotation for in-context learning from variety of perspectives: comparisons to finetuning methods ($4.1), varying language model sizes (.4.2 test data domain shifts ($4.3), prompt retrieval methods ($4.4) 4) and alternative
selective annotation methods ($4.5).
4.1 IN-CONTEXT LEARNING VS. FINETUNING
(Trendom((oBBRRi))
Fi-Vote-KIReBERTa
(CL-randem(GPT-])
EL-vote-k(GPT)
HellaSwag
70-
50
Accuracy
30
30
18
100
300
800
18
100
300
800
18
100
300 800
Annotation size
Figure 2: Comparisons between the in-context learning and finetuning paradigms over varying anno- lation budgets on three representative tasks: HellaSwag commonsene reasoning, MRPC paraphrase detection, and MWoZ dialogue state tracking. Four configurations are presented: finetuning with examples that randomly selected to annotate (FT-random) selected by vote-I selective annotation method ( FT-vote-k) and in-context learning with maddomly-selected annotation (ICL- random) or vote-k selection (ICL-vote-k). See I for experimental details. Selective annotation largely improves the in-context learning performance compared to randomly-selected annotation even when the annotation budget is 18. In-context learning with wisely-selected labeled samples is much better few-shot practice than strong finetuning method.
As discussed earlier, in-context learning an alternative learning paradigm to conventional finetuning. Through the lens of our two-step framework, we observed that selective annotation and prompt retrieval are key to the success of in-context learning. new question now arises: how does in- context learning compare with finetuning under limited annotation budgets? We empirically compare the two paradigms in this section.
We experiment with three representative tasks: MRPC (classification). HellaSwag (multiple-choitt and MWoZ (dialogue). Strong, state-of-the-ar pretrained models are used for finetuning: large-sized RoBERTa (Liu et al., 2019) for MRPC and HellaSwag and DS2-T5 (Shin et al. 2022 for MWoZ.. In-context learning uses GPT-J for MRPC, GPT-J and OPT 175B (Fig 4 for HellaSwag, and Codex- davinci-002 for MWoZ. Note that do not aim to conduct head-to-head comparisons with exactly the same pretrained model; finetuning a large left-to-right language model (e.g., GPT-J and GPT-3) is computationally (and thus financially) infeasible in many cases. Here we examine the two paradigms from the practical perspective and benefit from the advantage of in-context learning, which requires no parameter updates of massive language models.
Fig. 2 compares the two paradigms across varying annotation sizes ({18, 100, 300, 800}). Over all three tasks, we observe that in-context learning with vote-k selection outperforms the finetuning performance of state-of-the-ee similar performance to vote-k with |C| = 18 or 100, finetuning requires 1000 annotated examples for HellaSwag and 800 for MWoZ (10-100x annotation cost). Note that the in-context learning performance usually converges when 100 or 300 examples are carefully selected and annotated, suggesting that large annotated dataset is unnecessary for in-context learning to achieve strong performance. Interestingly, selective annotation helps in-context learning, but not finetuning. This result is consistent with recent work showing that many active learning algorithms perform similarly
pretrained language models. Specifically, we find that to achieve

---

Published as a conference paper at ICLR 2023
to random baseline, when pretrained language models finetuned (Karamcheti al. 2021 D'Arey Downey 2022). They proposed that might be due to outliers and the instability finetuning on limited number of annotated samples. We hypothesize that in-context learning with similarity- prompt retrieval is more robust to outliers and small annotation sizes because only the most similar examples retrieved for each test instance. We find two pieces of evidence for this hypothesis. First, $ showss that random (as opposed to similarity-basee prompt retrieval does not benefit from vote-k selective annotation. Second, Appendix B helps finetuning to benefit from vote-k.
we show that explicitly removing outliers also
4.2
LANGUAGE MODELS WITH VARIOUS SIZES
random
HellsSwag
SST-5
MiVo7
50
50
45
45
40
40
35
35
30
25
GPT-Neo
GPT-J
GPT-3
GPTNNeo
GPT:J
GPT-3
cushman
davinci
Model size
Figure 3: Comparisons of various models with 100 annotated examples. Vote-k selective annotation consistently improves in-context learning with pretrained language models of varying sizes.
shows performance with varying sizes of language models (GPT-Neo 2B, Black al. 202 GPT-J 6B, Wang & Komatsuzaki 202T GPT-3, Brown et al., (020) on HellaSwag commonsense reasoning, SST-5 sentiment analysis, and MWoZ dialogue state tracking. In general, when smaller model is used, the performance gap between random and vote-k selection is larger. In the HellaSwag task, vote-k outperforms randomly-selected annotation by 7.5% using GPT-Nco, but only 2.6% using GPT-3. Nonetheless, we see consistent performance gains from vote-k selection over varying sizes.
Fig. 3
4.3 EFFECTS OF DOMAIN SHIFT
Recent work observed that when a large, pretrained language model is finetuned, the performance gain from active learning limited (D' D'Arcy& Downey (022). but it can be larger there is domain shift between training and evaluation (Tamkin et al. (022). We have demonstrated that selective annotation consistently improves in-context learning, but here we explore cases of domain shifts.
Method
CivilComments
Amazon
[C]
Selection
Random
Domain
Random
Domain
100
Random
73.8
66.8
50.3
30.7
100
Vote-k
79.3
76.7
56.3
39.0
100
A Absolute gain
+5.5
+9.9
+6.0
+8.3
Table 3: Effects of domain shift. Random splits and domain splits are compared (Koh et al. 2021
Following Tamkin et 2022 we use two natural language datasets from the WILDS benchmark (Koh et al., 2022): Civil Commennsstts (toxicity classification; Borkan et al. (019)) and Amazon (review classification; Ni et al. 2019). Each comes with both random split and domain split: the former splits data randomly and the latter is based on the domains (demographic identities for CivilComments and users for Amazon), simulating cases where a model is deployed in a new scenario unseen during annotations. Similar $3.3 conduct experiments with GPT-J under two settings: random/vote-it selective annotation, followed by similarity- prompt retrieval. Both selective annotation and prompt retrieval conducted on the source domain.
7

---

Published as a conference paper at ICLR 2023
Tab. shows results. We that the gain from vote-k is pronouneed under the domain splits: e.g.. 9.9 vs. 5.5 accuracy point improvements on CivilComments. This suggests that selective annotation and prompt retrieval are particularly crucial when there is domain shift in the evaluation data, as in many realistic scenarios (Koh et al. 2021 Longpre et al. 2022).
4.4 RANDOM PROMPT RETRIEVAL
Method
Dataset
[
Selection
Retrieval
HellaSwag
SST-5
MWoZ
100
Vote-k
Similar
70.7
53.0
51.4
100
Random
Similar
65.2
44.2
47.2
100
Vote-k
Random
62.5
41.6
35.6
100
Random
Random
63.2
40.6
43.8
Table Comparison of random and similar prompt retrieval. Random retrieval fails benefit from diverse and representative annotated examples from vote-k.
We have performed kmilaaiity-bssed prompt retrieval so far. Here we experiment with a random baseline for the prompt retrieval step quantify the effect of prompt retrieval (Tab. 4 Interestingly, when random prompt retrieval is performed, vote- does not necessarily improve upon the randomly- selected annotation baseline: c.g., 62.5 vs. 63.2 on HellaSwag and 35.7 vs. 43.8 on MWoZ. This suggests that random prompt retrieval fails to benefit from diverse, representative 100 samples, selected by vote-k selective annotation. Combining selective annotation and prompt retrieval is thus crucial for the success of in-context learning.
4.5
ALTERNATIVE SELECTIVE ANNOTATION METHODS
Random
MFL
K-means
Diversity
Least-conf
Conf-only
Fast vote-k
Vote-k
HellaSwag
65.2
66.5
67.6
68.2
68.4
68.6
69.5
70.7
SST-5
44.2
45.6
47.2
48.5
46.2
48.3
51.9
53.0
MWoZ
47.2
48.3
48.8
49.2
49.4
49.2
50.2
51.4
Table 5: Comparisons of various selective annotation methods with 100 annotated examples. Perfor- mance is averaged three random trials. Vote-k outperforms all the other selective annotation methods. Fast vote-k; a faster version of voke-k without the need for confidence score computations, can achieve similar performance vote-k while being more computationally efficient.
Here explore six additional methods for selective annotation:
Maximizing facility location (MFL; Lin & Bilmes 2009) aims at optimizing the represen-
tativeness of the selected samples. Since this objective satisfies the submodular objective, maximization can be approximated via greedy algorithm (see Appendix G.2).
K-means (Lloyd] 1982) groups examples into clusters, and selects the centroid example from each cluster.
Diversity focuses on maximizing the diversity of the embeddings for selected examples in the first step (Appendix G.3).
Least-conf (Lewis& & Gale (994) iteratively adds least-confident examples the annotated
set.
Conf-only ablations that apply confidence based stratification similar to vote-k; but examples are sampled randomly from each group without graph- based scoring.
Fast vote-k is a fast, efficient alternative to our vote-k method ($2.1) that does not use confidence scores. It picks M samples with the largest vote-k scores. It avoids using the pretrained language model to compute a confidence score for every instance, resulting in a 10+ times speedup.
Notice that MFL, diversity, and least-confidence do not have hyperparameters other than the an- notation budget. As shown in Tab. vote-k outperforms all the other methods. It is noteworthy,
8

---

Published as a conference paper at ICLR 2023
however, that fast vote-k can achieve similar performance vote-k. Fast vote-k is thus attractive method for researchers and practitioners with a limited computational budget. Like vote-k, MFL also optimizes representativeness and Diversity also optimizes diversity. In particular, MFL defines representativeness as a sum over distances from the selected examples to all other examples, and Diversity defines diversity the distances between selected examples. Since they do not significantly outperform randomly-selected annotation, we conjecture that jointly optimize diversity and represen- tativeness is needed for selective annotation. Moreover, the way vote-k defines and diversity are also different from the baselines: vote-k defines representativeness as the number of neighbors during similarity- -based based prompt retrieval, which is effectively tailored in-context learning; vote-k directly optimizes for the diversity of selected samples using the in-context learning model's prediction confidence.
RELATED WORK
In-Context Learning In-context learning with large language models has recently received an
increasing amount of interest, partly due to its flexibility and sample efficiency (Liu et al. 2021). Several recent works proposed methods to improve in-context learning in many aspects: e.g., meta-
training (Chen et al. [202]] Min et al. (2022b), task instructions (Efrat & Levy 2020 Mishra et al.
2022 Wei et al.
2021 Sanh et al. (022)) or task formulation (Holtzman al., 2021] Zhao et al. 2021 Min et al. 2022a). In this paradigm, the choice of in-context (i.c., demonstration) examples
has been shown crucial (Liu et al. 2022, Rubin et al., 2022] Lu et al.) [2022), while recent work raised questions as to the degree to which correct labels are necessary (Min et al.] (022c). This work proposes an annotation-efficient in-context learning framework by focusing on the choice of examples and its implications on the annotation cost.
Active Learning Active learning aims to enable machine learning models to achieve similar or
greater performance with fewer labeled training instances (Cohn et al.) 1994 Settles 2009). Our selective anmotation step for in-context learning shares the same goal of reducing the annotation cost. Most active learning methods involve iterative parameter updates (e.g.. (e.g..Wang al. 2017 Kasai
let al. 2019). which are computationally expensive for large language models used in in-context learning. Similar our vote-k algorithm, Lin & Bilmes (2009) used the facility location objective to optimize representativeness. We observed that this objective largely underperforms vote-k for in-context learning, probably due to the fact the vote-k (1) effectively tailored to the prompt retrieval step of in-context learning and (2) directly optimizes the diversity of selected samples (see $4.5)
More recently, the effectiveness of active learning has been questioned when large-scale pretrained models are finetuned for various tasks (Karamcheti et al. [222]] D Arcy & Downey] 2022). Our experiments ($3) showed that selective annotation helps reduce the annotation cost of in-context learning, departing from the recent observations on finetuning with active learning. We hypothesize that it is because in-context learning with samilarrty-basee prompt retrieval is more robust to outliers since each test instance only retrieves its most similar examples. This is supported by $4.4 where random prompt retrieval does not benefit from selective annotation.
CONCLUSION
Much recent work illustrated the ability of large language models to adapt to new tasks simply from a few demonstration examples. We presented in-depth studies on the implications of this ability for dataset annotation through the lens of selective annotation and introduced an annotation-efficie practice. The best selective annotation method explored in this paper, our vote-k method, selects diverse, representative examples to annotate. In terms of the task performance, vote-k improves the performance on 10 diverse tasks by a large margin. Moreover, vote-k selective annotation yields similar performance to ofate-of-the-a art supervised finetuning with 10-100x less annotation cost. We further show that the effectiveness of vote-k is consistent with different language model sizes and domain shifts between training and test data. We hope that our findings will help researchers and practitioners efficiently design new natural language tasks and beyond.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
We thank Sewon Min, Pradeep Dasigi, Yanda Chen, Yushi Hu, Alisa Liu, and the ARK group at UW for their helpful feedback on this work.
REFERENCES
Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. The fifth PASCAL recognizing textual entailment challenge. In Proc. of TAC.
2009. itt mistt..c....toooo....ooo.. http:////...e.e..oo/.//....///// RTES
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Autoregressive Language Modeling with Jesh-Tensor,low, 2021. URL https://zenodo
org/tecord/5551202
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for classification. In Companion Proceedings of WWW, 2019. URL ht tps: /t::://doo /ttp:///o...// org/
3317593
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Rad- ford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Proc. of NeurlPS, 2020. URL httpt: / pesttps://proceedinng neurips cccppaper/2222//Flle 1457cca4bbccb1177111188BBBB -ag.
Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, lñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašié. MultiWOZ. large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proc. of EMNLP, 2018. URL https: http//aaxvv..r./////
abs 11100.110..0 00278
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christo- pher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mc- Grew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained code, 2021. URL https: /retps://arxiv... org abs 2000 03374
Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. Meta-learning via language model in-context tuning. In Proc. of ACL, May 2022. URL https://axxiv.rrr ors/abs/////aoaaaoooo/ooo 2110.07814
David A. Cohn, Les E. Atlas, and Richard E. Ladner. Improving generalization Imppoovinggennnnrllzzioooooono with active learning. Machine Learning, 1994. URL http:/ppp//aao..oo///000////////
Mike D' Arcy and Doug Downey. Limitations of active learning with deep transformer language models, 2022. URL /tt.///wwee///eeeeeeee/e/e/eeeea
Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transfonmers for language understanding. In Proc. of NAACL, 2019. URL
https:://axxiv. http://..rggab/oos/o 04805
10

---

Published as a conference paper at ICLR 2023
Bill Dolan, Chris Quirk, and Chris Brockett. Unsupervised construction of large paraphrase corpora:
Exploiting massively parallel news sources. In Proc. of COLING, 2004. URL https://
aclanthology org/C04-1055
Avia Efrat and Omer Levy. The Turking test: Can language models understand instructions?, 2020. 2020.
URL [RRlhttpss:/acxxv..oom wttporggggb///oooor////// 11982
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Proc. of ACI, 2021.
295
Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. Surface form compe- tition: Why the highest probabiliity answer isn't always right. In Proc. of EMNLP, 2021. URL
https://erxiv.. www.aarwwwwwirraaaaaa//aooooaa//
2210/ 08315
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. SMART: Robust and efficient fine-tuning for pre- pre-trained natural language models through principled reg- ularized optimization. In Proc. of ACL, 2020. URL
ac maam n. 197
Siddharth Karamcheti, Ranjay Krishna, Li Fei-Fei, and Christopher Manning. Mind your outliers! investigating the negative impact of outliers on active learning for visual question answering. In Proc. of ACL, 2021. htRLhtts::/ //tp:v::////xv.../////av org/ http://ab/2221...000 02331
Jungo Kasai, Kun Qian, Sairam Gurajada, Yunyao Li, and Lucian Popa. Low-resource deep entity resolution with transfer and active learning. Proc. of ACL, 2019. URL https://arxx:
org/abs/ 9069966600 08042
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In Proc. of ICML, 2021. URL https::/prroeedding mlr wwtppppwsseeeeeppepppppppp
html
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield. Michael Collins, Ankur Parikh, Chris Al- berti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: URL https::///// htt////reeeehhhee
benchmark for question answering research. TACL, 2019.
poogle/pubbsppb4//
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A lite BERT for self-supervised learning of language representations, In Proc. of ICIR, 2020. URL httpt:ttttsxiiiis wwt. abs 1909.11999999 11942
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. DBpedia a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, 2015. URL/att::///// aksw. wwt/ wttp:/pppe..//////////./.. http:///wwww_peaaaaaaabbaaaa
David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In SIGIR '94 pp. 3-12. Springer, 1994.
Chin Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Proc. of Text Summarization Branches Out, 2004. URL hhttps://wwz aww.aclwee.... org/ anthology www.:www///tttooooogo.o//////
W04 10141]
Hui-Ching Lin and Jeff A. Bilmes. How to select a good training-data subset for tran- scription: submodular active selection for sequences. In Proc. of INTERSPEECH, 2009. URL htttts://citesseeees ist. wttpp/eeeeeeeeeeed//////ooooeeae
1. 1 49. www.6444rrerreeeeyrerrrr
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Proc. of DeeLIO 2022, 2022. [thpps
7/ rxiv. / wttpi/orrg/ab//220..000

---

Published as a conference paper at ICLR 2023
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. 2021. Jtttts:s: org/ abs 2107. 13586
Yinhan Liu, Myle Ott, Naman Jingffi Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining approach, 2019. URL htt.t:ttt//sxiiiiiee http:///ggggab///a/g/a
11692
Stuart Lloyd. Least squares quantization in pem. IEEE transactions on information theory, 28(2):
129-133 137, 1982.
Shayne Longpre, Julia Reisler, Edward Greg Huang, Yi Lu, Andrew Frank, Nikhil Ramesh, and Chris DuBois. Active learning over multiple domains in natural language tasks, 2022. URL
https: orgoo/gga/sssoo 22222022522222
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where find them: Overcoming few-shot prompt order sensitivity. In Proc. of ACL, 2022.
08786
Katerina Margatina, Giorgos Vernikos, Loic Barrault, and Nikolaos Aletras. Active learning by acquiring contrastive examples. arXiv preprint arXiv:2109, 03764374 2021.
Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Noisy channel language model prompting for few-shot text classification. In Proc. of ACL, 2022a. URL /tttps://arxiy
org/abs/2108 04106
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetalCI: Learning to learn in context. In Proc. of NAACL, 2022b. URL https://axxiv.orxt wwtp/ag/aaaaa..o..///////////aae 15944
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work?, 2022c.
URL httph:tts///// http//xxvv../// org. /2002 12837]
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In Proc. of ACL, 2022. URL httpt://///:
orq/abs/ / 2104 08773
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proc. of EMNLP, 2018.
URL https:://axxi...otm org/ abs 1808888000 08745
Jiammo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-lbbeled reviews and fine-grat -reegge aspects. In Proc. of EMNLP, 2019. URL URL tttp/attpps://alltttt..ooy
org/D19-1019
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,0000 for machine comprehension of text. In Proc. of EMNLP, 2016.
abs/1606. 05250
Nils Reimers and Iryna Gurevych. Centence-BERT: Sentence embeddings using Siamese BERT- networks. In Proc. of EMNLP, 2019. URL ptatttps://ccaaatiit low.:lllyyll......ogo
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning retrieve prompts for in-context learning. In Proc. of NAACL, 2022. URL https:
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika. Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang. Mike Tian-Jian Jiang, Han Wang. Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Techan, Stella Biderman, Leo Gao. Tali Bers, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. In Proc. of ICLR, 2022. URL https: // arxiv.org/ abs fbs/2110..m 08207
12

---

Published as a conference paper at ICLR 2023
Burr Settles. Active learning literature survey. Computer sciences technical report, Univer- 2009. URL
sity of p
wwt. settles
activelearning
Jamin Shin, Hangyeol Yu, Hyeongdon Moon, Andrea Madotto, and Juneyoung Park. Dialogue summaries as dialogue states (ds2), template-guided summarization for few-shot dialogue state tracking. In ACL FINDINGS, 2022.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng. and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of EMNLP, 2013, URL
http://aaaaanooooo...oo.. www.oloooooorg/1333 1170
Alex Tamkin, Dat Nguyen, Salil Deshpande, Jesse Mu, and Noah D. Goodman. Active learning helps pretrained models learn the intended task, 2022. URL
arXiv. 2204 08491
Laurens der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Ma- chine Learning Research, 9:2579-2605, 2008. URL httttttpp/www.... jml r. wwt.orgg/ppp.. org/papers/v9/
vandermatterOB..Attm.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proc. of ICLR., 2019.
aa//////ss//////./////// 07461
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model.
jax 2021.
Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, and Liang Lin. Cost-effective active learning for deep image classification. TCSVT, 2017.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai. and Quoc V. Le. Finetuned language models are zero-shot learners. In Proc. of ICLR, 2021. URL wtttt:://aaxiiit..:. www.xxiii.org/abbb. /2199. 01652
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL, 2018. URL httptttt//xxii.....rt
abs/1704. 05426
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. HuggingFace's transformers: State-of-the- art natural language processing. Proc. of EMNLP: System Demonstrations, October 2020. URL wrg. 2020. emnlp-demos
httpps://aat httpss//aalaaathhoooooo
John M. Zelle and Raymond J. Mooney. Learning to parse database queries using inductive logic programming. Proc. of AAAI, 1996. URL http:://w.........////bbbaaaaaa.
aaai http://ww..aali..r.bbbbaaaaaaaaa
1996/aaai966 1ataai9661110 166.155..2.. php
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence?
Proc. ACL, 2019. URL
07830
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li. Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer.
Opt: Open pre-trained transformer language models. ArXiv, hbs/2205.01068. 2022.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Samcer Singh. Calibrate before use: Improving few-shot performance of language models. In Marina Meila and Tong Zhang (eds.), Proc. of ICML. 2021.
h/tp::i.:///rvvv..//a www.riiwi..oraaaooooooe abs/21a....22 09690
Ruiqi Zhong, Tao Yu, and Dan Klein. Semantic evaluation for text-to- SQL with distilled test suites. In Proc. of EMNLP, 2020. URL http:ppp///aaiii..orrraab//////p
02840
13

---

Published as a conference paper at ICLR 2023
Dataset
Task
Examples
A woman is outside with bucket and dog. The dog is running around trying avoid bath. She...
Commonsense Reasoning
A) rinses the bucket off with soap and blow dry the dog's s head.
HellaSwag
B) uses a hose to keep from getting soapy.
/C) gets the dog wet, then it runs away again.
D) gets into bath tub with the dog.
Sales rose 37 per cent year-on-year to 1.76hn, beating expectations. Sales for the quarter beat expectations, rising 37 percent year-on-year 1.76 billion curos. Paraphrase
Paraphrase Detection
MRPC
A warm, funny, engaging ilm..PPositive Suffers from the lack a compelling narrative.-Negatie
SST-5
Sentiment Analysis
Dialogue State Tracking
am looking for ALexender b&b
MWoZ 2.4
Dialogue state: alexander hed and breakfast
What is the area of California?
GeoQuery
Semantic Parsing
SELECT state. FROM state WHERE state.
state_name= state_name-CCallfforid california'
The keeled box turtle (Cuora mouhoti syn. Pyxidea mouhotii) is species of turtle in the family Geoemydidac. It is native to Asia where it occurs in China India Laos Burma Vietnam Thailand and Bhutan. Other common names include keel-backed terrapin and jagged-shelled turtle.
DBpedia
Topic Classification
Topic: animal
The F/A-18 17-F18 program eliminated over 40 percent of the parts used to build predecessor aircraft make the design more robust for manufacturing and identified critical manufacturing processes, bringing them under control before the start of production. The new design with robustness also increased the safety of machines. neutral
Natural Language Inference
MNLI
Judie Vivian, chief executive at ProMedica, medical service company that helps sustain the 2-year-old Vietnam Heart Institute in Ho Chi Minh City (formerly Saigon), that so about 1,500 children have received treatment. The previous name of Ho Chi Minh City was Saigon. / entailment
Natural Language Inference
RTE
Natural Questions
when was music first played on the radio / 1917
Open-Domain QA
Bliss said there was a shortage nconatal nurses and doctors, and safety standards were not being met. Dr Jenny Calvert, of the Wales Nconatal Network, said they working to further develop medical training neonatology help recruit more trainee doctors. Summary: Neonatal services across Wales are overstretched under pressure with the safety of vulnerable babies at risk, according to charity.
XSUM
Summarization
Table 6: All of the datasets with examples used in our experiments. The 10 datasets span various formations, including classification (SST-5, Socher et al. 2013) MRPC, MRlC.Dolan et al. 2004). multiple-choice selection (HellaSwag, Zellers et al. 2019), and code/text generation (MWoZ 2.4. Budzianowski et al. 2018 GeoQuery, Zelle & Mooney 1996 NQ, Kwiatkowski et al. 2019).
Appendices
A DATASETS AND TASKS
14

---

Published as a conference paper at ICLR 2023
B PROMPT TEMPLATES
B.1 HELLASWAG
Input:
The topic is Grooming dog, Two wemen attempt to wash two dogs. they get in the tub with the dogs and do shampoo, soap, and then rinse the dogs.
The topic Bathing dog. couple is outside with bucket and dog. The dog is running around trying avoid bath. they
Output:
get the dog wet, then it runs away again.
B.2 MRPC
Input:
Are the following two sentences 'equivalent' or 'not equivalent'? This around the time Congress was debating resolution granting the President broad authority Within four days the House and Senate overwhelmingly endorsed resolution granting the president authority to go to war answer:net equivalent
wage war
Are the following two sentences 'equivalent' or net equivalent'? Kerry last month outlined U.N. resclution authorizing military force under u.s. command and transferring responsibility to the United Nations for the political and humanitarian efforts
Kerry outlined last month UN resclution authorizing military force under command and transferring responsibility for political and humanitarian efforts to the UN answer:
Output:
equivalent
B.3 SST5
Input:
How do you feel about the follcwing sentence?
the movie 's blatant derivativeness is one reason it 's so lackluster answer:negative
How do you feel about the follewing sentence? the movic something-borrowed construction feels less the product loving well integrated homage and more like a mere excuse for the wan thinly sketched story answer:
Output:
negative
15

---

Published as a conference paper at ICLR 2023
B.4 MULTIWoZ
Input:
CREATE TABLE hotel(
text,
internet text CHECK (internet IN (dontcare, yes, no)
/.
example rows:
SELECT FROM hotel LIMIT 4; name pricerange type parking bock_number_ov days book day book_e area stars internet
and guest house moderate guest house dontcare 3 friday east 4 yes
Using valid SQLite, answer the following multi-turn conversational questions for the tables provided above.
Example #1
[context] hotel-area: west, hotel-stars: 3, hotel-internet: yes [system] the hobsons house is available in that area Q: [user] that sounds like it will work starting wednesday
book that for 3 nights
SQL: SELECT FROM hotel WHERE bock_day wednesday AND bock_people AND book_number_of_Gaa = AND name hobsons house;
Example #22
ccontext | hotel-parking: yes, hetel-pricerange: moderate, hotel-type: guest house, hotel-stars:
[system] there are 9 in the area i recommend the warkworth house
Q: [user] can you book that 1 for nights starting wednesday SQL: SELECT FROM
Output:
hotel WHERE book_day = wednesday AND book_number_of_days 4 AND nane warkworth house;
B.5 GEOQUERY
Input:
CREATE TABLE "border info" ("state name" text, "border" text)
/+
state_name
border
alabama tennessee
alabama georgia
alabama florida
Using valid SQLite, answer the following questions for the tables provided above.
which state has the longest river
SELECT RIveRaliasO, TRAVERSE FROM RIVER AS RIVERalias WHERE RIVERa1 as80. LENGTH SELECT MAX( RIVERal iasl. LENGTH ) FROM RIVER AS RIVERa list.
what is the longest river in the state with the highest point SELECT
16

---

Published as a conference paper at ICLR 2023
Output:
RIVERal as0. RIVER_NAEE FROM HIGHLOW HIGHLoWalias) RIVER
RIVERaliasO WHERE HIGHLOMaliasO HIGHEST ELEVATION SELECT MAX EIGHLONaliast. HIGHEST ELEVATION FROM HIGHLOW As HIGHLOwalissl AND WIIGHESTEEEEVAAIOON RIVERaliasO TRAVERRS = HICHLONaliasO. STATE NAME ORDER BY RIVERaliasO.
LENGTH DESC LIMIT
B.6 DBPEDIA
Input:
Cupressus funebris; content: Cupressus funebris (Chinese Weeping Cypress) is species of cypress native to southwestern and central China. It may also occur naturally in vietnam. plant
title:
title: Keeled box turtle; content: The keeled box turtle (Cuora mouhotii syn, Pyxidea mouhotii) is species of turtle in the family Geoemydldac. It native to Asia where it occurs China India Laos Burma Vietnan Thailand and Bhutan, other common names include keel-backed terrapin and lagged-shelle turtle.
Output:
animal
B.7 MNLI
Input:
Ideally, the design fixes for the failures should be corrected prior to manufacturing production units.. Based on that information, the claim The fixes should be addressed before they reach the assembly line if this was a smart plan. "True", "False", or "Inconclusive"? answer:
The F/A-18-E/F program eliminated over 40 percent of the parts used build predecessor aircraft to make the design more robust for manufacturing and Identified critical manufacturing processes, bringing them under control before the start of production.. Based on that information, is the claim The design with robustness also increased the safety of machines. "True", "False", or "Inconclusive"? answer:
Output:
Inconclusive
B.8 RTE
Input:
After giving nearly 5, 000 people second chance at life, doctors are celebrating the 25th anniversary of Britian's first heart transplant which was performed at camoridgeshire's Papworth Hospital in 1979. \par question: The first heart transplant in Britian was performed in 1979. True or False? answer:True
Judie Vivian, chief executive at ProMedica, medical service company that helps sustain the 2-year-old Vietnam Heart Institute in Ho Chi Minh City (formerly Saigon), said that so far about 1,500 children have received treatment..
question: The previous name of Chi Minh City saigon.. True or
17

---

Published as a conference paper at ICLR 2023
False?
answer:
Output:
True
B.9 NATURAL QUESTION
Input:
write an answer: who invented the radio during the industrial revolut on other
Guglielmo Marconi, 1st Marquis of Marconi
Write an answer: when was music first played on the radic
Output:
other
1917
B.10 XSUM
Input:
write short summary
Health Minister Mark Drakeford said the money would be used to improve areas of concern, including out-cf-hours help and access to psychological treatment.
money won't get the help they need in timely fashion, " she said. TL;DR; An extra [Unicode token] 7, 6m a year will be invested to improve mental health services for children and young people in Wales.
write short summary:
Bliss said there was a shortage of neonatal nurses and dectors, and safety standards were not being met.
Dr Jenny Calvert, the Wales Necnatal Network, said they working to further develop medical training neonatology help recruit more trainee docters.
Th;DR:
Output:
Neonatal services across Wales are overstretched and under pressure with the safety of vulnerable babies at risk, according to a charity.
18

---

Published as a conference paper at ICLR 2023
C DETAILED MAIN RESULTS
This section provides detailed version of our main results in Table where the maximum perfor- mance and minimum performances among the three trials are reported. Results are shown in Table and Table
Method
Classification
Selection
MRPC
SST-5
MNLI
DBpedia
RTE
100
Random Vote-k 70.7/72.3/69.1
63.5/66.0/60.5
37.4/41.00 .0/33.2
89. 99.8 .0/88.3 1/55/3.9//8/ 4
100
53.0/54.7/51..2
47.3/50.0/44.5
93.4/94. 1/92.6
55.5/57.0/53.9
18
Random
59.6/64.8/52.7
39.8/46. 1/37.1 36.7/40.6/303
77.6/82.0/7199 41.0/44.5/37.1
50.4/53.5/45.7
18
Vote-k 64.2/67.6/59.0
47.6/50.0/44.5
87.1/90.6/85.2
54.3/56.2/51.5
Table 7: Main result Table 2
with the mean/max/min results reported across three trials
Method
Multi-Choice
Dialogue
Generation
[C]
Selection
HSwag
MWoZ
GeoQ
NQ
XSum
100
Random
65.2/66.4/63.3
47. 47.2/49.2/44.5
78.6/80.5/77..3
30.8/32.8/28.1
15.3/16.4/14.8
100
Vote-k 70.7/71.5/69.5
51.4/53. 1/49.6
$2.8/83.6/82.1
33.6/35.2/31.6 17.2/17.6/16.4
18
Random
62.5/66.4/57.4
33.6/39.5/25.0
62.4/65.2/577
29.8/33.6/26.
13.6/14.5/12.5
18
Vote-k
67.4/71.1/64.8
42.8/47.7/40.2
72.5/74.2/69.5
32.3/33.6/30.1
15.2/16.0014.5
Table 8: Main result Table with the mean/max//in results reported across three trials.
D EVALUATE HELLASWAG ON OPT-175B MODEL
Here show that vote-k also improves model performance for OPT-175B
random
vote
82
81
80
Tccuracy
79
76
75
74
18
100
300
800
Annotation size
Figure 4: OPT-175B performance of ICL-random and ICL-vote-k on HellaSwag
E REMOVING OUTLIERS FOR FINETUNING
Here show that explicitly removing outliers also helps finetuning benefit from vote-k:
F DIVERSITY AND REPRESENTATIVENESS OF SELECTED SAMPLES
We hypothesized that both representativeness and diversity are crucial for selective annotation ($2.1).
Here we evaluate the diversity and representativeness of samples that selected by different methods,
19

---

Published as a conference paper at ICLR 2023
Outliers not removed 10% outliers removed
FT-random FT-vote-k FT-random FT-vote-k
HellaSwag MRPC
55.6
53.5
56.8
59.6
56.3
55.6
57.9
60.4
Table 9: Effects of vote-k in finetuning(FT) with annotation budget of 100. 10% outliers removed
means that we removed 10% of examples farthest to the training data, measured by average cosine similarity. The selection is conducted after example removal. After removing outliers, vote-k selection improves the model few-shot performance.
using the methods from prior work active learning (Margttina et al. 202T); diversity and representativeness use token overlap embedding cosine similarities. As shown in Table T0J K-means improves both the diversity and the representativeness, as compared random selection, while vote-k further improves it.
)) their measures of
Method
DIV-I
DIV-F
REPR.
Selection
HellaSwag
SST-5
MWoZ
HellaSwag
SST-5
MWoZ
HellaSwag
SST-5
MWoZ
Random
0.1820.007
0.0990003
0.3680.00s
0.4150000
0.3170001
0.6750.006
0,558a.007
0.4246.00
0.6960001
K-means 0.1 1844..00 o06
0. 0.1022002
0.372o.003
0,4170.00
0.317a.02
0.6760.00
0.559
0,423e.0zz
0.6980.03
Vote-k
0.191
0.108
0.379
0.425
0.321
0.683
0.565
0.426
0.702
Table DIV-I refers to diversity in input space, which measures the diversity of selected data in the input feature space, i.e., raw text; DIV-F refers to diversity in feature space, which measures the diversity in the dense feature space, i.c., sentence embeddings; REPR. refers to representativeness, which measures the representativeness of selected data. Subscripts stand for standard deviation.
G DETAILS OF SELECTIVE ANNOTATION METHODS
In this section, we provide details of selective annotation methods used in Section 4.5
G.1
VOTE-k SELECTIVE ANNOTATION
Algorithm
1
describes the vote-k selective annotation method introduced in Section 2.1
G.2 GREEDY ÁLGGORITHM FOR MAXIMIZING FACILITY LOCATION
Lin & Bilmes 2009) proposed to maximize the facility location objective to optimize representa- tiveness of the selected samples. Since this objective satisfies the submodular property, they applied a greedy algorithm as an approximation. Algorithm 2 describes the selective annotation method adapted from this greedy algorithm.
G.3 EMBEDDING DIVERSITY
This method aims to find diverse samples to annotate using embedding vectors. We first compute vector representation for each unlabeled training instance by Sentence-BEe BERT (Reimers Gurevych 2019), which is variant of BERT (Devlin et al., 2019),, finetuned to detect paraphrases 5 For instance, consider an example from SST-5 sentiment analysis 6 very well-made, funny and entertaining picture. We simply run Centence-BERT on this text input and average the resulting vectors over the words to obtain a vector representation.
Once embeddings are computed for all training data, instances. The intuition here is that a diverse set of annotated examples facilitates the subsequent prompt retrieval step since similar in-context examples can be found for many test instances. To find
them find diverse set of training
htttp::/huggiingfaaee http:///waaaaaaeooooooooooooooa/ htt -mpnet base-va
20

---

Published as a conference paper at ICLR 2023
Algorithm 1 Voke-k Selective Annotation
1: Input: X' {
of unlabeled samples; M: the number of samples to be selected: LM: inference
language model.
2: Initialization: [ = Ø, 2 = X. = (V, E), where V A' and (u, E E if e one of u's nearest
vertices in terms of the cosine similarity between the embeddings.
3: while (C] MM/10 do
4: v'" = arg maXueu Eve R s(v), where s(v) = -(-tEe((ceEE)) p>1
5:
L -= CU {
6: U = u {u'}
7: end while
8: for L do
Compute the confidence score of cach instance.
9: score(u) = Σ. logp(e: Aee z; θ), where p is LM prediction function and θ is LM parameters
10: end for
for j ...... 9/10M
12: U1 = indices((] ((i 1) jUI/M]
&. Divide the examples by confidence score.
u' arg max.EM, Eee( (v.(e( s(V), where s(e) =p [[tEC[(v.)ERR) p >
L LU {u'}
15: end for
16: Return: C: selected samples.
Algorithm 2 Greedy Algorithm for Facility Location Objective
Input: 21 {a.) : set of unlabeled samples;
number samples be selected.
2: Initialization: L = Ø0 U = V. Vi, = -1: maximum similarity of æ; to selected samples.
3: while (C] < .M do
w' = arg maxueu E (max (0, (.))( - pz])
5:
CU {u"}
6: 4=
7: Vi,pi max (pi, cos(sr,
update maximum similarity of each æ; selected samples
end while
9:
Return: L: selected samples.
a set of diverse embeddings, we take a simple, iterative approach: in every iteration, we choose an instance furthest from the already chosen ones. Specifically, let L and // denote the sets of already chosen (i.e., labeled) samples and unlabeled samples, respectively. Suppose also that M the target number of labeled examples (i.e., the annotation budget). Then, in every iteration, unlabeled sample that has the largest total cosine distance from C: arg min Σeec cos (u, E). Here abuse and e to mean both the instances and their embedding vectors from Sentence-BErN BERT. The first labeled sample is randomly selected from the 3K unlabeled examples ($3.I) and the iterative process continues until L) = M.
choose the
H LABEL DISTRIBUTION IN SELECTIVE ANNOTATION
We calculate the ratio between the numbers of the most frequent class and the least frequent class in the selected 100 instances. For example, 53.3 in the Ori. (original dataset) column indicates that the original dataset contains 53.3 times more examples with the most frequent label than those with the least frequent label. As shown in Table 11 with random selection, the ratio is similar to the original dataset. With selective annotation, the imbalance problem is significantly alleviated.
I T-SNE VISUALIZATION OF SELECTIVE ANNOTATION
We compare examples from selective annotation and full training data using the I-SNE visualiza- tion yan der Maaten & Hinton (2008). As shown in Figure vote-k selects diverse (better coverage) and representative (excluding outliers) instances from the task space across datasets.
21

---

Published as a conference paper at ICLR 2023
Ori. Random MFL. K-means Diversity Least-conf Conf-only Fast vote-k Vote-k
Amazon
53.3
59.0
55.0
52.0
20.5
48.0
26.5
14.0
11.0
CivilComments 7.8
11.5
9.0
7.3
5.7
15.7
6.7
5.3
4.6
Table 11:
The ratio between the number of examples with the most frequent class and those with the least frequent class, when different selective annotation methods are applied choose 100 instances. Ori. denotes the ratio in the original dataset. vote-k significantly reduces the ratio and alleviates the label imbalance problem.
full data selected esamples
full data selected examples
0,4
0.2
02
0,0
0.0
0,2
0.2
-2.4
-0.2
0.0
0.4
HellaSwag, random selection
(b) HellaSwag, vote-k selection
0.6
full data
0.6
full dota
selected examples
selected examples
0.4
0.4
0.2
0.2
0.0
0.0
-02
0.2
-0.4
0.4
-0.4
-.2
0.0
0.2
(c) SST-S, random selection
(d) SST-S, vote-k selection
Figure 5: t-SNE visualization of the 100 randomly/vote-k selected examples the dataset space. vote-1 selects more diverse and representative examples, compared to random selection.
SIMILARITY BETWEEN CONTEXT EXAMPLES AND INPUT
We calculate the average cosine similarity between the context examples and the input text in three representative datasets, HellaSwag, SST-5 and MWoZ. As shown in the Table 12 without similarity- based retrieval in the second step, the context examples and input text are not very similar when the first step is vote-k selection. We suspect the reason is that vote-k selects diverse instances and random retrieval includes irrelevant examples in the prompt. This explains the poor performance of vote-k with random retrieval in Table
K SELECTED EXAMPLES
In Table E 4
provide few examples from random selection and vote-k selection, when the
annotation size is 18.
22

---

Published as a conference paper at ICLR 2023
Method
Dataset
C
Selection
Retrieval
HellaSwag
SST-5
MWoZ
100
Vote-k
Similar
0.3393
0.1747
0.4881
100
Random
Similar
0.3134
0.1535
0.2212
100
Vote-k
Random
0,0057
0,0570
0.0006
100
Random
Random
0,0180
0.0065
0.0016
Table 12: Average cosine similarity scores between the context examples and evaluation input. With similarity retrieval in the second step, vote-k selection always helps to find more similar examples in the prompt.
Datasct
Randomly selected examples
Vote-k selected examples
simple poignant leavened with humor, it 's that affirms the nourishing aspects of love and companionship -Possitive
take away the controversy and 's not much more watchable than mexican soap opera. -Very Negative any rock pile will do for a set Negattvv a visual spectacle full of stunning images and effects .-> Positive the consistently funny of the austin powers films Posiiiive
this is nicely handled affair film about human darkness etched with light -Irb- yet unsentimental -rrb- touch Poosiiive what is the filmmakers point Neutral what really happened ?-Neutral admirable sometimes exceptional film.-> -eery Positive
SST-5
occasionally funny, sometimes inspiring often boring Neutral
Table 13:
Demo examples in SST-5 from random selection and vote-k selection. In general, vote-k selects more diverse and representative instances, with a more balanced label distribution.
23

---

Published as a conference paper at ICLR 2023
Dataset
Randomly selected examples
Vote-k selected examples
A man is completing rubiks cube. timer...
Mj's mommy is playing around with her hair smoothing it out, She goes to the bathroom with clips in her hair and gets out treatments and shampoos. she...
A) begins showing the amount of time it will take it to complete the disk.
B) is counting down on the side of the cube.
A) laughs until she is full of tears.
B) puts little blow dryer on her hair and turns it on over and over.
/C) is sitting on the table next to him.
D) goes hand at the end of the board while the man continues to solve and figure out the cubes.
/C) begins to apply the hair things to her
hair and combing it out, she starts part her hair and put curlers.
A man completing rubiks cube. A timer is sitting on the table next to him. he...
D) gets in the bathroom dynain her hair with an iron in her shirt. The batter is fighing with the man and other
A) takes the cube and couches it for 10 seconds.
people trye calm him down. lo of
B) finishes the rubiks cube and sets it
people wearing black unifroms...
aside.
/ A) are running in the field fighting.
C) starts hurting himself trying to solve
B) are all sitting on the floor.
the rubiks cubc.
C) are playing cornstarch ball against
/D) the completed rubiks cube down on the table.
each other the field.
D) are cools the batter.
A client tap with the finger inside a square. Then, the person stacks all the cards and puts the tokens inside the squares. then people...
A group of athletes row on canoes during a race in between buoys on waterway. the men...
A) pass over a wooden structure in the river.
/ A) start to gamble, while the woman distribute the cards and pick up tokens.
B) paddle while crashing through endless waves in the river.
B) play hole dig while other people watches.
HellaSwag
/C) cross the final numbered huoys and
C) pass disk on the tile tile.
glide while slowing down after the race.
D) five card get tokens inside the square.
D) go over large cliff's into a lagoon.
After. the client shows with the hand, then the person make gestures with the hand showing the table. next the woman...
A pair of handlebars are detached from the bike and is laying down flat on table. the man...
/ A) pick up the cards.
A) unstraps the handlebars and puts them back up and begins to pedal of his bike.
B) change the angle of the arm and shows
her knee cucumbers.
B) rolls them up and ties the handles with pulley.
C) puts brushes sides of the table and open the places one by one with the hand giving a thumbs up and different gestures to the client.
C) then shows with his knife and begins to a process of cutting them in half and putting them back to the bike.
D) place the hlindfold on the table, then she lay her back.
/D) then picks up the handlebars and sticks
them back into the bike tightening them with a key.
He takes scour and begins to scour the wall paper. He uses a paint roller to soak the wall paper for easy removal. he...
A man is completing a rubiks cube, a timer...
A) points to several things the wall and
A) begins showing the amount of time it will take it complete the disk.
sprays the wall paper.
B) takes scissors and measures the wall paper closcly.
B) is counting down on the side of the cube.
/C) then demonstrates how steam can be
/C) is sitting on the table next to him.
used too to loosen the wall paper.
D) goes hand at the end of the board
D) wipes the wall paper down against the
while the man continues to solve and figure out the cubes.
wall.
Table 14: Demo examples in HellaSwag from random and vote-k selection. In general, vote-k selects more diverse and representative instances
24


Reviews:
[{"review_id": "zgwPPma1DH", "reviewer": "Reviewer_bT5a", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method to select a pool of diverse examples to be annotated before evaluation with in-context learning. During evaluation, a cosine-similarity based method is used to retrieve supporting examples from this pool. The combination of these methods outperforms randomly selecting examples. \n\nKey contribution of the paper is the selective annotation method called vote-k to make a pool of diverse examples.\n", "summary_of_the_review": "The proposed method to collect a pool of examples and annotate them performs well in combination with similarity based retrieval for support examples for in-context learning. The authors motivated this approach well, and a detailed analysis is carried out. Some parts about the proposed method are not clear, and there are a few missing and potentially misleading pieces in the paper.", "strength_and_weaknesses": "### Strength\n\nThe efficient approach to select a pool of examples to annotate from unlabeled data which are shown to be diverse. The authors evaluate their method on several tasks including classification, reasoning, dialogue, semantic parsing and generation.\n\nThe authors also provide a detailed analysis of their method compared to other baselines.\n\n### Weaknesses\nI have a few comments:\n\n* Do the authors take the cost of encoding every unlabeled training instance using Sentence-BERT into account when comparing methods? I believe this was not mentioned in the paper.\n* In section 2.1, It is not very clear how the first M/10 examples are labeled. It is mentioned that “the current labeled $\\mathcal{L}$ has M/10 samples” which is used to label other instances. Are they manually labeled?\n* It is stated that the vote-k method encourages diversity. Since all the labels are removed first, does this method encourage or keep inclusivity of all the classes? Have the authors looked at whether their method favors examples from certain classes?\n* In section 2.1, “we conduct experiments three times and report the average score”. What/where is the randomness? Examples selected or model prediction? \n* Figure 2 shows a direct comparison between fine-tuned RoBERTa large and models such as DaVinci-002 while it is stated that the authors do not aim to conduct a head-to-head comparison. What is the comparison the authors are making here and perhaps could it be shown better with another figure? Results from this direct and not fair comparison are also mentioned in the abstract which could be misleading.\n* In section 4.4, I think some more detail is needed as to why the combination of vote-k and similarity retrieval works, but vote-k alone with random selection of supporting examples has almost the same score as random selection.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gRltDu_A77", "reviewer": "Reviewer_ZPh1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies NLP tasks in the context of large-language models that are capable of in-context learning. Specifically, the work advocates for constructing prompts dynamically, based on textual similarity with input query, relying on a small pre-annotated corpus. The paper focuses on describing a new technique for selecting good annotation candidates from an unannotated pool of examples to serve as a database for in-context learning. Specifically, they propose a graph-based algorithm, vote-k, which maps unlabeled examples into a similarity graph. The work then describes how retrieval of the examples can be done at inference time to construct prompts for in-context learning. The paper describes several experiments on different large language models (Codex-da-vinci, GPT-J, OPT-175B, etc.) using 10 different datasets. Finally, the paper also compares the vote-k selection strategy with existing selective annotation strategies previously used in active-learning or few-shot learning regimes.\n", "summary_of_the_review": "Overall, I think this is a well produced work with clear writing and interesting and insightful experiments. The proposed vote-k algorithm is pretty simple to implement and has potential to be of practical use to users of in-context learning. I do think there are some unanswered questions that I list above, including (1) whether this scheme would be robust to a truly unlabeled or uncurated corpus, and (2) the relationship of vote-k with simpler k-means and other clustering algorithms. Given that these issues do not seem overly problematic, I’m recommending the acceptance of this work.\n\n", "strength_and_weaknesses": "## Strengths\n\n1. In-context learning is a hot, rapidly evolving area of research and of high interest to the ICLR and NLP communities.  This paper presents a thorough set of evaluations of the impact of the choice of examples to include in the prompt, demonstrating that this can have a huge impact in final model behavior.\n\n1. The paper is clear and easy to read. The evaluation section contains several interesting results that should be valuable to share with the broader community.\n\n1. The gains of vote-k over other existing active learning methods presented is quite strong (Section 4.5).\n\n1. The system proposed seems to be practical and relatively easy to implement in practice. As such, there is a chance that it could be applied in practice in real world applications of in-context learning.\n\n1. The idea of automatically constructing graphs of unlabeled examples using similarity measures is relatively old in the label propagation literature. The proposed use-case in this work is interesting and novel.\n\n## Weaknesses\n\n1. The main weakness of the paper is the lack of a thorough description and analysis (including ablations) of the vote-k algorithm and its relationship to existing clustering algorithms. I would have hoped for a more complete discussion in Section 2.1, including the contents of Appendix G which seem important for readers to fully understand the proposed vote-k algorithm.\n\n1. One particularly confusing part of vote-k: it was not immediately clear to me how this differs or is better than simple k-means clustering to discover centroids? Once a weighted graph is constructed (as described in Section 2.1, using cosine similarities of sentence-level representations), it should be possible to run any graph-based algorithm for finding centroids. I think a naive baseline (that does not use model prediction confidence scores) would be to use k-means.\n\n1. Related to the point above, I would have liked to see more ablations on the vote-k selection process. How important is it to use the model confidence “stratification” strategy in the algorithm? This is particularly useful to know since, given a very large set of unlabeled examples `|U|`, it can be costly to run all `u in |U|` model predictions to collect confidence scores.\n\n1. One source of bias that was not addressed in the paper relates to the choice of unlabeled examples (|U|). In this study, the unlabeled corpus for all 10 datasets has been extracted from an existing labeled/supervised corpus (that has been previously annotated). This may not seem like it, but I believe there is huge amount of supervision and work simply *choosing* which examples to label in the first place.  In practice, however, for a new task/dataset, this will not be the case if we want to do only “selective annotation”. I was quite unsure (maybe a little skeptical) that the proposed procedure would work for a “random” or uncurated corpus of unlabeled examples.\n\n1. There is also no discussion on label distribution in the vote-k process. Again, I think the current experimental evaluation is taking advantage of curated training datasets where labels tend to be nicely distributed/represented. So focusing on performing input similarity and model confidence may be enough. For example, for some tasks, a fairly random unlabeled corpus may contain a huge number of “negative” examples (say you want to automatically find pairs of sentences for entailment - most random pairs of sentences will be NEUTRAL). Measuring similarities of inputs alone does not seem to be sufficient to match the expected posterior distribution of labels? Can you please elaborate on this point?\n\n1. With respect to the experimental setup, it is a bit jarring the number of combinations of (dataset, model) pairs (Table 1). I understand that it is expensive/infeasible to run all datasets on all models, and I don’t have a good suggestion to improve on this. But I fear that the end result is that subsequent/future work that builds on vote-k may need to re-evaluate your implementation on a partial set of dataset/tasks.\n\n## Other questions\n\n1. Question about potential clarification in initialization of Algorithm 1. Maybe I did not understand this part of the algorithm, but when running initially |U| contains all examples, and |L| is empty. My understanding is that the score in line 4 of Algorithm 1 (also equation in Section 2.1) generates score of 0 for every node since no u in |U| are yet connected to a node in |L|. Is the argmax using some stable ordering when all scores are 0 to guarantee determinism? Does this choice not influence the final set of examples in |L| ?\n\n1. Recently, the Chain-of-Thought prompting (https://arxiv.org/pdf/2201.11903.pdf) has been used by many papers using in-context learning, with reported improvements in model performance (the paper seems to have been cited already 100 times since its release earlier this year). One question this work does not address is whether model sensitivity/variance to retrieved prompts is also present when used in conjunction with chain-of-thought. And whether the vote-k scheme would be useful to limit the number of chain-of-thought annotations needed to improve in-context learning performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dGEZTIZWEg", "reviewer": "Reviewer_3Ufu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors propose a two-step technique for improved few-shot learning with large language models on a variety of NLP tasks:\n1. *Selective annotation*, an unsupervised similarity-based procedure referred to as vote-*k*, that selects diverse and representative examples that are subsequently labeled, and\n2. *Prompt retrieval* at test time that retrieves in-context examples from the labeled set that are most similar to each test instance.\n\nvote-*k* is found to improve performance on 10 diverse tasks by a large margin compared to random selections. Under similar budget constraints, in-context learning is shown to be a much better few-shot learner compared to standard model fine-tuning. vote-*k* is also shown to outperform existing methods for selective annotation.", "summary_of_the_review": "I have rated this submission as a marginal accept for now. Look forward to the authors' responses to my questions.  ", "strength_and_weaknesses": "Strengths:\n1. The authors present a simple technique for in-context learning with large language models that achieves consistently good results across a variety of NLP tasks.\n2. The proposed ideas in isolation do not yield significant improvements. The authors find that the main ingredients for the success of in-context learning are a combination of selective annotation with similarity-based prompt retrieval.\n3. The experimental section is thorough. Along with ablations and trying LMs of varying sizes, their technique is compared against many other existing selective annotation approaches and shown to consistently outperform the latter.\n\nWeaknesses: Some of the design choices need to be elaborated on further and additional analysis-based experiments would also be useful. I elaborate on these further below for the authors to address.\n\n- For the classification tasks, what do the label distributions look like over the labeled subset $\\mathcal L$? Since the selective annotation is based entirely on similarities derived from sentence embeddings, there is nothing explicit ensuring that the label distribution over the selected subset is not skewed. Or, is the label distribution on the annotated subsets derived via vote-*k* indeed skewed for some tasks and the performance improvements are mainly coming from improvements on the labels that are well-represented? Some discussion of this would be useful.\n- It seems like the optimal value of $k$ in vote-*k* would depend on the number of instances in the unlabeled set that changes with the tasks. A single value of $k=150$ was chosen for experiments across all tasks. Could the authors justify this choice further?\n- From the diversity and representativeness measures shown in Table 10 in Appendix F, the difference between Random and vote-*k* does not appear to be very large. In the least, it's unclear how to assess the differences shown in this table. A qualitative analysis might be more revealing here. It might be interesting to see some examples, especially when the annotation budget is 18, of the kinds of instances that get selected depending on the task. t-SNE plots of these selected examples in the larger context of unlabeled instances might also be a good visualization to show. Please comment.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0lls2iOtV9q", "reviewer": "Reviewer_dEqn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an active learning approach to select annotated examples for in-context learning in large PLMs. The proposed method is made up of three components: (1) nearest neighbors: diverse examples are selected from a graph constructed with nearest neighbors; (2) pre-trained language model: more examples are selected according to the scores of the PLM; and (3) prompt retrieval. Empirical studies on a number of public datasets demonstrate the effectiveness of the proposed method in comparison with random selection. ", "summary_of_the_review": "The paper proposes an active learning approach for in-context learning for large PLMs.  \n\nThe method is simple yet effective, and the insights from the experiments are interesting. However, I have a few concerns:\nRegarding to comparison: \n(1) can you compare vote-k with the following alternative? (a) replace vote-k with random selection; (b) run PLM and obtain confident scores; and (c) select annotations according to the confident scores. \n(2) can you compare the proposed method with some soft prompt tuning method? \n\nRegarding to presentation.\nFrom the description in Page 3, it seems that the remaining 9M/10 examples are selected according to the confident scores, but  Line 11-16 in Algorithm 1 has nothing to do with the confident scores. \n", "strength_and_weaknesses": "Strength:\nThe proposed method is simple yet effective.\nThe paper is well-written and thus easy to follow.\nThe empirical studies provide interesting insights about how the method works. \n\nWeaknesses:\nSome important alternatives and baselines are not involved in the comparison.\nThere is inconsistency in presentation. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster