Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Authors: Neel Guha
Mayee F Chen
Kush Bhatia
Azalia Mirhoseini
Frederic Sala
Christopher Re
Venue: NeurIPS 2023 poster (2023)
Keywords: language models
prompting
embeddings
weak supervision
TL;DR: We use the embedding spaces of small language models (e.g. BERT) to identify and correct errors in the few-shot predictions of larger language models.

Abstract: Recent work has shown that language models' (LMs) prompt-based learning capabilities make them well suited for automating data labeling in domains where manual annotation is expensive. The challenge is that while writing an initial prompt is cheap, improving a prompt is costly---practitioners often require significant labeled data in order to evaluate the impact of prompt modifications. Our work asks whether it is possible to improve prompt-based learning _without_ additional labeled data. We approach this problem by attempting to modify the predictions of a prompt, rather than the prompt itself. Our intuition is that accurate predictions should also be consistent: samples which are similar under some feature representation should receive the same prompt prediction. We propose Embroid, a method which computes multiple representations of a dataset under different embedding functions, and uses the consistency between the LM predictions for neighboring samples to identify mispredictions. Embroid then uses these neighborhoods to create additional predictions for each sample, and combines these predictions with a simple latent variable graphical model in order to generate a final corrected prediction. In addition to providing a theoretical analysis of Embroid, we conduct a rigorous empirical evaluation across six different LMs and up to 95 different tasks. We find that (1) Embroid substantially improves performance over original prompts (e.g., by an average of 7.3 points on GPT-JT), (2) also realizes improvements for more sophisticated prompting strategies (e.g., chain-of-thought), and (3) can be specialized to domains like law through the embedding functions.

Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Neel Guha Stanford University
Mayee Chen Stanford University
Kush Bhatia* Stanford University
Azalia Mirhoseini Anthropic
Frederic Sala
University of Wisconsin Disconsin-Madison
Christopher Ré Stanford University
Abstract
Recent work has shown that language models' (L.Ms) prompt-based learning ca- pabilities make them well suited for automating data labeling in domains where manual annotation is expensive. The challenge is that while writing initial prompt is cheap, improving a prompt costly-practttiones often require signifi- cant labeled data in order to evaluate the impact of prompt modifications. Our work asks whether it is possible to improve prompt- based learning without additional labeled data. We approach this problem by attempting to modify the predictions prompt, rather than the prompt itself. Our intuition is that accurate predic- tions should also be consistent: samples which are similar under some feature representation should receive the same prompt prediction. We propose EMBROID, method which computes multiple representations of dataset under different embedding functions, and uses the consistency between the LM predictions for neighboring samples to identify mispredictions. EMBROID then uses these neigh- borhoods to create additional predictions for each sample, and combines these predictions with simple latent variable graphical model in order generate final corrected prediction. In addition to providing theoretical analysis of EMBROID, we conduct rigorous empirical evaluation across six different LMs and up 95 different tasks. We find that (1) EMBROID substantially improves performance over original prompts (e.g., by an average of 7.3 points on GPT-JT), (2) also realizes improvements for more sophisticated prompting strategies (e.g.. shain-of-tthughh and (3) can be specialized domains like law through the embedding functions.
Introduction
Acquiring labeled data for domains like medicine and law is essential to training machine learning models performing basic data analysis (c.g., "how many contracts contain a choice-of-fie ofoorummmmmmo-mmmmmmmmmm clause" or "how many patient medical histories discuss an adverse reaction to a drug?") [5] [8] [9]
However, building large labeled datasets is difficult, and efforts like 255 show that manual labeling with domain experts is cost-prost prosiprrhibiiive. Recent works have begun exploring if language models
(LMs) could learn annotation tasks in-context [6] and replace manual labeling scale [3] 15 23 B1]
The promise of this approach is that LMs' in-context capabilities enable them to learn tasks from descriptions of the the task (i.e., prompts). However, the challenge is that producing high performance
'heee authors contributed equally this work.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Existing work has studied how to engineer better prompts through various textual modifications.
We study how to idantify and flpp erroneous prompt predictions, wilhout using laboled data.
X1
capital
Language Model
Prediction
Prediction: No
X2
Chelsou lost these yoals
Prompt: Prediction No team?
Embroid
Prediction:
goals the FIF final
Prediction Vee...
Prodiction: Yes
Embroid
RoBERTa Nearest Ne geghborr
Or vater
BERT Nearest Neighbora for:;
CERT embedding
Corrected predictior:
RoBeRTo emberding oding space vote: Yes
for function
Combine computed with iniginall predictionn using graphical model generate corrected prediction
compute majority voze for each embecdinng function
Figure I: The EMBROID method for prompt-patching.
prompts still expensive, as practitioners require labeled data in order measure the impact of modifications to prompt [51]. Existing work has thus focused on how domain experts can optimally construct prompts for a task (prompt-engineering) while minimizing reliance on labeled data [1 58 64). Yet, because language models are sensitive to even small changes in prompt
language, these techniques are imperfect and still produce erroneous predictions 22 38 (7).
Our work approaches the challenge of improving prompt performance without labels from an orthogonal perspective: given the predictions of any prompted LM, can identify and correct mis-predictions using unlabeled data? We describe this as the problem of prompt prompt-patching In the context of data annotation tasks, prompt-patching methods should meet three goals. First, they should be theoretically explainable, so that practitioners can understand when and how to apply them. Second, they should be fast, that practitioners can efficiently integrate them into existing workflows. Finally, they should rarely be wrong, so that they don't worsen the performance of predictions.
Our work presents EMBROID: method for automatically identifying and correcting LM predictions with unlabeled data and no expert supervision. Recent work has shown that for many tasks, samples close-by in embedding spaces (produced by models like BERT) have the same label (7). EMBROID applies this intuition to the prompt-patching regime. Specifically, after LM predictions for all samples have been generated, EMBROID retrieves the À most similar samples for each input, under N different embedding functions. For each embedding function, EMBROID computes a saaled-modffied majority vote over the LM's predictions for the k retrieved samples. EMBROID then combines these N votes with the original LM prediction for the test sample using simple latent variable graphical model that learned with fast method-of -f-mmmenss estimator [4] The intuition behind EMBROID that good prompts are smooth with respect to their predictions over a dataset-sample samples which are proximate under an embedding function should receive consistent predictions. Thus, modifying the predictions of prompt to increase neighborhood agreement can improve the accuracy of those predictions. Lastly, because single embedding space may imperfectly capture similarities between samples, retrieving neighbors from multiple embedding spaces improves robustness (28 40
Because EMBROID relies on weak-supervision- the subject of recent rigorous study 77 is possible to thcoretically analyze and explain why and when EMBROID will improve performance. In particular, we find that performance is function of the quality of the embeddings and the performance of the initial prompt. We also empirically study EMBROID, conducting experiments over six LMs, on up to 95 tasks, with several different prompt strategies. We find that EMBROID rarely worsens performance, and often improves F1 by a substantial margin. For instance, EMBROID improves GPT-3.5 by an average of 4.9 points Fl per task, and GPT-JT by an average of 7.3 points per task. The magnitude of EMBROID's 's gains are such that it enables a 1.3B parameter model to outperform an instruction-tune 6.7B parameter model. EMBROID also complementary to advanced prompt engineering strategies, and achieves performance improvements when applied to prompts designed

---

using chain-of -itt-of--hae thought 164), AMA 21, and selective annotation [58]. Finally, EMBROID can be extended to specialized domains like law, through the use of already-available domain specific embeddings.
Succinetly, contributions in this paper are: (1) EMBROID, simple prompt-patching framework for improving LM predictions over text classification tasks; (2) a theoretical analysis of EMBROID which explains performance improvements in terms of embedding smoothness and base accuracy; and (3) an empirical evaluation of EMBROID covering up to 95 tasks and six different LMs.
2 Related work
Improving LM performance Improving the in-context generalization abilities of LMs has been intensely studied. The first family of approaches focuses on adapting LMS in order make them more amenable to prompting. This includes task-specific finetuning [26] 27 47 training on instruction data [9]. , RLHF [50], and weight-surgery methods which attempt "correct" incorrect information stored in model weights [10 [2] 43 44]. A second family of approaches explores strategies for optimizing prompts to models, either through the specific textual features of the prompt [29] 45
64]. the use of task decompositions or LM recursion [2], implicit prompt representations 33 37)
or external databases [46). Prompt-patching in contrast, focuses on identifying mistakes in the predictions generated from particular prompt. The most related approaches are aggregation methods,
in which the outputs of multiple prompts are combined with an ensembling method [2] 41] We find
that EMBROID outperforms many such baselines, and can be applied to enhance their outputs.
Weak supervision EMBROID leverages statistical techniques developed in the weak supervision literature. The objective in weak supervision is to generate probabilistic labels for unlabeled data by combining the predictions of multiple noisy heuristics [4 53 54 57 62 66]. EMBROID'S novelty is that it uses embeddings construct additional synthetic predictions, which are combined with the original predictions. In contrast, recent weak supervision approaches which incorporate embeddings use them produce more fine-grained accuracy parameters [7], detect and discard training points (34),
and as the basis for label propagation with final weak supervision predictions [5
3 Problem setup and background
Problem setup Our problem setup comprises three elements: an unlabeled dataset, predictions from a LM for each sample in this dataset, and embedding representations of our dataset. Our goal is to improve the accuracy of LM predictions, by using the embedding representations to identify and correct predictions likely to be incorrect. Because recent work has explored how predictions from multiple prompts can be combined for a task (2). we present a generalized version of EMBROID in which we have access to multiple LM predictions. In our empirical evaluation however, we show that EMBROID performs well regardless of the number of predictions per sample available.
More formally, we focus on a binary classification task where x E X' denotes sentence or paragraph and E y = {-1, 1] is the binary label. We assume we are given an unlabeled dataset D = {z, } of nt, points. Each point is sampled i.i.d. from a distribution Pz. and there exists true underlying distribution IP on the joint (z. y). Following the true few-shot regime [5I], we assume the only labels available are those used in the prompt. We denote language model (e.g., GPT-3)
ALLM. and task-specific prompt as é. which prepends task instructions to input J (e.g.. "Does the clause contain an audit provision? Yes or No."). We consider the "prompt" to include both the task description and the in-context samples, consistent with [70]. The prediction this prompt induces for ÄLLM over x is ()xx)) E > Varying @ by changing the task description, in-context demonstrations, or punctu- ation will alter the prediction generated for x. For a set of m prompts (Ø),.... (m.) we denote their respective predictions
ALLM (d (x))] For convenience, we denote ) as (x(2) or X; when the x is obvious. and similarly use X instead of A(x). We distinguish between two regimes: the angle-promppt regime with m we have access to LM prediction for each point æ, while in the multi-prompp prompt regime with m > 1, we have access to multiple predictions.
x as a vector of weak sources A(x) = (x))),
We assume a task-specific mapping function which allows a practitioner to associate a text generation from an LM to particular class prediction in V
3

---

Algorithm EMBROID: Correcting LLMs with embeddings
Input: Unlabeled data D, LLM predictions X(x) for each & D, embedding models & (Ei,..., Ex) shrinkage parameter т, nearest neighbors parameter
for all unlabelled x: E 'D do
for all embedding models E; E do
Compute k-nearest neighbors (2)
Compute smoothed neighborhood prediction ( using A, (N), and T using eq. end for end for Solve graphical model Pr(y, A(x), 2mm())) in eq. (3) with triplet method over D (Algorithm 2
for all unlabeled x E D do
Sample ia Pr(v))(x);; Am (m))) end for
Output: Label set Y (Ux D
We assume access to N embedding models [Ei...., Ex]. each represented as fixed mapping E; : X' + Z; from an input I to an embedding vector 2. These auxiliary embedding models provide representations of which encode different types of similarity information. Through model repositories like HuggingFace [65], it is possible to download number of models which generate These models have the property
representations for text sentences (e.g., BERT or RoBERTa [12 42
that semantically similar sentences are close-by in embedding space
28 40
Weak supervision background Weak supervision uses a graphical model to combine votes from multiple noisy sources into single prediction, by estimating the accuracy of each source. It mod- els Pr(y. X(x)) as a latent variable graphical model and uses ğ = argmax, (rrnlx,,
produce label estimates, where Pr represents the learned model. The graphical model is based on graph G (V, E), where V = y J À and E consists of edges from y to each Aj. We assume no dependen- cies between sources, although simple extensions can incorporate them [6 The formal graphical model is:
Pr(y, A(x)) = - exp( Ogy + +8 A(x)y)
(I)
(II)
where the partition function used for normalization, (T) represents a label balance term with parameter Oy controlling the prior of Pr(y = 1). and (7T) represents the source accuracy term where each 0; an accuracy parameter for the ith source. Note that from this model, sources are conditionally independent: X; IL \şly for any i, E [m/]. Our use of this model has two steps. First, we must learn the accuracy parameters
Pr(y, A(x)) without access to y. We use the triplet method introduced in [14,, which is efficient method-of of-mmentss estimator for the parameters. Then, at inference we compute Pr(y)A(z)) (z))). Appendix B contains more details.
4 EMBROID
the embedding models & to compute additional votes for each Let (x) C 'D be the k-nearest neighbors of sample x under the embedding function Ej. We define the smoothed neighborhood prediction vector ) 0, 13 follows, with (x)
First, EMBROID
being the ith element:
{,1(()
Σ \x(#)
(ENN,.((*)
{+(l(() > 7
(2)
Aum. [i](x)
{+(((() <
o.W.

---

where Tf E [-1,1] and Ti
1 act as shrinkage parameters for A which control the level of
agreement amongst the neighbors necessary to generate a particular vote. The scalar Aam. [=](2) (=)x) is the average vote of X; amongst the neighbors of x: in F.j. When A is sufficiently positive, i.e.,
Aam.j [ T." EMBROID sets A (z) to be positive vote. When (z) is sufficiently
negative, i.e., Amm.. (+)( < T- EMBROID sets {am.)(](xx) to be negative vote. Otherwise, Asm.j
{ (2r) is set to be an abstain. The intuition is that Asm (a) will be an accurate vote over whenever two conditions are met: (1) the LM is generally accurate, i.e., > is usually correct, and (2)
Ej is smooth, i.c., nearest- neighbors share the same task label.
Next, we augment our base model in equation ( to incorporate these auxiliary neighborhood predictions Aum [Aam. Asm.N)) E (-1,0, Nm computed using the embeddings:
Pr(y. A, Aum) Z exp
OgN 020 An
ay Pamm.V)
(3)
1=1
where the vector a; Rn represents the quality parameters for the jth embedding model when used with the m different prompts. To solve this model and produce label estimates, we note that has the same format as ① if we concatenate À and Xum into
of weak sources. Therefore, use the triplet method from [4] to learn parameters and output estimates Pr(y|A(x), Amm (x))
for each x E D at inference time (see Appendix B for details).
Parameters 9 and jj in (3) allow us to trade-off two different sources of information -onnn---on one presented by directly prompting an I.M to obtain a label and the other by incorporating similarity information from the embedding models-ann to further account for varying error modes among the embedding models. Our use of the neighborhood predictions in (3) yields more expressive model than the standard weak supervision framework solely LLM predictions in (I). which can recover when 0, and can thus help make corrections to the LLM predictions. practice, we find that setting B[A:] (i.e., the average source vote) yields good performance (Appendix G
5 Theoretical analysis
We analyze EMBROID, discussing the advantages of using Asm in addition to A, and show that embedding smoothness and base prediction accuracy play a critical role information gain. Appendix F provides synthetics demonstrating these tradeoff's and comparing weak-supervision baselines.
First,
we provide
result on the generalization error of our model Define the generalization error as the expected cross-entropy loss, log (xx). >x)()))]
Pr(y) A, Asm).
We use [A represent [A, Asm] and denote by 0max = max, E [A((r)y] the largest accuracy (scaled to [),, of any source, and by bmin min,, {E [A,A,] É [ the minimum expected pairwise product between any two sources. Assume that all sources are better than random, e.g., Pr(A: y) 0.5. These terms and assumptions are from using the triplet method.
L(,, Asm; D) E (.),(
to
Proposition 5.1. Suppose that the data x. y, X, Amm follows the model in error of Pr(v| A, >mm) can be decomposed into
3 The generalization
C(N + 1)m
L(X, Asm: D) < H A, Asm)
+o(1/nw),
Th
Irresucible Eroor
Variance
where
3(1
812
min
In the bound above, the variance term comes from estimation error when learning the parameters via the triplet method. The irreducible error depends on quality of and >am If knowledge of the LLM prediction and neighborhood prediction significantly reduces uncertainty in y, the conditional entropy term H Axx) Asm (z)) is low.
Information gain from using both X, Amm We compare upper bounds on generalization error when both X, Anm modeled versus when only X is modeled, as in (I) corresponding to classical weak
5

---

supervision. Based on the bound in Proposition 5.1 modeling both À and Asm increases the variance term by a constant multiplicative factor.
Here, we examine how the irreducible error is affected, that is, the difference H(y H(y X, > ). Since this quantity is always nonnegative, focus bounding the pointwise difference in condi- tional entropy-which we call the information gain-ffor a given 2o on which the LLM is incorrect. For simplicity, suppose have one embedding E. An embedding the label if
M-smooth with respect to
Pr(ū = cly = c, [[ [ < 2 MÆ(e),
where e V. > 0 and (.) E [0, 1] is decreasing in its input. Note that this definition requires knowledge of ground-truth labels, and is thus impossible to use as metric for selecting which embeddings to use.
Define 3; = Pr(A: = y) as the accuracy of X; and PA = Pr(y = 1 (zx)) ) as the prediction on zo
given only access A. Let E₁ maX |E(x) E()) be the maximum distance between To and its & neighbors. Without loss of generality, assume the label on zo is 1.
Theorem 5.2. Assume that E is M-smooth. The pointwise information gain on zo is
Hzy))(ro) (xo)) Amm (xo)) z
2(1 px)
[
0.5) 0.5
where BuNu. = Pre-NN, (Ae(0) = y) 2 B.ME(ss) is the neighborhood accuracy.
few observations on the bound are order.
Improvement over WS: If the neighborhood accuracy bounded sufficiently far from and & is large, using EMBROID has better irreducible error than just using X. For example, setting m = 1, = 10, SNN,.. = 0.7, and Px = 0.25 gives us an improvement of 0.076 nats.
Smoothness: If E is highly smooth, then Mg($s) will be large and irreducible error will be small.
Base prediction accuracy: If the original sources À have high accuracy (3,), irreducible error will be small.
Additionally, we observe that if px a high-quality prediction close to the true label 1, the information gain is small. Choice of the & parameter presents performance trade-off: increasing k will increase EK and incorporate
aawher--wwa,, less reliable predictions, but will also reduce the noise of the majority vote. We also comment on the information gain when using both X and Asm over just Asm in Appendix C
6 Results
Our empirical evaluation focuses on three questions: (1) How robust is EMBROID'S performance across LMs? (2) How does EMBROID, as prompt-patching method, compare to high performance prompt-engineering methods? (3) How sensitive EMBROID to the embeddings and dataset size?
We study tasks where sentence embeddings can capture information relevant to the task, leading us focus on sentence classification datasets. We consider collection of 95 class-balanced sentence classification tasks, derived from binarizing existing multi-class legal, scientific, and general domain classification benchmarks like CUAD, AGNews, DBpedia- 14, FewRel, and several others [2]] Example tasks include, "Classify if the following texts discuss a recording label" or "Classify if the following contractual clauses contain an audit rights provision."
Tasks
25
80
67 69
We hope explore multi-class extensions and more complex reasoning tasks in future work.

---

LM
Win rate (%)
Avg. Improvement (F1)
J1-Jumbo (176B)
72.2
10.6
API-Access Models
GPT-3.5 170B)
80.6
4.9
Open Source Bloom (7.1B)
91.2
10.1
OPT (6.7B)
91.2
11.6
Instruction Tuned
GPT-JT (6B)
89.1
7.3
Table 1:
We evaluate the extent to which EMBROID improves the original prompt on different models in terms of win rate and relative improvement (defined in-line). All models are with three trials. For each model, we report the percentage of tasks (across all trials) for which EMBROID improves, and the average improvement (in Fl points), Additional details provided in Appendix.
Choice of embedding models Following prior work illustrating the benefits domain specific representation (20,7]] EMBROID uses different embeddings for each task domain. For law tasks, rely on two BERT-variants trained on different legal corpora [24] 711. For science tasks, we rely on
three BERT-variants trained science, biology, and medical texts 3 17 35). For general domain
tasks, we rely on BERT, Roberta, and SentenceBert embeddings 12 55
Prompts Prompts are constructed using fixed instructions, and by manually selecting three random samples (from each class) as in-context demonstrations (Appendix E... We follow the true few-shot regime [51], in that we assume the only labeled data available to the data scientist are the labels used for in-context demonstrations, Prior work has found this regime most realistically represent real-world workflows.
and J1- Jumbo 39 Because API models raise significant privacy and compliance concerns for data scientists working with sensitive data [16], we also evaluate on open-source models. We select models in the 6-7B parameter range, as these are the largest models which fit commonly available 40GB A100 machines. Specifically, we evaluate Bloom [56) and OPT [68]. Given the increasing popularity of instruction-tuning we also evaluate GPT-JT (60), an 6. 7B parameter instruction tuned version of GPT-J. Because of cost-constraints, we evaluate API-access models on representative selection of 12 tasks, while evaluating all other models on the full suite 95 tasks. Appendix D provides details.
Models We evaluate on two API-access models: GPT-3.5
6.1 By how much does prnnpt-patchinggg improve performance?
Performance across LM families We examine if EMBROID achieves improvements for different types of LMs. For each LM, we select three different combinations of in-context demonstrations (i.e., three prompts), generate predictions for each prompt, and apply EMBROID to independently each prompt's predictions. This produces 3 285 trials for open-source models, and 3 12 36 trials for API-models. We report win-rate, i.e., the proportion of trials for which EMBROID outperforms the original predictions, and improvement, i.e., the average difference in F1 points (across all trials) between EMBROID and the original predictions.
As Table illustrates, EMBROID improves performance for substantial proportion of prompts, by substantial margin, across all models. On GPT-3.5 for instance, EMBROID achieves win-rate of 80.6%, with an average of improvement of 4.9 points. EMBROID also improves for open source models, with a win-rate 91.2% on OPT-6.7 and an average improvement of 11.6 points. Finally, EMBROID achieves similar gains on an instruction tuned model, with win-rate of 89.1% 1% and an average improvement of 7.3 points.
Performance when prompts are good We additionally investigate how EMBROID'S performance improvements change as a function of the performance of the base prompt. Hypothetically, one could imagine that better performing prompts are smoother with respect embeddings, thus diminishing (or negating) EMBROID. In Figure 2 (upper left), we plot the improvement of EMBROID against the performance of the base prompt for GPT-JT. Even when the base prompt performs well (i.e., F1 0.8), EMBROID improves 89% of tasks by an average of 4.1 points.
7

---

LM
MV
Liger
FlyingSquid
AMA
EMBROID-1
EMBROID-3
J1-Jumbo
47.4
48.7
50.5
60.7
60.4
64.5
GPT-3.5
81.4
82.5
82.1
84.7
83.9
86.0
Bloom-7.1B
54.6
55.8
54.3
63.0
64.7
69.1
OPT-6.7
46.1
46.8
46.3
56.3
59.8
64.2
GPT-JT 69.3
69.4
70.1
74.6
75.1
79.0
Table 2: We evaluate how EMBROID compares to common ensemble approaches for improving prompt prediction performance. All ensemble baselines are run with three sets of predictions. EMBROID-I run with one set of predictions, and EMBROID-3 is run with three set predictions. For each method, report the average macro-FI over all tasks. We observe that EMBROID- I is competitive with ensemble methods which use many more predictions, while EMBROID-3 outperforms all other methods by substantial margin.
Measuring performance in parameter count trend in recent literature has been to measure the magnitude of improvements to prompt performance in terms of parameter count [2]. by showing how a particular method makes a smaller method equivalent performance a larger model. We find that EMBROID enables the non-instructed tuned 1.3B GPT-Neo model to outperform instruction tuned 6.7B model; across all trials, GPT-JT scores an average of 67.8, while EMBROID +GPT-Neo-1.31 scores an average of 68.5.
6.2 Comparing prompt-patching to prompt-engineering
Our work distinguishes between prompt-construction -eihoddsswhihhh control how a prompt is generated-and prompt-patching methods attempt identify and correct errors in the predictions produced by prompt. We use EMBROID to further study the difference between these frameworks in two ways. First, we compare EMBROID'S performance improvement over a base prompt to that of several specialized prompting strategies. Second, we examine the extent to which EMBROID- when applied the predictions produced by these prompting strategies can generate further performance improvements. We study three prompting strategies:
1. Ensemble strategies, in which the predictions of multiple prompts are combined using an unsupervised ensembling model. Specifically, compare to ensembling methods pre- viously studied for L.MS (AMA (2) and majority vote [41), one ensembling method which incorporates embedding information (Liger (7)), and one well regarded weak supervision baseline (FlyingSquid [14).. Each bascline is run over the predictions generated by three different prompts.
2. Chain-of-thought prompting 64] in which for each in-context demonstration, we provide step-by-step explanation for the demonstration's 's label.
3. Selective annotation (SA) [8 in which
embeddings select subset of & data samples to label, and then, for each input sample, retrieve the most similar samples (under some embedding function) from this pool use as in-context demonstrations.
Ensemble methods We evaluate two versions of EMBROID. In the first version, run EMBROID with the predictions of only one prompt (EMBROID-1). In the second version, we run EMBROID with the predictions of three different prompts (EMBROID-3). Note that this requires performing inference over the few-shot LM three times for each sample, which can be expensive. This baseline comparable to applying EMBROID to the outputs of an ensemble method. In Table observe that EMBROID-1 is competitive with the ensemble baselines (while using substantially fewer predictions), while EMBROID-3 consistently outperforms these baselines (across different LMs).
Chain-of-though We compare EMBROID to chain-of- (CoT) prompting for a subset of a representative subset of 10 tasks on GPT-3.5. For each task, we manually construct a base prompt consisting of six demonstrations, and CoT prompt where an explanation is provided for each demonstration. We first find that EMBROID'S 's performance improvement over the base prompt exceeds that of chain-of-though prompting (Table 3). Using EMBROID to modify the base prompt is better

---

Base prompt
+CoT
EMBROID
CoT EMBROID
76.3
80.1
81.9
85.4
Table We evaluate EMBROID compared to, and applied to, CoT prompting on GPT-3.5 for a subset of 10 tasks. We report the average across the studied tasks.
0.85
Embroid
0.6
0.75
SA
Embroid
0.4
Rand
0.70
Rand Embroid
0.4
0.6
0.8
1.0
20
60
80
100
Original Prompt Performance
Number of labeled paints
Embroid improvement (FI)
0.20
0.20
Embroid Insprovement
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
-0.05
0.055
0.6
0.7
0.8
0.9
1000
2000
3000
Embedding smoothness
Dataset size
Figure 2: Upper left: The EMBROID F1 plotted against the F1 score for the original prompt for GPT-JT. Even for high performing prompts, EMBROID is capable of improving performance. The dashed line y = æ is plotted for visual aid. Upper right: A comparison of EMBROID to selective annotation (SA) over all tasks for GPT-JT. Bottom left: For each task (using GPT-JT), we plot the performance improvement of EMBROID against the average smoothness of the embeddings used. We observe positive correlation performance improvement of EMBROID against the size of the task.
0.39). Bottom right: Across all tasks, measure the
on average than CoT' prompting, outperforming CoT on six out the ten tasks. Second, applying EMBROID to predictions generated by a CoT prompt yields further improvements, outperforming vanilla CoT predictions on eight of ten tasks.
Selective annotation (SA) We compare EMBROID selective annotation with label budget of [6, 25, 50, 100] (Figure 2 upper-right). For each task, we run selective annotation using a domain specific embedding. EMBROID (applied to prompt with randomly chosen samples) outperforms selective annotation with label budget of 25 samples. When label budget of 100 samples is available, EMBROID improves the performance of prompt constructed using selective annotation 88% tasks, by an average of 4.3 points.
6.3 Ablations
Finally, we perform several ablations of EMBROID to study how performance changes as a function of (1) the domain specificity of the embedding used, (2) the quality of the embedding spaces used, and (3) the size of the dataset. Additional ablations are presented in Appendix G

---

Domain specific embeddings improve performance We compare how performance on the legal and science tasks changes when we shift from domain specialized embeddings to general domain embeddings. On law tasks for GPT-JT, we find that using two legal embedding spaces outperforms using BERT and RoBERTa for 77% of tasks, by up science tasks for GPT-JT, we find that using two science embedding spaces [3] 35] outperforms using BERT and RoBERTa for 92% of tasks, by up to 4.3 points Fl on certain tasks.
points Fl on certain tasks [24 71]. For
Embedding quality Building on Section 5 we compare EMBROID's performance improvement over the base prompt to the average smoothness of the embedding spaces with respect to each task (Figure 2). We observe positive correlation: smoother embedding spaces are associated with larger performance gains (with a Pearson coefficient of r = 0.39). Applying this insight, we explore how performance changes when extremely high quality embeddings are added. For subset of 19 tasks generate OpenAI meext-embedding aade--da-0000 -00a embeddings, and find that adding them to EMBROID improves performance by up 13 points F1 (at average of points across all studied tasks).
Dataset size Finally, we study how EMBROID'S performance improvement changes as the dataset size changes. Because EMBROID relies nearest-neighhors in different embedding spaces, might expect performance to be poor when the dataset being annotated is small. In Figure (bottomm (bottom right), that EMBROID achieves performance improvements for "small" datasets with only several hundred samples.
Conclusion
We study the problem of improving prompt-based learning, by developing a method (EMBROID) for detecting and correcting erroneous predictions without labeled data. We validate EMBROID across range of datasets and LMs, finding consistent improvement in many regimes. We take a moment to address the societal impact of our work: while we do not foresee any direct harmful impacts arising from our work, we caution that any use of language models in meaningful applications should be accompanied by conversations regarding risks, benefits, stakeholder interests, and ethical safeguards.
8 Acknowledgements
We are grateful to Arjun Desai, Avanika Narayan, Benjamin Spector, Dilara Soylu, Gautam Machiraju, Karan Goel, Lucia Zheng. Rose E. Wang, Sabri Eyuboglu, Sarah Hooper, Simran Arora, Maya Varma, and other members the Hazy Research group for helpful feedback and conversations on this project.
We gratefully acknowledge the support of NIH under No. U54EB020405 (Mobilize), NSF under Nos. CCF1763315 (Beyond Sparsity), CCF1563077 (Volume Velocity). 1937301 (RTML) and CCF21067077 US DEVCOM ARL under No. W911NF-21-2-00 ing); ONR under No. N000141712286 (Unifying Weak Supervision); ONR N00014-20- 1-2480: Understanding and Applying Non-Euclidean Geometry in Machine Learning; (00014201227) (NEP- TUNE); NXP, Xilinx, LETICEA, Intel, IBM, Microsoft, NEC, Toshiba, TSMC, ARM, Hitachi, BASE, Accenture, Ericsson, Qualcomm, Analog Devices, Google Cloud, Salesforce. Total, the HAI-GCP Cloud Credits for Research program, the Stanford Data Science Initiative (SDSI), the Wisconsin Alumni Research Foundation (WARF), the Center for Research Foundation Models (CRFM), and members of the Stanford DAWN project: Facebook, Google, and VMWare. The U.S. Government is authorized reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.
(Interactive Human-AI Team-
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views, policies, or endorsements, either expressed or implied, of NIH, ONR, the U.S. Government.
10

---

References
[1] Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. Large language models are few-shot clinical information extractors, 2022.
[2] Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, and Christopher Ré. Ask me anything: A simple strategy for prompting language models, 2022.
[3] Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv: 10676, 2019.
[4] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2 07258, 2021.
[5] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 20/9 world wide web conference, pages 491-500 2019.
[6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--901, 2020.
[7] Mayee F. Chen, Daniel Fu, Dyah Adila, Michael Zhang, Frederic Sala, Kayvon Fatahalian, and Christopher Ré. Shoring up the foundations: Fusing model embeddings and weak supervi- sion, 2022.
[8] Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. On the relation between sensitivity and accuracy in in-context learning. arXiv preprint arXiv:2209. SCV:22209.0766,, 2022.
[9] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetueed language models. arXiv preprint arXiv:22 10. /416, 2022.
[10] Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2 104, 104.08696, 2021.
[11] Franck Dernoncourt and Ji Young Lee. Pubmed 200k ret: dataset for sequential sentence classification in medical abstracts. arXiv preprint arXiv:1710.0007/ 2017.
[12] Jacob Devlin, Ming-Wor Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04805. 2018.
[13] Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq Joty, and Boyang Li. gpt-3 a good data annotator? arXiv preprint arXiv:2212.10450 2022.
[14] Daniel Fu, Mayee Chen, Frederic Sala, Sarah Hooper, Kayvon Fatahalian, and Christopher Re. Fast and three-rious: Speeding up weak supervision with triplet methods. Hal Daumé and Aarti Singh, editors, Proceedings of the 37th International Conference Machine Learning, volume 1 19 of Proceedings of Machine Learning Research, pages 2280-3291. PMLR, 13-18 Jul 2020.
[15] Karan Goel, Sabri Eyuboglu, Arjun Desai, James Zou, and Chris Ré. Meerkat and the path to foundation models as a reliable software abstraction. 2023.
[16] Karla Grossenbacher. Employers should consider these risks when employees use chat- gpt. www.ssaarccssaaaoorr.dddelle/goe employees--ss--chtggp. 2023.
[17] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specifie language model pretraining for biomedical natural language processing, 2020.

---

[18] Neel Guha, Daniel E Ho, Julian Nyarko, and Christopher Ré. Legalbench: Prototyping collaborative benchmark for legal reasoning. arXiv preprint arXiv:2209.06120. 2022.
[19]
Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael I. Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, and Zehua Li. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models, 2023.
[20] Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo. Iz Beltagy, Doug Downey, and Noah A Smith. Don't stop pretraining: adapt language models to domains and tasks, arXiv preprint arXiv:2004. 1096, 2020.
[21] Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-ard evaluation. arXiv preprint arXiv: 10147, 2018
[22] Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.1366 2021.
[23] Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. Annollm: Making large language models to be better crowdsourced annotators, 2023.
[24] Peter Henderson, Mark S Krass, Lucia Zheng, Neel Guha, Christopher D Manning, Dan Jurafsky, and Daniel Ho. Pile law: Learning responsible data filtering from the law and a 256gb open-souree legal dataset. arXiv preprint arXivv2207.00222 2022.
[25] Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. CUAD: an expert-annotated NLP dataset for legal contract review. CoRR, abs/2103.06200 2021.
[26] Cheng-Yu Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen- Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes, 2023.
[27] Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. arXiv preprint arXiv:2210.11610 2022.
[28] Qiao Jin, Bhuwan Dhingra, William W Cohen, and Xinghua Lu. Probing biomedical embed- dings from language models. arXiv preprint arXiv:1904.02188 2019.
[29] Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. arXiv preprint arXiv:2205.11882. 2022.
[30] Martin Krallinger, Obdulia Rabal, Saber Ahmad Akhondi, Martín Pérez Pérez, Jesus Santamaría, Gael Pérez Rodríguez, Georgios Tsatsaronis, Ander Intxaurrondo, J. A. Lopez, Umesh K. Nandal, Erin M. van Buel, Ambika Chandrasckhar, Marleen Rodenburg, Astrid Lægreid, Marius A. Doornenbal, Julen Oyarzábal, Anália Lourenço, and Alfonso Valencia. Overview of the biocreative vi eeeemccal-rooeinn interaction track. 2017.
[31] Taja Kuzman, Igor Mozetic, and Nikola Ljubešic. Chatgpt: Beginning of an end of manual linguistic data annotation? use case of automatic genre identification. ArXiv, abs/2303.03953, 2023.
[32] Stanford Legal Design Lab. Legal issues taxonomy, 2023,
[33] Suffolk Law School Legal Innovation & Technology Lab. Spot's training data, 2022.
12

---

[34] Hunter Lang, Aravindan Vijayaraghavan, and David Sontag. Training subset selection for weak supervision, 2022.
[35] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-122 12400 2020.
[36] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning, 2021.
[37] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation, 2021.
[38] Perey Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga. Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint 1.Xiv:2211.09110 2022.
[39] Opher Lieber, Or Sharir, Barak Lenz, and Yoav Shoham. Jurassic-l: Technical details and evaluation. White Paper. A/2/ Labs, 1, 2021.
[40] Lucy H Lin and Noah A Smith. Situating sentence embedders with nearest neighbor overlap. arXiv preprint arXiv: wwt.i/999117770 2019.
[41] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
[42] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1rXiv 2019.
[43] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, 2022.
[44] Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in transformer. arXiv preprint arXiv:2210.07222, 2022.
[45] Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannanch Hajishirzi. Refram- ing instructional prompts to gptk's 's language. arXiv preprint arXiv:2109.07830, 2021.
[46] Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory-based model editing scale. In International Conference on Machine Learning, pages 15817-15831. PMLR, 2022.
[47] Akihiro Nakamura and Tatsuya Harada. Revisiting fine-tuning for few-shot learning. arXiv preprint 1rXiv:1910.00216, 2019.
[48] OpenAl. Gpt-4 technical report, 2023.
[49] Laurel Orr. Manifest.
2022.
[50] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02I: 2022.
[51] Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. Advances in neural information processing systems, 34:1 1054-11070, 2021.
[52] Rattana Pukdee, Dylan Sam, Maria-Florina Balcan, and Pradeep Ravikumar. Label propagation with weak supervision. arXiv preprint 2rXiv:2210.03594 2022.
[53] A. J. Ratner, B. Hancock, J. Dunnmon, F. Sala, S. Pandey, and C. Ré. Training complex models with multi-task weak supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, 2019.
13

---

[54] Alexander Ratner. Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. Snorkel: Rapid training data creation with weak supervision. Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases, volume 11, page 269. NIH Public Access, 2017.
[55] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. arXiv preprini urXiv:19088 10084, 2019.
[56] Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 2022.
[57] Changho Shin, Winfred Li, Harit Vishwakarma, Nicholas Carl Roberts, and Frederic Sala. Universalizing weak supervision. In International Conference on Learning Representations (ICLR), 2022.
[58] Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang. Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. Selective annotation makes language models better few-shot learners, 2022.
[59] Rohan Taori, Ishaan Gulrajani. Tianyi Zhang. Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpaca: A strong, replicable cstruction-followinng model. 2023.
http::/crfm.stanfoonee http://ffntttaooooooddd////////0 http:///waaee00000//////aaaaaaaa
[60] Together. Releasing v1 of gpt-jt powered by open-source ai.
xxyget/erxxy/ Accessed: 2023-01-
blog/releasing- v1- gpt- jppt-jt-paa by-ppooweedo---ppeeee------e source-- ai
25.
[61] Paroma Varma, Frederic Sala, Ann He, Alexander Ratner, and Christopher Re. Learning dependency structures for weak supervision models. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1428-6427. PMLR, 09-15 Jun 2019.
[62] Harit Vishwakarma and Frederic Sala. Lifting weak supervision to structured prediction. Advances in Neural Information Processing Systems, 35:37563-37574 2022.
[63] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11177 2022.
[64] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le. and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.1190 11903, 2022.
[65] Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface transformers: State-of-the- the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38 45, 2020.
[66] Renzhi Wu, Sh Chen, Jieyu Zhang, and Xu Chu. Learning hyper label model for programmatic weak supervision. arXiv preprint arXiv:2207. 13545, 2022.
[67] Jieyu Zhang, Yue Yu, Yinghao Li, Yujing Wang, Yaming Yang, Mao Yang, and Alexander Ratner. Wrench: A comprehensive benchmark for weak supervision. arXiv preprint arXiv:2 109.77777 2021.
[68] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint WrXiv:2205.01066 2022.
[69] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification, Advances in neural information processing systems, 28, 2015.
14

---

[70] Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models, 2021.
[71] Lucia Zheng, Neel Guha, Brandon Anderson, Peter Henderson, and Daniel E Ho. When does pretraining help? assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, pages 159-168, 2021.
15

---

A
Notation
The glossary is given in Table below.
Symbol
Used
Input sentence paragraph x X.
y
Binary task label y E y = (-1,+1).
Unlabeled dataset D = {ai) 1hh of Th, points.
P, Px
The joint distribution of (æz y) and the marginal on x, respectively.
Number of labeled in-context examples used in querying the LLM (5-10 examples).
ALLM()),
Users interact with language model ÄLLM via prompt on x. Number prompts that we have access to.
m
A(x) = (rr),,.... Am (z]] where (x) is shorthand for (
E
The set of N embedding models, & = (Ei...., EN) where each embedding is represented fixed mapping E; X' Z,.
Partition function for normalization (T)
0g. 9
0y is a label balance parameter and each 6, a scalar accuracy parameter for the ith source in (T) The k-nearest neighbors of x in embedding space Ej.
NN,
Aum
(x) D.mm....... x | E {-1, 1] N' where Amm.y = [>am. [1] ..l............. Am [ma]]] and Aam ) (2) the
smoothed neighhorhood prediction (z) in F. (eq. (2)
Shrinkage parameters for determining when >mm. [z) set to 0, -1, -1,or
Og
Vector of accuracy parameters for Fig when used with prompts in (
L(A, Aun.. D)
Generalization error EMBROID expected cross-entropy loss).
amax
The largest scaled accuracy of any source, amax = max; [r..))
bmin
The smallest expected pairwise product between any two sources, Dinn min., [E[A,,,,
ME(-)
An embedding is M -moooth if Pr(ğ = cly = c, [[x) B I E) 2 Ma(e) for all e y and any &
where M=(-) E [0, 1] is decreasing in its input.
8.
The accuracy of A, 8; Pr(A: y)-
PA
The prediction on z given only access px Pr(y 1|x()))). The maximum distance between zo and neighbors, En = ma
EN
[E(x) - E(
Table 4: Glossary of variables and symbols used in this paper.
16

---

B Weak supervision background
In this section, we provide details on the inference and learning procedures for solving the graphical model defined in equation ①T. The content from this section is derived from [1 and 7
Pseudolabel inference. To perform inference, we compute Pr(y)A(x)) for some x E A. This is done via Bayes' rule and the conditional independence of weak sources:
Pr(y 1|A(=)) I Pr(A(x))l==) 1) Pr
(5)
Pr(A(x))
We assume that the class balance is known; for datasets, the class balance is Pr(y 1) 0.5. More generally, it can be estimated [53]. The latent parameter of interest in this decomposition is Pr(A: l 1). which corresponds to the accuracy of X4.
Algorithm 2 Triplet method [14]
Input: Dataset D, weak sources X(x).
for i E [m] do for j, E [m]][i do Estimate É [>rAg] over D, and similarly estimate I [AcAa] and È [AgAu].
Compute aP. end for Calculate average à. Mean (a,
EAAAAA AR]
j.k Vj, [m]
Compute estimated accuracy Pr(A, = y) = 2 end for
l+a.
Output: Accuracies Pr(A: = y) for all i E [m]]
Source parameter estimation: Triplet method. Previous approaches have considered how to estimate Pr(A; 1/y 1) via the triplet method [14), which exploits conditional independence properties. First, by the properties the graphical model in (T) it holds that the accuracy of X; is symmetric: Pr(A; 1ly = = Pr(A, = -lyy 11) Pr(A; y) (Lemma 4 of 77. Therefore,
Pr(A, = 1 = 1) can be written in terms of [Ar]] with E [A,y] = 2 Pr(A = 1ly = 1) - 1.
Define = EE [Aay]. The graphical model in ( tells us that Agy 1 if if A 1 Aylv, which holds for all i E [ı7]] (Proposition 1 of [4]). As a result, E [A x E [A,y] E \^Aッッ²²| E [AAggle which a quantity that can be computed from observed LLM predictions. That is, we have that a,a, E [A,Ay]. If we introduce a third λę, we can generate system of equations over a; aj. 0g in terms of their pairwise rates of agreements:
aga, E [ArAg]
(6)
agag = [AcAe]
(7)
ayaą E DAdde]
(8)
Solving, we get that
E [A IE [APA E [Ayde]
la:|
(9)
and likewise for a,,, ag- If we assume that each weak source is better than random over the dataset, then a, = |asl > 0, so we can uniquely recover the accuracy of each source by selecting two other sources and computing the above expression by using empirical expectations over D. We then set
Pr(A; 1 1) 1+ũ; and plug this into the expression for Pr(y 1|A(x)) in (5).
2
This approach is formally described in Algorithm
17

---

C
Proofs
C.1 Proof of proposition 5.1
We note that [X, Amm can be viewed as set sources in the weak supervision set up used in Therefore,
apply Theorem I from 7 to our problem setting, noting that we do not perform their clustering step and that our predictions do not abstain and output in addition to {-1,1). We have a total of (N 1)m sources, so
L( sm: D) S Hy Amm) + 3(1 62 blin
1
(N + 1)m
+ o(1/na)) (10)
88 min (1 a m
J min
D2
muin
C.2 Proof of theorem
We write the change in point- point-wise wise irreducible error follows:
H (y A(zo)) (y)))(x)) Asm (xo) = E - log Pr(y) (3) A (xo)) + log Pr(yp ((xo), Asm (20))] (11)
Pr(y ( Asm (zo))
E
log
(12)
Pr(v(>(aa))
Pr(A(za): Amm (Io) Pr(y)
Pr(A(zo))
E
log
(r()) Asm (2o)
Pr(A(ro)ly) Pr(y)
(13)
Pr( (1o) ((xo)) E
log
(14)
Pr(Asm (20) A(zo))
Next, we use the fact that ((ro) IL Aam (2o) by to simplify the expression into
(ra))m((aallu
log Pr(Asm(zo)le) = Pr(y 1|A(z))) Pr(Asm(zo)ly = - Pr(y = - -1|((z)))
(15)
The exact Asm (zo) is unknown but is drawn from the distribution Pr Pr(Asmli = 1) since xp 's label is Then, this expression becomes expectation over Asmi
Pr A by 1)
Exmn ly=1
log Pr Pr(Asm |y 1)px + Pr Pr(Asmli 1)(1 -P>)
(16)
Given that our Aam is high-quality, suppose that Aam (oo) all equal 1 with high probability, and then we can lower bound our expression by
Pr(Agm sm 1ly =
Pr(Aam = 1ly 1) log Pr(dsm 1ly 1)px + Pr(Asm = 1ly = -1(1 p>)
(17)
The key quantity of interest is Pr(Asm = 1ly = = ITm Pr(Mem, = 1ly = 1). We focus on bounding Pr(>_m, 1ly = 1) next. Suppose that the neighbors of 3o are Iı, Fg. Define Pý = =Pr()s(a;) = 1 = 1) for all j E [k]. Note that Ax(2)) IL X (xyy bly for any j.j E [K] (while Asm. as a whole is dependent on y. individual neighbors are still conditionally independent). Then, the event that Aam. [i] 1/y 1 as least as likely as the event that Binomial(k; min p.) 2 Let Pmin = mine Ps, and assume that Pmii 2 Then,
h 1ly = 1) Pr Binomial (k., Pmii) 2 2.) = Pr (
Σx,
Pr(Aem. [1]
(18)
1=1
)
Pr
X, 2 Puin
(Pmia
(19)
1=1
18

---

where X, ~ Beroouli(Puuin) Next, let = Pmin
We can apply Hoeffding's 's inequality to get
2
Pr [Binomial(k, Pmin) 2
k 2
Pr
X2 Pmin - -6==1
=1 -1-11 Pr
X3 S Pmii - 6) 2 1 exp(-260
(20)
= 1 - expl 1xpp-22k(pmm))) (pmin - 0.5)
(21)
All that's left is to lower bound Pmin- Without loss of generality, suppose that Pmin corresponds to an arbitrary Pj = Pr(A.(a,)) = 1yy 1). We can decompose this probability into
Pr(Ax(2,)) = 1ly (r,), 1,y(2,) = 1ly 1) Pr(A,(a,) 1.,(),)) -1yy 1)
(22)
-Pr(A(s,,) = l)((,)) 1) Pr(u(r,) = 1yy = 1) + Pr(M)(2))) 110(0))) = -1) Pr(v((x;) = - = 1).
(23)
Since Pr(A,(2,) = 1)b(2;) = 1) is over all 2j ~ Pr this quantity is just equal to the accuracy of Xi dg. Next, recall that [xE E(x) I Ek. where Ek maxx,eNN(e)) [[E();) E [z. is the maximum distance from the l neighbors z. Then, we can write Pr(v(zy) = 1/y
as Pr(v(xy)) = Bly = 1, [[x, - E(x) S Ex) 2 Mε(es), since we have assumed that E M-smooth. We now bound Pmin:
Pmin 2 a.Ma((E)) + (1 - a=) Mε(Ex))-
(24)
Therefore, we have that
Pr(Aam 1ly = 1) 2 I exp|-2k(a, - 0.5)
(25)
Before we plug in Pr(Aam = 1ly 1) into 17.. we simplify the expression. Note that Pr(Aam
1ly 1) can be written [ P; for some Pt, and Pr(>gm 1ly -1) 1) be written as IT (1 p:). A simple proof by induction shows that I (1 p.)
I Il Ps- Therefore,
can write that (17) is lower bounded by
Pr Pr(Agm 1yy= 1)
Pr(Asm 1/y 1) log Pr(Asm = 11y = 1)px + (1 - Pr(Asm = 1ly = 1))(1 - PA)
(26)
Let's abbreviate Pr(Asm = 1/y = 1) as and define the function
f(x) = x log
(27)
æp> + (1 x11-x)(1- PA)
We note that for 2 0.5, f=r is convex and can thus be lower bounded by f(x) (00.5 0.5). We compute f(xx)
æyx # (11-))I---p> f'(5.)) 2(1 px). Therefore, f(x) 2 2(1 - (x
0.5). Our final bound on the pointwise difference in irreducible error on zig is
H (v)A(z))) (x)
(amo)) (2o)) 2 2(1 Px)
II 1 2xp(-2M(a.M((e)) - 0.5
(28)
Information gain from using Aum over Aum We briefly comment on the opposite direction-
how much does using both LLM predictions and neighborhood predictions help over just using neighborhood predictions?
The quantity we aim to lower bound is (yl)amm (zo)) H A(ro), Asm (2o)) for a point of interest To. We can write this quantity
Pr(A(ro) (v)
H (z)))))) - HyylA(oo), Aam (xo)) = E log
(29)
Pr(A (z D (ro)
19

---

Without loss of generality, suppose that the true label on Zo is y 1, and that for each A3. the neighborhood around Ho consists of balanced mix of X, 1 and X; -1. Then, with high probability we have that Pr(y 1 (2o)) Pa R 0.5. From our proof of Theorem 5.2 5.2 we can
thus write
Pr(A(zo)1y== = 1)
H (y) Asm (2o)) H (1) (v))( Amm (x0)) - Ex(eo) log Pr(A(22))1 = 1) 1) Pr(A(wo)lu- = - -1)(1 Pa
(30)
Pr(A(zo) 11y 1)
2 Pr(A(zo) 1 = 1)) log Pr(A(zo) = 11y = 1)p... + Pr(A(zo) 1/y = -1)(1 PA.m)
(31)
22(1 PA (Pr(A( = 1/y 1) 0.5)
(32)
on Tp has high accuracy and Px is low, then we can have significant point-wise information gain from modeling both À and Amm rather than just am-
20

---

D Datasets
Motivation. We study the performance of our method across diverse collection of task definitions. In our setting, task definition denotes a specific classification that data scientist wishes to perform. For instance, data scientist working on quantifying the breadth of legal issues that individuals face may wish to identiff which posts in online forum refer implicate legal issues related to housing.
This evaluation strategy motivated by the observation that task definitions vary in their smoothness across embedding spaces, different embeddings may do better job of capturing features relevant for the task. For instance, out-of-the-bee Sentence- BERteTcc--RRTTTe embeddings are better than traditional BERT at capturing the topicality of sentence (55). By focusing we can better forecast how our method might perform for new tasks that practitioners may need to create classifiers for. We also avoid issues with leakage that may arise the practice of finetuning LLMs on tasks increases [9]
broad range of task definitions,
In total, we study 95 distinct task definitions, encompassing 100,418 total samples. Each task varies between 108 and 3308 samples.
Legal tasks. The emergence of LLMS is exciting for law and finance, where expert-annotations especially difficult acquire 4 18 19]. Drawing on recent benchmarks and released datasets
framing the potential use cases for LLMs in law, we study the following datasets:
CUAD [25]: : The original CUAD dataset consists of 500 contracts spanning an array of sectors, with clauses manually into one of 41 legal categories. Following [8] we adapt the original dataset for clause-by-clause classification. We turn each clause type into a binary classification task, where the objective is to distinguish clauses of that type from clauses of other types (i.e. "negatives"),. Negative clauses are sampled randomly so as to make the task class balanced. We ignore clauses for which there insufficient annotations in the original dataset.
Learned Hands [33: The Learned Hands dataset consists of legal questions that individuals publicly posted to an online forum (r/legaladvice). The questions have been coded by experts into legal categories according to the Legal Issues Taxonomy [3). We consider several such issue classes, and create binary classification task for each issue. Negative clauses are sampled randomly so as to make the task class balanced. Because these questions can be long, we truncate them 50 tokens.
Science tasks. LLMs have generated similar excitement for medical and science informatics applica- tions [ 4] We study established classification/eetrattio benchmarks.
ChemProt consists of sentences from PubMed abstracts describing chemical- protein relationships. We study seven relations, and create a binary task for each one. Each task is class balanced, with negatives sampled from the other relations.
Chemprot [30]:
The RCT dataset consists of PubMed abstracts for papers discussing randomized control trials, where sentences in the abstract are annotated according to their semantic role (e.g., background, methods, results, etc). There five roles, and create binary task for each one. Each task is class balanced, with negatives sampled from the other relations.
RCT [IT]
General domain tasks. Finally, we study the following "general domain" tasks, derived from popular sentence classification and information extraction benchmarks.
This is a relationship classffiction/eettaation dataset, where each sample corresponds to sentence mentioning the relationship between two entities. We select 20 relations, and for each relation construct a binary classification task with 700 positive instances of the relation, and 700 randomly sampled sentences (corresponding to other relations).
FewRel [21]:
We study the YouTube spam detection task from the WRENCH benchmark. This task requires classifying YouTube comments as spam/not spam.
Spam Detection [67|
Toxic content detection [
This task requires classifying whether posted comments are toxic not. We use sampled subset of the CivilComments dataset.
21

---

AG News [69]:
The original dataset organizes news snippets into four categories: World, Sports. Business, and Science/Technone Tennnccocccholoyy..... We create separate task for each category. Negatives are sampled from the remaining classes.
DBPedia [69]
DBPedia is a 14-way ontology classification dataset. We convert this into 14 distinct tasks, corresponding each of the ontology types.
Table 5: Legal tasks
Task
Description/Intent
Size
Affiliate License-Licensee (CUAD)
Does the clause describe a license grant to a li- censee (incl. sublicensor) and the affiliates of such licensec/sublieensor?
208
Anti-Assignment (CUAD)
Does the clause require consent or notice of a party if the contract assigned to third party?
1212
Audit Rights (CUAD)
Does the clause discuss potential audits?
1224
Cap On Liability (CUAD)
Does the clause specify a cap on liahility upon the breach of a party's ' obligation?
1262
Change Of Control (CUAD)
Does the clause give one party the right to termi- nate if such party undergoes change of control?
426
Competitive Restriction Exception (CUAD)
Does the clause mention exceptions carveouts to Non-Compete, Exclusivity and No-Solicit of Customers?
226
Covenant Not To Sue (CUAD)
Does the clause mention if party is restricted contesting validity of the counterparty's s ownership of intellectual property?
318
Effective Date (CUAD)
Does the clause mention when the contract comes effective?
Exclusivity (CUAD)
Does the clause mention an exclusive dealing com- mitment with the counterparty?
Expiration Date (CUAD)
Does the clause mentions date when the con-
892
tract's term expires?
Governing Law (CUAD)
Does the clause mentions which statc/country's laws govern interpretation of the contract?
910
Insurance (CUAD)
Does the clause mention a requirement for insur- ance?
1040
Ip Ownership Assignment (CUAD)
Does the clause mention if intellectual property created by one party become the property of the counterparty?
590
Irrevocable Or Perpetual I License (CUAD)
Does the clause describe license grant that is irrevocable perpetual?
300
Joint Ip Ownership (CUAD)
Does the clause provide for joint shared owner- ship intellectual property between the parties the contract?
198
License Grant (CUAD)
Does the clause describe license granted by one party to its counterparty?
1430
Liquidated Damages (CUAD)
Does the clause award either party liquidated dam- ages for breach or a fee upon the termination a contract (termination fee)?
Minimum Commitment (CUAD)
Does the clause specifies minimum order size or minimum amount or units pertime period that one party must buy from the counterparty?
778
22

---

Table 5 continued from previous page
Task
Description/nntet
Size
No-Solicit Employees (CUAD)
Does clause restricts a party's 's soliciting or hir- ing employees and/or contractors from the coun- terparty, whether during the contract or after the contract ends (or both).
150
Non-Compete (CUAD)
Does the clause restrict the ability of party compete with the counterparty operate in tain geography or business or technology sector?
450
Non-Disparagement (CUAD)
Does the clause require party disparage counterparty?
108
-on-Transferable License (CUAD)
Does the clause limit the ability of a party to trans- license being granted a third party?
558
Notice Period Terminate Renewal (CUAD)
Does clause requires notice period termi- nate renewal?
Post- Termination Services (CUAD)
Does clause subject a party to obligations ter the termination or expiration of a contract, in- cluding
816
termination transition, payment, transfer of IP, wind-down, last-buy, or similar com- mitments?
Renewal Term (CUAD)
Does the clause mention renewal term for after the initial term expires?
398
Revenue-Profi Sharing (CUAD)
Does the clause require party share revenue or profit with counterparty for any technology, goods, or services?
784
Rofi-Rofo-Rofi (CUAD)
Does the clause provide a party with a right first refusal?
Source Code Escrow (CUAD)
Dnes clause requires one party to deposit source code into escrow with a third party, which can be released to the counterparty upon the oc- currence of certain events (bankruptcy, insolvency. etc.)?
126
Termination For Convenience (CUAD)
Does the clause state that party can terminate this contract without cause (solely by giving a notice and allowing a waiting period to expire)?
442
Uncapped Liability (CUAD)
Does clause state a party's liability un- capped upon the breach of its obligation in the contract?
Volume Restriction (CUAD)
Does the clause describe fee increase or con- sent requirement if one party's use of the prod- uct/services exceeds certain threshold?
328
Warranty Duration (CUAD)
Does the clause mentions the duration of any war- ranty against defects or errors technology, prod- ucts, or services provided under the contract?
326
BU (Leamed Hands)
Does the discuss issues relating business intellectual property?
200
CO (Leamnd Hands)
Does the text discuss issues relating to courts and lawyers?
194
(Learned Hands)
Does discuss issues relating to criminal issues?
ES (Learned Hands)
Does text discuss issues relating to estates 182 wills?
FA (Learned Hands)
Does text discuss issues relating to family divorce?
23

---

Table 5 continued from previous page
Task
Description/nntet
Size
HE (Learned Hands)
Does text discuss issues relating to health?
HO (Learned Hands)
Does the text discuss issues relating housing? 1270
MO (Learned Hands)
Does the text discuss issues relating payments
740
or debt?
TO (Learndd Hands)
Does the text discuss issues relating to accidents or harassment?
454
TR (Learned Hands)
Does the text discuss issues relating to cars 516
traffic?
WO (Learned Hands)
Does the discuss issues relating employ- ment or job?
726
Table Science tasks
Task
Description/Intent
Size
Agonist (Chemprot)
Does the sentence describe agonist relationship?
896
Antagonist (Chemprot)
Does the sentence describe antagonist relationship?
1330
Downregulator (Chemprot)
Does the sentence describe downregulator relationship?
3038
Part of (Chemprot)
Does the sentence describe part-of relationship?
1210
Regulator (Chemprot)
Does the sentence deseribe regulator relationship?
2876
Substrate (Chemprot)
Does the sentence describe substrate relationship?
2384
Upregulator (Chemprot)
Does the sentence describe upregulator relationship?
2404
Background (RCT)
Does the sentence describe background on the study?
2000
Conclusions (RCT)
Does the sentence state conclusion?
2000
Methods (RCT)
Does the sentence describe scientific experimental method?
2000
Ohjective (RCT)
Does sentence descrihe goal of the study?
2000
Results (RCT)
Does sentence describe experimental results?
2000
Table 7: General domain tasks
Task
Description/Intent
Size
Business (AGNews)
Does the article discuss business news?
2000
Sports (AGNews)
Does the article discuss sports news?
2000
Technology (AGNews)
Daes the article discuss technology news?
2000
World (AGNews)
Does article discuss global affairs or world events?
2000
Civil Comments
Does sentence contain toxic hateful content?
2500
Album (DBPedia)
Is the entity discussed in the sentence example of album?
2000
Animal (DBPedia)
Is the entity discussed in the sentence example of animal? 2000
Artist (DBPedia)
Is the entity discussed in the sentence an example of artist? 2000
Athlete (DBPedia)
Is the entity discussed in the sentence an example of athlete?
2000
Building (DBPedia)
Is the entity discussed in the sentence an example a build- 2000
ing?
Company (DBPedia)
Is the entity discussed in the sentence an example of a com- pany?
2000
Educational institution (DBPedia)
Does the sentence discuss a school, university, or college?
2000

---

Table 7- continued from previous page
Task
Description/nntet
Size
Film (DBPedia)
Is entity discussed the sentence an example film?
2000
Mean of transportation (DBPedia)
Does the sentence discuss car, ship, train, plane?
2000
Natural place (DBPedia)
Is the entity discussed in the sentence example of natural
2000
landscape or environment?
Office holder (DBPedia)
Is the entity discussed in the sentence example of office holder?
2000
Plant (DBPcdia)
Is the entity discussed in the
example of plant? 2000
Village (DBPedia)
Is the entity discussed in the sentence example of village?
2000
Written work (DBPedia)
Is the entity discussed in the sentence writing?
2000
Architect (FewRel)
Daes the text mention an architect?
600
Composer (FewRel)
Does the text mention a musical composer?
600
Country (FewRel)
Does the text mention country?
Developer (FewRel)
Does the text mention development?
600
Director (FewRel)
Does the text mention film director?
600
Distributor (FewRel)
Does the text mention film distributor?
600
Father (FewRel)
Does the text mention father?
600
Genre (FowRel)
Does the text mention the genre of song or artist?
600
Instrument (FewRel)
Does text mention instrument?
League (FewRel)
Does the mention sports competition, league, divi- sion?
600
Military Branch (FewRel)
Does the text mention military branch?
Movement (FewRel)
Does the text mention art movement?
600
Occupation (FewRel)
Does the text mention professional occupation?
600
Participating Team (FewRel)
Dnes the text mention sports team?
600
Platform (FewRel)
Does the text mention an online platform?
600
Sibling (FewRel)
Does text mention sibling?
Successful Candidate (FewRel)
Does the text mention election winner?
600
Taxonomy Rank (FewRel)
Does the text mention laxonomy class of animals?
600
Tributary (FewRel)
Does the text mention tributary?
600
Winner (FewRel)
Does the text mention competition winner?
600
YouTube Spam Detection
Does the comment ask the user to check out another video?
1836
25

---

E
Prompts
We construct a prompt for each task by randomly selecting three examples of each class use as in-context demonstrations [5]]. We manually defined "instructions" for each task. An example of an abridged prompt shown in Figure
Instruction Classification
Prompt
the entity discussed in the sentence example of an animal?
Sentence: Hugh Christopher Chris Brown is Canadian singer- songwriter and iutt-tinsttuueetaltst.. Label: No
Sentence: " Forficula auricularia the is an omnivorous insect in the family Forficulidae. The European earwig survives variety environments
earwig European earwig
common hausehold insect in North America. The name earwig comes from false superstition that these insects crawl into human ears and enter the brain; in fact they harmless humans." " Label: Yes
Sentence: Northern Dancer (May 27 1961 November 16 1990) was a Canadian-bred Thoroughbred racehorse who won the Kentucky Derby Preakness Stakes and who became the most successful sire of the 28th Century. The National Thoroughbred Racing Association calls him one of influential sires Thoroughbred history.A bay colt Northern Dancer was by Nearctic and his dan Natalma was by Native Dancer. In 1952 Edward P.* Label:
Figure 3: An example of the instruction classification prompt for the dipedia abpedia_animal animal task, with two in-context demonstrations. Here, the task instructions are in red, the in-context demonstrations are in blue, and the sample for which we want a label is in green.
26

---

F
Synthetics
We conduct synthetic experiments which provide additional insights on EMBROID.
For our setup, we create two equal clusters of data of 500 points each, C1 and C2. in R2, We assign labels the points in each cluster i.i.d. according a probability p. where Pr(y = 1/r Ci) p and Pr(y = 1 E C2) = p. When p = 0.5, both the clusters have a uniform label distribution (non- smooth) while p ensures each cluster has one class (smooth). We set k 20, 7+ P(): - 1) and T_ = P(A, = -)).
Improvement over weak supervision. We show that EMBROID offers improvement over methods that only use A. We fix = 0.8 and 3, = Pr(A, = y) 0.6 for each i E [m]] We compare EMBROID against the standard weak supervision approach from [14] which requires m 2 in
Figure 4a
Smoothness. EMBROID'S 's performance depends on the smoothness of the embedding as defined in eq. 44). We consider one LM prediction m and vary the smoothness p from 0.5 to 1.0 and generate predictions using 3; = 0.6. Figure 4b exhibits that EMBROID's accuracy is positively correlated with the embedding smoothness.
Base prediction accuracy. Finally, we show that EMBROID'S performance depends on the base prediction accuracy, 3r. We consider m and set p 0, B, 0.8. We use parameter p to denote the probability that A; is incorrect on points in cluster C1. As varies from 0 to 1, the predictions of X₂ become biased towards and effectively reduces 34 to 0.5. In Figure 4c we observe that as the base prediction accuracy decreases, EMBROID'S 's performance decreases and eventually goes below the base LM performance.
Varying number of
Varying embockingg smoothiess
Varying LLM skew
1.0
Entecid
Enteoid
0.72
Walk Suparvisiu
0,50
0.9
Acrurayy
0.70
0.75
Emteeiid LLM
6.70
0.7
0.66
0.68
0.6
0.64
0.55
4
0.6
0.7
0.3
0.9
1.0
0.0
0.2
0.4
0.5
0.8
1.0
Bourees
Embeddängg amoodhaess
Prediction seew
(a)
(b)
(c)
Figure 4: Synthetic experiments. (a) Comparison of EMBROID to weak supervision by varying number of LLM sources. Increasing sources consistently improves both EMBROID and WS and the gains remain constant. (b) EMBROID'S 's performance as embedding smoothness (eq. varies. EMBROID's 's accuracy linearly improves as a function of embedding smoothness. (c) EMBROID'S performance with varying probability of LLM being incorrect in C1. As the LM becomes incorrect, the cluster becomes less homogeneous and this degrades EMBROID'S 's performance.
G Ablation
We perform additional ablations of EMBROID. We focus on two aspects:
The role of T -/+
* The impact of weak supervision in combining >am.j.
G.1
Ablation over T
In our experiments, we set 7, T. E[As], or the average vote of source AІ. This has the following effect on the neighborhood vote Asm.j [i] for source A, under Ey:
When the average vote for a source X; neighborhood under F. for x is more negative than the average overall vote for a source, then > [z) = -1.
27

---

0.7
0.6
0.5
0.4
-1.0
-0.5
0.0
0.5
1.0
Figure 5: We analyze how Fl changes for different settings of ,, for GPT-JT on task. Observe that setting Ti = E[A,] (denoted as the orange triangle) produces close-to-optimal perfor-
mance.
LM
Base prompt
Majority vote aggregation
EMBROID
jl-jumbo openai text-davinci-003
0,498
0.569
0,604
0.806
0.844
0.855
bloom-7bl
54.7
61.2
64.7
opt-6.7b
48.2
56.1
59.8
GPT-JT-66-v1
67.8
73.2
75.1
Table 8: We evaluate how EMBROID compares to a majority vote aggregation over neighborhood vote vectors. We report macro-F1 average across all tasks and prompts, mirroring results in Table T
When the average vote for source in neighborhood under Eg for is more positive than the average overall vote for a source, then > [z] = 1.
In general, we find that this setting provides good performance, while requiring no additional tuning or validation. For example, the Figure below compares setting of /,, against the F1 score for GPT-JT on ag news baseeessssseess....
G.2 Role of weak supervision aggregation
We quantify the extent to which performance gains are derived from (1) the computation of > as opposed to (2) the use weak-supervision [14] combine Aum and X. Specifically, we replace Equation 3 with a simple majority vote classifier which combines the original prediction with the computed neighborhood votes.
G.3 Choice of embeddings
We provide additional discussion how practitioners may go about choosing embeddings for EMBROID. At the outset, we note that selccting/evaluating embeddings particularly difficult in regime, where practitioners do not have access to labeled data. Importantly, this motivates Embroid's "mixture of embeddings" approach. Previous methods (e.g., Liger (7)) operate on only single embedding space. Thus, practitioners must be certain that the embedding space chosen is good for a task, and a poorly chosen embedding space will reduce performance. Because EMBROID ensembles different embedding spaces, practitioners can be less certain in the quality of any individual embedding space.
A natural strategy may be to start with an initial large set of candidate embeddings, and drop embeddings according to task-specific measurements (akin feature selection). Several criteria may be used to determine which embeddings to maintain:
28

---

Looking at the MLM loss of the embedding model (e.g. BERT) on the task data. Prior literature on domain specificity has suggested that this may promising heuristic [7].
Looking the performance of embedding models other related tasks may also be helpful
Looking at the extent to which potential embedding space generates embeddings which are geometrically similar to embeddings already included in Embroid. If the embedding space is similar, then the additional value of incorporation low.
The number of embeddings used presents bias-variance tradeoff. Under Proposition 5.1 increasing the number embedding spaces (1) increases the variance due to estimation error but lowers (2) the conditional entropy of H (yy X). Precisely characterizing this tradeoff is challenging because the variance is an upper bound, and H (ylA) can only be estimated. However, these bounds/cstimates can be used derive heuristic stopping rules: for instance, add embedding spaces until the marginal decrease in the conditional entropy is less than the upper bound on the marginal increase in variance.
29

---

H Experiments
H.1 Implementation details
Compute Inference for API-access models (c.g., GPT-3.5 and J1-Jumbo) were run using the HELM API [38]. Inference for open source models (OPT, GPT-JT, and Bloom) were run using the Manifest library 49 on 40GB A A00 NVIDIA GPU machines.
Hyperparameters
EMBROID run with 10, 7. P(): 1), and TÚ P(A: =
H.2 Space demand
We provide additional information on the space requirements for EMBROID. First, we clarify that EMBROID requires only one large model. This model can be on the order of GPT-3 (176B parameters), or a much smaller open-source equivalent like GPT-JT (7B parameters). Next, EMBROID requires embeddings from 2-3 additional models. However, relatively small. For instance, we rely on BERT-base and SentenceBERT models-both of which are approximately 110 million parameters. Finally, the latent variable graphical model has parameters for class prior and I+N parameters per LLM prediction, where is the number of embedding spaces. So, when EMBROID is run single LLM'' predictions with two embedding spaces, the total number of parameters is 2 + (1+2)*1
results show, these models can be
H.3 Runtime
In terms of time complexity, observe that computing LLM predictions is the largest bottleneck.
On a dataset 1800 samples for instance:
Computing predictions on GPT-3.5 takes 1440 seconds (24 minutes).
Computing embeddings using BERT takes seconds.
Computing nearest-neighbors using the (unoptimized) scikit-learn library takes 3 seconds.
Solving the Embroid graphical model takes less than a second.
A AP--model tasks Due cost constraints, study the API access models (GPT-3.5 and J1-Jumbo) on a subset of tasks. These are:
ag. ne_sewrwo
ag news__sssssss
abpadia__ducationnal institution
dbpetia..ttlleccccepppoor..uaa
chemprot.upregullato
rct_objective
rct_methods
CUAD CUAD_Audit Rights
CUAD_Non-Compete
learned_
learned_hands_It
few_rel_architect
few_rel_league
H.4 Robustness across models
We provide the results for each LM on each task as CSV files in the supplemental attachment. We visualize the improvements T by plotting the original prompt performance on the x-axis, and the performance after EMBROID the y-axis (Figure 6).
30

---

j1 jjmbbo
opt-6.7b
1.0
0.8
0.6
0.4
GPT-JT-6B-vl
openai text-davinci-003
1.0
Prompt+ Eemppt-EmboiddTP F1
0.8
0.6
0.4
bloom-7b
gpt-neo-1.3B
1.0
0.8
0.6
0.4
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
Prompt F1
Figure 6: We visualize the improvement from EMBROID over the base prompt for each model's tasks. All models except for gpt-neo- 1.3B were run thrice task, with each run using different in-context samples for the prompt. Each dot corresponds to a task. The x-axis measures the average macro F1 of the base prompt, and the y-axis measures the average macro FI of EMBROID (across all runs). Because GPT-3.5 and -Jumb are studied on a subset of tasks, there are fewer dots the plot.
H.S Comparison AMA
We visualize the improvements in Table 2 below, by plotting the performance of AMA on the x-axis,
and the performance of EMBROID-3 the y-axis (Figure
H.6
Chain-of-thought experiments
We compare EMBROID to chain-of-thought (CoT)
64 on the following tasks:
ag_ www.nws_bbsingg
ag_news_sports
CUAD_Affiliate License-Licensee
CUAD_Audit Rights
abpedia_album
dbpedia_building
few_rel_architect
few_rel_country
learned_hands_
31

---

j1 jjmbbo
opt-6.7b
1.0
0.8
0.6
0.4
GPT-JT-6B-vl
openai text-davinci-003
1.0
Prompt+ Eemppt-Emmoodd F1
0.8
0.6
0.4
bloom-7b
gpt-neo-1.3B
1.0
0.8
0.6
0.4
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
Prompt F1
Figure We visualize the improvement from EMBROID over AMA for each model's tasks. Each dot corresponds to a task. The x-axis measures the average macro Fl of AMA, and the y-axis measures the average macro FI of EMBROID. Because GPT-3.5 and J1 JumJo are studied on a subset of 12 tasks, there are fewer dots in the plot.
We manually write logical chains for each prompt. Because CoT primarily succeeds on "large" models (64 we focus our experiments on GPT-3.55.
H.7
Comparison to GPT-4
made publicly available after this work completed. Nonetheless, of- fer preliminary results evaluating EMBROID on predictions produced by GPT-4. We evaluated EMBRooDEnnroid-GGTT- subset of 14 tasks. These are:
GPT-4 48
ag news_wowd
newsess_ss
abpedia__ducattional institution
dbpedie_ttheercchepppoo...g....
chemprot.uprrggulatoo
rct_objective
rct_methods
CUAD_Audit Rights
CUAD NUDD_Non-Compete
learned_hands
learned_hand _____aand____________
32

---

fes_rel_architect
few_rel_league
We note that overall, GPT-4 appears saturate our evaluated tasks, scoring greater than 95% on 5 of them. Given that we evaluate on publicly accessible benchmarks, this could be explained by leakage into GPT-4's pre-training data, We find that EMBROID is strictly better than the base prompt on 42% of tasks, and effectively equivalent or better (i.e., no worse than point F1) on 73% of tasks. On two tasks Embroid improves by 4+ points.
Though EMBROID is absolute gains over base prompts may diminish for stronger base LMs like GPT- our evaluation and findings focused open-source models because of their practical feasibility. Such models are cheaper and better suited for sensitive domains, as they allow data to remain on- premises. This is a particularly salient concern for applications in medicine, law, or government. On these models, we found that EMBROID offered substantial gains, with a win-rate > 88% and an average improvement points FI.
H.8 Comparison to self-consistency
We evaluate EMBROID against self-consistency [63 over GPT-3.5, and find that (1) it can provide competitive performance self-consistency, and (2) that it can improve predictions generated from self-consistency.
We study a subset of five tasks: ag_news_ og_news_bustiee businesusees,, CUAD_Audiz Rights, dbpedia_a aambm,, lew_rel_architect learned Searned_hands... _HO. For self-consistency, we generate predictions per sample, using temperature of 0.7. For each task, consider "base" prompt which does not chain-of-thought, but otherwise uses the same task instructions and in-context demonstrations. For 4/5 tasks, tasks, EMBROID applied to the predictions generated from self-consistency improve performance, by median of six points.
applied to the output of the base prompt outperforms self-consistency. On all
We additionally note one important distinction between self-consistency and EMBROID. While self-consistency requires multiple predictions from a LM for single sample (thus accruing additional cost hardware usage or API calls), Embroid requires only one prediction per-sample.
H.9 Multi-class experiments
We present initial results evaluating EMBROID in the multi-class setting. We create a multi- class version of the contract classification task by combining five different binary classifica- tion datasets: CUAD_A Audiitt right, BUAD_GGverning law, CUAD_Non WwwNNooncomeeeee restriction, and CUAD_ BUAD_Expiration date. The total size of this dataset is 1858, with the largest class (602 samples) occurring nearly 3.5x as much as the smallest class (165 samples).
We evaluate one-vs-all version of EMBROID, where EMBROID is applied five times-each time predicting one the classes. A final class prediction is generated by comparing each the class- specific EMBROID classifier's predicted probability for the class, and choosing the class with the highest probability, Overall, we find that this variant of EMBROID improves the quality of the base prompt (by points on aalaanedd-acuuraee and 1.6 points on macro-Fl).
33


Reviews:
[{"review_id": "NlBAd5M3Ji", "reviewer": "Reviewer_RD7v", "summary": "The authors focus on few-shot prompted classification using language models. In this setting, they focus on the challenge of developing optimal few-shot (in-context) prompts for language models in domains where data collection is prohibitively expensive. Rather than engineering the prompts themselves, the proposed Embroid method aims to *correct* prompted language model predictions using semantic similarity predictions of similar samples using other pre-trained language models. Embroid operates as a form of mixture-of-experts involving both the core language model as well as a family of other embedding models to edit LM annotations and enforce consistency. The authors perform a theoretical analysis of how/why Embroid works re: embedding smoothness, and demonstrate how Embroid improves accuracy across a range of language models and tasks.\n", "questions": "N/A", "limitations": "Yes", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- To my knowledge, this paper proposes a novel way of using noisy similarity heuristics as a \"vote of confidence\" to regularize language model predictions\n- The methods sections (3, 4) are clearly written and equation intuitions are well-explained. This helps greatly in understanding why the Embroid method may work and helps in understanding the theoretical analysis in Section 5.\n- The authors report a significant improvement overall (win rate) and average across tasks over a variety of models after applying Embroid across their extensive task suite.\n- The experiments section is structured well, and the research questions naturally flow from one to another. It is encouraging to see that Embroid seems to work well together with chain-of-thought prompting, prompt ensembling, and other methods of improving the prompt (selective annotation).", "weaknesses": "- There is a bit of confusion in the problem setting description about what constitutes the \"prompt\". (108/109) suggests that the prompt includes the description and in-context examples. Is the main challenge the authors target the task description engineering, choice of in-context examples, or both?\n- Regarding the smoothness observation in Section 5: is there a simple method to quantify smoothness of embedding models on a given dataset? And does this give rise to selection strategies or methods to drop out embeddings from a given task? Essentially I'm wondering whether there is a theoretical result for the optimal number or selection of embeddings used in Embroid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank RD7v for their review! We are glad to hear they appreciated the novelty of our method, its intuitive appeal, and the structure of our evaluation. Our response primarily serves to answer the questions they raised. \n\n**Confusion regarding the definition of “prompt”.** We consider the “prompt” to include both the task description and the in-context samples, consistent with Zhao et al (2021). Because Embroid corrects the predictions of the prompt–and is thus agnostic to the content of the prompt itself–we believe it can address challenges in both the choice of in-context samples and in task description engineering. We will add a clarification in §2.3 our draft to make this clearer.\n\n**Embedding smoothness.** Please see the general response above.\n\n*References*\n\nZhao, Zihao, et al. \"Calibrate before use: Improving few-shot performance of language models.\" International Conference on Machine Learning. PMLR, 2021.\n"}, {"review_id": "kD4H8eErb4", "reviewer": "Reviewer_YSHe", "summary": "The authors propose Embroid, a promising method that exploits availability of diverse pre-trained LLMs to create something akin to (but better than) an ensemble approach for prompt-patching. ", "questions": "See weaknesses section.", "limitations": "The authors address limitations like dependence on domain-specific embeddings, the quality of embeddings, and the availability of at least a moderate number of data points (in the hundreds). ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- a promising and easy-to-use method for prompt-patching\n- robust theoretical analysis\n- extensive empirical evaluation", "weaknesses": "- Evaluation was performed on models which are non-SOTA and the improvement may possibly be explained by gaps in their training that are filled by the domain-specific pre-trained models. It would be helpful to evaluate Embroid with a stronger general model (e.g. GPT4) to see if performance gains decrease significantly. It could also be helpful to evaluate Embroid on models that have been pre-trained on the particular domains of interest, to see if lack of domain-knowledge for the general model is what enables improvement when leveraging domain-specific pre-trained embedding models. \n- Evaluation was performed on only binary sentence classification tasks. It seems like extending to multi-class classification should be doable (at a minimum this can be done by doing multiple binary one-vs-all comparisons, though a cleaner extension would of course be better) and would go a long way in validating Embroid on more realistic settings. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank YSHE for their review! We are glad to hear they appreciated the ease of use, our theoretical analysis, and our empirical evaluation. \n\n**Evaluating on stronger general models.** Our original work found Embroid had a substantial win rate (80.6%) and average improvement (4.9 points F1) on GPT-3.5, a highly performant model which–at the time of paper submission–was SoTA amongst publicly available API-models. \n\nWe evaluated Embroid+GPT-4 on a subset of 14 tasks. The tasks chosen are identical to those listed in Appendix H.1 of the original submission. We note that overall, GPT-4 appears to saturate our evaluated tasks, scoring greater than 95% on 5 of them. Given that we evaluate on publicly accessible benchmarks, this could be explained by leakage into GPT-4’s pre-training data.\n\nWe find that Embroid is strictly better than the base prompt on 42% of tasks, and effectively equivalent or better (i.e., no worse than 1 point F1) on 73% of tasks. On two tasks Embroid improves by 4+ points. We will add the above analysis to the Appendix.\n\n**Evaluation on domain specific models.** Surprisingly, there is (to the best of our knowledge) no publically available few-shot language model for law that could be used to generate predictions. This was, in fact, a motivation for exploring Embroid’s ability to offer domain-specialization capabilities. We will add experiments to our paper evaluating Embroid’s performance improvements for domain-specific medical models.\n\n**Importance of open-source models.** Though Embroid’s absolute gains over base prompts may diminish for stronger base LMs like GPT-4, our evaluation and findings focused on open-source models because of their practical feasibility. Such models are cheaper and better suited for sensitive domains, as they allow data to remain on-premises. This is a particularly salient concern for applications in medicine, law, or government. On these models, we found that Embroid offered substantial gains, with a win-rate > 88% and an average improvement > 7 points F1. \n\n**Evaluation on multi-class.** We found that Embroid can be applied to the multiclass setting through a one-vs-all approach. Please see the general response for more details.\n"}, {"review_id": "ahLCm6By8B", "reviewer": "Reviewer_Bw2X", "summary": "This paper presents EMBROID, a prompt-patching method that corrects erroneous predictions for a prompt via agreements over KNN examples. For this, it employs N different embedding models to get smoothed neighborhood prediction vector. The smoothed information is integrated to combine the voting with quality parameters. They conduct a formal theoretical analysis for EMBROID to get its behaviors and corresponding results. Experimental results demonstrate the efficacy of the proposed method, improving baseline predictions. It boosts the performance when multiple prompts are used and can be complementary used with other prompting methods such as chain-of-thought. Moreover, they show the proposed method works well even with the dataset size is relatively small.", "questions": "Please position the proposed method compared to the Self-Consistency method.", "limitations": "It would be good to show that the proposed method can be easily adapted to multi-class classification or even generative tasks.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces a novel method to improve the prompt-based learning paradigm, introducing a broad agreement procedure over the predictions.\n2. The intuition of the idea and corresponding analyses are good.\n3. Strong empirical results", "weaknesses": "- The method is limitedly evaluated only in the binary classification tasks.\n    - Multi-class classification or generative tasks are not tested at all.\n- It requires multiple embedding models for each correction.\n- The proposed method always assumes unlabeled datasets.\n- The paper does not include discussion about the self-consistency [1] method, another voting method to improve prompt accuracy.\n\n\n[1] ****Self-Consistency Improves Chain of Thought Reasoning in Language Models (Wang et al.)****", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Bw2X for their review! We are glad to hear they appreciated Embroid’s novelty, the analysis, and our empirical evaluation. \n\n**Evaluation on multi-class.** We found that Embroid can be applied to the multiclass setting through a one-vs-all approach. Please see the global review for more details.\n\n**Comparison to self-consistency.** We evaluate Embroid against self-consistency, and find that (1) it can provide competitive performance to self-consistency, and (2) that it can improve predictions generated from self-consistency. We study a subset of five tasks, and will add an expanded study of additional tasks to the paper. For self-consistency, we generate 5 predictions per sample, using a temperature of 0.7. For each task, we consider a “base” prompt which does not use chain-of-thought, but otherwise uses the same task instructions and in-context demonstrations. For 4/5 tasks, Embroid applied to the output of the base prompt outperforms self-consistency. On all tasks, Embroid applied to the predictions generated from self-consistency improve performance, by a median of six points. \n\nWe additionally note one important distinction between self-consistency and Embroid. While self-consistency requires multiple predictions from a LM for a single sample (thus accruing additional cost in hardware usage or API calls), Embroid requires only one prediction per-sample. \n\n**Embroid requires multiple models for each correction and assumes unlabeled datasets.** We agree with the reviewer that these requirements are essential to Embroid. However, we believe that for a substantial number of applications, both requirements are easy to satisfy. \n\nFirst, repositories like Huggingface contain a number of BERT-variants for specialized domains like law or medicine. Computing embeddings from BERT-models for Embroid is relatively cheap–on an NVIDIA T4 (16GB) computing embeddings on a dataset of ~1800 takes 5 seconds.\n\nSecond, we observe that for many applications in medicine and law, unlabeled data is readily accessible. For instance, the CUAD dataset contains only 500 annotations per-clause type, yet nearly 34GB of unlabeled contractual data. \n\nWe will update our discussion in the paper to clarify the setting we focus on.\n"}, {"review_id": "Lc2OPdwWj2", "reviewer": "Reviewer_XCbh", "summary": "The paper aims to improve prompt performance through a technique of prompt patching where multiple neighbourhood instance of a given datapoint (retrieved through the embedding space of BERT or RoBERTa like model) are used in prediction from the LLM along with the original instance. Finally a majority vote is incorporated. Empirically, the method performs well in the case of binary classification tasks and outperforms the Chain-of-thought prompting technique.\n\nExtra note: I want to highlight that I have not published in the area of prompting for LLMs and my review is based on the recent readings in the area.", "questions": "See Weaknesses", "limitations": "While I don't see any significant limitations, the final version of the paper can be updated with results on newer instruction-tuned models. ", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The main strength of the paper lies in the simplicity of the method and also that it works well in practice (empirically validated). \n- The theoretical analysis well supports the empirical analysis which is a strong point. \n- Combination of Chain-of-thought and prompt patching results in strong improvements.\n- Strong ablations and baselines are considered!", "weaknesses": "Overall, I believe the paper is empirically strong, with good theoretical intuition. The writing in Sec. 6.2 can be improved to reflect the workings of the baseline. I believe the reader might get confused with the Embroid-1 / Embroid-3 notation when comparing with the baselines. It will be better to highlight exactly how Embroid-1 / Embroid-3 differ from the baselines in terms of the exact number of prompts used. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank XCbh for their review! We are glad to hear they appreciated Embroid’s simplicity, its theoretical analysis, and the empirical results. \n\nWe will update the paper with results on new instruction-tuned models. We will also update the writing in Section 6.2 to clarify the workings of the different baselines, so that readers are not confused as to how many prompts are used.\n"}, {"review_id": "m3tTMnCXBN", "reviewer": "Reviewer_bLRH", "summary": "The authors in this paper propose EMBROID aim to improve language models (LM) (such as GPT-3.5) without additional labeled data. Unlike prompt-based models that focus on prompt designs, this paper attempts to modify the predictions of the data point $x$ by considering its neighboring samples. First, the proposed model applies a set of embedding encoders (such as BERT, and SentenceBERT) to construct the neighboring graph by the output embeddings, and then a latent variable model is used to generate the final prediction. This paper also provides the theoretical analysis about the generalization error and information gain. Finally, experiments on sentence classification datasets and ablation studies on output embeddings and dataset size show the improvements of the proposed model.", "questions": "(1) One of the main technical challenges is how to fuse the original prediction with the neighboring prediction, and this paper employs the recent latent variable graphical model to the end. Please explain the core difference from [1] and [2] theoretically.\n\n(2) The proposed model consists of several large models, such as three embedding functions, GPT-3.5, and latent graphical model. It will be better to provide the space and time complexity analysis.\n\n(3) Ablation studies on $\\alpha$ in Eq.(3) are needed to evaluate the proposed module.\n\n(4) Theorem 5.2 suggests a smooth embedding function E. I wonder how to select an optimal E?  Are BERT-like encoders smooth functions?\n\n[1] Mayee F. Chen et.al. Shoring up the foundations: Fusing model embeddings and weak supervision.\n\n[2] Daniel Fu et.al.  Fast and three-rious: Speeding up weak supervision with triplet methods.", "limitations": "None", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The motivation that improves the LM predictions with unlabeled data and no expert supervision is interesting for recent NLP community. Based on ensemble learning, this paper uses a set of textual encoders to define the neighborhood prediction vector for each data point and combines the original prediction with its neighborhood prediction to correct the final prediction. The idea is simple and reasonable.\n\n(2) This paper also provides theoretical analysis by deriving a bound of generalization error and calculating the pointwise information gain, which suggest the efficiency of the proposed modules mathematically under several assumptions.\n\n(3) Extensive empirical results with several baselines on sentence classification task show the consistent improvements.\n\n", "weaknesses": "(1) The novelty of this paper appears to be satisfactory. Building upon previous works on weak supervision [1][2], the authors employ a latent variable model in the context of prompt-based language models.The combination of these two approaches is intuitive, and there are only a few significant technical challenges to address in solving the combined equation (Eq. 3).\n\n(1) This paper limits the proposed model to the classification task (from the method description and empirical results). Given that the generation ability (such as question answer) of a language model is also important, the proposed model seems to only work on the classification case.\n\n(2) Genral textual benchmarks such as SuperGLUE can enhance the convincingness of the experiment.\n\n[1] Mayee F. Chen et.al. Shoring up the foundations: Fusing model embeddings and weak supervision.\n\n[2] Daniel Fu et.al.  Fast and three-rious: Speeding up weak supervision with triplet methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank XCbh for their review! We are glad to hear they appreciated Embroid’s motivation, the theoretical analysis, and the extensive validation. \n\n**Limited number of technical problems are solved.** We would like to clarify the novelty of our work. Specifically, our contribution is synthesizing distinct technical insights from different areas of study (e.g., weak supervision and embedding-based approaches), and showing how they can combine into a simple yet powerful method. We note that while both fields of study have existed, our work is the first to (1) show how weak supervision-approaches can be applied to settings where we only have one set of predictions, and (2) ensemble embeddings to account for potential noise in individual embedding spaces. \n\n**Evaluation on multi-class.** We found that Embroid can be applied to the multiclass setting through a one-vs-all approach. Please see the general response for more details.\n\n**General textual benchmarks.** Embroid is applicable for tasks where practitioners can identify pre-existing embeddings which are smooth for the task (see §5). The tasks contained in the SuperGLUE benchmark fall outside of this scope, because these tasks are natural language inference tasks. Embedding proximity for samples in these tasks is unrelated to task labels\n\n**Explaining difference from Chen et al 2022.** Chen et al (2022) introduces Liger. Liger differs from Embroid in several ways. \n1. Liger requires the user to specify a single embedding space, while Embroid incorporates multiple embedding spaces. As a result, Liger requires the embedding space chosen be high quality. In contrast, Embroid is more robust to lower quality embedding spaces, because these embedding spaces are aggregated. \n2. Moreover, Liger requires multiple predictions for each sample, each of which must come from a different hand-written labeling function (LF). In contrast, Embroid requires only a single prediction for each sample–generated from a few-shot LM–and relies on embedding spaces to manufacture additional predictions which can be combined.\n3. Liger operates on label functions, and not the predictions of few-shot LMs. These label functions are written by hand, and have fundamentally different properties from few-shot LMs. For instance, they abstain, usually have high precision, and lower recall. \n4. Liger uses embedding spaces to (1) propagate votes for each LF from labeled samples onto its neighbors, and (2) uses these embedding spaces to learn clusterings over the data, on each of which a different graphical model is learned. In contrast, Embroid uses embedding spaces to evaluate disagreement between neighboring predictions, and only learns a single graphical model for the entire dataset. \n5. Finally, we note that Embroid, when run in the same setting as Liger (i.e., with access to multiple predictions per sample), exceeds Liger’s performance (Table 2).\n\n**Explaining difference from Fu et al 2020.** Fu et al (2020) introduces FlyingSquid. While Embroid uses FlyingSquid to solve the graphical model posed, Embroid introduces several innovations beyond the contributions made in Fu et al. First, Embroid incorporates information from embedding spaces, which Fu et al. do not consider. Second, Embroid requires only one noisy vote per sample (bootstrapping additional votes from the embedding spaces), while FlyingSquid requires multiple. Finally, Fu et al evaluate FlyingSquid in the classical hand-written LF regime.\n\n**Space and time complexity.** We provide additional information on the space and time complexity of Embroid. First, we clarify that Embroid requires only one large model. This model can be on the order of GPT-3 (176B parameters), or a much smaller open-source equivalent like GPT-JT (7B parameters). Next, Embroid requires embeddings from 2-3 additional models. However, as our results show, these models can be relatively small. For instance, we rely on BERT-base and SentenceBERT models–both of which are approximately 110 million parameters. Finally, the latent variable graphical model has 2 parameters for class prior and 1+N parameters per LLM prediction, where N is the number of embedding spaces. So, when Embroid is run on a single LLM’s predictions with two embedding spaces, the total number of parameters is 2 + (1+2)*1 = 5.\n\nIn terms of time complexity, we observe that computing LLM predictions is the largest bottleneck. On a dataset of ~1800 samples for instance: \n\n- Computing predictions on GPT-3.5 takes 1440 seconds (24 minutes). \n- Computing embeddings using BERT takes 5 seconds.\n- Computing nearest-neighbors using the (unoptimized) scikit-learn library takes ~3 seconds. \n- Solving the Embroid graphical model takes less than a second.\n\nWe will add this analysis to our Appendix.\n\n**Ablation on Eq 3.** We note that ablating Eq. 3 to isolate the impact of adding the last term (which incorporates embedding information) is essentially equivalent to either (1) comparing the original predictions to Embroid, or (2) comparing the FlyingSquid model over three sets of predictions (per sample) to Embroid. (1) is provided for §6.1 (Table 1), where we observe that Embroid improves upon the original predictions for multiple models, by a significant margin. (2) is provided by §6.2 (Table 2), where we find that Embroid improves upon Flying Squid by between 4 to 20 points, depending on the LM used.\n\n**How should embedding spaces be selected?** Please see the general response.\n\n**Are BERT embeddings smooth?** Figure 2 (bottom left) plots average embedding smoothness for each task, as computed using Eq 4. We find that for a number of tasks, BERT-variant embeddings are smooth. "}]

Author rebuttal:
We thank all reviewers for their feedback. We are glad reviewers recognized and appreciated the novelty/simplicity of our method [Rd7v, Bw2X, YSHE, XCbh, bLRH], our theoretical analysis [Rd7v, YSHE, XCbh, bLRH], and our empirical validation [Rd7v, YSHE, Bw2X, XCbh, BLRH]. 

We have made a number of changes in response to the reviewers’ comments and questions: 
1. We have added an evaluation in the multi-class setting, where we find Embroid offers improvements over the base prompt (by up to 3.5 points on balanced-accuracy).
2. We have added exposition to clarify questions related to embedding smoothness and the choice of embeddings. 
3. We have added experiments evaluating Embroid on GPT-4 and with self-consistency. We find that Embroid can still improve upon GPT-4 (by up to 8 points for some tasks), and that Embroid improves self-consistency for all studied tasks.

**Multi-class evaluation.** Multiple reviewers [YSHE, Bw2X, bLRh] asked whether Embroid could be applied to multi-class problems in a one-vs-all setting. We provide initial results on a multi-class version of the contractual classification task, involving five unbalanced classes (where the largest class is 3.5x larger than the smallest class). Overall, we find that the one-vs-all variant of Embroid improves the quality of the base prompt (by 4 points on balanced-accuracy and 1.6 points on macro-F1). We will add a more comprehensive study with additional datasets to our paper.

**Embedding smoothness.** Several reviewers [RD7v, bLRH] raised questions regarding properties of the embedding spaces and how they should be selected for Embroid. In the revised draft, we will expand our discussion of hyperparameters in the Appendix (Appendix G) to include an overview of embedding selection. We offer more concise responses to reviewer questions below.

At the outset, we note that selecting/evaluating embeddings is particularly difficult in our regime, where practitioners do not have access to labeled data. Importantly, this motivates Embroid’s “mixture of embeddings'' approach. Previous methods (e.g., Liger [7]) operate on only a single embedding space. Thus, practitioners must be certain that the embedding space chosen is good for a task, and a poorly chosen embedding space will reduce performance. Because Embroid ensembles different embedding spaces, practitioners can be less certain in the quality of any individual embedding space. 

*Is there a simple method to quantify smoothness for an embedding?*

Smoothness for an embedding is typically calculated using Eq. 4 (L188). This requires knowing the true label $y$ for each point. When practitioners have access to a dev set, smoothness can be estimated using this set of points. Our work assumes a pure unlabeled setting (also known as “true few-shot”) where no dev set is accessible. Thus, computing Eq. 4 is impossible. We will clarify this fact in §5.

*Are there methods/selection strategies to drop out embeddings from a given task?*

In the expanded discussion in our paper, we will discuss potential methods for selecting possible embedding spaces to include in Embroid. These include: 
- Looking at the MLM loss of the embedding model (e.g. BERT) on the task data. Prior literature on domain specificity has suggested that this may be a promising heuristic [27].
- Looking to the performance of embedding models on other related tasks may also be helpful [27].
- Looking at the extent to which a potential embedding space generates embeddings which are geometrically similar to embeddings already included in Embroid. If the embedding space is similar, then the additional value of incorporation is low. 

*Is there a theoretical result for the optimal number of selection of embeddings used in Embroid?*

The number of embeddings used presents a bias-variance tradeoff. Under Proposition 5.1, increasing the number of embedding spaces (1) increases the variance due to estimation error, but lowers (2) the conditional entropy of $H(y | \lambda)$. Precisely characterizing this tradeoff is challenging because the variance is an upper bound, and $H(y | \lambda)$ can only be estimated. However, these bounds/estimates can be used to derive heuristic stopping rules: for instance, add embedding spaces until the marginal decrease in the conditional entropy is less than the upper bound on the marginal increase in variance. We will add this discussion to Section 5. 

Meta-review: This paper proposes a method for improving prompt-based few-shot classification without additional labeled data. The method focuses on computing multiple embedding representations and using consistency of predictions within neighborhoods as a way to identify LM mispredictions. The paper is well motivated, the proposed approach is simple & presented clearly, and shown to be effective for different model & task settings. The authors also did a pretty good job in their rebuttal and discussions addressing reviewer comments, which further helped clarify & also spotlight some of the central questions (around extension to other tasks, comparison against other related works on self-consistency, and the viability of the method for practical tasks). One area for potential future work is to consider more complex task settings (go beyond even multi-class problems, for instance QA) & study the practical tradeoff between smoothness (or a weakened requirement) vs. efficacy of this approach.

Overall, this is a good contribution with an interesting approach validated by strong experimental results & analyses using contemporary models/tasks in a practical setting. The authors are encouraged to use the provided feedback and incorporate the points detailed in comments to improve the final version of the paper.
Decision: Accept (poster)