Title: Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Venue: NeurIPS 2023
Accepted: yes
Keywords: language models

prompting

embeddings

weak supervision
Abstract: Recent work has shown that language models' (LMs) prompt-based learning capabilities make them well suited for automating data labeling in domains where manual annotation is expensive. The challenge is that while writing an initial prompt is cheap, improving a prompt is costly---practitioners often require significant labeled data in order to evaluate the impact of prompt modifications. Our work asks whether it is possible to improve prompt-based learning _without_ additional labeled data. We approach this problem by attempting to modify the predictions of a prompt, rather than the prompt itself. Our intuition is that accurate predictions should also be consistent: samples which are similar under some feature representation should receive the same prompt prediction. We propose Embroid, a method which computes multiple representations of a dataset under different embedding functions, and uses the consistency between the LM predictions for neighboring samples to identify mispredictions. Embroid then uses these neighborhoods to create additional predictions for each sample, and combines these predictions with a simple latent variable graphical model in order to generate a final corrected prediction. In addition to providing a theoretical analysis of Embroid, we conduct a rigorous empirical evaluation across six different LMs and up to 95 different tasks. We find that (1) Embroid substantially improves performance over original prompts (e.g., by an average of 7.3 points on GPT-JT), (2) also realizes improvements for more sophisticated prompting strategies (e.g., chain-of-thought), and (3) can be specialized to domains like law through the embedding functions.

Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification

Abstract

Recent work has shown that language models' (LMs) prompt-based learning capabilities make them well suited for automating data labeling in domains where manual annotation is expensive. The challenge is that while writing an initial prompt is cheap, improving a prompt is costly-practitioners often require significant labeled data in order to evaluate the impact of prompt modifications. Our work asks whether it is possible to improve prompt-based learning without additional labeled data. We approach this problem by attempting to modify the predictions of a prompt, rather than the prompt itself. Our intuition is that accurate predictions should also be consistent: samples which are similar under some feature representation should receive the same prompt prediction. We propose EMBROID, a method which computes multiple representations of a dataset under different embedding functions, and uses the consistency between the LM predictions for neighboring samples to identify mispredictions. EMBROID then uses these neighborhoods to create additional predictions for each sample, and combines these predictions with a simple latent variable graphical model in order to generate a final corrected prediction. In addition to providing a theoretical analysis of EMBROID, we conduct a rigorous empirical evaluation across six different LMs and up to 95 different tasks. We find that (1) EMBROID substantially improves performance over original prompts (e.g., by an average of 7.3 points on GPT-JT), (2) also realizes improvements for more sophisticated prompting strategies (e.g., chain-of-thought), and (3) can be specialized to domains like law through the embedding functions.

1 Introduction

Acquiring labeled data for domains like medicine and law is essential to training machine learning models or performing basic data analysis (e.g., 'how many contracts contain a choice-of-forum clause" or 'how many patient medical histories discuss an adverse reaction to a drug?') [15, 18, 19]. However, building large labeled datasets is difficult, and efforts like [25] show that manual labeling with domain experts is cost-prohibitive. Recent works have begun exploring if language models (LMs) could learn annotation tasks in-context [6] and replace manual labeling at scale [13, 15, 23, 31]. The promise of this approach is that LMs' in-context capabilities enable them to learn tasks from descriptions of the the task (i.e., prompts ). However, the challenge is that producing high performance

Embroid:UnsupervisedPredictionSmoothingCanImproveFew-ShotClassification

Existingworkhasstudied howtoengineerbetter

Westudyhowtoidentifyandfliperroneousprompt

promptsthroughvarioustextualmodifications.

predictions,withoutusinglabeleddata.

X1

The capital of India is Delhi.

LanguageModel

Prediction:No

Prediction:No

M

X2

Chelsealostbythreegoals

in theFA CupFinal.

:Prompt:"Does the

→Prediction: No

Embroid

→

Prediction:Yes

sentence discuss

：

：

Xn

France scoredfourgoalsin

the FIFA final.

asoccerteam?"

→Prediction:Yes

Prediction:Yes

Embroid

RoBERTaNearest NeighborsforX2

Original vote:No

BERTembedding spacevote:Yes

Corrected prediction:Yes

RoBERTaembeddingspacevote:Yes

Xn-1

(1)RetrieveknearestsamplesforXundereachfunctionand

(2)Combine computed votes with original LM prediction using

graphicalmodeltogeneratecorrectedprediction

prompts is still expensive, as practitioners require labeled data in order to measure the impact of modifications to a prompt [51]. Existing work has thus focused on how domain experts can optimally construct prompts for a task ( prompt-engineering ), while minimizing reliance on labeled data [41, 58, 64]. Yet, because language models are sensitive to even small changes in prompt language, these techniques are imperfect and still produce erroneous predictions [2, 8, 38, 51, 70].

Our work approaches the challenge of improving prompt performance without labels from an orthogonal perspective: given the predictions of any prompted LM, can we identify and correct mis-predictions using unlabeled data? We describe this as the problem of prompt-patching . In the context of data annotation tasks, prompt-patching methods should meet three goals. First, they should be theoretically explainable, so that practitioners can understand when and how to apply them. Second, they should be fast, so that practitioners can efficiently integrate them into existing workflows. Finally, they should rarely be wrong, so that they don't worsen the performance of predictions.

Our work presents EMBROID: a method for automatically identifying and correcting LM predictions with unlabeled data and no expert supervision. Recent work has shown that for many tasks, samples close-by in embedding spaces (produced by models like BERT) have the same label [7]. EMBROID applies this intuition to the prompt-patching regime. Specifically, after LM predictions for all samples have been generated, EMBROID retrieves the k most similar samples for each input, under N different embedding functions. For each embedding function, EMBROID computes a scaled-modified majority vote over the LM's predictions for the k retrieved samples. EMBROID then combines these N votes with the original LM prediction for the test sample using a simple latent variable graphical model that is learned with a fast method-of-moments estimator [14]. The intuition behind EMBROID is that good prompts are smooth with respect to their predictions over a dataset-samples which are proximate under an embedding function should receive consistent predictions. Thus, modifying the predictions of a prompt to increase neighborhood agreement can improve the accuracy of those predictions. Lastly, because a single embedding space may imperfectly capture similarities between samples, retrieving neighbors from multiple embedding spaces improves robustness [28, 40].

Because EMBROID relies on weak-supervision-the subject of recent rigorous study [7]-it is possible to theoretically analyze and explain why and when EMBROID will improve performance. In particular, we find that performance is a function of the quality of the embeddings and the performance of the initial prompt. We also empirically study EMBROID, conducting experiments over six LMs, on up to 95 tasks, with several different prompt strategies. We find that EMBROID rarely worsens performance, and often improves F1 by a substantial margin. For instance, EMBROID improves GPT-3.5 by an average of 4.9 points F1 per task, and GPT-JT by an average of 7.3 points per task. The magnitude of EMBROID's gains are such that it enables a 1.3B parameter model to outperform an instruction-tuned 6.7B parameter model. EMBROID is also complementary to advanced prompt engineering strategies, and achieves performance improvements when applied to prompts designed

using chain-of-thought [64], AMA [2], and selective annotation [58]. Finally, EMBROID can be extended to specialized domains like law, through the use of already-available domain specific embeddings.

Succinctly, our contributions in this paper are: (1) EMBROID, a simple prompt-patching framework for improving LM predictions over text classification tasks; (2) a theoretical analysis of EMBROID which explains performance improvements in terms of embedding smoothness and base accuracy; and (3) an empirical evaluation of EMBROID covering up to 95 tasks and six different LMs.

2 Related work

Improving LM performance Improving the in-context generalization abilities of LMs has been intensely studied. The first family of approaches focuses on adapting LMs in order to make them more amenable to prompting. This includes task-specific finetuning [26, 27, 47], training on instruction data [9, 59], RLHF [50], and weight-surgery methods which attempt to 'correct' incorrect information stored in model weights [10, 22, 43, 44]. A second family of approaches explores strategies for optimizing prompts to models, either through the specific textual features of the prompt [29, 45, 64], the use of task decompositions or LM recursion [2], implicit prompt representations [36, 37], or external databases [46]. Prompt-patching, in contrast, focuses on identifying mistakes in the predictions generated from a particular prompt. The most related approaches are aggregation methods, in which the outputs of multiple prompts are combined with an ensembling method [2, 41]. We find that EMBROID outperforms many such baselines, and can be applied to enhance their outputs.

Weak supervision EMBROID leverages statistical techniques developed in the weak supervision literature. The objective in weak supervision is to generate probabilistic labels for unlabeled data by combining the predictions of multiple noisy heuristics [14, 53, 54, 57, 62, 66]. EMBROID's novelty is that it uses embeddings to construct additional synthetic predictions, which are combined with the original predictions. In contrast, recent weak supervision approaches which incorporate embeddings use them to produce more fine-grained accuracy parameters [7], detect and discard training points [34], and as the basis for label propagation with final weak supervision predictions [52].

3 Problem setup and background

Problem setup Our problem setup comprises three elements: an unlabeled dataset, predictions from a LM for each sample in this dataset, and embedding representations of our dataset. Our goal is to improve the accuracy of LM predictions, by using the embedding representations to identify and correct predictions likely to be incorrect. Because recent work has explored how predictions from multiple prompts can be combined for a task [2], we present a generalized version of EMBROID in which we have access to multiple LM predictions. In our empirical evaluation however, we show that EMBROID performs well regardless of the number of predictions per sample available.

More formally, we focus on a binary classification task where x ∈ X denotes a sentence or paragraph and y ∈ Y = {-1 , 1 } is the binary label. We assume we are given an unlabeled dataset D = { x i } n u i =1 of n u points. Each point x is sampled i.i.d. from a distribution P x , and there exists a true underlying distribution P on the joint ( x, y ) . Following the true few-shot regime [51], we assume the only labels available are those used in the prompt. We denote a language model (e.g., GPT-3) as λ LLM , and a task-specific prompt as ϕ , which prepends task instructions to input x (e.g., 'Does the clause contain an audit provision? Yes or No."). We consider the 'prompt' to include both the task description and the in-context samples, consistent with [70]. The prediction this prompt induces for λ LLM over x is λ LLM ( ϕ ( x )) ∈ Y . 1 Varying ϕ by changing the task description, in-context demonstrations, or punctuation will alter the prediction generated for x . For a set of m prompts [ ϕ 1 , . . . , ϕ m ] , we denote their respective predictions on x as a vector of weak sources λ ( x ) = [ λ LLM ( ϕ 1 ( x )) , . . . , λ LLM ( ϕ m ( x ))] . For convenience, we denote λ LLM ( ϕ i ( x )) as λ i ( x ) or λ i when the x is obvious, and similarly use λ instead of λ ( x ) . We distinguish between two regimes: in the single-prompt regime with m = 1 , we have access to a LM prediction for each point x , while in the multi-prompt regime with m> 1 , we have access to multiple predictions.

Algorithm 1 EMBROID: Correcting LLMs with embeddings

Input: Unlabeled data D , LLM predictions λ ( x ) for each x ∈ D , embedding models E = { E 1 , . . . , E N } , shrinkage parameter τ , nearest neighbors parameter k for all unlabelled x ∈ D do

for all embedding models E j ∈ E do

Compute k-nearest neighbors NN j,k ( x )

Compute smoothed neighborhood prediction λ sm ,j ( x ) using λ , NN j,k ( x ) , and τ using eq. (2)

end for end for

Solve graphical model Pr( y, λ ( x ) , λ sm ( x )) in eq. (3) with triplet method over D (Algorithm 2).

for all unlabeled x ∈ D do

end for

Output: Label set ˆ Y = { ˆ y x | x ∈ D}

We assume access to N embedding models E = [ E 1 , . . . , E N ] , each represented as a fixed mapping E i : X ↦→ Z i from an input x to an embedding vector z . These auxiliary embedding models provide representations of x which encode different types of similarity information. Through model repositories like HuggingFace [65], it is possible to download a number of models which generate representations for text sentences (e.g., BERT or RoBERTa [12, 42]). These models have the property that semantically similar sentences are close-by in embedding space [7, 28, 40].

Weak supervision background Weak supervision uses a graphical model to combine votes from multiple noisy sources into a single prediction, by estimating the accuracy of each source. It models Pr( y, λ ( x )) as a latent variable graphical model and uses ˆ y = argmax y ˆ Pr( y | λ ( x )) to produce label estimates, where ˆ Pr represents the learned model. The graphical model is based on a graph G = ( V, E ) , where V = y ∪ λ and E consists of edges from y to each λ j . We assume no dependencies between sources, although simple extensions can incorporate them [61]. The formal graphical model is:

where Z is the partition function used for normalization, ( I ) represents a label balance term with parameter θ y controlling the prior of Pr( y = 1) , and ( II ) represents the source accuracy term where each θ i is an accuracy parameter for the i th source. Note that from this model, sources are conditionally independent: λ i ⊥ ⊥ λ j | y for any i, j ∈ [ m ] . Our use of this model has two steps. First, we must learn the accuracy parameters of Pr( y, λ ( x )) without access to y . We use the triplet method introduced in [14], which is an efficient method-of-moments estimator for the parameters. Then, at inference we compute ˆ Pr( y | λ ( x )) . Appendix B contains more details.

4 EMBROID

First, EMBROID uses the embedding models E to compute additional votes for each x . Let NN j,k ( x ) ⊂ D be the k -nearest neighbors of sample x under the embedding function E j . We define the smoothed neighborhood prediction vector λ sm ,j ( x ) ∈ {-1 , 0 , 1 } m as follows, with λ sm ,j [ i ]( x ) being the i th element:

where τ + i ∈ [ -1 , 1] and τ -i ∈ [ -1 , 1] act as shrinkage parameters for λ i which control the level of agreement amongst the neighbors of x necessary to generate a particular vote. The scalar λ sm ,j [ i ]( x ) is the average vote of λ i amongst the neighbors of x in E j . When λ sm ,j [ i ]( x ) is sufficiently positive, i.e., λ sm ,j [ i ]( x ) > τ + i , EMBROID sets λ sm ,j [ i ]( x ) to be a positive vote. When λ sm ,j [ i ]( x ) is sufficiently negative, i.e., λ sm ,j [ i ]( x ) < τ -i , EMBROID sets λ sm ,j [ i ]( x ) to be a negative vote. Otherwise, λ sm ,j [ i ]( x ) is set to be an abstain. The intuition is that λ sm ,j [ i ]( x ) will be an accurate vote over x whenever two conditions are met: (1) the LM is generally accurate, i.e., λ j is usually correct, and (2) E j is smooth , i.e., nearest-neighbors share the same task label.

Next, we augment our base model in equation (1) to incorporate these auxiliary neighborhood predictions λ sm = [ λ sm , 1 , . . . , λ sm ,N ] ∈ {-1 , 0 , 1 } Nm computed using the embeddings:

where the vector α j ∈ R m represents the quality parameters for the j th embedding model when used with the m different prompts. To solve this model and produce label estimates, we note that it has the same format as (1) if we concatenate λ and λ sm into one set of weak sources. Therefore, we can use the triplet method from [14] to learn parameters and output estimates ˆ Pr( y | λ ( x ) , λ sm ( x )) for each x ∈ D at inference time (see Appendix B for details).

Parameters θ and α j in (3) allow us to trade-off two different sources of information-one presented by directly prompting an LM to obtain a label and the other by incorporating similarity information from the embedding models-and to further account for varying error modes among the embedding models. Our use of the neighborhood predictions in (3) yields a more expressive model than the standard weak supervision framework solely on LLM predictions in (1), which we can recover when k = 0 , and can thus help make corrections to the LLM predictions. In practice, we find that setting τ + i = τ -i = E [ λ i ] (i.e., the average source vote) yields good performance (Appendix G).

5 Theoretical analysis

We analyze EMBROID, discussing the advantages of using λ sm in addition to λ , and show that embedding smoothness and base prediction accuracy play a critical role in information gain. Appendix F provides synthetics demonstrating these tradeoffs and comparing to weak-supervision baselines.

First, we provide a result on the generalization error of our model ˆ Pr( y | λ , λ sm ) . Define the generalization error as the expected cross-entropy loss, L ( λ , λ sm , D ) = E y, λ ( x ) , λ sm ( x ) , D [ -log ˆ Pr( y | λ ( x ) , λ sm ( x ))] . We use [ λ 1 , . . . , λ ( N +1) m ] to represent [ λ , λ sm ] and denote by a max = max i E [ λ i ( x ) y ] the largest accuracy (scaled to [ -1 , 1] ) of any source, and by b min = min i,j { E [ λ i λ j ] , ˆ E [ λ i λ j ] } the minimum expected pairwise product between any two sources. Assume that all sources are better than random, e.g., Pr( λ i = y ) > 0 . 5 . These terms and assumptions are from using the triplet method.

Proposition 5.1. Suppose that the data x, y, λ , λ sm follows the model in (3) . The generalization error of ˆ Pr( y | λ , λ sm ) can be decomposed into

In the bound above, the variance term comes from estimation error when learning the parameters via the triplet method. The irreducible error depends on quality of λ and λ sm . If knowledge of the LLM prediction and neighborhood prediction significantly reduces uncertainty in y , the conditional entropy term H ( y | λ ( x ) , λ sm ( x )) is low.

Information gain from using both λ , λ sm We compare upper bounds on generalization error when both λ , λ sm are modeled versus when only λ is modeled, as in (1) corresponding to classical weak

supervision. Based on the bound in Proposition 5.1, modeling both λ and λ sm increases the variance term by a constant multiplicative factor.

Here, we examine how the irreducible error is affected, that is, the difference H ( y | λ ) -H ( y | λ , λ sm ) . Since this quantity is always nonnegative, we focus on bounding the pointwise difference in conditional entropy-which we call the information gain-for a given x 0 on which the LLM is incorrect. For simplicity, suppose we have one embedding E . An embedding E is M -smooth with respect to the label if

where c ∈ Y , ε > 0 and M E ( · ) ∈ [0 , 1] is decreasing in its input. Note that this definition requires knowledge of ground-truth labels, and is thus impossible to use as a metric for selecting which embeddings to use.

Define β i = Pr( λ i = y ) as the accuracy of λ i and p λ = Pr( y = 1 | λ ( x 0 )) as the prediction on x 0 given only access to λ . Let ε k = max ˜ x ∈ NN k ( x ) ∥ E ( x ) -E (˜ x ) ∥ be the maximum distance between x 0 and its k neighbors. Without loss of generality, assume the label on x 0 is y = 1 .

Theorem 5.2. Assume that E is M -smooth. The pointwise information gain on x 0 is

where β NN k ,i = Pr ˜ x ∼ NN k ( λ i (˜ x ) = y ) ≥ β i M E ( ε k ) is the neighborhood accuracy.

A few observations on the bound are in order.

Improvement over WS: If the neighborhood accuracy is bounded sufficiently far from 1 2 and k is large, using EMBROID has better irreducible error than just using λ . For example, setting m = 1 , k = 10 , β NN k ,i = 0 . 7 , and p λ = 0 . 25 gives us an improvement of 0 . 076 nats.

Smoothness: If E is highly smooth, then M E ( ε k ) will be large and irreducible error will be small.

Base prediction accuracy: If the original sources λ have high accuracy ( β i ), irreducible error will be small.

Additionally, we observe that if p λ is a high-quality prediction close to the true label 1 , the information gain is small. Choice of the k parameter presents a performance trade-off: increasing k will increase ε k and incorporate farther-away, less reliable predictions, but it will also reduce the noise of the majority vote. We also comment on the information gain when using both λ and λ sm over just λ sm in Appendix C.

6 Results

Our empirical evaluation focuses on three questions: (1) How robust is EMBROID's performance across LMs? (2) How does EMBROID, as a prompt-patching method, compare to high performance prompt-engineering methods? (3) How sensitive is EMBROID to the embeddings and dataset size?

Tasks We study tasks where sentence embeddings can capture information relevant to the task, leading us to focus on sentence classification datasets. We consider a collection of 95 class-balanced sentence classification tasks, derived from binarizing existing multi-class legal, scientific, and general domain classification benchmarks like CUAD, AGNews, DBpedia-14, FewRel, and several others [21, 25, 30, 67, 69]. 2 Example tasks include, 'Classify if the following texts discuss a recording label' or 'Classify if the following contractual clauses contain an audit rights provision.'

Table 1: We evaluate the extent to which EMBROID improves the original prompt on different models in terms of win rate and relative improvement (defined in-line). All models are run with three trials. For each model, we report the percentage of tasks (across all trials) for which EMBROID improves, and the average improvement (in F1 points). Additional details provided in Appendix.

Choice of embedding models Following prior work illustrating the benefits of domain specific representation [20, 71], EMBROID uses different embeddings for each task domain. For law tasks, we rely on two BERT-variants trained on different legal corpora [24, 71]. For science tasks, we rely on three BERT-variants trained on science, biology, and medical texts [3, 17, 35]. For general domain tasks, we rely on BERT, Roberta, and SentenceBert embeddings [12, 42, 55].

Prompts Prompts are constructed using fixed instructions, and by manually selecting three random samples (from each class) as in-context demonstrations (Appendix E). We follow the true few-shot regime [51], in that we assume the only labeled data available to the data scientist are the labels used for in-context demonstrations. Prior work has found this regime to most realistically represent real-world workflows.

Models We evaluate on two API-access models: GPT-3.5 ( text-davinci-003 ) and J1Jumbo [39]. Because API models raise significant privacy and compliance concerns for data scientists working with sensitive data [16], we also evaluate on open-source models. We select models in the 6-7B parameter range, as these are the largest models which fit on commonly available 40GB A100 machines. Specifically, we evaluate Bloom [56] and OPT [68]. Given the increasing popularity of instruction-tuning, we also evaluate on GPT-JT [60], an 6.7B parameter instruction tuned version of GPT-J. Because of cost-constraints, we evaluate API-access models on a representative selection of 12 tasks, while evaluating all other models on the full suite of 95 tasks. Appendix D provides details.

6.1 By how much does prompt-patching improve performance?

Performance across LM families We examine if EMBROID achieves improvements for different types of LMs. For each LM, we select three different combinations of in-context demonstrations (i.e., three prompts), generate predictions for each prompt, and apply EMBROID to independently each prompt's predictions. This produces 3 × 95 = 285 trials for open-source models, and 3 × 12 = 36 trials for API-models. We report win-rate , i.e., the proportion of trials for which EMBROID outperforms the original predictions, and improvement , i.e., the average difference in F1 points (across all trials) between EMBROID and the original predictions.

As Table 1 illustrates, EMBROID improves performance for a substantial proportion of prompts, by a substantial margin, across all models. On GPT-3.5 for instance, EMBROID achieves a win-rate of 80.6%, with an average of improvement of 4.9 points. EMBROID also improves for open source models, with a win-rate of 91.2% on OPT-6.7 and an average improvement of 11.6 points. Finally, EMBROID achieves similar gains on an instruction tuned model, with a win-rate of 89.1% and an average improvement of 7.3 points.

Performance when prompts are good We additionally investigate how EMBROID's performance improvements change as a function of the performance of the base prompt. Hypothetically, one could imagine that better performing prompts are smoother with respect to embeddings, thus diminishing (or negating) EMBROID. In Figure 2 (upper left), we plot the improvement of EMBROID against the performance of the base prompt for GPT-JT. Even when the base prompt performs well (i.e., F1 > 0 . 8 ), EMBROID improves on 89% of tasks by an average of 4.1 points.

Table 2: We evaluate how EMBROID compares to common ensemble approaches for improving prompt prediction performance. All ensemble baselines are run with three sets of predictions. EMBROID-1 is run with one set of predictions, and EMBROID-3 is run with three set of predictions. For each method, we report the average macro-F1 over all tasks. We observe that EMBROID1 is competitive with ensemble methods which use many more predictions, while EMBROID-3 outperforms all other methods by a substantial margin.

Measuring performance in parameter count A trend in recent literature has been to measure the magnitude of improvements to prompt performance in terms of parameter count [2], by showing how a particular method makes a smaller method equivalent in performance to a larger model. We find that EMBROID enables the non-instructed tuned 1.3B GPT-Neo model to outperform an instruction tuned 6.7B model; across all trials, GPT-JT scores an average F1 of 67.8, while EMBROID +GPT-Neo-1.3B scores an average of 68.5.

6.2 Comparing prompt-patching to prompt-engineering

Our work distinguishes between prompt-construction methods-which control how a prompt is generated-and prompt-patching methods-which attempt to identify and correct errors in the predictions produced by a prompt. We use EMBROID to further study the difference between these frameworks in two ways. First, we compare EMBROID's performance improvement over a base prompt to that of several specialized prompting strategies. Second, we examine the extent to which EMBROID-when applied to the predictions produced by these prompting strategies-can generate further performance improvements. We study three prompting strategies:

Ensemble strategies, in which the predictions of multiple prompts are combined using an unsupervised ensembling model. Specifically, we compare to two ensembling methods previously studied for LLMs (AMA [2] and majority vote [41]), one ensembling method which incorporates embedding information (Liger [7]), and one well regarded weak supervision baseline (FlyingSquid [14]). Each baseline is run over the predictions generated by three different prompts.

Chain-of-thought prompting [64], in which for each in-context demonstration, we provide a step-by-step explanation for the demonstration's label.

Selective annotation (SA) [58], in which we use embeddings to select a subset of k data samples to label, and then, for each input sample, retrieve the most similar samples (under some embedding function) from this pool to use as in-context demonstrations.

Ensemble methods We evaluate two versions of EMBROID. In the first version, we run EMBROID with the predictions of only one prompt (EMBROID-1). In the second version, we run EMBROID with the predictions of three different prompts (EMBROID-3). Note that this requires performing inference over the few-shot LM three times for each sample, which can be expensive. This baseline is comparable to applying EMBROID to the outputs of an ensemble method. In Table 2, we observe that EMBROID-1 is competitive with the ensemble baselines (while using substantially fewer predictions), while EMBROID-3 consistently outperforms these baselines (across different LMs).

Chain-of-thought We compare EMBROID to chain-of-thought (CoT) prompting for a subset of a representative subset of 10 tasks on GPT-3.5. For each task, we manually construct a base prompt consisting of six demonstrations, and a CoT prompt where an explanation is provided for each demonstration. We first find that EMBROID's performance improvement over the base prompt exceeds that of chain-of-thought prompting (Table 3). Using EMBROID to modify the base prompt is better

0.4

0.6

0.8

1.0

Original Prompt Performance

0.4

0.6

0.8

1.0

Embroid

20

40

60

80

100

Number of labeled points

0.70

0.75

0.80

0.85

F1

SA

SA + Embroid

Rand

Rand + Embroid

0.6

0.7

0.8

0.9

Embedding  moothne

-0.05

0.00

0.05

0.10

0.15

0.20

Embroid improvement (F1)

Law

General

Science

0

1000

2000

3000

Dataset size

-0.05

0.00

0.05

0.10

0.15

0.20

Embroid Improvement

on average than CoT prompting, outperforming CoT on six out of the ten tasks. Second, applying EMBROID to predictions generated by a CoT prompt yields further improvements, outperforming vanilla CoT predictions on eight of ten tasks.

Selective annotation (SA) We compare EMBROID to selective annotation with a label budget of [6 , 25 , 50 , 100] (Figure 2, upper-right). For each task, we run selective annotation using a domain specific embedding. EMBROID (applied to a prompt with randomly chosen samples) outperforms selective annotation with a label budget of 25 samples. When a label budget of 100 samples is available, EMBROID improves the performance of a prompt constructed using selective annotation on 88% of tasks, by an average of 4.3 points.

6.3 Ablations

Finally, we perform several ablations of EMBROID to study how performance changes as a function of (1) the domain specificity of the embedding used, (2) the quality of the embedding spaces used, and (3) the size of the dataset. Additional ablations are presented in Appendix G.

Domain specific embeddings improve performance We compare how performance on the legal and science tasks changes when we shift from domain specialized embeddings to general domain embeddings. On law tasks for GPT-JT, we find that using two legal embedding spaces outperforms using BERT and RoBERTa for 77% of tasks, by up to 6 points F1 on certain tasks [24, 71]. For science tasks for GPT-JT, we find that using two science embedding spaces [3, 35] outperforms using BERT and RoBERTa for 92% of tasks, by up to 4.3 points F1 on certain tasks.

Embedding quality Building on Section 5, we compare EMBROID's performance improvement over the base prompt to the average smoothness of the embedding spaces with respect to each task (Figure 2). We observe a positive correlation: smoother embedding spaces are associated with larger performance gains (with a Pearson coefficient of r = 0 . 39 ). Applying this insight, we explore how performance changes when extremely high quality embeddings are added. For a subset of 19 tasks we generate OpenAI text-embedding-ada-002 embeddings, and find that adding them to EMBROID improves performance by up to 13 points F1 (at an average of 2 points across all studied tasks).

Dataset size Finally, we study how EMBROID's performance improvement changes as the dataset size changes. Because EMBROID relies on nearest-neighbors in different embedding spaces, we might expect performance to be poor when the dataset being annotated is small. In Figure 2 (bottom right), we see that EMBROID achieves performance improvements even for 'small' datasets with only several hundred samples. ¯

7 Conclusion

We study the problem of improving prompt-based learning, by developing a method (EMBROID) for detecting and correcting erroneous predictions without labeled data. We validate EMBROID across a range of datasets and LMs, finding consistent improvement in many regimes. We take a moment to address the societal impact of our work: while we do not foresee any direct harmful impacts arising from our work, we caution that any use of language models in meaningful applications should be accompanied by conversations regarding risks, benefits, stakeholder interests, and ethical safeguards.

References

Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. Large language models are few-shot clinical information extractors, 2022.

Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, and Christopher Ré. Ask me anything: A simple strategy for prompting language models, 2022.

Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 , 2019.

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 , 2021.

Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference , pages 491-500, 2019.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Mayee F. Chen, Daniel Y. Fu, Dyah Adila, Michael Zhang, Frederic Sala, Kayvon Fatahalian, and Christopher Ré. Shoring up the foundations: Fusing model embeddings and weak supervision, 2022.

Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. On the relation between sensitivity and accuracy in in-context learning. arXiv preprint arXiv:2209.07661 , 2022.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696 , 2021.

Franck Dernoncourt and Ji Young Lee. Pubmed 200k rct: a dataset for sequential sentence classification in medical abstracts. arXiv preprint arXiv:1710.06071 , 2017.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 2018.

Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq Joty, and Boyang Li. Is gpt-3 a good data annotator? arXiv preprint arXiv:2212.10450 , 2022.

Daniel Fu, Mayee Chen, Frederic Sala, Sarah Hooper, Kayvon Fatahalian, and Christopher Re. Fast and three-rious: Speeding up weak supervision with triplet methods. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 3280-3291. PMLR, 13-18 Jul 2020.

Karan Goel, Sabri Eyuboglu, Arjun Desai, James Zou, and Chris Ré. Meerkat and the path to foundation models as a reliable software abstraction. https://hazyresearch.stanford.edu/blog/2023-03-01-meerkat, 2023.

Karla Grossenbacher. Employers should consider these risks when employees use chatgpt. https://news.bloomberglaw.com/us-law-week/employers-should-consider-these-risks-whenemployees-use-chatgpt, 2023.

Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing, 2020.

Neel Guha, Daniel E Ho, Julian Nyarko, and Christopher Ré. Legalbench: Prototyping a collaborative benchmark for legal reasoning. arXiv preprint arXiv:2209.06120 , 2022.

Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, and Zehua Li. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models, 2023.

Suchin Gururangan, Ana Marasovi´ c, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. Don't stop pretraining: adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964 , 2020.

Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. arXiv preprint arXiv:1810.10147 , 2018.

Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654 , 2021.

Xingwei He, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. Annollm: Making large language models to be better crowdsourced annotators, 2023.

Peter Henderson, Mark S Krass, Lucia Zheng, Neel Guha, Christopher D Manning, Dan Jurafsky, and Daniel E Ho. Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset. arXiv preprint arXiv:2207.00220 , 2022.

Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. CUAD: an expert-annotated NLP dataset for legal contract review. CoRR , abs/2103.06268, 2021.

Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes, 2023.

Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. arXiv preprint arXiv:2210.11610 , 2022.

Qiao Jin, Bhuwan Dhingra, William W Cohen, and Xinghua Lu. Probing biomedical embeddings from language models. arXiv preprint arXiv:1904.02181 , 2019.

Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. arXiv preprint arXiv:2205.11822 , 2022.

Martin Krallinger, Obdulia Rabal, Saber Ahmad Akhondi, Martín Pérez Pérez, Jesus Santamaría, Gael Pérez Rodríguez, Georgios Tsatsaronis, Ander Intxaurrondo, J. A. Lopez, Umesh K. Nandal, Erin M. van Buel, Ambika Chandrasekhar, Marleen Rodenburg, Astrid Lægreid, Marius A. Doornenbal, Julen Oyarzábal, Anália Lourenço, and Alfonso Valencia. Overview of the biocreative vi chemical-protein interaction track. 2017.

Taja Kuzman, Igor Mozetic, and Nikola Ljubešic. Chatgpt: Beginning of an end of manual linguistic data annotation? use case of automatic genre identification. ArXiv, abs/2303.03953 , 2023.

Stanford Legal Design Lab. Legal issues taxonomy, 2023.

Suffolk Law School Legal Innovation & Technology Lab. Spot's training data, 2022.

Hunter Lang, Aravindan Vijayaraghavan, and David Sontag. Training subset selection for weak supervision, 2022.

Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics , 36(4):1234-1240, 2020.

Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning, 2021.

Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation, 2021.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 , 2022.

Opher Lieber, Or Sharir, Barak Lenz, and Yoav Shoham. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs , 1, 2021.

Lucy H Lin and Noah A Smith. Situating sentence embedders with nearest neighbor overlap. arXiv preprint arXiv:1909.10724 , 2019.

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys , 55(9):1-35, 2023.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems , 2022.

Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Massediting memory in a transformer. arXiv preprint arXiv:2210.07229 , 2022.

Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. Reframing instructional prompts to gptk's language. arXiv preprint arXiv:2109.07830 , 2021.

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory-based model editing at scale. In International Conference on Machine Learning , pages 15817-15831. PMLR, 2022.

Akihiro Nakamura and Tatsuya Harada. Revisiting fine-tuning for few-shot learning. arXiv preprint arXiv:1910.00216 , 2019.

OpenAI. Gpt-4 technical report, 2023.

Laurel Orr. Manifest. https://github.com/HazyResearch/manifest , 2022.

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 , 2022.

Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. Advances in neural information processing systems , 34:11054-11070, 2021.

Rattana Pukdee, Dylan Sam, Maria-Florina Balcan, and Pradeep Ravikumar. Label propagation with weak supervision. arXiv preprint arXiv:2210.03594 , 2022.

A. J. Ratner, B. Hancock, J. Dunnmon, F. Sala, S. Pandey, and C. Ré. Training complex models with multi-task weak supervision. In Proceedings of the AAAI Conference on Artificial Intelligence , Honolulu, Hawaii, 2019.

Alexander Ratner, Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. Snorkel: Rapid training data creation with weak supervision. In Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases , volume 11, page 269. NIH Public Access, 2017.

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks. arXiv preprint arXiv:1908.10084 , 2019.

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili´ c, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 , 2022.

Changho Shin, Winfred Li, Harit Vishwakarma, Nicholas Carl Roberts, and Frederic Sala. Universalizing weak supervision. In International Conference on Learning Representations (ICLR) , 2022.

Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. Selective annotation makes language models better few-shot learners, 2022.

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpaca: A strong, replicable instruction-following model. https://crfm.stanford.edu/2023/03/13/alpaca.html, 2023.

Together. Releasing v1 of gpt-jt powered by open-source ai. https://www.together.xyz/ blog/releasing-v1-of-gpt-jt-powered-by-open-source-ai . Accessed: 2023-0125.

Paroma Varma, Frederic Sala, Ann He, Alexander Ratner, and Christopher Re. Learning dependency structures for weak supervision models. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 6418-6427. PMLR, 09-15 Jun 2019.

Harit Vishwakarma and Frederic Sala. Lifting weak supervision to structured prediction. Advances in Neural Information Processing Systems , 35:37563-37574, 2022.

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 , 2022.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 , 2022.

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages 38-45, 2020.

Renzhi Wu, Shen-En Chen, Jieyu Zhang, and Xu Chu. Learning hyper label model for programmatic weak supervision. arXiv preprint arXiv:2207.13545 , 2022.

Jieyu Zhang, Yue Yu, Yinghao Li, Yujing Wang, Yaming Yang, Mao Yang, and Alexander Ratner. Wrench: A comprehensive benchmark for weak supervision. arXiv preprint arXiv:2109.11377 , 2021.

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 , 2022.

Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems , 28, 2015.

Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models, 2021.

Lucia Zheng, Neel Guha, Brandon R Anderson, Peter Henderson, and Daniel E Ho. When does pretraining help? assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law , pages 159-168, 2021.

A Notation

The glossary is given in Table 4 below.

B Weak supervision background

In this section, we provide details on the inference and learning procedures for solving the graphical model defined in equation (1). The content from this section is derived from [14] and [7].

Pseudolabel inference. To perform inference, we compute ˆ Pr( y | λ ( x )) for some x ∈ X . This is done via Bayes' rule and the conditional independence of weak sources:

We assume that the class balance is known; for our datasets, the class balance is Pr( y = 1) = 0 . 5 . More generally, it can be estimated [53]. The latent parameter of interest in this decomposition is Pr( λ i = 1 | y = 1) , which corresponds to the accuracy of λ i .

Algorithm 2 Triplet method [14]

Input: Dataset D , weak sources λ ( x ) .

for i ∈ [ m ] do for j, k ∈ [ m ] \ i do

Estimate ˆ E [ λ i λ j ] over D , and similarly estimate ˆ E [ λ i λ k ] and ˆ E [ λ j λ k ] .

end for

Calculate average ˆ a i = Mean ( ˆ a i j,k ∀ j, k ∈ [ m ] \ i ) .

Compute estimated accuracy ˆ Pr( λ i = y ) = 1+ˆ a i 2 .

Output: Accuracies ˆ Pr( λ i = y ) for all i ∈ [ m ] .

end for

Source parameter estimation: Triplet method. Previous approaches have considered how to estimate Pr( λ i = 1 | y = 1) via the triplet method [14], which exploits conditional independence properties. First, by the properties of the graphical model in (1), it holds that the accuracy of λ i is symmetric: Pr( λ i = 1 | y = 1) = Pr( λ i = -1 | y = -1) = Pr( λ i = y ) (Lemma 4 of [7]). Therefore, Pr( λ i = 1 | y = 1) can be written in terms of E [ λ i y ] with E [ λ i y ] = 2Pr( λ i = 1 | y = 1) -1 .

Define a i = E [ λ i y ] . The graphical model in (1) tells us that λ i y ⊥ ⊥ λ j y if λ i ⊥ ⊥ λ j | y , which holds for all i, j ∈ [ m ] (Proposition 1 of [14]). As a result, E [ λ i y ] × E [ λ j y ] = E [ λ i λ j y 2 ] = E [ λ i λ j ] , which is a quantity that can be computed from observed LLM predictions. That is, we have that a i a j = E [ λ i λ j ] . If we introduce a third λ k , we can generate a system of equations over a i , a j , a k in terms of their pairwise rates of agreements:

and likewise for a j , a k . If we assume that each weak source is better than random over the dataset, then a i = | a i | > 0 , so we can uniquely recover the accuracy of each source by selecting two other sources and computing the above expression by using empirical expectations over D . We then set ˆ Pr( λ i = 1 | y = 1) = 1+ˆ a i 2 and plug this into the expression for Pr( y = 1 | λ ( x )) in (5).

This approach is formally described in Algorithm 2.

Solving, we get that

C Proofs

C.1 Proof of proposition 5.1

We note that [ λ , λ sm ] can be viewed as a set of sources in the weak supervision set up used in [7, 14]. Therefore, we can apply Theorem 1 from [7] to our problem setting, noting that we do not perform their clustering step and that our predictions do not abstain and output 0 in addition to {-1 , 1 } . We have a total of ( N +1) m sources, so

C.2 Proof of theorem 5.2

We can write the change in point-wise irreducible error as follows:

Next, we use the fact that λ ( x 0 ) ⊥ ⊥ λ sm ( x 0 ) | y to simplify the expression into

The exact λ sm ( x 0 ) is unknown but is drawn from the distribution Pr( λ sm | y = 1) since x 0 's label is 1 . Then, this expression becomes an expectation over λ sm :

Given that our λ sm is high-quality, we suppose that λ sm ( x 0 ) all equal 1 with high probability, and then we can lower bound our expression by

The key quantity of interest is Pr( λ sm = 1 | y = 1) = ∏ m i =1 Pr( λ sm , [ i ] = 1 | y = 1) . We focus on bounding Pr( λ sm , [ i ] = 1 | y = 1) next. Suppose that the k neighbors of x 0 are x 1 , . . . , x k . Define p j = Pr( λ i ( x j ) = 1 | y = 1) for all j ∈ [ k ] . Note that λ i ( x j ) ⊥ ⊥ λ i ( x j ′ ) | y for any j, j ′ ∈ [ k ] (while λ sm , [ i ] as a whole is dependent on y , individual neighbors are still conditionally independent). Then, the event that λ sm , [ i ] = 1 | y = 1 is as least as likely as the event that Binomial ( k, min i ∈ [ k ] p i ) ≥ k 2 . Let p min = min i ∈ [ k ] p i , and assume that p min ≥ 1 2 . Then,

where X j ∼ Bernoulli ( p min ) . Next, let δ = p min -1 2 . We can apply Hoeffding's inequality to get

All that's left is to lower bound p min . Without loss of generality, suppose that p min corresponds to an arbitrary p j = Pr( λ i ( x j ) = 1 | y = 1) . We can decompose this probability into

Since Pr( λ i ( x j ) = 1 | y ( x j ) = 1) is over all x j ∼ P x , this quantity is just equal to the accuracy of λ i , a i . Next, recall that ∥ E ( x j ) -E ( x ) ∥ ≤ ε k , where ε k = max x i ∈ NN ( x ) ∥ E ( x ) -E ( x i ) ∥ is the maximum distance from the k neighbors to x . Then, we can write Pr( y ( x j ) = 1 | y = 1) as Pr( y ( x j ) = 1 | y = 1 , ∥ E ( x j ) -E ( x ) ∥ ≤ ε k ) ≥ M E ( ε k ) , since we have assumed that E is M -smooth. We can now bound p min :

Therefore, we have that

Before we plug in Pr( λ sm = 1 | y = 1) into (17), we simplify the expression. Note that Pr( λ sm = 1 | y = 1) can be written as ∏ m i =1 p i for some p i , and Pr( λ sm = 1 | y = -1) can be written as ∏ m i =1 (1 -p i ) . A simple proof by induction shows that ∏ m i =1 (1 -p i ) ≤ 1 -∏ m i =1 p i . Therefore, we can write that (17) is lower bounded by

Let's abbreviate Pr( λ sm = 1 | y = 1) as x and define the function

We note that for x ≥ 0 . 5 , f ( x ) is convex and can thus be lower bounded by f ( x ) ≥ f ′ (0 . 5)( x -0 . 5) . We compute f ′ ( x ) = 1 -p λ xp λ +(1 -x )(1 -p λ ) , so f ′ (0 . 5) = 2(1 -p λ ) . Therefore, f ( x ) ≥ 2(1 -p λ )( x -0 . 5) . Our final bound on the pointwise difference in irreducible error on x 0 is

Information gain from using λ , λ sm over λ sm We briefly comment on the opposite directionhow much does using both LLM predictions and neighborhood predictions help over just using neighborhood predictions?

The quantity we aim to lower bound is H ( y | λ sm ( x 0 )) -H ( y | λ ( x 0 ) , λ sm ( x 0 )) for a point of interest x 0 . We can write this quantity as

Without loss of generality, suppose that the true label on x 0 is y = 1 , and that for each λ i , the neighborhood around x 0 consists of a balanced mix of λ i = 1 and λ i = -1 . Then, with high probability we have that Pr( y = 1 | λ sm ( x 0 )) = p λ sm ≈ 0 . 5 . From our proof of Theorem 5.2, we can thus write

If λ on x 0 has high accuracy and p λ sm is low, then we can have significant point-wise information gain from modeling both λ and λ sm rather than just λ sm .

D Datasets

Motivation . We study the performance of our method across a diverse collection of task definitions . In our setting, a task definition denotes a specific classification that a data scientist wishes to perform. For instance, a data scientist working on quantifying the breadth of legal issues that individuals face may wish to identify which posts in an online forum refer implicate legal issues related to housing.

This evaluation strategy is motivated by the observation that task definitions vary in their smoothness across embedding spaces, as different embeddings may do a better job of capturing features relevant for the task. For instance, out-of-the-box Sentence-BERT embeddings are better than traditional BERT at capturing the topicality of a sentence [55]. By focusing on a broad range of task definitions, we can better forecast how our method might perform for new tasks that practitioners may need to create classifiers for. We also avoid issues with leakage that may arise as the practice of finetuning LLMs on tasks increases [9].

In total, we study 95 distinct task definitions, encompassing 100,418 total samples. Each task varies between 108 and 3308 samples.

Legal tasks . The emergence of LLMs is exciting for law and finance, where expert-annotations are especially difficult to acquire [4, 18, 19]. Drawing on recent benchmarks and released datasets framing the potential use cases for LLMs in law, we study the following datasets:

CUAD [25]: The original CUAD dataset consists of 500 contracts spanning an array of sectors, with clauses manually into one of 41 legal categories. Following [18], we adapt the original dataset for clause-by-clause classification. We turn each clause type into a binary classification task, where the objective is to distinguish clauses of that type from clauses of other types (i.e. 'negatives'). Negative clauses are sampled randomly so as to make the task class balanced. We ignore clauses for which there are insufficient annotations in the original dataset.

Learned Hands [33]: The Learned Hands dataset consists of legal questions that individuals publicly posted to an online forum (r/legaladvice). The questions have been coded by experts into legal categories according to the Legal Issues Taxonomy [32]. We consider several such issue classes, and create a binary classification task for each issue. Negative clauses are sampled randomly so as to make the task class balanced. Because these questions can be long, we truncate them at 50 tokens.

Science tasks . LLMs have generated similar excitement for medical and science informatics applications [1, 4]. We study established classification/extraction benchmarks.

Chemprot [30]: ChemProt consists of sentences from PubMed abstracts describing chemicalprotein relationships. We study seven relations, and create a binary task for each one. Each task is class balanced, with negatives sampled from the other relations.

RCT [11]: The RCT dataset consists of PubMed abstracts for papers discussing randomized control trials, where sentences in the abstract are annotated according to their semantic role (e.g., background, methods, results, etc). There are five roles, and we create a binary task for each one. Each task is class balanced, with negatives sampled from the other relations.

General domain tasks . Finally, we study the following 'general domain' tasks, derived from popular sentence classification and information extraction benchmarks.

FewRel [21]: This is a relationship classification/extraction dataset, where each sample corresponds to a sentence mentioning the relationship between two entities. We select 20 relations, and for each relation construct a binary classification task with 700 positive instances of the relation, and 700 randomly sampled sentences (corresponding to other relations).

Spam Detection [67]: We study the YouTube spam detection task from the WRENCH benchmark. This task requires classifying YouTube comments as spam/not spam.

Toxic content detection [5]: This task requires classifying whether posted comments are toxic or not. We use a sampled subset of the CivilComments dataset.

AG News [69]: The original dataset organizes news snippets into four categories: World, Sports, Business, and Science/Technology. We create a separate task for each category. Negatives are sampled from the remaining classes.

DBPedia [69]: DBPedia is a 14-way ontology classification dataset. We convert this into 14 distinct tasks, corresponding to each of the ontology types.

E Prompts

We construct a prompt for each task by randomly selecting three examples of each class to use as in-context demonstrations [51]. We manually defined 'instructions' for each task. An example of an abridged prompt is shown in Figure 3.

InstructionClassification

Prompt

Is the entity discussed in the sentence an example of an animal?

Sentence:" Hugh Christopher Chris Brown is a Canadian singer-

songwriter and multi-instrumentalist."

Label: No

Sentence:"Forficula auricularia the common earwig or European earwig

is an omnivorous insect in the family Forficulidae. The European earwig

survives in avarietyofenvironments and is acommon household insect

in North America.The name earwig comes from a false superstition that

e      a   e

are harmless to humans."

Label:Yes

Sentence:" Northern Dancer(May 27 1961-November 16 1990） was a

Canadian-bredThoroughbredracehorsewhowontheKentuckyDerbyand

PreaknessStakesand whobecamethe mostsuccessfulsireofthe2oth

Century. The National Thoroughbred Racing Association calls him one of

the most influential sires in Thoroughbred history.A bay colt Northern

Dancer was by Nearctic and his dam Natalma was by Native Dancer.In

Label:

F Synthetics

We conduct synthetic experiments which provide additional insights on EMBROID.

For our setup, we create two equal clusters of data of 500 points each, C 1 and C 2 , in R 2 . We assign labels to the points in each cluster i.i.d. according to a probability p , where Pr( y = 1 | x ∈ C 1 ) = p and Pr( y = 1 | x ∈ C 2 ) = 1 -p . When p = 0 . 5 , both the clusters have a uniform label distribution (nonsmooth) while p = 1 ensures each cluster has one class (smooth). We set k = 20 , τ + = P ( λ i = 1) and τ -= P ( λ i = -1) .

Improvement over weak supervision . We show that EMBROID offers improvement over methods that only use λ . We fix p = 0 . 8 and β i = Pr( λ i = y ) = 0 . 6 for each i ∈ [ m ] . We compare EMBROID against the standard weak supervision approach from [14], which requires m ≥ 3 , in Figure 4a.

Smoothness . EMBROID's performance depends on the smoothness of the embedding as defined in eq. (4). We consider one LM prediction m = 1 and vary the smoothness p from 0 . 5 to 1 . 0 and generate predictions using β i = 0 . 6 . Figure 4b exhibits that EMBROID's accuracy is positively correlated with the embedding smoothness.

Base prediction accuracy . Finally, we show that EMBROID's performance depends on the base prediction accuracy, β i . We consider m = 1 and set p = 0 , β i = 0 . 8 . We use a parameter ρ to denote the probability that λ i is incorrect on points in cluster C 1 . As ρ varies from 0 to 1 , the predictions of λ i become biased towards 1 and effectively reduces β i to 0 . 5 . In Figure 4c, we observe that as the base prediction accuracy decreases, EMBROID's performance decreases and eventually goes below the base LM performance.

Varying LLM skew

2

4

6

8

Number of Sources

0.62

0.64

0.66

0.68

0.70

0.72

0.74

Varying the number of sources

Embroid

Weak Supervision

(a)

0.6

0.7

0.8

0.9

1.0

Embedding smoothness

0.55

0.60

0.65

0.70

0.75

0.80

0.85

Varying embedding smoothness

Embroid

(b)

0.0

0.2

0.4

0.6

0.8

1.0

Prediction skew

0.5

0.6

0.7

0.8

0.9

1.0

Embroid

LLM

(c)

G Ablation

We perform additional ablations of EMBROID. We focus on two aspects:

The role of τ -/ τ + .

The impact of weak supervision in combining λ sm ,j .

G.1 Ablation over τ

In our experiments, we set τ + i = τ -i = E [ λ i ] , or the average vote of source λ i . This has the following effect on the neighborhood vote λ sm ,j [ i ] for source λ i under E j :

When the average vote for a source λ i in a neighborhood under E j for x is more negative than the average overall vote for a source, then λ sm ,j [ i ]( x ) = -1 .

-1.0

-0.5

0.0

0.5

1.0

τ +

0.4

0.5

0.6

0.7

F1

When the average vote for a source λ i in a neighborhood under E j for x is more positive than the average overall vote for a source, then λ sm ,j [ i ]( x ) = 1 .

In general, we find that this setting provides good performance, while requiring no additional tuning or validation. For example, the Figure 5 below compares a setting of τ + i / τ -i against the F1 score for GPT-JT on ag_news_business.

G.2 Role of weak supervision aggregation

We quantify the extent to which performance gains are derived from (1) the computation of λ sm , as opposed to (2) the use weak-supervision [14] to combine λ sm and λ . Specifically, we replace Equation 3 with a simple majority vote classifier which combines the original prediction with the computed neighborhood votes.

G.3 Choice of embeddings

We provide additional discussion on how practitioners may go about choosing embeddings for EMBROID. At the outset, we note that selecting/evaluating embeddings is particularly difficult in our regime, where practitioners do not have access to labeled data. Importantly, this motivates Embroid's 'mixture of embeddings' approach. Previous methods (e.g., Liger [7]) operate on only a single embedding space. Thus, practitioners must be certain that the embedding space chosen is good for a task, and a poorly chosen embedding space will reduce performance. Because EMBROID ensembles different embedding spaces, practitioners can be less certain in the quality of any individual embedding space.

A natural strategy may be to start with an initial large set of candidate embeddings, and drop embeddings according to task-specific measurements (akin to feature selection). Several criteria may be used to determine which embeddings to maintain:

Looking at the MLM loss of the embedding model (e.g. BERT) on the task data. Prior literature on domain specificity has suggested that this may be a promising heuristic [71].

Looking to the performance of embedding models on other related tasks may also be helpful [38, 71].

Looking at the extent to which a potential embedding space generates embeddings which are geometrically similar to embeddings already included in Embroid. If the embedding space is similar, then the additional value of incorporation is low.

The number of embeddings used presents a bias-variance tradeoff. Under Proposition 5.1, increasing the number of embedding spaces (1) increases the variance due to estimation error, but lowers (2) the conditional entropy of H ( y | λ ) . Precisely characterizing this tradeoff is challenging because the variance is an upper bound, and H ( y | λ ) can only be estimated. However, these bounds/estimates can be used to derive heuristic stopping rules: for instance, add embedding spaces until the marginal decrease in the conditional entropy is less than the upper bound on the marginal increase in variance.

H Experiments

H.1 Implementation details

Compute Inference for API-access models (e.g., GPT-3.5 and J1-Jumbo) were run using the HELM API [38]. Inference for open source models (OPT, GPT-JT, and Bloom) were run using the Manifest library [49] on 40GB A100 NVIDIA GPU machines.

Hyperparameters EMBROID was run with k = 10 , τ + i = P ( λ i = 1) , and τ -i = P ( λ i = -1) .

H.2 Space demand

We provide additional information on the space requirements for EMBROID. First, we clarify that EMBROID requires only one large model. This model can be on the order of GPT-3 (176B parameters), or a much smaller open-source equivalent like GPT-JT (7B parameters). Next, EMBROID requires embeddings from 2-3 additional models. However, as our results show, these models can be relatively small. For instance, we rely on BERT-base and SentenceBERT models-both of which are approximately 110 million parameters. Finally, the latent variable graphical model has 2 parameters for class prior and 1+N parameters per LLM prediction, where N is the number of embedding spaces. So, when EMBROID is run on a single LLM's predictions with two embedding spaces, the total number of parameters is 2 + (1+2)*1 = 5.

H.3 Runtime

In terms of time complexity, we observe that computing LLM predictions is the largest bottleneck. On a dataset of 1800 samples for instance:

Computing predictions on GPT-3.5 takes 1440 seconds (24 minutes).

Computing embeddings using BERT takes 5 seconds.

Computing nearest-neighbors using the (unoptimized) scikit-learn library takes 3 seconds.

Solving the Embroid graphical model takes less than a second.

API-model tasks Due to cost constraints, we study the API access models (GPT-3.5 and J1-Jumbo) on a subset of tasks. These are:

ag_news_world

ag_news_sports

dbpedia_educational institution

dbpedia_athletechemprot_regulator

chemprot_upregulator

rct_objective

rct_methods

CUAD_Audit Rights

CUAD_Non-Compete

learned_hands_HE

learned_hands_HO

few_rel_architect

few_rel_league

H.4 Robustness across models

We provide the results for each LM on each task as CSV files in the supplemental attachment. We visualize the improvements in Table 1 below, by plotting the original prompt performance on the x-axis, and the performance after EMBROID on the y-axis (Figure 6).

0.4

0.6

0.8

1.0

j1-jumbo

opt-6.7b

0.4

0.6

0.8

1.0

GPT-JT-6B-v1

openai_text-davinci-003

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

bloom-7b1

0.4

0.6

0.8

1.0

gpt-neo-1.3B

Prompt F1

Prompt+Embroid F1

H.5 Comparison to AMA

We visualize the improvements in Table 2 below, by plotting the performance of AMA on the x-axis, and the performance of EMBROID-3 on the y-axis (Figure 7).

H.6 Chain-of-thought experiments

We compare EMBROID to chain-of-thought (CoT) [64] on the following tasks:

ag_news_business

ag_news_sports

CUAD_Affiliate License-Licensee

CUAD_Audit Rights

dbpedia_album

dbpedia_building

few_rel_architect

few_rel_country

learned_hands_HO

learned_hands_MO

0.4

0.6

0.8

1.0

j1-jumbo

opt-6.7b

0.4

0.6

0.8

1.0

GPT-JT-6B-v1

openai_text-davinci-003

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

bloom-7b1

0.4

0.6

0.8

1.0

gpt-neo-1.3B

Prompt F1

Prompt+Embroid F1

We manually write logical chains for each prompt. Because CoT primarily succeeds on 'large' models [64], we focus our experiments on GPT-3.5.

H.7 Comparison to GPT-4

GPT-4 [48] was made publicly available after this work was completed. Nonetheless, we offer preliminary results evaluating EMBROID on predictions produced by GPT-4. We evaluated EMBROIDEmbroid+GPT-4 on a subset of 14 tasks. These are:

ag_news_world

ag_news_sports

dbpedia_educational institution

dbpedia_athletechemprot_regulator

chemprot_upregulator

rct_objective

rct_methods

CUAD_Audit Rights

CUAD_Non-Compete

learned_hands_HE

learned_hands_HO

few_rel_architect

few_rel_league

We note that overall, GPT-4 appears to saturate our evaluated tasks, scoring greater than 95% on 5 of them. Given that we evaluate on publicly accessible benchmarks, this could be explained by leakage into GPT-4's pre-training data. We find that EMBROID is strictly better than the base prompt on 42% of tasks, and effectively equivalent or better (i.e., no worse than 1 point F1) on 73% of tasks. On two tasks Embroid improves by 4+ points.

Though EMBROID 's absolute gains over base prompts may diminish for stronger base LMs like GPT4, our evaluation and findings focused on open-source models because of their practical feasibility. Such models are cheaper and better suited for sensitive domains, as they allow data to remain onpremises. This is a particularly salient concern for applications in medicine, law, or government. On these models, we found that EMBROID offered substantial gains, with a win-rate > 88% and an average improvement > 7 points F1.

H.8 Comparison to self-consistency

We evaluate EMBROID against self-consistency [63] over GPT-3.5, and find that (1) it can provide competitive performance to self-consistency, and (2) that it can improve predictions generated from self-consistency.

We study a subset of five tasks: ag_news_business , CUAD_Audit Rights , dbpedia_album , few_rel_architect , learned_hands_HO . For self-consistency, we generate 5 predictions per sample, using a temperature of 0.7. For each task, we consider a 'base' prompt which does not use chain-of-thought, but otherwise uses the same task instructions and in-context demonstrations. For 4/5 tasks, EMBROID applied to the output of the base prompt outperforms self-consistency. On all tasks, EMBROID applied to the predictions generated from self-consistency improve performance, by a median of six points.

We additionally note one important distinction between self-consistency and EMBROID. While self-consistency requires multiple predictions from a LM for a single sample (thus accruing additional cost in hardware usage or API calls), Embroid requires only one prediction per-sample.

H.9 Multi-class experiments

We present initial results evaluating EMBROID in the multi-class setting. We create a multiclass version of the contract classification task by combining five different binary classification datasets: CUAD_Audit right , CUAD_Governing law , CUAD_Non-compete , CUAD_Volume restriction , and CUAD_Expiration date . The total size of this dataset is 1858, with the largest class (602 samples) occurring nearly 3.5x as much as the smallest class (165 samples).

We evaluate a one-vs-all version of EMBROID, where EMBROID is applied five times-each time predicting one of the classes. A final class prediction is generated by comparing each of the classspecific EMBROID classifier's predicted probability for the class, and choosing the class with the highest probability. Overall, we find that this variant of EMBROID improves the quality of the base prompt (by 4 points on balanced-accuracy and 1.6 points on macro-F1).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors in this paper propose EMBROID aim to improve language models (LM) (such as GPT-3.5) without additional labeled data. Unlike prompt-based models that focus on prompt designs, this paper attempts to modify the predictions of the data point $x$ by considering its neighboring samples. First, the proposed model applies a set of embedding encoders (such as BERT, and SentenceBERT) to construct the neighboring graph by the output embeddings, and then a latent variable model is used to generate the final prediction. This paper also provides the theoretical analysis about the generalization error and information gain. Finally, experiments on sentence classification datasets and ablation studies on output embeddings and dataset size show the improvements of the proposed model.\n\nSTRENGTHS:\n(1) The motivation that improves the LM predictions with unlabeled data and no expert supervision is interesting for recent NLP community. Based on ensemble learning, this paper uses a set of textual encoders to define the neighborhood prediction vector for each data point and combines the original prediction with its neighborhood prediction to correct the final prediction. The idea is simple and reasonable.\n\n(2) This paper also provides theoretical analysis by deriving a bound of generalization error and calculating the pointwise information gain, which suggest the efficiency of the proposed modules mathematically under several assumptions.\n\n(3) Extensive empirical results with several baselines on sentence classification task show the consistent improvements.\n\nWEAKNESSES:\n(1) The novelty of this paper appears to be satisfactory. Building upon previous works on weak supervision [1][2], the authors employ a latent variable model in the context of prompt-based language models.The combination of these two approaches is intuitive, and there are only a few significant technical challenges to address in solving the combined equation (Eq. 3).\n\n(1) This paper limits the proposed model to the classification task (from the method description and empirical results). Given that the generation ability (such as question answer) of a language model is also important, the proposed model seems to only work on the classification case.\n\n(2) Genral textual benchmarks such as SuperGLUE can enhance the convincingness of the experiment.\n\n[1] Mayee F. Chen et.al. Shoring up the foundations: Fusing model embeddings and weak supervision.\n\n[2] Daniel Fu et.al.  Fast and three-rious: Speeding up weak supervision with triplet methods.\n\nQUESTIONS:\n(1) One of the main technical challenges is how to fuse the original prediction with the neighboring prediction, and this paper employs the recent latent variable graphical model to the end. Please explain the core difference from [1] and [2] theoretically.\n\n(2) The proposed model consists of several large models, such as three embedding functions, GPT-3.5, and latent graphical model. It will be better to provide the space and time complexity analysis.\n\n(3) Ablation studies on $\\alpha$ in Eq.(3) are needed to evaluate the proposed module.\n\n(4) Theorem 5.2 suggests a smooth embedding function E. I wonder how to select an optimal E?  Are BERT-like encoders smooth functions?\n\n[1] Mayee F. Chen et.al. Shoring up the foundations: Fusing model embeddings and weak supervision.\n\n[2] Daniel Fu et.al.  Fast and three-rious: Speeding up weak supervision with triplet methods.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper aims to improve prompt performance through a technique of prompt patching where multiple neighbourhood instance of a given datapoint (retrieved through the embedding space of BERT or RoBERTa like model) are used in prediction from the LLM along with the original instance. Finally a majority vote is incorporated. Empirically, the method performs well in the case of binary classification tasks and outperforms the Chain-of-thought prompting technique.

Extra note: I want to highlight that I have not published in the area of prompting for LLMs and my review is based on the recent readings in the area.

STRENGTHS:
- The main strength of the paper lies in the simplicity of the method and also that it works well in practice (empirically validated). 
- The theoretical analysis well supports the empirical analysis which is a strong point. 
- Combination of Chain-of-thought and prompt patching results in strong improvements.
- Strong ablations and baselines are considered!

WEAKNESSES:
Overall, I believe the paper is empirically strong, with good theoretical intuition. The writing in Sec. 6.2 can be improved to reflect the workings of the baseline. I believe the reader might get confused with the Embroid-1 / Embroid-3 notation when comparing with the baselines. It will be better to highlight exactly how Embroid-1 / Embroid-3 differ from the baselines in terms of the exact number of prompts used.

QUESTIONS:
See Weaknesses

LIMITATIONS:
While I don't see any significant limitations, the final version of the paper can be updated with results on newer instruction-tuned models.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

comments: SUMMARY:
This paper presents EMBROID, a prompt-patching method that corrects erroneous predictions for a prompt via agreements over KNN examples. For this, it employs N different embedding models to get smoothed neighborhood prediction vector. The smoothed information is integrated to combine the voting with quality parameters. They conduct a formal theoretical analysis for EMBROID to get its behaviors and corresponding results. Experimental results demonstrate the efficacy of the proposed method, improving baseline predictions. It boosts the performance when multiple prompts are used and can be complementary used with other prompting methods such as chain-of-thought. Moreover, they show the proposed method works well even with the dataset size is relatively small.

STRENGTHS:
1. This paper introduces a novel method to improve the prompt-based learning paradigm, introducing a broad agreement procedure over the predictions.
2. The intuition of the idea and corresponding analyses are good.
3. Strong empirical results

WEAKNESSES:
- The method is limitedly evaluated only in the binary classification tasks.
    - Multi-class classification or generative tasks are not tested at all.
- It requires multiple embedding models for each correction.
- The proposed method always assumes unlabeled datasets.
- The paper does not include discussion about the self-consistency [1] method, another voting method to improve prompt accuracy.


[1] ****Self-Consistency Improves Chain of Thought Reasoning in Language Models (Wang et al.)****

QUESTIONS:
Please position the proposed method compared to the Self-Consistency method.

LIMITATIONS:
It would be good to show that the proposed method can be easily adapted to multi-class classification or even generative tasks.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

comments: SUMMARY:
The authors propose Embroid, a promising method that exploits availability of diverse pre-trained LLMs to create something akin to (but better than) an ensemble approach for prompt-patching.

STRENGTHS:
- a promising and easy-to-use method for prompt-patching
- robust theoretical analysis
- extensive empirical evaluation

WEAKNESSES:
- Evaluation was performed on models which are non-SOTA and the improvement may possibly be explained by gaps in their training that are filled by the domain-specific pre-trained models. It would be helpful to evaluate Embroid with a stronger general model (e.g. GPT4) to see if performance gains decrease significantly. It could also be helpful to evaluate Embroid on models that have been pre-trained on the particular domains of interest, to see if lack of domain-knowledge for the general model is what enables improvement when leveraging domain-specific pre-trained embedding models. 
- Evaluation was performed on only binary sentence classification tasks. It seems like extending to multi-class classification should be doable (at a minimum this can be done by doing multiple binary one-vs-all comparisons, though a cleaner extension would of course be better) and would go a long way in validating Embroid on more realistic settings.

QUESTIONS:
See weaknesses section.

LIMITATIONS:
The authors address limitations like dependence on domain-specific embeddings, the quality of embeddings, and the availability of at least a moderate number of data points (in the hundreds).
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors focus on few-shot prompted classification using language models. In this setting, they focus on the challenge of developing optimal few-shot (in-context) prompts for language models in domains where data collection is prohibitively expensive. Rather than engineering the prompts themselves, the proposed Embroid method aims to *correct* prompted language model predictions using semantic similarity predictions of similar samples using other pre-trained language models. Embroid operates as a form of mixture-of-experts involving both the core language model as well as a family of other embedding models to edit LM annotations and enforce consistency. The authors perform a theoretical analysis of how/why Embroid works re: embedding smoothness, and demonstrate how Embroid improves accuracy across a range of language models and tasks.\n\nSTRENGTHS:\n- To my knowledge, this paper proposes a novel way of using noisy similarity heuristics as a \"vote of confidence\" to regularize language model predictions\n- The methods sections (3, 4) are clearly written and equation intuitions are well-explained. This helps greatly in understanding why the Embroid method may work and helps in understanding the theoretical analysis in Section 5.\n- The authors report a significant improvement overall (win rate) and average across tasks over a variety of models after applying Embroid across their extensive task suite.\n- The experiments section is structured well, and the research questions naturally flow from one to another. It is encouraging to see that Embroid seems to work well together with chain-of-thought prompting, prompt ensembling, and other methods of improving the prompt (selective annotation).\n\nWEAKNESSES:\n- There is a bit of confusion in the problem setting description about what constitutes the \"prompt\". (108/109) suggests that the prompt includes the description and in-context examples. Is the main challenge the authors target the task description engineering, choice of in-context examples, or both?\n- Regarding the smoothness observation in Section 5: is there a simple method to quantify smoothness of embedding models on a given dataset? And does this give rise to selection strategies or methods to drop out embeddings from a given task? Essentially I'm wondering whether there is a theoretical result for the optimal number or selection of embeddings used in Embroid.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-28"}