Zero-Label Prompt Selection
Authors: Chonghua Liao
Yanan Zheng
Zhilin Yang
Venue: Submitted to ICLR 2023 (2023)

Abstract: Natural language prompts have been shown to facilitate cross-task generalization for large language models. However, with no or limited labeled examples, the cross-task performance is highly sensitive to the choice of prompts, while selecting a high-performing prompt is challenging given the scarcity of labels. To address the issue, we propose a Zero-Label Prompt Selection (ZPS) method that selects prompts without any labeled data or gradient update. Specifically, given the candidate human-written prompts for a task, ZPS labels a set of unlabeled data with a prompt ensemble and uses the pseudo-labels for prompt selection. Experiments show that ZPS improves over prior methods by a sizeable margin in zero-label performance. We also extend ZPS to a few-shot setting and show its advantages over strong baselines such as prompt tuning and model tuning.

Under review as a conference paper at ICLR 2023
ZERO-LABEL PROMPT SELECTION
Anonymous authors
Paper under double-blind review
ABSTRACT
Natural language prompts have been shown facilitate cross-task generalization for large language models. However, with no limited labeled examples, the cross-task performance is highly sensitive to the choice of prompts, while select- ing high-performing prompt is challenging given the scarcity of labels. To ad- dress the issue, we propose a Zero-Label Prompt Selection (ZPS) method that selects prompts without any labeled data gradient update. Specifically, given the candidate -wiiten prompts for a task, ZPS labels a set of unlabeled data with prompt ensemble and uses the pseudo-labels for prompt selection. Ex- periments show that ZPS improves over prior methods by a sizeable margin in zero-label performance. We also extend ZPS few-shot setting and show its advantages over strong baselines such as prompt tuning and model tuning.
INTRODUCTION
Recently, extensive studies have shown that large language models (LLMs) have promising perfor- mance for few-shot learning (Brown et al., 2020; Zhao al., 2021; Schick & Schütze, 2021; Gao et al., 2021), and they even show strong generalization abilities to new tasks without any annotated data (Brown et al., 2020; Wei al., 2021; Sanh et al.. 2021). Different from conventional fine-tuning methods that require expensive parameter updates for each downstream task, prompts are employed to provide in-context information or task instructions, which helpful for guiding models to per- form each task. Manually-writte prompts are often used to specify the task and unify the format of inputs.
However, the performance of different prompts during evaluation can vary from near state-of-the-ar to random guess; e.g., using a non-optimal prompt can cause performance drop of up 60 points on the CB task (Zhao et al., 2021). Previous work mainly relies on using multiple prompts (Brown al., 2020; Wei al., 2021; Sanh al., 2021) or prompt ensemble (Zhou et al., 2022) to enhance the performance and robustness when generalizing to test tasks, while omitting the fact that using multiple prompts leads to a substantially increased computational cost, which hinders the practical deployment of LLMs. These challenges make prompt selection important problem.
There have been efforts on improving model performance via searching for a better prompt. For example, Jiang al. (2020) proposed two automatic methods to augment prompts. They further ex- plored combining the generated diverse prompts with ensemble methods. Shin et al. (2020) designed gradient-based based search method to find trigger words in prompt. Gao et al. (2021) developed way to use a span-corruption pretraining objective for prompt generation. Deng et al. (2022) presented Rpprompt, prompt search method with reinforcement learning which relies trained with carefully designed reward function. Prasad et al. (2022) designed an iterative prompt search algorithm that relies on human-defined edit rules improve the few-shot performance, Xu et al. (2022) proposed GPS, genetic prompt searching algorithm that leveraged generative lan- guage models for prompt augmentation. Nevertheless, the main drawback of such methods that they all require additional labeled set to serve as a prompt scoring set or to provide the rewards or gradient signals. It remains challenging when no labeled samples are available. Thus, a crucial question arises:
policy network
Is it possible to select a high-performing prompt without any labeled data or gradient update?
In this paper. we answer this question affirmatively. To tackle the aforementioned problem, we propose 7 Label Prompt Selection-a simple-yet-effecttie technique for selecting high-

---

Under review as a conference paper at ICLR 2023
Rosed on this reviow, woukd recommend
product?
Review: (fcontent))
(1)
100%
negurive positive
pre-trained Language Hodel
tone this prodact review?
75%
This sound taack was beautfful!
Review: ((conteut)
g008
(3)
50%
Review: [cconteut] profuctt review convey negative or positive sertiment?
Unlabeled Data
Prompts
Prompt Ensemble
The main pipeline of ZPS. (1) A number of prompted unlabeled data are fed into the pretrained language model to get logits and predictions. (2) The pseudo labels are obtained from the prompt ensemble. (3) The pseudo labels are used to calculate the pseudo accuracy of prompts. And the one with the highest pseudo accuracy is selected. Some details like prompt filtering are omitted for brevity and can be found in the text.
Figure 1:
performing prompt in zero-label setting. As illustrated in Figure 1, given set of candidate human- written prompts P and unlabeled dataset X' for task, the ensemble of prompts is used to tate the unlabeled data. Finally, the pseudo-labels generated from the prompt ensemble are used for prompt selection. We also extend the idea of ZPS few-shot setting and validate our advantages over strong baselines such as prompt tuning and model tuning. In the few-shot setting, we further explore the role of pseudo-labeled data: pseudo-labeled data can not only be used for prompt selec- tion, but checkpoint selection well. By using pseudo-labeled data for checkpoint selection, there no need to split a subset from limited labeled data as a validation set, which means more labeled data can be used for training to boost model performance.
Our contributions are summarized as follows.
We propose a novel Zero-Label Prompt prompts without an extra validation set any parameter update.
Selection (ZPS) algorithm to select high high-performing
We show that ZPS can be used in a plug-and-play manner to boost zero-label and few-shot per- formance. Extensive experiments show that ZPS leads to substantial performance boost for both the zero-label and few-shot settings.
2 RELATED WORK
Recently, there have been many advances in deep learning with pseudo-labeling. leeudo-labeling (Lee et al., 2013; Reed al., 2015: Shi al., 2018) employs model make pre- dictions for unlabeled samples (Yarowsky, 1995b; McClosky et al., 2006). Iscen et al. (2019) showed that pseudo-labels can also be created by label propagation instead of direct network predictions. Shi et al. (2018) incorporate the idea of confidence levels for unlabeled samples to discount influences from uncertain samples. Another line of work is self-training (III, 1965: Yarowsky, 1995a; Riloff, 1996), which trains model with both labeled and pscudo-labeled data for few iterations. Some modifications like strong data augmentation (Zoph et al., 2020), learning an equal-or-larger student model (Xie al., 2020), and using additional noise (Xie et al., 2020; He al., 2020) are shown to be beneficial for self self- training. Another popular technique ensemble distillation (Hinton et al., 2015), which means distilling knowledge in an ensemble into single model.
Pseudo-labeling.
2

---

Under review as a conference paper at ICLR 2023
Zero-label Learning. Different from pseudo-labeling where labeled data is usually available, zero-label learning transfers a pretrained model to unseen tasks with unlabeled data only. Instead of directly predicting labels for unlabeled data, Wang et al. (2021) proposed to use unlabeled data to generate synthetic data without tuning the generative models. Zhou et al. (2022) designed a prompt- consistency loss for finetuning on unlabeled data. Lang al. (2022) used co-training fine-tune both large language model and smaller task-specific model, where the different views of the data are obtained from different prompts. These methods study generating synthetic data or improved training techniques, while our ZPS focuses on an orthogonal aspect- how to select high-perfonee perforrefominn prompts.
Prompt Search. Past works attempt to improve prompt quality via prompt tuning (Liu et al., 2021b; Lester et al., 2021; Li & Liang, 2021; Vu et al., 2021; Gu et al., 2021; 1: Mokady et al., 2021; Qian et al., 2022; An et al., 2022). Soft-prompts continuous nature of soft-prompts makes them hard to be interpreted. Other prior work has looked into optimizing discrete manual prompts in various aspects, such selecting priming examples (Zhao et al., 2021; Liu et al., 2021a), ordering examples (Zhao et al., 2021; Lu et al., 2022: Kumar & Talukdar, 2021), and prompt search (Jiang et al., 2020; Gao al., 2021; Shin et al., 2020; Prasad et al., 2022; Xu et al., 2022). The aforementioned methods require updating parameters and a validation set. Our ZPS, on the contrary, is a tuning-free method while labeled data is required.
optimized via gradient descent. However, the
METHOD
In this section, provide detailed description of the presented approach, Zero-Label Prompt Selection (ZPS). ZPS mainly consists of three parts: prompt filtering, ensemble, and selection. After filtering out some low-performing prompts, the ensemble of the remaining prompts is used for prompt selection. We first start with our problem definition of zero-label learning.
3.1 ZERO-LABEL LEARNING
Suppose an LLM has been pretrained. Given a new task and a set of unlabeled data X associated with the task, we use the LLM to perform the new task without any label. Zero-label learning a reasonable setting since task-relevant unlabeled data are usually accessible at test time. This setting has also been adopted in previous work (Meng al., 2022) to test the ability of cross-task generalization.
3.2 PROMPT FILTERING
In this part, we introduce prompt filtering. We assume that we are given the pretrained model M, a set of prompts P. (Tr, Vi) E P, and a set of unlabeled data X' = {21...., Fr, with size n. Formally, a prompt P = (T, is a pair composed of two parts: a prompt template and the corresponding verbalizer. The prompt template is defined as function that transforms the unlabeled data into a sequence. For example, given a prompt template T of a sentiment classification task "Based on this review, would the user recommend this product? Review: { {content)) } Answer:", will fill a predicted text sample into the "( [[coontent)"" )" blank. Further, the verbalizer V is defined as function that maps each label to phrase. For instance, the verbalizer corresponding to T will map 1 to "Yes", and 0 to "No". As a result, the LLM performs the tasks by predicting the mapped phrases.
Now we want to filter low-performing prompts before we proceed to the next step. Given an unla- beled sample 2g, the LLM outputs the conditional probability for a given label yy:
P(y Ik, P.) = (M:(()))) [T(())))
where we use PM denote the LLM probability. We define the confidence score of prompt as
C3 = Σ (p(vt)) æx, P.) - p(12) | P.))
TEER'
where y(1) and 3(2) denote the choices with the highest and the second highest LM probability, separately. The confidence scores represent how confident the L.M is about the predictions with
3

---

Under review as a conference paper at ICLR 2023
specific prompt. Let (c1,-.. Cp) denote the confidence scores of all p prompts. We use k-means to cluster these scores into two sets and discard the set of prompts with lower scores.
3.3 PROMPT ENSEMBLE AND SELECTION
In this subsection, after clustering and filtering, will introduce how use the rest of the prompts to form prompt ensemble, and utilize the prompt ensemble for prompt selection. We then provide three possible ways to combine prompts a prompt ensemble. We note that directly serving prompt ensemble at inference time will increase the computational cost latency substantially, which not practical for many applications. In this work, we mainly focus on the setting where only one prompt can be served.
The key idea our approach is from one of most classic techniques-ensent 1990; Krogh & Vedelsby, 1994). We ensemble all available prompts to obtain pseudo label for each sample. Since we cannot directly calculate accuracy on a labeled validation set, we obtain a "pseudo accuracy" by caleulating the agreement between model predictions and pscudo labels, as illustrated in Figure I. Specifically, the prediction of an input Iy given the prompt P; is
(Hansen Salamon,
Vt.k argmax Ply Fk. P.).
yEY
We use the notation = (fii,... Din) ) to denote all the predictions of prompt P; and we use denote the prediction from the prompt ensemble. The "pseudo accuracy" of the prompt P; is computed as
acc; = (Nec
where use face denote the accuracy function that calculates the proportion of identical predic- tions between the two vectors. Then we select the prompt with the highest "pseudo accuracy" as the prompt use for the target task. Intuitively, the pseudo labels assumed have higher accuracy compared to the prediction from single prompt, which provides a stronger signal for selection.
There are many possible ways to form a prompt ensemble for obtaining ye- Here we discuss three possible ensemble strategies.
bog-robbability mean. Given an unlabeled example dhe we use the average of log probabilities to calculate the score of choice and obtain the ensemble prediction Uc,k:
s v)
Σ P.EP
log Ply Tk, P.),
Gr,k = (egmaxs(Fee)) vEy
Probability mean. It is also possible to get the score by averaging the LLM probabilities from different prompts. Specifically, we have
1 Σ P(y æk, P.).
s s(Zk, 8)
P P.EP
Majority vote. This method aggregates the prediction of each prompt given input æ, and out- puts the choice with the maximum occurrences. Formally,
s(zk,
Σ
1(Ara =)
P.EP
where 1(-) is the indicator function and Sr.s denotes the prediction of prompt P, given the input zg.
3.4 ZPS FOR FEW-SHOT LEARNING
In addition zero-label learning, we extend 7PS the few-shot learning setting. In this setting, are given labeled set with m samples, say Dm. in addition to the unlabeled data Y and the LLM M. Our goal is develop strategy to boost the performance of the model. Traditional model tuning often splits Dn, into to parts of the same size, Dm//2 and D' Dm//2-
training set and a validation set. m/2 is then used for checkpoint selection The model is first trained D Dm and the validation set D'
4

---

Under review as a conference paper at ICLR 2023
and prompt selection. However, since the number of labeled examples is limited, using validation set means less data is used for training. With ZPS, we propose to use pseudo-labeles data for both checkpoint and prompt selection. As result, all labeled examples are used for tuning the models, which leads to better generalization performance.
EXPERIMENTS
In this section, we conduct extensive quantitative experiments to evaluate our 7PS against baseline methods. We mainly consider two experimental settings, zero-label learning and few-shot learning, as described in Section 3. addition, we also investigate and analyze several influential factors and hyper-parameters in ZPS.
4.1 EXPERIMENTAL SETUP
4.1.1 DATASETS
We use the TO benchmark (Sanh et al., 2021) that consists of 39 training tasks of task types, and 11 test tasks of 4 task types. Both the training and the test sets are disjoint in task types. Specifi- cally, the test tasks include natural language inference (RTE (Dagan et al.. 2006), CB (De Marneffe al., 2019), ANLI/RI-R (Nie et al., 2020)), coreference resolution (WSC (Levesque al., 2012), Winogrande (Sakaguchi et al., 2020)), sentence completion (COPA (Roemmele et al., 201 1), Sto- ryCloze (Mostafazadeh et al., 2017). Hellaswag (Zellers et al., 2019)), and word sense disambigua- tion (WiC (Pilehvar & Camacho-Collados, 2019)). We also use the prompt candidates provided in TO, which are constructed using PromptSource (Bach et al., 2022). For the zero-label setting, we construct the unlabeled data by removing labels of the training- split data for each test task. For the few-shot setting, we randomly sample 32 labeled data from the training split.
4.1.2 BASELINES
For the zero-label setting, we compare our ZPS with the following baseline methods.
To TO (Sanh et al., 2021) employs prompted multi-task pretraining for cross-task generalization, which provides a framework for all methods considered in our experiments. TO does not provide way to select prompts without labels, and the average performance of all candidate prompts is used.
Self-Training Self-training first uses a trained TO to label the unlabeled data, and then uses the pseudo-labeled data to further finetune the TO model (Sanh et al., 2021). The whole process is repeated multiple times. The self-training method also reports average performance over multiple prompts.
Ensemble Distillation We use a prompt ensemble (Hinton et al., 2015) to pseudo label the data and distill the knowledge into multiple prompts using prompt tuning (Lester al., 2021). The tuned soft prompt embeddings are concatenated to different prompts and the mean performance of all prompts is reported.
For the few-shot setting, we mainly compare ZPS with the following two categories of baseline methods, respectively methods with parameter tuning, including model tuning and prompt tuning, and methods without parameter tuning, including in-context learning, GRIPS, and GPS.
In-Context Learning (ICL) ICL (Brown et al., 2020) is a few-shot method which requires a few input-output pairs priming demonstrations guide models to perform the test task. Given prompt P and a test sample, few prompted labeled samples are concatenated to the prompted test sample to form an input. The average performance of all prompts is reported.
GRIPS GRIPS (Prasad et al., 2022) is an edit-based prompt search approach. It iteratively mu- tates the prompts with human-definen edit rules. Then, the augmented prompts with the highest scores on the validation set are selected, It also reports the average performance on multiple aug- mented prompts.
GPS GPS (Xu et al., 2022) is also gradient- prompt search method that adapts the genetic algorithm and uses generative models to augment prompts. We report the average performance of the selected prompts.
5

---

Under review as a conference paper at ICLR 2023
Model Tuning (MT) Model tuning finetunes all parameters of language model for each task. We follow the feo-shot setting in Zheng et al. (2021) to use half of the data for training and the other half for model selection.
Prompt Tuning (PT) Prompt tuning (Liu et al., 2021b; Lester et al., 2021) tunes continuous embedding while the LLM is frozen. Training and validation splits are identical to model tuning.
4.1.3 TRAINING DETAILS
For fair comparison, keep the number of labeled samples 32 for all few-shot methods. For all methods that require gradient update (PT, MT, self- self-trrining), we use the Adafactor Optimizer and set the batch size as We set the learning rate 5e-5 for MT and self-training, and 0.05 for PT. For PT, we set the number of soft prompt tokens as 3 for zero-label ensemble distillation and I for few-shot learning. For ICI., we randomly select 2 examples from the training set of each task to compose the priming prompt. The above hyperparameters are chosen based on validation results. For GRIPS and GPS, we also follow the hyper-parameters reported in Prasad et al. (2022) and Xu et al. (2022).
4.2 MAIN RESULTS AND ANALYSIS
Zero-Label Performance. In this section, we compare ZPS with the baselines mentioned in the last subsection on the 11 test tasks. As shown in Table 1, our ZPS outperforms the TO baseline, which shows the effectiveness of prompt selection. Our zero-label ZPS even has a considerable advantage over some few-shot methods (GRIPS, ICL, PT) in Table 2, which shows that the selection of prompts crucial factor in model performance. Moreover, combining ZPS with ensemble distillation further boosts performance and outperforms using ensemble distillation alone, which indicates that ZPS is able to select nigh-performing prompts in a prompt tuning scenario.
Natural Language Inference
Sentence Completion
Co-reference WSC
WSD
Method
RTE
ANLII
ANL12
ANLI3
COPA
Hella.
Story.
Wino.
WiC
Avg.
Self-training
84.12
85.71
42,41
39.35
43.04
84.01
50.55
97.49
49.04
55.20
55.49
62.40
TO
80.97
70.12
43.16
38.68
41.87
90.02
33.55
92.84
61.06
59.70
56.13
60.74
EnsD
83.86
75.48
42.22
38.78
41.67
92.11
39.71
95.65
58.65
59.18
59.69
62.45
TO + ZPS (Ours)
86.28
80.36
45.10
40.50
42.92
92.00
34.33
95.83
62.50
59.75
59.40
63.54
EnsD ZPS (Ours)
85.20
82.14
44.00
39.60
44.08
93.00
40.99
95.72
60.58
59.83
61.60
64.25
Table Main results on zero-label performance of different methods. All methods are based on the TO-1IB model. "EnsD" denotes ensemble distillation.
Few-Shot Performance. Next, turn experiments where 32 labeled samples available for each task. Table shows that MT + ZPS outperforms all other strong few-shot baselines on the average performance of all test tasks. This reveals the effectiveness of ZPS. We notice that the performance of ICL with TO is significantly worse than other few-shot baselines, which is probably because the priming prompt used in ICL is quite different from task descriptions used in the multitask training stage of TO. The performance of GRIPS and PT are similar, slightly better than the TO zero- shot baseline, while GPS achieves the strongest performance among all few-shot methods that do not require parameter updating. In our MT + GPS, with the help of pseudo labels, more labeled data can be freed to perform training instead of model selection (traditional MT requires portion of the labeled set to select checkpoints (Zheng al., 2021)). The superior effectiveness of ZPS validates that pseudo-labeled data can not only select prompts, but select model checkpoints as well.
4.3 ABLATION STUDY
In this section, we perform several ablation experiments to explore the influential factors of our method. We keep the other factors fixed while studying the effect of specific factor.

---

Under review as a conference paper at ICLR 2023
Natural Language Inference
Sentence Completion COPA
Coreference WSC
WSD
Avg.
RTE
CB
ANLII
ANLI2
ANLI3
Story. Hella.
Wino.
WiC
GPS
83,86
80.00
46.47
39.91
43.01
93.43
43.96
95.03
65.48
61.96
58.82
64.72
GRIPS
81.59
76.07
44,41
39.32
42.10
91.41
33.11
94.16
61.54
58.61
57.23
61.78
ICI.
72.42
65.24
37.58
33.05
37.17
84.07
26.77
90.18
64.42
54.21
49.33
55.86
PT
81.48
70.24
42.67
38.67
40.55
92.59
39.12
95.46
60.87
59.79
58.90
61.85
MT
78.34
81.25
47.10
38.43
45.75
91,45
52.91
94.55
65.38
69.53
53.29
65.27
MT ZPS (Ours)
80.87
85.71
45.50
41.60
50.33
93.00
58.54
93.80
57.69
69.93
53.92
66.45
Table 2: Main results on few-shot performance of different methods. All methods are based on the TO- TB model. For a fair comparison, we keep the size of the labeled set for each task as 32. Model Tuning (MT) and Prompt Tuning (PT) tune the full model parameters and the continuous prompt embedding separately. In-Context Learning (ICL), GRIPS (Prasad et al., 2022) and GPS (Xu al., 2022) do not require any parameter update.
4.3.1 ENSEMBLE STRATEGIES
As shown Table 3, among these three ensemble strategies, log-probability mean attains the best performance. This also echoes with the finding in Jiang et al. (2020) that given any prompt, log probabilities can penalize the choice that is very unlikely.
Natural Language Inference
Sentence Completion
Coreference WSC Wino.
WSD WiC
Avg.
RTE
CB
ANLII ANLI2 ANLI3
COPA Hella.
Story.
Majority vote
86.28
78.57
44.30
39.30
42.92
92.00
33.03
95.35
62.50
59.75
59.40
63.04
Probahility mean
86.28
78.57
44.30
39.30
42.92
92.00
33.79
95.83
56.73
59.75
59.40
62.74
Ours (log prob mean)
86.28
80.36
45.10
40.50
42.92
92.00
34.33 95.83
62.50
60.54
58.78
63.54
Table 3: Performances of different ensemble strategics.
4.3.2 PROMPT FILTERING
Prompt filtering plays an important role in our method. We compare the performance of using and not using filtering in Table 4. It shows that prompt filtering boosts the performance of ZPS for almost all tasks. Since prompt filtering is followed by a prompt ensemble, filtering out possible low-performing prompts contributes to a higher accuracy of the ensemble and leads better per- formance. Moreover, even without prompt filtering, our ZPS still holds strong advantages over the other baselines in Table I, indicating the stability and robustness of our method.
Natural Language Inference
Sentence Completion Story.
Coreference WSC Wino.
WSD WiC
Avg.
RTE
CB ANLII ANLI2 ANLI3
COPA
Hella.
w/o filter
81.95 78.57
44.80
39.30
44.41
92.00
34.33
95.35
62.50
59.70
56. 13 62.96 58.78 63.54
w/ filter
86.28 80.36
45.10
40.50
42.92
92.00
34.33
95.83
62.50
60.54
Table 4: Results of ZPS with and without filtering.
4.3.3 ROBUSTNESS TO ADVERSARIAL PROMPTS
Since ZPS relies on prompt ensemble, it is possible that the quality of the candidate prompts will affect its performance. We want to test whether ZPS will still outperform the average performance of candidate prompts when there are more low-performing prompts in the candidates. To this end, we create adversarial low-performing prompts by adapting GPS (Xu et al., 2022). In the genetic algorithm, the generated prompts with the lowest performance on the validation set are selected as the wwrrerrorming prompts. A portion of the original prompts are replaced with the same number of low- wwwrffforring prompts. We then report the performance of ZPS- seleeted prompts and the average of candidates. The replacement process is repeated with 5 different random seeds. We also vary
7

---

Under review as a conference paper at ICLR 2023
the ratio of low-pen low-performing prompts from 0.1 to 0.8. As shown in Table 5, ZPS yields consistent improvements over not using prompt selection (i.e., the mean of candidate prompts). This reflects that ZPS is useful consistently under different levels of adversary and noise.
Ratio
0.1
0.2
0.5
0.8
ZPS
62.20
62.07
58.98
51.45
No prompt selection
59.95
58.45
54,90
50.93
Table 5: Robustness adversarial prompts. We replace the original candidate prompts with ad- versarial low-perfon oow-eerormming prompts generated by GPS (Xu et al., 2022). We vary the ratio of low- performing prompts from 0.1 to 0.8. ZPS yields consistent improvements over not using prompt selection.
4.3.4 SCALE
Another important factor is the model scale. We conduct experiments TO-3B and TO- Larg)(770M) to verify the effectiveness of ZPS on different models. Table shows that ZPS is able to boost the performance on models with different scales.
Natural Language Inference
Sentence Completion COPA Hella.
Coreference WSC
WSD
Avg.
RTE
CB
ANLII ANL12 ANLI3
Story.
Wino.
WiC
TO
60.61
48.81
35.11
33.28
33.52
73.44
27.76
84.91
65.00
50.91
51.27
51.32 52.93
3B
ZPS (ours)
58.48
60.71
36.60
34.40
33.33
76.00
28.49
87.39
64.42
51.78
50.63
TO
72.67
56.55
32.77
32.15
34.38
85.36
27.18
93.04
63.94
54.35
50.33
54.79
Large
ZPS (ours)
79.06
67.86
31.20
31.10
34.25
88.00
29.16
93.43
65.38
53.43
49.84
56.61
Table The performance of ZPS with different scales.
4.3.5 USAGE OF LABELED AND PSEUDO-LABELED DATA
In this part, we examine different ways of using labeled and pseudo-labeled data in the few-shot setting. Given 32 labeled examples D32 from the training set and all unlabeled examples X' for each task, we consider the following cases:
16 + 16: This is classic approach to use labeled data in model tuning. The labeled set is splited into two equal-sizes halves, Die and Dig, as a training set and a validation set. The validation set is then used perform checkpoint selection and prompt selection.
Natural Language Inference ANLI1
Sentence Completion COPA
Coreference WSC
WSD
Avg.
RTE
CB
ANLI2
ANLI3
Hella.
Story.
Wino.
WiC
Mean
78.19
72.38
49,16
38.87
45.94
90.45
46.36
94.55
63.65
70.04
53.18
88
16 16
Val select
78.34
81.25
47.10
38.43
45.75
91.45
52.91
94.55
65.38
69.53
53,29
65.27
Pseudo select
80,51
73.21
48.20
39.50
46.25
91.00
52.91
94.23
64.42
70.09
52.51
64,80
32 pseudo
Mean
81.91
79.88
38.89
36.20
42.13
93.65
51.94
94.12
56.54
57.47
55.11
62.53
Val select
82.43
79.76
38.80
36.50
42,30
96.00
51.51
94.07
61.54
57.73
55.33
63.27
train
Pscudo select
81.95
82.14
36.50
43.17
95.00
51.51
94.33
57.69
57.46
55.33
63.10
32 pscudo
Mean
83.72
80.60
48,97
37.98
46.01
92.55
48.06
95.79
65.48
70.54
55,38
66.00
Pscudo select
83.75
78.57
49.40
37,50
45.58
93.00
56.10
96.26
62.50
70.40
55.96
66.28
More
Mean
85.00
40.74
50.16
92.80
51.62
93.67
62.88
70.12
56.54
66.32
pseudo val
Pseudo select
80.87
85.71
45.50
41.60
50.33
93.00
58.54
93.80
57.69
69.93
53.92
66.45
Table 7: An ablation study on different strategies to use labeled and pseudo-labeled data. "Mean" denotes the mean of prompts, "val select" means selecting the prompt on the labeled validation set, and "pseudo select" means prompt selection seleciin on the pseudo-labeled labeled data. We come to the conclusion that it is best to use labeled data for training and use pseudo-labele data for checkpoint and prompt selection. "More pseudo val" with "pseudo select" what we adopt in ZPS.
8

---

Under review as a conference paper at ICLR 2023
32 pseudo train: We first annotate the unlabeled test data X by TO to get pseudo labels. Then, select the pseudo-labeled samples with the top-32 highest confidence. This set is denoted as Xz2- The entire labeled set 322 is only used as a validation set.
32 pseudo val: This
is similar to "32 pseudo train", despite that the entire labeled set D22 is used as a training set for model tuning. In this case, we use X32 for checkpoint selection.
More pseudo val (the approach we adopted in our ZPS): The only difference between this case and "32 pseudo val" is that we increase the size of the pseudo-labeles validation set.
Table presents the results of the aforementioned cases. These four cases all show significant improvements compared with the TO zero-label baseline, which demonstrated the effectiveness of labuli.... "32 pseudo train" performs the worst among all cases, indicating that the pseudo- labeled samples still contain much noise that is harmful for training. By comparing "16 + 16" and "32 pseudo val",, we show that the prompt selecting ability of ZPS is comparable to that of labeled validation set. Moreover, "Pseudo val" methods perform much better than others, which validates our hypothesis. These results indicate that the gain of training with the labeled set is greater than using the labeled set for checkpoint selection. However, since the pscudo-labeled data is noisy, increasing the size of pseudo-labeles validation data can only provide marginal performance gain. Overall, adopt "more pseudo val" with "pseudo select" in our ZPS for the best performance.
4.4 CASE STUDY
Prompt
Dio Ace.
Pseudo Acc.
Acc.
Given [premise)], } Is guaranteed true that "([nypothesis] J"" or no?
87.88
92.06
81.95
Suppose {prremise}) Can we infer that "{ "([hyootheiss))" Yes or no?
87.88
91.34
81.95
((premise)) nn Question: ([hypothesis} True False?
84.85
94.58
82.31
( [(premise) J \n)nn Question: Does this imply that "( ([hypttheis)) !"? Yes or no?
90.91
90.97
82.31
Given {fpremise)) Should we assume that "( ([hypothesiss))" is true? Yes no? that [premise }} Does it follow that (hypothesis) Yes no?
90.91
94.22
81.95
90.91
81.95
73.29
{(premise)) } Based on the previous passage, is it true that "( ((hhyptthesis))? Yes or no?
90.91
94.95
86.28
((premise) } we justified saying that "{ "(hhpoohhssis))"" )" no?
84.85
86.64
75.81
Given that [premise]) Therefore. it must be true that "[ ((yypothesss) !"? Yes or no?
90.91
94.58
82.31
Table 8: The performance of RTE prompts illustrated by: (1) Die Acc., the accuracy of a validation set with 16 labels (2) Pseudo Acc., the pseudo accuracy obtained by ZPS, and (3) Posthoe Acc., the true accuracy on the test set.
In Table 8, we present 10 RTE prompts and the following quantities (1) D16 Acc., which is the accuracy of a prompt on small validation set with 16 labeled samples. (2) Pscudo Acc., which is calculated by ZPS and (3) Posthoc Acc., which is the real performance on the test set. The results show that D16 has poor prompt selection abilities and there are no obvious correlations between Die Acc. and Posthoe Ace. On the contrary, our ZPS can distinguish the difference prompt performances and have better correlations with the real test performance.
CONCLUSIONS
We propose a novel Zero-L.abel Prompt Selection (ZPS) algorithm to select a high-performing prompt in the challenging setting where no labeled data is available. We demonstrate that 7PS can be used a song-and-play manner for different cases (zero-label generalization, ensemble distilla- tion, and few-shot model tuning). We also perform detailed ablation studies to verify the robustness and effectiveness with respect to different factors.
REFERENCES
Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, and Jian-Guang Lou. Input-tuning. Adapting unfamiliar inputs to frozen pretrained models. arXiv preprint 2xXi::2003.03111 2022.

---

Under review as a conference paper at ICLR 2023
Stephen H. Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M. Saiful Bari, Thibault Févry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun. Srulik Ben-David, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Alan Fries, Maged Saeed AIShaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Mike Tian-Jian Jiang, and Alexander M. Rush. Promptsource: An integrated development environment and repository for natural language prompts. CoRR, abs/2202.01279, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment chal- lenge. In Joaquin Muiiionero-Candela Ido Dagan, Bernardo Magnini, and Florence d' Alché Buc (eds.), Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classi- fication, and Recognising Tectual Entailment, pp. 177-190, Berlin, Heidelherg, 2006. Springer Berlin Heidelberg, ISBN 778-3-540-334228-
Marie-Catherie De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: In- vestigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pp. 107-124, 2019.
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205. 22488, 2022.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3816-3830, Online, August 2021. Association for Computational Linguis- tics. doi: 00.18653/v1/2000 www.oooo.....ooo...999.. URL https: //s:://aca http///aaaatthhhch anthllogh org/2021. acl-long.299
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. Ppt: Pre-trained prompt tuning for few-shot learning. arXiv preprint 100i:22099003322 2021.
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence, 12(10):993-1001 1990.
Junxian He, Jiatao Gu, Jiajun Shen, and Mare' Aurelio Ranzato. Revisiting self-training for neural sequence generation. In ICLR. OpenReview.net, 2020.
Geoffrey Hinton, Oriol Vinyals. Jeff Dean, et al. Distilling the knowledge in neural network. arXiv preprint arXivv/503.0255 02531, 2(7), 2015.
H. Scudder III. Probability of error of some adaptive ceatteetrnooontiionn machines. IEEE Trans. Inf. Theory, 11(3):363-37,, 1965.
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondfej Chum. Label propagation for deep semi- supervised learning. 2019 IEEE/CVF Conference on Computer Vision and Patern Recognition (CVPR), pp. 5065-5074, 2019.
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-4338 2020. doi: 10. 16.1162/tacl.a a 00324. URL https: ac anlllnnthhoogo org.//og/g// tacl - 11.28....
Anders Krogh and Jesper Vedelsby. Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems, 1994.
Sawan Kumar and Partha Talukdar. Reordering examples helps during priming-based few-shot learning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4507-4518, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/ v12202 findings-acl.395. URL https: /at.aaaaannnooooo.ooooaaaooooola www.aatttoooooooooo/o2oeo2e orgg20211 Findings-acl. 395.
10

---

Under review as a conference paper at ICLR 2023
Hunter Lang, Monica N Agrawal, Yoon Kim, and David Sontag. Co-training improves prompt- based learning for large language models. In International Conference on Machine Learning, pp- 11985-12003. PMLR. 2022.
Dong-Hyun Lee al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges representation learning, ICML, volume 3, pp. 896, 2013.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2 10l.:204400811,, 2021.
Hector Levesque, Emnest Davis, and Leora Morgenstern. The winograd schema challenge. Thir- feenth International Conference the Principles of Knowledge Representation and Reasoning- Citeseer, 2012.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv: 101. 00900, 2021.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? arXiv preprint 1XXi::2101006804 2021a. URL
https: /arxivv... org/ abs /210//2111. 06804.
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv preprint arXiv:2103. 2021b.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2022. URL https: org/ abs 2204.11140000 08786.
David McClosky, Eugene Charniak, and Mark Johnson. Effective self-training for parsing. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pp. 159-159, New York City, USA, June 2006. Association for Computational Linguistics.
Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. Generating training data with language models: Towards zero-shot language understanding. ArXiv, abs/2202.04538, 2022.
Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image captioning. arXiv preprint wwx.::2111.097. 2021.
Nasrin Mostafazadeh, Michael Roth, Annie Louis, Nathanael Chambers, and James Allen. LS- DSem 2017 shared task: The story cloze test. In Proceedings of the 2nd Workshop on Link- ing Models of Lexical, Sentential and Discourse-level Semantics, pp. 46-51, Valencia, Spain, April 2017. Association for Computational Linguistics. doi: 10. (0.18653/1/W11-000066 URL
https: hatp::/://allaahhloogyy. orgooo/g///oooo Mrgg///1-090..
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adver- sarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pp.
Online, July URL
2020. Association for Computational Linguistics. doi: 1111653//122220accmmmmiiaa https: https://aclantte htt////aaaaaaattaaa.aaaaa org//o20.022... acl.maainnn..aaaaaaaaa..aa 441.
Mohammad Taher Pilehvar and Jose Camacho-Collados WiC: the word-in-contex. dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume (Long and Short Papers), pp. 1267-1273 Minneapolis, Min- nesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-111 1128, URL https: //a/aaaaannhhlooo wwwphhhwll....o.o.ooooeoo 1128.
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient-free, edit-based in- struction search for prompting large language models. arXiv preprint arXiv:2203.07281, 2022.
Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. Controllable natural language generation with contrastive prefixes. arXiv preprint arXiv:2202, 2022.

---

Under review as a conference paper at ICLR 2023
Scott E. Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. ICLR (Work- shop), 2015.
Ellen Riloff. Automatically generating extraction patterns from untagged text. In AAAI/IAAI, Vol. 2. pp. 1044-1049, AAAÍ Press / The MIT Press, 1996.
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible alterna- tives: An evaluation of commonsense causal reasoning. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, pp. 90-95, 2011.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. In AAAI, pp. 8732-8740. AAAI Press, 2020.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, An- toine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110008000) 2021.
Timo Schick and Hinrich Schüize. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chupter of the Association for Computational Linguistics: Main Volume, pp. 255-269, Online, April 2021. Association for Computational Linguistics. doi: 10. 10.118811/2222111 eacl- m0in.20. URL https: //aclanthology ogy. org/ 2021 eacl -ma i in.... 20.
Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng MaXiaoyu Tao, and Nanning Zheng. Transductive semi-supervised deep learning using min-max max features. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 299-315, 2018.
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Auto- Prompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pp. 4222-4235, Online, November 2020. Association for Computational Linguis- tics. doi: 10. 10.18653/w1/2020.emm comnlp-main.3. main.346. URL https: http:////aaaaaanaa////a/o org/2020. emn.p-main. 346.
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. Spot: Better frozen model adaptation through soft prompt transfer. arXiv preprint 10.09:21 2021.
Zirui Wang. Adams Wei Yu, Orhan Firat, and Yuan Cao. Towards zero-label language learning.
arXiv preprint arXiv:2109.09193 2021.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint 109.022109.01652. 2021.
Qizhe Xie, Minh- Minh-Thhag Luong, Eduard H. Hovy, and Quoc V. Le. Self-training with noisy student improves imagenet classification. In CVPR, pp. 10684-10695... IEEE, 2020.
Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang. Ze- roprompt: Scaling prompt- based pretraining 1,000 tasks improves zero-shot generalization. arXiv preprini arXiv:2201.0m6 006910, 2022.
David Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. ACL.,
pp. 189-196. Morgan Kaufmann Publishers/ / ACI,, 1995a.
David Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd Annual Meeting of the Association for Computational Linguistics, pp. 189-196, Cam- bridge, Massachusetts, USA. June 1995b. Association for Computational Linguistics. doi: 00.31159811558.98666
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can ma- chine really finish your sentence? In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics. pp. 4791-4800 Florence, Italy, July 2019. Association for Com- putational Linguistics. doi: 10. 10.18653/v1/P119 1472. URL https: / /aaclannhooogyy wwwooyy..o... /rg/ P1 1472.
12

---

Under review as a conference paper at ICLR 2023
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pp- 12697-12706. PMLR, 2021.
Yanan Zheng, Jing Zhou, Yujie Qian, Ming Ding, Jian Li, Ruslan Salakhutdinov, Jie Tang, Sebastian Ruder, and Zhilin Yang. Fewnlu: Benchmarking state-of-the-art methods for few-shot natural language understanding. CoRR. (bbs/210..12742, 2021.
Chunting Zhou, Junxian He, Xuezhe Ma, Taylor Berg- Kirkpatrick, and Graham Neubig. Prompt consistency for zero-shot task generalization. arXiv preprint vrXiv:2205.00049 2022.
Barret Zoph, Golnaz Ghiasi, Tsung-Yi Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoc Le. Rethinking pre-training and self-training. In NeurlPS, 2020.
13


Reviews:
[{"review_id": "pLx6fAZytbW", "reviewer": "Reviewer_g5DF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces zero-label prompt selection (ZPS), which selects the optimal prompt for an unseen task without any labeled data or gradient update. ZPS obtains pseudo labels of an unseen task through prompt ensemble, calculates pseudo accuracy by comparing pseudo labels with the prediction of each prompt, and selects the prompt candidate with the highest pseudo accuracy. This paper shows the effectiveness of ZPS in both zero-shot and few-shot training settings.", "summary_of_the_review": "This paper proposed a simple but effective method to select the optimal prompt for an unseen task. The proposed method, Zero-label prompt selection (ZPS) boosts the performance of the T0 model and outperforms baselines such as self-training for zero-shot setting and model tuning and GRIPS method for few-shot setting. Although the paper is straightforward and well-written, the evaluation setting has some weaknesses. First, the assumption that a lot of unlabeled data for the target task is unrealistic. Also, the experiments of this paper are limited to the T0 model and promptsource library although the method could be applied to other types of model: more extensive experiments are needed to show the effectiveness of the proposed model. ZPS also has a practical limitation that the performance is dependent on the prompt candidate; if there are no \"good\" prompts, the effectiveness would be small. Lastly, for few-shot setting experiment, the baselines should be trained with the same number of labeled instances to show that ZPS is effective for few-shot setting as well.", "strength_and_weaknesses": "Strengths:\n1. The proposed method is simple and straightforward. It is easy to reproduce the proposed method.\n2. The effectiveness of the proposed method is shown in both zero-shot and few-shot settings.\n3. The paper is well-written and easy to understand.\n\n\nWeakness:\n1. It seems like the whole unlabeled training instances are used for ZPS (zero-shot). This cannot be seen as a realistic setting. I wonder if ZPS for zero-shot setting also shows effectiveness when there are not many unlabeled training instances (~16).\n2. The paper claims that they do not serve prompt ensemble during inference and instead select a single prompt to use the selected prompt for inference due to computational cost or latency. However, applying ZPS also increases the computational cost and latency if it uses many unlabeled training instances to select a single prompt (sometimes even more than prompt ensembling during inference when the training instances are larger than evaluation instances). Therefore, the performance of ZPS should also be compared against prompt ensembling during inference for a fair comparison. \n3. Is ZPS applicable for other prompt libraries (different from promptsource) or other model families (GPT-3, OPT, etc)? I think that showing that ZPS works for LMs that are not fine-tuned to follow instructions is important because the prompt sensitivity is known to be much higher and problematic for general pretrained LMs. \n4. The performance of zero-label prompt selection is highly dependent on the quality of prompt candidates. If there are not any \"good\" prompts that are manually created by humans, the upper bound of the performance would be low. The implicit assumption made by the paper, \"there is at least 1 good prompt\", is not a realistic assumption for the real-world use case. Also, ZPS cannot be applied to benchmarks that only consist of a single prompt such as BIG-bench because ZPS does not generate but select an optimal prompt instead. It would underperform GRIPS baseline which \"edits\" the instruction when all of the candidate prompts show low performance without edit operation.\n5. The paper does not compare ZPS with the oracle performance (the performance when selecting the best-performing prompt). The performance should also be compared with oracle performance and also the accuracy of how often the oracle prompt is selected. This accuracy directly shows the performance of \"prompt selection\".\n6. For few-shot ZPS, MT+ZPS uses twice more training instances (32) compared to baselines and uses unlabeled data for checkpoint and prompt selection. However, although I agree that we can use more labeled data for training by applying ZPS on unlabeled data for validation, we cannot accurately see MT+ZPS is more effective than MT itself; two factors, which are 1) effect of using more labeled data for training and 2) effect of applying ZPS, are not separated. Therefore, it is hard to claim that ZPS is effective on few-shot setting by only this result. There should be baseline results on 32 examples as well to see 2).  \n7. Although ZPS is effective for 11B scale, for 3B scale, it underperforms naive T0 on 4 out of 11 datasets which is free from any prompt ensemble techniques and the need for unlabeled data. This shows that the effectiveness of ZPS is not universal.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "k1fgg7Azter", "reviewer": "Reviewer_xSND", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a zero-label prompt selection (ZPS) algorithm and extend it to few-shot tasks. The algorithm can help select high-performing prompts and be used in both zero-label and few-shot settings conveniently.", "summary_of_the_review": "The core idea of the paper focuses on prompt selection. Yet, it lacks valid explanation and experiments to show why the use of the confidence score can filter low-performing prompts. Although the experiments show that the introduction of this algorithm improves the result of baselines, it lacks comparison with same type of algorithms (i.e., prompt selection algorithms). ", "strength_and_weaknesses": "Strength:\n1. The paper is well organized to be understood clearly. \n2. A large number of experiments are conducted on downstream tasks and the results show that the introduction of ZPS can boost the performances of baseline methods.  \n\nWeakness:\n1. In the part of prompt filtering, the confidence score and the role of clustering are not fully explained from the mathematical perspective. The relationship between the confidence score and the performance of prompts is not clear.\n2. The proposed prompt selection algorithm is too empirical and has limited novelty. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "GeX5OyTRuV", "reviewer": "Reviewer_34nz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Zero-Label Prompt Selection (ZPS), an algorithm to select the best prompt without any labeled data. In practice, they first use a heuristic rule to filter out low-quality prompts and obtain a candidate prompt set. Given these candidate prompts, they use them to assign pseudo labels for some unlabeled X in the dataset, using different ensemble strategies. This will result in a \"probing set\", which can be used as a pseudo validation set to select the best prompt. They also extend ZPS to support few-shot settings. ", "summary_of_the_review": "This paper targets an interesting problem and presents a simple yet effective solution. However, highly-similar work already exists (but missing) which hurts this submission's novelty. This missing related work has to be discussed and even compared to demonstrate the effectiveness of ZPS. ", "strength_and_weaknesses": "Strength:\n\n1. The problem is well-motivated. How to select prompts in the zero-shot setting is a very interesting and important research problem. \n2. The solution is simple, yet effective.\n3. The evaluation is comprehensive and the analysis is interesting.\n\nWeakness:\n\n1. An important and highly-related baseline is missing.\nDespite being a very interesting research problem, there are actually some existing works that try to address this problem. For instance, [1]  choose the prompt that maximizes the mutual information between the input and the corresponding model prediction. Their method has shown surprising performance in prompt selection. As another example, Lu et al., 2022 also try to construct a pseudo validation set for prompt selection.  Although their method of creating the validation set is quite different from ZPS, the intuition behind is very similar and worth further discussion in the related work. The existence of [1] hurts the novelty of ZPS, and also brings an important baseline to compare with. \n\n\n[1] An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6ee5m9Apr79", "reviewer": "Reviewer_5fB6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Zero-Label Prompt Selection (ZPS) that selects prompts without any labeled data or gradient update. ", "summary_of_the_review": "Despite concerns about the novelty, I recommend this paper because of its potential impact on prompt learning.", "strength_and_weaknesses": "## Strengths\n\n1. This paper explores pseudo-labeling at scale. \n1. This paper is well-written and easy to follow.\n1. The performance is good compared to self-training and the original T0.\n1. This paper sheds light on how to evaluate the effectiveness of a prompt, which previously highly relies on human expertise.\n1. This paper also explores robustness and ensemble strategies. I found the analysis interesting and informative.\n\n## Weaknesses\n\n1. The technical novelty is somewhat limited. The technique of determining pseudo-accuracy is not new.\n1. This paper is built on an assumption that a small validation set is not available. However, I would like to see how it's compared to select prompts on a very small validation set (e.g., with only 20 or 50 examples annotated by humans), which may be a more realistic setting.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject