PromptBoosting: Black-Box Text Classification with Ten Forward Passes
Authors: Bairu Hou
Joe O'Connor
Jacob Andreas
Shiyu Chang
Yang Zhang
Venue: Submitted to ICLR 2023 (2023)

Abstract: We describe PromptBoosting, a query-efficient procedure for building a text classifier from a neural language model (LM) without access to the LM’s parameters, gradients, or hidden representations. This form of “black-box” classifier training has become increasingly important as the cost of training and inference in large-scale LMs grows. But existing black-box LM classifier learning approaches are themselves computationally inefficient, typically specializing LMs to the target task by searching in a large space of (discrete or continuous) prompts using zeroth-order optimization methods. Instead of directly optimizing in prompt space, PromptBoosting obtains a small pool of prompts via a gradient-free approach and then constructs a large pool of weak learners by pairing these prompts with different elements of the LM’s output distribution. These weak learners are then ensembled using the AdaBoost algorithm. The entire learning process requires only a small number of forward passes and no backward pass. Experiments show that PromptBoosting achieves state-of-the-art performance in multiple black-box few-shot classification tasks, and matches or outperforms full fine-tuning in both few-shot and standard learning paradigms, while training 10x faster than existing black-box methods.

Under review as a conference paper at ICLR 2023
PROMPTBOOSTING: BLACK-BOX TEXT CLASSIFICA- TION WITH TEN FORWARD PASSES
Anonymous authors Paper under double-blind review
ABSTRACT
We describe PROMPTBOOSTING query-efficient procedure for building text classifier from a neural language model (L.M) without access to the LM's param- eters, gradients, hidden representations. This form of "black-box" classifier training has become increasingly important as the cost of training and inference in large-scale LMs has grown. But existing black-box LM classifier learning ap- proaches are themselves computationally inefficient, typically specializing LMs to the target task by searching in a large space of (discrete or continuous) prompts using zeroth-order optimization methods. Instead of directly optimizing in prompt space, PROMPTBOOK BOOSTING obtains a small pool of prompts via a gradient-fre ffree ap- proach, and then constructs large pool of weak learners by pairing these prompts with different elements of the LM's output distribution. These weak learners are then ensembled using the ADABOOST algorithm. The entire learning process re- quires only small number of forward passes per batch and no backward pass. Experiments show that PROMPTBOOSTING in multiple black-box few-shot classification tasks, and matches or outperforms full fine-tuning in both few-shot and standard learning paradigms, while training 10x faster than existing black-bor methods.
achieves state-of-the-art performance
INTRODUCTION
Prompt-based learning has emerged as effective method to adapt pretrained language mod- els (LMs) for downstream natural language processing (NLP) tasks. A typical prompt-learning paradigm involves appending specially-designed sequence, called prompt, to the input to pre- trained LM, which will thereby be repurposed for given downstream task. Compared to the stan- dard fine-tuning, prompt-based based learning is much more parameter-efficieett.
Most prompt-based learning methods require searching for the optimal prompt for the downstream task. When gradient information of the pre-trained LM is available, such optimization be performed by standard gradient- based methods (Liu et al., 2021 [L Liang) 2021 Lester et al.,
easily
2021 Zhhag et al. 2021] Liu al. 2022). However, in many real-world scenarios, the parameters, gradient or hidden representations of the LMs are not accessible, a.k.a. the black- -bx tuning setting, which makes gradient-baat prompt learning very challenging (Sun al. (022).
To tackle the challenges, the most common existing black-box solution is to resort to gradient-free optimization techniques search for the optimal prompt, such as the zeroth-order gradient approx- imation (Sun et al. 2022 Diao et al., 2022) and reinforcement learning ceamiingguuied optimization (Deng al. 2022). However, these methods would require a large number of queries of the LMs, which, considering the ever-growing size and computation cost of the pre-trained LMs, is highly inefficient and could lead to large approximation errors.
In this paper, we propose PROMPTBOOSTING a novel black-box prompt learning approach which does not rely on searching an optimal prompt, and which can thus drastically improve the computa- tional efficiency over the existing methods. Figure Iillustrates the pipeline of PROMPTBOOSTING Specifically, rather than optimizing over the prompts, PROMPTBOOSTING constructs a small pool of prompts via a gradient-free free approach. These prompts are sub-optimal because they not opti- mized for any downstream tasks. Then, PROMPTBOOSTING creates a large pool of weak learners by pairing each prompt with different elements of the LM's output distribution, which is commonly

---

Under review as a conference paper at ICLR 2023
Step 0: Prompt Gemeration
Step I: Weak Learner Construction
Step 2: Ensemble
Prompt Q
Weak Learner t
Weighted Loss
ADABoOST Ensemble
[Input] [Nask] film.
Verbalizer M
[Input] truly [Mask]
LM
+
novie.
Tuned Frozen
Final Black-Bor Box Classifier
Input
Prompt *
[Input] [Mask].
Random Draw
Figure 1: Overview of PROMPTBOOSTING.
known the verbalizer. Finally, these weak learners are ensembled using the ADABOOST algo- rithm, where the optimization each iteration is performed only over the verbalizer, not the prompt. The entire process only needs to evaluate the LM's output with each of weak prompts, it only in- volves a small number of forward passes per batch, and no backward pass.
We evaluated our method PROMPTBOOSTING achieves state-of-the-art performance and matches or even outperforms full fine-tuning in both few-shot and standard learning paradigms. Furthermore, PROMPTBOOSTING can run 10x faster than existing black-bor prompt-learning approaches, with only ten forward passes per batch. PROMPTBOOSTING can inspire a new black-bor prompt learning paradigm.
number of downstream tasks and LM's. The results show that
RELATED WORK
Prompt-based learning Prompt-based learning has emerged as new approach for adapting pre- trained LMs for downstream tasks fueled by the success of GPT-3 (Brown al. 2020). Since the prompts directly influence the performance prompt-based learning, recent studies have focused on how to find the best prompts given specific task. AutoPrompt (Shin et al. 2020) designs gradient- based discrete optimization method to search for the optimal prompt. LM-BFF (Gao et al., (021) leverages the pre-trained T5 model (Raffel al., (0200) automatically generate prompts and select the best one based on the performance on the validation set. Since verifying the automatically- generated prompts is still time-consuming, the PTR (Han al.) (202]) method incorporates logic rules to construct prompts and to encode prior knowledge into prompt- based learning.
Another line of work replaces the discrete prompt tokens with continuous embeddings that have their own parameters. P-tuning (Liu et al.) ) trains BiLSTM network to output continuous prompt embeddings. Prefix-tuning (Li Liang (021] inserts prompt embeddings each transformer layer in LMs and optimizes only the prompt embeddings during training. Prompt Tuning (Lester et al.) 2021) also keeps the LMs frozen prompt embeddings to each layer, P-tuning V2 (Liu et al. 2022) replaces the language model head in LMs with linear layer for classification and shows that soft prompt tuming scales to medium- sized LMs and hard sequence tagging tasks. Our work adopts the automatically generated discrete prompts for prompt-based learning.
adds prompt embeddings only in the input. addition to adding
Black-box Tuning Extremely large LMs such GPT-3 are only provided as service in the cloud, resulting inaccessible parameters and gradients of LMs. Furthermore, from the model provider's perspective, sharing hidden representations or gradients LMs may reveal the vulnerability of the model and lead to security problems (Tramer et al.] 2016). How to find the optimal prompts in such black-box tuning setting has attracted various explorations. BBT (Sun et al. 2022) employs the CMA evolution strategy, derívative-free optimization method, to optimize continuous prompt embeddings. However, BBT requires querying the LM tens of thousands of times even in few-shot settings. Furthermore, BBT's of soft prompts violates the black-box tuning setting because in realistic use one can query the LM service with only textual input instead of prompt embed- dings. Other concurrent work (Diao al. 2022) also gradient-free techniques optimize soft
2

---

Under review as a conference paper at ICLR 2023
prompts. RLPrompt (Deng al. 2022) is more realistic black-box tuning method where discrete prompt tokens are optimized through reinforcement learning and the performance on downstream tasks serves the reward. GrIPS (Prasad al.) 2022) performs phrase-level editing to generate dis- crete prompts for downstream tasks. Existing black-box tuning methods suffer from poor efficiency and sub-optimal performance. In contrast, our method achieves high efficiency by first generating only small set of prompts and achieves superior performance by then creating a set of weak learners from these prompts and ensembling them together via ADABOOST (Freund & Schapire 1997..
Model ensemble. Model ensembling is a commonly used technique machine learning. Prior to deep learning, Bagging (Breiman [996] 2001) and Boosting (Freund & Schapire 1997: Fried- man 2001) showed the power of model ensembling. One these methods, ADABOOST (Freund & Schapire 1997), sequentially learns series of weak learners and ensembles them for better generalization. During training, each weak learner is tweaked by leveraging examples that were misclassified by previous classifiers. Since the performance of each individual prompt can be weak, our method adopts ADABOOST as the framework for learning and ensembling multiple prompts.
Prompt ensemble. As has been pointed out by prior work (Lester et al. 2021 ensembling prompts is more efficient than ensembling entire fine-tuned models. Various ensemble strategies have been explored in past work. Uniformly averaging the predictions from different prompts has
been used for factual probing (Jiang et al. 2020), text generation (Yuan et al.
2021 Schick &
Schütze 2020), and classification tasks (Schick & Schütze 2021a Lester al. 2021) ). Further-
more, some methods adopt a weighted averaging strategy for better performance the weight of each different prompt can be learned during training (Jiang al.) 2020) Qin & Eisner 202
defined using some heuristics (Schick & Schütze [2021a|b b). Our method also falls into the prompt ensemble category. The main difference is each prompt-based model sequentially learned condi- tioned on the classification errors of prior models.
METHODOLOGY
In this section, we will describe the PROMPTBOOSTING algorithm. For notation, we use |A| to denote the size of finite set A; [A] to denote an index set (1,2, A}.
3.1 PROBLEM FORMULATION AND THE PROMPT-LEARNING FRAMEWORK
Consider a text classification downstream task. Denote Drr = U,[(,; vK)} as the training set, where æ; denotes the input text sequence and yi denotes the output label. We are given a pre- trained language model, denoted p; F"xx),, which, given the input æ,, produces probability distribution over the vocabulary set, V, at a given location. In this paper, only the output distribution where the input is [mask] is relevant, so p; R(V) x is just |V/-dimensional vector specifying the output probability at the [mask] location, where | denotes the vocabulary size. Our goal is to adapt the LM F* F*) to the downstream task using the downstream training set Dir-
We adopt the common prompt-learning framework, where the parameters of F*(-) are frozen (hence we add a superscript to emphasize this). Instead, the following two mechanisms are added to convert F* into text classifier for the given downstream tasks.
1. Prompt A prompt is sequence of tokens that is concatenated to the input. Formally, denote the prompt sequence as and the concatenated input sequence as æ; Then the LM is modified as F* (T))).l)).
Verbalizer To convert the output probability over the vocabulary into that over the classes, a verbalizer is introduced to assign each token into different classes. Formally, denote the number of classes of the downstream class as |)) then the verbalizer an [)|-by-|D matrix, denoted as M, where the element in row column represents the assignment weight of the y-token in the vocabulary into class c. Each row of M would sum up to one. The predicted probability of all the classes can then be expressed as Mp:-
3

---

Under review as a conference paper at ICLR 2023
To up, after the prompt and verbalizer are applied, the adapted LM becomes MF*(x:llq) (z:llq). Therefore, the prominttuning process boils down to learning an appropriate verbalizer M and prompt for the downstream task.
3.2 ALGORITHM OVERVIEW
Conventional black-box prompt learning methods commonly pre-set M while performing black-box optimization over q. which results in large computation cost. On the contrary, PROMPT- BOOSTING randomly chooses from small number of pre-generated prompts and performs op- timization over M instead. Due to the sub-optimality of pre-generated prompts and the limited representation power of M, the resulting classifiers are weak. However, this process is able to quickly generate a large pool of such weak learners, which can then be ensembled into strong learner using the ADABOOST approach. Since the optimization over M is computationally cheap, the ensembling process is still much more efficient than the conventional black-bor box methods.
More specifically, the ADABOOST algorithm iteratively generates T weak learners, and weaker learner / is optimized under its respective loss function, denoted as (e M), which is essentially weighted loss over the training set with larger weights on those that are misclassified by the previous weak learners (More details of the ADA BooST algorithm will be provided in Section 3.5) Then, as PROMPTBOOSTING consists of the following key steps.
shown in Figure]] |
Step 0: Generate a pool of prompts, Q = U,( ). using gradient-free method.
Step 1:
Construct 7" weak learners. For weak learner t, its prompt qe is uniformly randomly drawn from Q; its verbalizer M is determined by solving
min Le (g. M
s.t. Mc 2 0, Ve E [[]]], E [[2]].
Σ
M.. 1, [V]]
(I)
cE(D
Step 2: Ensemble the weak learners according ADABOOST.
Section (.3 3.ll will describe how to solve equation] Q, is generated.
!
Section 3.4 will describe how the pool of prompts,
3.3 LEARNING THE VERBALIZER
As discussed, the loss function C, in equation i essentially weighted sum of the individual loss over the training dataset Dir, i.e.
L.(g..
Σ
wril(wi, yei ge, M))
(EL.BEEDE
where war denotes the weight training data point for learning weak learner determined by ADABOOST; f( M) denotes the loss on data point (æ,, with the parameters set to qi and M. Since we focus on classification tasks, f(-) should ideally be the cross-entropy loss. However, the optimization problem in equation i essentially partition problem, which can easily lead to combinatorial complexity. To derive the tractable solution, we adopt the following strategy. First, solve equation with E(-) set to the ε loss, which, though not optimal for the classification task, bears a closed-form solution. Second, further screen the token assignment by maximizing the training set performance. The detailed method is described below.
Minimizing the L loss By replacing the E(-) in equation 2 with the & loss, closed-form solution can be derived, which can establish a basis for the subsequent steps for deriving a good verbalizer. Formally, let h; be the one-hot representation of the class label and let ा. = F=(a,llq)) represent the LM output probability with the prompt qe concatenated. Then, with the e, loss, equation becomes
Li(g,, M)
Σ PPPr
Whr h2; M
Σ (e.r
wi 1h. Mr.
(3)
Σ
Wei (T- MT)).
4

---

Under review as a conference paper at ICLR 2023
Here, represents all-one column vector of dimension 13%, and (-1)*t represents the element- wise power operation. The last equality is because each element of M is within [O, 1] and each element of h; is either 0 or 1, so can easily remove the absolute sign depending on the actual values of h;:
As shown in equation [] the loss function is linear with respect M, the optimization in equa- tion [ becomes a linear optimization problem with linear constraints, which has closed-form corner solutions. For notational brevity, define score matrix, S, as
S Σ (ELD
Wa; W (o0r:[(1))) [(-1)^11
which is the same size as M and is essentially the coefficients multiplied on M in equation Then, we state without detailed derivations that the solution to equation i such that each token is assigned to the class for which it gets the highest score among all the classes, i.e.
Ma = 1, if arg max Se'o, and 0 otherwise.
(5)
d'E|D''
Since the & loss does not generally work well for classification tasks, we empirically find that the verbalizer derived in equation 5 is of limited performance. However, this inspires that the score matrix, S, a good measure of how well each token should be selected for a class. In the following step, we will further screen the tokens with the help of the score matrix.
Screening the tokens One issue with the verbalizer in equation 5 that each token has to be assigned to one class, even those tokens that are not well indicative of any classes, Therefore, by removing the non-informative tokens and only retaining the best tokens for each class, we can improve the verbalizer performance. To reduce the computational complexity, we will retain only one token for each class. Specifically, we first identify a candidate set of tokens for each class by choosing the tokens with top-k scores for that class, i.e. the top-k elements in Se: for class c, where subscript e denotes the e-th row. Then, we evaluate all the possible combinations that include one token from the candidate set for each class (hence k!'| combinations in total) and choose the combination that achieves the best training accuracy (weighted by (wi)).
3.4 CONSTRUCTING THE PROMPT SET
To generate the pool of prompts, Q (step 0 in section 3.2). we adopt the optimization-free method proposed by Gao et al. (202)), which employs the T5 (Raffel al. (020)) model. Specifically, we first construct a small subset of the training set, denoted as Dyens to induce the prompt generation gen is exactly Dr, in few-shot setting). Then, for each data point (z,, E Pgens we construct an input to the T5 model as <input><A><laabe
token><B> (for sentence- -air classification tasks, token>>B><inputt2>) Here, <A> and <B> are
the input to T5 becomes <imput1><A><latt mask tokens in T5 representing spans to be filled in. <input>, <input1> and <input2> represent the input text æ;. <label token> is a pre-defined mapping to convert class labels to tokens in V. For example, positive label (34 1) in SST-2 (Socher et al. 2013) dataset is mapped to token great while negative label (z/ = 0) is mapped to terrible. Given this input, the T5 model fills in the spans for <A> and <R>, The decoding process aims maximize output probability conditioned on the input over D Dyen- Then the T5 generated outputs, denoted as <output A> and coutput B> will be converted into prompts and concatenated to the training input text, i.e. æ;1lq, in A> [mask] coutput B> (for sentence-pair tasks, the form becomes A> [mask] coutput B><input2>). As an example, on SST-2 dataset, one of the generated outputs by T5 is <output A> A truly, <output B> movie. Then the input sentence "I Love it. will be converted to "I love it. A truly [MASK] movie". With wide beam search width (by default we use 100), we select the top-10 generated prompts according to the log-likelihood to form the prompt pool, Q. All the generated prompts used in our experiments can be found in Table 6 in Appendix C Readers can refer to Gao et al. (2021) for further details. The entire generation process does not involve any optimization over the prompts, and thus is computationally efficient. It is worth noting that the aforementioned approach can be replaced with any other optimization-free prompt generation methods, such as manually creating the prompts, making PROMPTBOOSTIN flexible for realistic use.
the form of <input><<utppp sunput1><cutee
3.5 ENSEMBLING THE WEAK LEARNERS
5

---

Under review as a conference paper at ICLR 2023
We follow the ADABOOST algorithm ensemble the weak learners. As discussed. each weak learner minimizes weighted loss over the training set (equation 2). The final prediction is produced by taking a weighted average over the weak classi- fiers' output. Further details, including how the weights are computed, are shown
Algorithm 1 Model Ensemble in PROMPTBOOSTING
1: Input: prompt set Q = U,{q;), LM ("(),, Pir.
Output: weak learners U.(f(() and their weights U.(ar).
3: Set initial data weight to wt: 1//. Vi [[Pr-]]
4: for Iteration = 1,..., T do
5: Randomly draw a prompt q, from Q
6: Learn the verbalizer M, with weight {we)
in Algorithm [T] It is worth mentioning
7: Set weak learner / to f+(.) = M, M, (+1g))
that we can generate many weak learners at a very low computational cost, because we only need to evaluate the LM's output distribution with each of the pre-generated prompts in Q, beyond which no extra for- ward pass is needed when learning each weak learner. Since the number of pre- generated prompts is small, typically ten in our implementation, the entire learning process involves no more than ten forward passes per batch in the training set, no matter how many weak learners are generated.
8:
Compute weighted error
err (4) = EEErl we wey ( [Perl w0
Ptrl
9:
Compute the weight on
a: log 1-ery(()
enr(t) + + 1)
10:
Adjust dataset weight
((r+1)s = We exp(ar (usts(t))) vi [Dr.]] Re-normalize (w(++)))-
II:
12:
end for
EXPERIMENTS
4.1
EXPERIMENT SETUP
Datasets We conduct experiments on wide range of tasks including sentiment analysis (SST-2
(Socher et al. 2013) and MR (Pang & Lee 2005)), topic classification (TREC (Voorhees & Tice.
2000) and AG's News (Zhang al. 2015)), ), and natural language inference (SNLI(Bowne al.
2015 MNLI-m (Williams et al. 2018), RTE (Dagan et al.] 2005), and QNLI (Rajpurkar et al.
2016
The dataset statisties can be found in Table in Appendix
Evaluation Setting We mainly evaluate the performance of PROMPTBOOSTING in few-shot set- tings. This is reasonable especially for black-box model tuning scenarios, where the maximum allowed query times may be limited. We randomly sample & examples per class from the original training set to construct a k-shot training set D., for model training. Following previous work Gaao also construct the validation set Deat by
et al. 2021 Zhang et al.) 2021 Sun et al. 2022). domly sampling another l examples per class from the original training set (i.e., (Drll = [Deat]] D By default we set k 16 for our main experiments. Also, while previous work splits the training and validation sets in this way and we do so for direct comparison, we also explore integrating the validation set into the training set-in a truly few-shot setting, we should make full use of as many examples as we can, and we show this leads to an improvement in performance. As for evaluation, we use the whole testing set. For SNLI (Bowman al. 2015) and the datasets from the GLUE benchmark (Wang et al. 2018), we use the original validation set for evaluation.
Backbone Models In the main experiments, we adopt the widely-used RoBERTa- large model (Liu ct al. 2019 for evaluation in order to allow for direct comparison with baselines.
Baselines We compare PROMPTBOOSTING with fine-tuning and state-of- black-box tuning methods described below. For reference, we also include white-box prompt-based learning meth- ods that are designed for few-shot setting. Implementation details can be found in Appendix A (1) Fine-tuning is just standard model fine-tuning in a few-shot setting. (2) LM-BFF (Gao et al,, 2021) is a prompt-based fine-tuning method. In LM-BFF, all input will be transformed using au- tomatically generated prompts. Then the whole model is fine-tuned based on the transformed data. (3) DART (Zhang et al.] 2021) replaces the discrete prompts in LM-BFF with trainable prompt embeddings, which can reduce the prompt generation cost. (4) Feature-
MLP uses LMs as feature extractors, which is simple yet effective way to train LMs in black-box settings. We take the manual prompt from past work (Gao et al., 202T) and query the LMs to get the the masked token prediction p. Then 3-layer ML. P is trained the extracted features. (5) BBT (Sun et al., 2022) is

---

Under review as a conference paper at ICLR 2023
Table 1: Performance of proposed PROMPTROOSTING and baseline methods in few-shot setting (k: 16) measured by classification accuracy (%) All methods use RoBERTa-large (Liu ct al. 2019) as the backbone LM for fair comparison. Two white-hox methods are included for reference including LM-BFF (Gao et al.
2021 and DART (Zhang al] 2021]. Feature-MLP, BBT (Sun et al. 2022). and PrPrmmpt (Deng et al.
2022
the main black-box baselines. PROMPTBOOSTINGG- combines both training and validation for training Mean accuracy (and standard deviation) is reported over different splits.
Method
SST-2
MR
AG's News
TRFC
SNLI
MNLI
QNLI
RTE
Avg.
tuning
82.7 (3.6)
(1.4)
48.4 (4.8)
(6.4)
56.3 (1.5)
(3.9)
LM-BFE
(1.3
87.4 (0.6)
87.1
(2.7
76.5 (2.6)
(2.0)
64.4 (4.6)
66.6
78.3
DART
93.5 (0.5)
86.8 (1.0)
(3.8)
(1.6)
(26))
66.7 (3.7)
(9.5
78.1
Peature-MIA
(3.8)
82.1 (1.6)
(2.0)
(2.4)
(3.2)
(1.8
(4.5
(4.5
60.3
BBT
88.2
82.8 (2.6)
(2.7
(5.3
44.7 (4.0)
(2.8
56.8 (2.0)
(3.3)
60.6
RLPrompt
90.5 (1.5)
86.2 (2.5)
76.2 (2.7)
(3.3
(1.8
(4.7)
52.1 (2.9)
(2.2
59.8
PROMPE BUOSTING
(3.0)
84.6 (2:5)
85.2 (0.9)
(4.0
(3.5
(1.5)
(..)
(5.0
71.4
PROMPTBOOSTE BOOSTINNS-
(3.6)
84.7 (2.1)
(1.1
(1.4)
(2.072..7
(1.3
38.3 (2.8)
(2.4
Table 2:
Deployment efficiency proposed PROMPTBOOSTING and baseline methods few-shot setting
(k = 16). With all methods use RoBERTa-large (335M parameters) as the backbone LM, some basclines introduce additional parameters, leading to the slight variation in total parameters. Wall time is reported to measure the training time efficiency. Query efficiency is evaluated by #Forward and #Backward, which refer the number forward/backward passes per batch during training respectively.
Method
Triinable
param
Total param
Time
AG's News #Forward
#Backward
Acc
Time
RTE
#Forward
#Backward
Finc-tuning
335M
335M
86.2
13 min
100
100
54.4
19 min
100
100
LM-BFF
335M
335M
87.1
Smin
32
32
66.6
9 min
60
DART
335M
86.8
15 min
30
59.0
120
Foature-MLP
SM
340M
74.1
0.5 min
1
55.5
0.3 min
BBT
25k
335M
81.2
88 min
SK
49.1
52 min
8K
RLPrompt
420M
77.2
117 min
1K
52.2
90 min
1K
PROMPTBOOSTING
<Ik
335M
85.2
8 min
10
60.0
4 min
10
black-box tuning method for few-shot learning which employs zeroth-order gradients to optimize the continuous prompts. (6) RLPrompt (Deng et al.] (202)) models the black-box optimization of discrete prompts reinforcement learning problem and adopts Q-learning to find the best prompt.
Implementation Details The implementations of standard fine-tuning and feature-based MI.P are based on the Huggingface trans fo Formers library (Wolf et al. 2019). For all other baselines, we use their official implementations and hyper-parameters. Please refer to more details in Appendix A For our method, we sequentially train weak classifiers and add them to our ensemble- stop when validation performance plateaus or when we reach the maximum number weak classifiers.
4.2 EVALUATION RESULTS
Overall Comparison We first evaluate the effectiveness of PROMPTBOOSTING in few-shot set- ting with experiment results in Table [ Although there is some variance across datasets, PROMPT- BOOSTING achieves state-of-the-art performance compared existing black-box tuning methods.
We emphasize the effectiveness of model ensembling in PROMPTBOOSTING Firstly, on the SST-2 and MR datasets, which are sentiment analysis tasks, even individual weak learners in PROMPT- BOOSTING can achieve 100% accuracy on the training set, making the model ensemble inapplica- ble (note that AdaBoost cannot ensemble classifiers that achieve 100% accuracy). Therefore, we directly train 10 weak learners using 10 prompts on the unweighted training set and then select the weak learner that performs best on the validation set as the final model. Since the advantage of model ensemble is limited SST-2 and MR dataset, it is not surprising that PROMPTBOOSTINS performs slightly worse than BBT and RLPrompt. However, PROMPT BOSTING is still better than fine-tuning an-MLP, demonstrating the effectiveness of our proposed verbalizer learning method.
Secondly, on the other 6 datasets, PROMPTBOOSTING consistently outperforms all baselines. PROMPTBOOSTING also outperforms standard fine-tuning the 4 NLI tasks. It is worth noting that on the TREC dataset, all of the black-bo box baselines performs very badly except for PROMPT- BOOSTING, which even achieves level of accuracy close to that of white-box methods. One poten-
7

---

Under review as a conference paper at ICLR 2023
Acaraay
Accamey
Ioar plwerting
(a) Performance on SST-2 (b) Performance on MR (c) Performance on SNLI (d) Performance on MNLI
Figure 2: Model performance function of training set size different datasets. For NLI tasks
(SNLI and MNLI), we also include prompt refinement for better performance.
tial reason is that the TREC dataset is harder for prompt-based learning. For example, the manual prompt on the TREC dataset achieves only 32% accuracy Gao et al. (202I). According to our exper- iments, individual weak learners trained on the unweighted training set using verbalizer learning method can only achieve 30%-50% accuracy. However, after employing model ensembling, the performance is largely improved, demonstrating the effectiveness of PROMPTBOOSTING.
Finally, we incorporate a variant of PROMPTBOOSTING, namely PROMPTBOOSTING- which skips the hyper-parameter tuning and directly integrates the validation into training. The hyper- parameter, i.e., the number of weak classifiers, is determined manually according to its value when the validation set is available. By expanding the training set, one can observe slight improvement on the performance with lower variance.
Deployment efficiency Another concern with black-box model tuning is the deployment effi- ciency. As we have discussed above, directly adopting zeroth-order gradient optimization techniques suffers from the need query many times, making it less applicable in realistic scenarios. We vi- sualize the deployment efficiency of different methods in Table 2 AG's News and RTE datasets are adopted due to the average input length (see Table 4). The metries include parameter efficiency (number of trainable parameters and total parameters), wall time of training, and number of for- ward/backward passes per batch. In terms of trainable parameters, PROMPTBOOSTING optimizes only less than 1k parameters 2(00) and does not introduce any extra parameters. In contrast, RL- Prompt uses another network, DistilGPT2 (Sanh al. 2019), in addition the backbone RoBERTa model and consequently increases the training cost. In terms of wall time, PROMPTBOOSTING im- proves the efficiency over existing black-box tuning baselines (BBT and RLPrompt) by more than 10 times. The query time is also significantly lower. Only 10 forward passes per batch of train- ing data are required during the training of PROMPTBOOSTING, By contrast, our baselines require thousands of forward passes, which makes them hard to use in realistic scenarios.
Effect of training data size We also study the performance of PROMPTBOOSTING as the size of the training set increases (see Figure 2 Note that we still fix k = 16 for the validation set regardless of the training set size. Results on AG's News, TREC, QNLI, and RTE dataset are shown in Figure B in Appendix B The conclusions are in three dimensions. Firstly,
the SST- and MR datasets, PROMPTBOOSTING consistently outperforms fine-tuning with lower variance, demonstrating the effectiveness of our method. Secondly, the AG News and TREC datasets, PROMPTBOOSTING performs worse than fine-tuning. similar phenomenon also exists in past work Gao et al. (02T). ), where a white-box prompt-based few-shot learning method can achieve performance that is at most only comparable with fine-tuning. However, we remark that our method still maintains large advantages compared to all black-box baseline methods and achieves highly usable performance. Finally, as the amount of training data increases, the performance of fine-tuning improves and gradually outperforms our method on the four NLI datasets. This finding possibly due to the fact that pre- pre-trained LMs before fine-tuning are not good at tasks involving sentence pairs.
directly depends on the prompt. As has been shown in previous work. different prompts have significant in- fluence on the performance of prompt-hased methods (Shin al. 2020 Gao et al.) 2021 However,
Refinement of prompts The performance of the weak learner in PROMPTBoOSTEE
8

---

Under review as a conference paper at ICLR 2023
Table 3: Performance of PROMPTROOSTING with different numbers of prompts in few-shot setting (k Poj.PPPPBBOSSTIN means top-d prompts (sorted according to the heam search score) are used for model training. Mcan accuracy (and standard deviation) is reported over 5 different splits.
SST-2
MR
AG's
TREC
SNLI
MNLI
QNLI
Avg.
(1.0)
(5.0)
(3.7)
(4.3)
(4.0)
(2.0 (3.5)
(5.7
62.9
88.8
(1.9
83.5 (4.2)
(2.5
(3.5)
(4.9
56.5 (2.)
(7.0445)
70.2
(3.6
(2.5)
85.2 (0.9)
(4.6
(2.5 (3.3
(8.0
60.0
714
PROMPI SOMMP/BOSTTIN--21
88.1
8.30
(1.3)
(1.9
(3.9
55.201.)
(4.0
(.21
71.3
in PROMPTBOOSTINGG the prompts are fixed and will not be optimized during training. Therefore, we consider simple yet effective way to improve the performance through prompt refinement. Specifically, because we automatically generate 100 prompts for each dataset but only use 10 of them, we may select top-10 prompts following some heuristics to improve the quality of prompts. Before training, we first evaluate the performance on the validation set by training a weak classifier using the method in Section 3.3 on the unweighted few-shot training set. Then we construct the prompt pool by selecting the top-10 10 prompts according to the accuracy of the corresponding weak learner on validation set. Please note that the few-shot setting makes the refinement process very efficient. Later on, PROMPTBOOSTING is trained using the refined prompts. We mainly evaluate the effectiveness of the prompt refinement SNLI, MNLI, and QNLI dataset where the gap be- tween PROMPTBOOSTING data. Experiment results can found in Figure 2 One can observe consistent improvements on few-shot performance across three NLI tasks, especially on QNLI dataset where the performance of CROMPTBOOSTING was far from satisfactory without prompt refinement. Overall, the prompt refinement leads to trade-off between training cost and model performance.
and standard fine-tuning is relatively large with the increase of training
Effeet of the number of prompts our main experiments, we use 10 prompts by default. In- tuitively, large prompt pool increase the diversity of weak classifiers which could improve the performance. However, the training/inference cost will also increase more prompts are included for model training. We empirically study the relationship between the number of prompts and the model performance in Table 3 In general, more prompts benefit the performance for most datasets (except QNLI). We highlight the effectiveness of multiple prompts on AG's News and TREC dataset, on which the performance becomes better and more stable. As we have discussed in the few-shot experiments in Table individual prompt performs very bad on TREC dataset. This is also proved by PROMPTBOOSTING I that only achieves 4 41.3% accuracy. However, by using our prompt ensem- ble framework, the performance can be boosted to 84.6% when 10 prompts are provided. Finally, the performance improvement is relatively small when the number of prompts increase from 10 to 20, implying that 10 prompts should be good enough for PROMPTBOOSTING.
Full data training Due to its efficiency, PROMPTBOOSTING can also generalize to full data train- ing instead of just the sho-shot setting. We compare with fine-tuning on the entire training dataset for SST, MR, TREC, and RTE. Experimental results can be found in Table 5 the Appendix B
CONCLUSION
In this paper, we propose PROMPTBOOSTING, an effective black-box model tuning framework. Without access to the parameters and gradients of pre-trained LMs, PROMPTBOOSTING can adapt LMs for various downstream tasks. The efficient weak learner construction method, together with the ADABOOST model ensemble algorithm, makes PROMPTBOOSTING achieve state-of-the-an per- formance in black-box tuning setting with least 10x run-time efficiency.
For future directions, we will explore how to generalize PROMPTBOOSTING beyond the classifi- cation tasks to generation tasks. Also, will study how to combine the prompt ensemble idea in PROMPTBOOSTING with gradient- based optimization and further improve the performance of existing prompt-based learning methods.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
The authors have made great efforts to ensure the reproducibility of the experiment results in the paper. Firstly, the experiment settings, evaluation metrics, benchmarks, etc. are explained in detail in Section 4.1 In summary, eight public datasets with clear references are used for evaluation in few-shot setting. Secondly, the implementation details and hyper-paraan parerr-araeterrs of our method and baselines are clearly presented in Section B and Appendix A We also include all prompts that are used for our method in Appendix [ Thirdly, all experiments are based on 5 runs with different random seeds to improve the reliability. Finally, the pre-released code for the proposed method is also included the supplemental material.
10

---

Under review as a conference paper at ICLR 2023
REFERENCES
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632-642, 2015.
Leo Breiman. Bagging predictors. Machine learning, 24(2):123-140, 1996.
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D) Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pp. 177-190, 2005.
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsich, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205. /2548, 2022.
Shizhe Diao, Xuechun Li, Yong Lin, Zhichao Huang, and Tong Zhang. Black-box prompt learning for pre-trained language models. arXiv preprint arXiv:2201.08533 2022.
Yoav Freund and Robert E. Schapire. A Dacision-theoreet generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55:119- 1997.
Jerome H Friedman. Greedy function approximation: gradient boosting machine. Annals of statistics, pp. 1892, 2001.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the Ith International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pp. 3816 3816-3830, 2021.
Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. Ptr: Prompt tuning with rules for text classification. arXiv preprint arXiv:2 1259, 2021.
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438, 2020.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cexsing, pp. 3045-3059 2021.
Xiang Lisa Li and Percy Liang. Prefix-tuning Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume Long Pupers), pp. 4582-4599 2021.
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv:2103. 2021.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume Short Papers), 2022.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1907. 1692, 2019.
Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACI pp. 115-124, 2005.

---

Under review as a conference paper at ICLR 2023
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient- free, adit-based in- struction search for prompting large language models. arXiv preprint arXi::2203.07281, 2022. 2022.
Guanghui Qin and Jason Eisner. Learning how to ask: Querying Ims with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5203-5201111 5212, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, al. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., 21 22(1111))))) (140):1-67, 2020.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392. 2016.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC Workshop, 2019.
Timo Schick and Hinrich Schütze. Few-shot text generation with pattern-eplooiting training. arXiv preprint 1rXiv:2012. 1996,, 2020.
Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 255-269, 2021a.
Timo Schick and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2339-2352, 2021b.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222 4235, 2020.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language pro- cessing, pp. 1631-1642, 2013.
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. Black-box tuning for languge-mooiel-ass-serviee In Proceedings of ICML, 2022.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction (APIs). 25th USENIX security symposium (USENIX Security 16), pp. 601-618, 2016.
Ellen M Voorhees and Dawn M Tice. Building question answering test collection. In Proceed- ings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pp. 200-207, 2000.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: multi-task benchmark and analysis platform for natural language understanding. Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 2018
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pp. 1112-1122 1122, 2018.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's 's transformers: State-of tftth--rrtr- natural language processing. arXiv preprint 190.031110 2019.
12

---

Under review as a conference paper at ICLR 2023
Weizhe Yuan, Graham Neubig, and Pengfei Liu. Bartscore: Evaluating generated text text gener- ation. Advances in Neural Information Processing Systems, 11127263--22777 2021.
Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. Differentiable prompt makes pre-trained language models better few-shot learners. In International Conference on Learning Representations, 2021.
Xiang Zhang. Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification, In NIPS, 2015.
13

---

Under review as a conference paper at ICLR 2023
Table The dataset statistics. [ is the number of classes, Avg.#W the average number of words
in the input, and #Train/#Test refer the number of examples in training/testing dataset.
Category
Dataset
[>]
Avg. IW
#Train
#ess
SST-2
MR
20
single sentence
AG's News
47
120000
7600
TREC
5452
SNLI
3
22
549367
9842
MNLI
3
33
392702
9815
sentence pair
QNLI
3
41
104743
5463
RTE
3
59
2490
277
Table 5: Performance of full data training
SST-2
MR
TREC
RTB
Fine-tuning
95.5 (0.4)
91.5 (0.6)
97.2 (0.2)
81.9 (1.1) 71.7 (2.0)
PROMPTBOOSTING
94.1 (0.3)
89.7 (0.4)
90.5 (1.2)
A IMPLEMENTATION DETAILS
Dataset Statistics The dataset statisties can be found in Table For fair comparison, the few- shot training/aliddationneetngn split generation is strictly following the implementation of Gao et al. 2021
Training of baselines For standard fine-tuning, we adopt the Huggingface transformed li- brary (Wolf et al., 2019) to load RoBERTa-large backbone model and use its Trainer for fine- tuning. The learning rate is set to le-5. We use AdamW optimizer as the optimizer and the learning rate linearly decays to O. The training batch size is set to 16 and total training epochs is 100. For Feature-MLP method, we use a three-layer MLP with hidden dimension 100. The learning rate is set to le-3 without learning rate decay. We also train the MLP for 100 epochs. For other baselines, we use their the official implementation with default 2021). DART (Zhang al. (222I). BBT (Sun al. 2022). and RLPrompt (Deng et al. (022)). For RLPrompt, because of its low efficiency, we set its training epochs to 1000 instead of the 12000 used in their paper. This is reasonable since it takes nearly hours for RLPrompt to finish 1000 epochs of optimization.
including LM-BFF (Gao et al.)
B ADDITIONAL EXPERIMENTS
Performance on full dataset The high efficiency of PROMPTBOOSTING makes possible to gen- eralize to medium-sized datasets. We evaluate the performance of PROMPTBOOSTIE on SST-2, MR, TREC, and RTE datasets. We sample 10% of the original training set to construct the validation set and use the original validation set for testing if the labeled test set is unavailable. The experi- ment results can be found in Table PROMPTBOOSTING achieves comparable performance with standard fine-tuning on SST-2 and MR datasets, which is impressive given the fact that PROMPT- BOOSTING has no access to the parameters and gradients of the LM. For TREC dataset, standard fine-tuning outperforms PROMPTBOOSTING, but still remark that the performance is still highly usable in black-box setting. Finally, the gap between tively large on RTE dataset, which is consistent with our previous discovery that it seems pre-trained LMs are not good at sentence pair classification tasks before fine-tuning.
CROMPTBOоSTIO
and fine-tuning is rela-
Effect of training data size For AGNews, TREC, QNLI, and RTE datasets, we shown the perfor- mance of PROMPTBOOR BOOSTING the size of the training set increases in Figure B
14

---

Under review as a conference paper at ICLR 2023
Insaace Per Clasa
Instance Per
Inrarce Per Canas
Indaacce Per Cana
(a) Performance AG (b) Performance TREC (c) Performance QNLI (d) Performance RTE
Figure 3: Model performance function of training set size on different datasets. For QNLI dataset, we also include prompt refinement for better performance.
Table 6: Prompts used by PROMPTBOOSTING on different datasets.
SST-2
MR
[Input] It's [MASK]
[Input] It [MASK]
2
[Tnput]] A [MASK] movia.
[Tnput: Tt [MASK]
3
[Input] ] A [XASK] film.
[Input] A [MASK] piece of wark.
[Input]
[MASK] picca of work.
[Input] It's [MASK]
5
[Input] ] A truly [MASK] film.
[Input] & [MASK] waste of time.
[Input] ]
Thin is [MASK]
[Input] ] a truly [MASK] film.
[Tnput]
[MASK]
[Tnput] thought it
(MASK).
8
[Input] A [MASK] waste of time.
[Input] It'a a juat [MASK]
[Tnput]
[MASK]
[Tnput] A truly [MASK] movie.
10
[Input] A truly [MASK] movie.
[Input] The film is [MASK]
AG's News
TREC
[Input] This entry was posted in [MASX]
[Input] What [MASK]
[Input]] U.S. [MASK]
[Input]
the [MASK]
3
[Input] U.S. [MASK]
[Input] What [MASK]?
[Input This entry was posted in [MASK]
[Input] [MASX]
5
[Input] The [MAS3] Journal reports.
[Input] See [MASK]
[Input] The [MASX] Journal has more.
[Input] ] Which [XASK]
[Input The New York Times [NASK]
[MASK]
[Input] [MASX]?
8
[Input] ]
[Input] Full [MASK]
[Input]
[MASK] Report.
Inpuu]] many [KASK]
10
[Input] Read more [MASK] Insider.
Input]] 1. [MASK]
SNLI
MNLI
1
[Input [MASK] [Input2]
[Input 11
[XASK]
[Input2]
2
[Input]] [MSK]] [Tnput:]
[Input1]
[KASK] but [Tnput:2]
3
[Input]] [HASK] and [Input2]
[Input
[KASK] [Ingut2]
[Input [MASK], but [Irput2]
[Input
[MASK] [Input2]
5
[Tnput [MASK]: [Input2]
[Input
[MASK] But [Input:2]
[Input [MASK] one of [Input2]
[Input 1 ?
[MASK]] [Input2]
[Tnput [MASK] [Tnput2]
[Tnput
[VASK]
[Tnput
8
[Input [MAS]], just [Inpu:2]
[Input [KASK] and [Enput
[Tnput [MASK]
[Inpur2]
[Tnput [MASK]
[Tnput2]
10
[Inptt]] [ASK]; [Input2]
[Input]]
[ASK] [Input
QNLI
RIB
1
[1n
[ASK]] [Input2]
[Input [AAK]] [Ingut2]
[]] [ASK]] but [Irput2]
[Input
[AAK]] [Inout
3
[Input1]
[AAS]] [Input2]
[Input 1]
[[ASK] but [Input
[Input1]2 [MASK] but [Input2]
[Input
[KASK]
[Input2]
5
[Input1]? [MAS]] In fact, [Input2]
[Inputl]
[MASK] [Ingut2]
[Input11 [HASK]; [Input2]
[Inputl]
[KASK], the [Input2]
[] [MAK]] However, [Input2]
[Inputi].
[KASK] [Inout
8
[Input1] [ASK]] and [Input2]
[Input
[XASK] [Imput2]
[Input1] [ASK]] [Input2]
[Inputl].
[KASK] and [_nput2]
10
[Inputl]. [MSK]] [Input2]
[Inputi]. [KASK] but [Input2]
C GENERATED PROMPTS
In this section, we visualize the prompts we used in our experiments for each dataset in Table 6 Regardless of different few-shot training/validation splits, we use the same 10 prompts for model training.
15


Reviews:
[{"review_id": "m3NRpazS3w", "reviewer": "Reviewer_QZLQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes PROMPTBOOSTING, a novel black-box prompt learning approach that does not rely on searching for an optimal prompt and which can thus drastically improve the computational efficiency over the existing method. Specifically, the proposed approach obtains a small pool of prompts via a gradient-free approach and then constructs a large pool of weak learners by pairing these prompts with different elements of the LM’s output distribution. These weak learners are then ensembled using the AdaBoost algorithm. The learning process requires only a small number of forward passes per batch and no backward pass. Experiments show that PROMPTBOOSTING achieves state-of-the-art performance in multiple black-box few-shot classification tasks and matches or outperforms full fine-tuning in both few-shot and standard learning paradigms while training 10x faster than existing black-box methods. Overall, this paper is well-written and well-motivated. ", "summary_of_the_review": "Overall, I think this paper is well-written and well-motivated. Some small issues are the lack of detailed analysis of different prompt templates. I think black-box tuning is a promising solution for foundation models, and this work can help inspire future work.", "strength_and_weaknesses": "Strength：\n\n1.This paper proposes a novel approach that does not rely on searching for an optimal prompt and which can thus drastically improve the computational efficiency over the existing method. In my opinion, the proposed approach is well-written and well-motivated.\n\n2.Comprehensive experimental results show that the proposed approach is efficient and effective.\n\nWeaknesses:\n\nFrom my point of view, I think the prompt set may have a big influence on the performance. As the paper mentioned, \"the aforementioned approach can be replaced with any other optimization-free prompt generation methods,\" so can you provide the performance with other prompt generation methods?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bdQz9_J_OPa", "reviewer": "Reviewer_cHte", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper deals with the problem of prompt-style finetuning when gradient information is not available. The solution is using adaboost to quickly learn prompt-style learners. Experimental results on standard benchmark showed competitive results over previous black-box methods.", "summary_of_the_review": "Overall I like the research problem and the method proposed. Not very difficult or sophisticated solution but sounds working. I hope authors can add further studies to validate the root cause. Disclaimer is that I am not particularly familiar with fine-tuning under black-box methods. I do study black-box methods under other setups but not sure if this paper covers all recent works, and if recent works covered all possible black-box methods.", "strength_and_weaknesses": "- Strength\n 1. Clearly written, well motivated\n 2. Competitive results\n 3. Reasonable methods\n\n-Weakness\n1. I do believe the method will work but somehow I don't see a further discussion on the best scheme when boosting is used.\n2. not so many black-box method is deployed in this direction, and there is no thorough discussion of the potential applicability of previous black-box methods. That is, I believe zero-order method has a broader literature but this paper didn't connect with that well.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Frm57yAMh-", "reviewer": "Reviewer_QXXp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThis paper proposes a prompting-based method for building a text classifier from language models (LMs), which is called PromptBoosting. The idea of the proposed method is to first obtain a small pool of prompts via gradient-free approach, and then contract a large pool of weak learners by pairing the prompts with different elements of the LM’s output distribution. Then the adaboost algorithm is applied to ensemble the weak learners. Empirical studies claim that PromptBoosting can achieve state-of-the-art performance in different few-shot classification tasks while being 10x faster than existing black-box methods.", "summary_of_the_review": "The paper applies the adaboost algorithm to the existing prompting methods to get the better text classifier with pretrained LM. While the experimental results look promising, the lack of novelty and comprehensiveness of experiments are two concerns.", "strength_and_weaknesses": "Strength:\n\n1. The method is very intuitive in applying the boosting idea to the prompt-based methods such as Gao et al. (2021). Also, the paper is well written. So in general, this paper is easy to understand.\n2. Learning the verbalizer has a closed form solution, which makes it very efficient.\n\nWeaknesses:\n\n1. The novelty of the paper seems slim. It seems like a direct application of adaboost on the existing prompting method Gao et al. (2021).\n2. The evaluation is not convincing. \n    2.1. First of all, according to the implementation, this method can only be applied to text classification tasks, making it a bit narrow. \n    2.2. Besides, while there a quite a few text classification tasks, the authors only select 8 without giving the rational why they are selected. In contrast, Gao et at. (2021) evaluates their method on 16 tasks, mostly from (Super)GLUE. I suggest  the author at least extend the evaluation on more representative tasks to show the effectiveness is indeed universal.\n    2.3. Finally, there should be some simple baselines to compare with, e.g., prompt ensemble with majority vote. \u0010But they are never shown up in the experiments.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "56lovpgLsrY", "reviewer": "Reviewer_jDoU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed PromptBoosting, which works on black box setting for text classification with LLM and prompt. The idea is to build a set of weak learners and each of them is associated with a prompt (raw text) and the final model is trained with adaboost over these weak learners. The resulting weak learners achieve state-of-the-art performance in multiple black-box few-shot text classification tasks, and matches or even outperforms full fine-tuning in both few-shot and standard learning.", "summary_of_the_review": "The paper is well written and easy to read. The idea is simple and makes sense. There are lots of experiment provided in the paper showing good performance of the proposed method for text classification.  However I am mostly concerned about the limit of the scope of the work (limited to text classification only).", "strength_and_weaknesses": "Strength:\n\n1: Black box setting for LLM with prompt is an interesting direction and would have lots of applications, especially for the cases the LLM is too large.\n\n2: Paper is well written and the proposed idea makes sense. \n\n3: Comparing with many methods and datasets, and showing better results than other methods.\n\nWeakness:\n\n1: The scope of the work seems limited. This work seems can only be used for text classification (due to the Verbalizer step which needs the end task to be classification task only). Therefore a large amount of use cases for LLM with few shot learning settings such as summarization, etc cannot be directly applied with the proposed method.\n\n2: Missing some ablation studies. Such as: the prompts sets are generated by optimization-free method proposed by Gao et al. (2021). How about using other prompt sets, maybe just original training examples as a simple baseline? And the adaboost is used for ensemble weak learners, how about use other ensemble methods?  \n\n3: Some of the experimental results need further explanation.\n 3.1 Is the training time (wall time) including prompt set generating time? Or just the time for training weak learners? Could you provide time for each step of the proposed method?\n\n3.2 In Table 2, DART and LM-BFF need lots of backward and forward passes of the model, why the training time is close to the proposed method? Does it mean the forward and backward time is not the bottleneck? \n\n3.3. And also finetuning results are almost always worse than the proposed method (and DART and LM-BFF), which is beyond my expectation. It is better to have some explanation for that.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject