Prompt Exploration with Prompt Regression
Authors: Michael Feffer
Ronald Xu
Yuekai Sun
Mikhail Yurochkin
Venue: COLM (2024)
Keywords: prompt engineering, prompt selection, large language models, regression
TL;DR: We introduce a regression method to predict effects of prompt combinations as well as select effective prompts based on the regression model accordingly.

Abstract: In the advent of democratized usage of large language models (LLMs), there is a growing desire to systematize LLM prompt creation and selection processes beyond iterative trial-and-error. Prior works majorly focus on searching the space of prompts without accounting for relations between prompt variations. Here we propose a framework, Prompt Exploration with Prompt Regression (PEPR), to predict the effect of prompt combinations given results for individual prompt elements as well as a simple method to select an effective prompt for a given use-case. We evaluate our approach with open-source LLMs of different sizes on several different tasks.

Published as a conference paper at COLM 2024
Prompt Exploration with Prompt Regression
Michael Feffer *+
Ronald Xu
Software and Societal Systems Department Carnegie Mellon University
EECS Department Massachusetts Institute of Technology MIT-IBM Watson Al Lab
mfeffer@andres www.ccmu.cmmmm edu
ronaldxu@mit edu
Yuekai Sun
Mikhail Yurochkin
Department of Statistics University of Michigan quekai@umicht edu
IBM Research
MIT-IBM Watson Al Lab mikhail. her.aall.yuocchiiniiii com
Abstract
In the advent of democratized usage of large language models (LLMs), there is growing desire to systematize LLM prompt creation and selection processes beyond iterative trial-and-error. Prior works majorly focus on searching the space of prompts without accounting for relations between prompt variations. Here we propose a framework, Prompt Exploration with Prompt Regression (PEPR), to predict the effect of prompt combinations given results for individual prompt elements as well as simple method to select effective prompt for given use-case. We evaluate our approach with open-source LLMs of different sizes on several different tasks.
1 Introduction
Large language models (LLMs) have captured the public's attention and imagination over the past couple of years and poised to impact various sectors and industries going forward. By prompting and training LLMs in certain ways, researchers, practitioners, and end-users have adapted them to specific problems. This said, success has varied due to randomness quintessential to LLMs. Thus, despite growing interest in prompt engineering, processes involved deriving prompts differ little from iterative trial-and-error.
We focus on a specific type of prompt engineering problem. Namely, given an LLM, dataset of inputs, and prompt library composed of individual prompt elements that can be chained together, our goal is to predict how element combinations affect LLM outputs and use these predictions derive an optimal prompt for the given task. Though this problem does not involve searching over an entire language space, the solution space still grows exponentially with the number of prompt library elements, rendering brute-force approaches practically intractable. Therefore, to address this prompt library search problem, we propose our method, Prompt Exploration with Prompt Regression (PEPR).
Using PEPR involves three steps that are outlined in Figure 1, and PEPR itself comprises two procedures. After building prompt library for the given task, the prompt regression part of PEPR derives parameter weights for each prompt library element based on how much it affects LLM outputs. Using these weights, the next part of PEPR, prompt selection, chooses prompt elements relative to desired behavior. Finally, the overall prompt is recovered after this prompt selection step. Both the prompt regression and prompt selection parts can leverage either reference text generations human-labeled preference information to yield prompts in line with the provided data.
Work performed while doing an internship at IBM Research.
*hheee authors contributed equally.
1

---

Published as a conference paper at COLM 2024
2. PEPR
1. Build Prompt Library
You unpert matl umallcs..
2. Only ase jnur on neva.
A 0.75
You are enroppo mudiemutios.
3. Reepodd altbeeets.
Onty emot yo snswer.
m-0
On yunr erton inyourannens
(Teepond giberihh.
Theppons with liberte..
Araaee quaedione muthuuls..
muthitulty.
4. 03635
Thintarpp by syrthp.
The step stop..
3. Get prompt
You 5. Think step shep
A. Derive prompt regressiiun
Scloct elcmonts based regression desired behavior
Figure 1: Overview of PEPR utilization. After building prompt library, the prompt regression step of PEPR uses the prompt library in conjunction with reference text or preference information to determine the influence of each prompt element overall output. promptselectionn
The second step, prompt selection, uses the parameters derived from prompt regression and data corresponding to desired behavior to select prompt elements in line with this behavior. Finally, the overall prompt is recovered from prompt selection.
Overall, our contributions as follows:
To our knowledge, we are (or are among) the first to work on this prompt library search problem, which we define more rigorously in the following sections.
We provide mathematical definitions and formulae for the prompt regression and prompt selection components of PEPR.
We validate both PEPR components using several open-source LLMs across variety of different datasets and tasks, and
We outline clear next steps for future work in this area.
2 Related Work
Prompt engineering We draw inspiration from the work of Prasad et al. (2023), who note that not all parts of prompt may be helpful for obtaining good LLM output. As such, they split prompts into parts and iteratively search for the most effective prompts through substituting, adding, and removing these parts. Our approach is similar in that search for the best way to combine elements from a prompt library given a prompt regression model, but we do not consider substitution or addition operations. We also discuss our findings with those of Sclar et al. (2023) and Zhao et al. (2021) in mind, as they discover that minute prompt formatting like spacing and punctuation can have nontrivial effects on LLM performance. While we do not explore differences at such granular levels, we are concerned with obtaining the best prompt for a given task by building from set of prompt elements, which we argue is similar problem and note their takeaways accordingly. Other prior work considers prompt engineering in the form of iterating on in-context learning examples (e.g., Liu et al. (2022)), prompt template (e.g,, Jiang et al. (2020)), keywords (e.g., Shin et al. (2020)), and task description (e.g., Zhang et al. (2023)).
a recent paper, Shi et al. (2024) consider the problem of general prompt iteration with limited resources by framing the issue of choosing a prompt from given set as a special case of a multi-armed bandit problem. While we also note resource consumption and utilize fixed set of potential prompt elements in our work, we diverge from their findings and contributions in that PEPR explores how these elements interact with each other and predicts the effects of adding or removing elements from the prompt on resulting performance without the need to evaluate every prompt variation. In this sense, we conjecture we are among the first to work on this and related problems.
Language model alignment Our work builds on-top of foundational LLM research explor- ing reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022), in-context learning (Brown et al., 2020), and Constitutional AI (Bai et al., 2022). Namely, our prompt regression and optimization processes for preference data are inspired by preference learn- ing approaches found in the work of Ouyang et al. (2022). Moreover, our application of
2

---

Published as a conference paper at COLM 2024
the Brarley-Terry (BT) model in (3.4) is similar to the application in the direct preference optimization (DPO) work of Rafailov et al. (2023), but we note that we diverge from their work in that our score/ score/reward function differs from theirs (as they additionally incorporate log-probabilities of the unprompted base model in their formulation) and furthermore they perform LLM training based on their BT model application whereas we perform regression and optimization over a prrmpt library. Regarding the work of Brown et al. (2020) with in-context learning and Bai et al. (2022) with Constitutional AI, our prompt regression and optimization methods should work with prompt libraries composed of in-context learning examples or LLM principles. Our work is therefore also similar that of Sun et al. (2023) who introduce pipeline for fine-tuning and aligning LLMs to principles, but we again note that we diverge from these works that we perform no LLM training but rather build an effective prompt from a library. Thus, our work also extends that of Liu et al. (2022) who note that not all in-context learning examples are equally effective. a similar fashion, our method aims to build an effective prompt from prompt library whose elements may differ in both the type and magnitude of effect they have on the LLM in question.
3 Prompt Regression
Here, we detail the theory and assumptions behind prompt regression, the first of two parts in our approach prompt optimization, and illustrate our empirical results.
3.1 Prompt Regression Methodology
Consider LLM π and library of prompts s (pi....,,PP Pk) whose prompt elements Px each steer the LLM in certain ways (e.g., in Constitutional Al (Bai et al., 2022) or via in-context learning (Brown et al., 2020)). Our main goal is be able to easily predict the effect of an arbitrary combination of prompts from the library on the model's behavior.
Prompt regression for log-probability data Let I {4 E {0,1)} where 4 = =1 implies that element Px is included in the prompt s(I). We posit that the tog-probability of response given input x and prompt s(Z) ) is a convex combination of the log-probabilities, or logprobs, of response y given x and elements px I:
log ( (s))))))) R
Σ k:4
(Z) log Tly (Pxx
(3.1)
where A(I) E - is a set of weights corresponding to each element.' 1 To model these weights efficiently, we assume that the elimination or addition of an element from the prompt library does not affect interactions between the other prompts. This assumption motivated by social choice theory, where is known as the independence of irrelevant alternatioes (Ray, 1973). Our model of the weights is thus
Ak
(I)
(3.2)
EmcI Am'
where ==
= AR(IIm 1] 1), iee, the weight of the prompt element when prompting with the complete prompt library s. We learn { via a simple constraint regression:
log π (s,x;)) - Σ 140 2
min AEAR-1
5f = --logre 7yyi (:)x
(3.3)
Note that prompt elements can also be defined to depend on the input provided to the LLM (i.e,, Ph (x)) Our approaches remain the same mathematically, and we consider such prompts in some of our experiments (eg., with CAMEL Biology and Physics datasets) show technical feasibility.
3

---

Published as a conference paper at COLM 2024
We note two key advantages of our method:
1. (3.1) allows us to estimate the effect of any of the 2 prompt combinations while only evaluating K + prompts for fitting A;
2. solving (3.3) does not require knowledge of the reference (correct) for the inputs {xi) (egg,, when there is a finite set of meaningful generations, such as classifica- tion or multiple-choice QA, one can simply plug every possible y for every input xj when fitting the regression coefficients).
We denote this method as PEPR-R as it utilizes reference (log-probability) data.
Prompt regression for preference data Aligning LLMs to human preferences is often based on preference data, egg,, RLHF (Ouyang al., 2022), where annotators choose preferred response from several options. Classification and multiple-choice QA can also be viewed as preference data where the correct answer is preferred over others. We demonstrate how our method can be used to automate prompt engineering using preference data.
Let (xx,y) リ/ be dataset of inputs x; and potential LLM responses y and y?, where y is preferred over u?: We adopt Bradley-Terry (BT) model (Bradley, 1984):
P(11 2 1/2 x; xt,ss
+ exp ((olooo(()))) (yy2 (s(I), - log ( (s(I) (3.4)
where indicates preference, B 0 is temperature parameter, and our score reward function is log-probability of the associated response, Instead of performing constraint regression to minimize the difference between estimated and actual log- nog--robbbiiities, we can substitute log (y; (s, x;)) with
log (1 ł y/2 sZ), x log (y)) (s,x;)) log ( (s,x;)),
the approximate log-likelihood of desired preferences and do the same with o% = log Aty (P),::)). The resulting objective function is
log ( (s,x)) log (y? (5,x;)) E =
min ACAK-1
8 = log T(y (pk, xi)x log 7ly? (pk, (3.5)
The resulting prompt regression model is therefore
log (ly (s) - log Tly (s(
Σ Ax (og (s(Z), log (s()),x))), (3.6)
k=1==11
Similarly to regressing on log-probabilitie, we do require knowledge of which of y and y preferred with preference data. This allows us to use additional "unlabeled" data to learn A. We name this method PEPR-P as it uses preference (log- dog-probability difference) data.
3.2 Prompt Regression Experiments
We conducted several experiments to test the independence of irrelevant alternatives as- sumption by evaluating both versions of PEPR in their ability to predict prompt effects using four different types of datasets and several open-source LLMs.
2Th total number of terms in (3.3) is nC, where C is the number of possible outputs for an input. 'The generalized version the BT model, the Plackett-Luce (PL) model (Plackett, 1975; Luce, 1959), can be used to model preference data with more than two responses.
"The approximation is accurate assuming larger
4

---

Published as a conference paper at COLM 2024
Toy Dataset We sample 100 prompts from the databricks-dolly--ee dataset (Conover et al., 2023) and embed a subset of prompt library elements in the system prompt of evaluated LLMs to generate responses like those of pirate. We then apply PEPR with the full library to build a system prompt aligned with pirate responses.
HateCheck We use test set data from the HateCheck dataset (Röttger et al., 2020) to validate our method. The creators of this dataset note that while hate speech detectors typically do well with overt hate speech, they perform poorly in nuanced scenarios ge reclaimed slurs, hate speech counters). To this end, we crafted a prompt library to address these limitations and applied PEPR to build system prompts to respond to examples with content. Note that we run prompt selection once per data
hateful or non-hate depending class for each of the 11 classes present in the dataset (i.e., we subset data and build a system prompt for that class accordingly, but we regress with all data regardless of class).
CAMEL We leverage the Biology and Physics datasets generated from the CAMEL exper- iments (Li al., 2023) to test the method on text generation. We sample 100 points from each dataset and apply PEPR to uncover the prompts that generate the best outputs from LLMs in line with those of subject matter experts.
Natural Instructions We data corresponding to several tasks from the Natural Instruc- tions (NT) dataset (Mishra et al., 2022). Specifically, we randomly select 100 data points each from tasks 020 (recognize if the answer to a question is in some context), 195 (sentiment analysis), and 199 (determine if sentences agree with one another) and apply PEPR to build prompts that produce aligned outputs from LLMs.
Models For all experiments, we evaluated our approach using models from the Llama- 2-chat family of LLMs (Touvron et al., 2023). As their instruction following and system prompt features lend well to our goal of building the best prompts for particular use-cases, we opt for the chat versions of these language models over their base model variants.
For all models and datasets, used prompts and obtained logprobs and promptt created from our prompt library as system differences associated with desired and un- desried generations based on these prompts. After obtaining results from individual prompt elements and the entire prompt library (fed as a single prompt), we performed regression and estimated the behavior of prompts of size 2, 3, and 4. We then compared the predicted results to the behavior of the -round-truth prompt combinations (of the same sizes and element compositions). Note that for datasets with open-ended text responses (such as some of the NI tasks and our Toy Dataset), we work with average logprobs across response tokens (i.e., log- probbbbbayyyty of response divided by number of tokens) as well as their differences.
Results Figure 2 shows results of subset of prompt regression experiments, namely results from the NI Task 195 and HateCheck datasets (see Appendix B for plots from other regression experiments). Specifically, for each model-dataset-PEPt configuration, we produced rredicted-versus-true scatterplots corresponding to prompt behavior predictions and ground-trutt values in addition to MAE and correlation after grouping results by number of prompt elements. Evidently, prompt regression performance worsens for larger models (in terms of lower correlation and increased MAE) regardless of PEPR type for NI Task 195, but it improves with increased model size (based on the same metrics) for both types for HateCheck. As a result, there are no clear trends across model size for either of the PEPR methods. This said, PEPR-P outperforms PEPR-R as correlation values are the same or higher in all cases except for one (HateCheck with Llama-2-7B-chat), Overall, both PEPR-P and PEPR-R have high correlation and low ertor, in turn suggesting our assumption of the independence of irrelevant alternatives is justified (at least in most cases). 6
"Though PEPR-P induces larger MAF than PEPR-R, we neither recommend nor discuss this comparison as error is semantically different across the two methods (i.e,, error in predicting log- probability is categorically different from error in predicting log-probability difference). 'If interactions or coupling between prompt elements were to have major impact, a linear model would not do well here.
5

---

Published as a conference paper at COLM 2024
Dataset
NI Task 195
HateCheck
PEPR-R
PEPR-P
PEPR-R
PEPR-P
7B
Farrest 231 n0l dec 2311
Jame 281 LEER RIAA mat
Syme U. SRI
13B
pore Mt Wen juar 21
Tomer mart
70B
Figure 2: Predicted-versus-truu value plots corresponding to prompt regression experiments two datasets, NI Task 195 and HateCheck. These in turn illustrate the ability of our prompt regression model to predict LLM outputs when prompt elements and corresponding coefficients are marginalized out of the model (e.g,, for regression model reflecting the effects of 10 prompts, we aim to illustrate how its predictions of the prompt with elements and 5 compare to ground-truth outputs). While there appear to be no clear trends across model size, PEPR-P appears to do the same or better than PEPR-R in most cases. Addition- ally, both versions of PEPR have low error and high correlation in general, suggesting that our assumption of the independence of irrelevant alternatives typically holds.
Table 1: Overview of differences between prompt regression and prompt selection. Prompt regression does not require knowledge of ground-truth, whereas prompt selection does to build an appropriate prompt. PEPR-R formulae are used here; replacing logprobs with differences yields an equivalent table for PEPR-P.
Prompt Regression
Prompt Selection
Input
nt, {xi], Pi...., Pxi possible generations of for x,s Learn dg s.t. for all
n,, (x:), Pi...., correct generation yj for x;s Learn 4 s.t.
Process
Σ As loggognlgg z(y] X Pk) z log T(yl (x Pk)
max Σ Elogn(y: ( (s(I,x;))
7
Σ A log mly Pt) z log PI.....
Output Regression model parameters Aç
Prompt element selectors k
4 Prompt Selection
Here, we show how PEPR can be used for prompt engineering by describing the second part of our approach: prompt selection (see Table 1 for differences from regression).
6

---

Published as a conference paper at COLM 2024
4.1 Prompt Selection Methodology
Just as prompt regression can leverage either reference generations or preference data to predict LLM outputs, so too can prompt selection to choose an effective subset of prompt elements from the library. We describe each version of selection accordingly.
Prompt selection via log-probability data prompt regression Let (i) be a dataset of inputs and corresponding reference outputs. To select the prompt maximizing the likeli- hood of reference output generations, we apply our model from (3.1) and solve
max Σ log (y: (s(I, - max 1 I
EAAA Ex444
(4.1)
I I 1
where Rx = E and 8 = ( (pk, x;)) as defined in (3.3). This is an instance of an integer program which are typically hard to solve, but, fortunately, we can cast it as simple linear program by relaxing 4 [0,1] and applying the Charnes-Cooper transformation for finear-fractinna programming (Charnes & Cooper, 1962). The resulting solution is guaranteed be on the boundary of the feasible set, i.e, recovering binary 4 E {0, 1} prompt selection variables. See Appendix A for details.
Prompt selection via preference data prompt regression Now suppose we have a dataset (xx))),)))) similar the prior one except that instead of reference outputs y,, we have preference data in the form of potential outputs v for each input x; such that y > y/- Our goal is to find prompt maximizing the approximate log-likelihood of desired preferences using the prompt regression model corresponding to preferences (3.6):
max E log Ply! ž y xx;,,((])} 7 max ΣΣ (x) m(y) (pxxx:)) - log n(S? (p)x
This is the same as (4.1) substituting of = log (y! (;k, log (y, (p),
4.2 Prompt Selection Experiments
To evaluate our approach, we run both versions of PEPR against each dataset to select prompts from prompt libraries and compute performance metrics to assess effectiveness relevant to the given task. For our Toy Dataset, our performance metric is accuracy, i.e., whether the model is more likely to generate the desired pirate output (for each instance, 1 if so, 0 otherwise). For HateCheck sections and Natural Instructions tasks, metric is top-1 accuracy, i.e,, whether the prompted model exhibits correct behavior based on the given input. For the CAMEL datasets, we utilize BERTScore (Zhang et al., 2020) to measure generation quality. The CAMEL datasets do not have preferences, thus only PEPR-R is applicable. We utilize the same models, datasets, and logprobs described in Section 3.2 with the caveat that the optimizing data employed corresponds to desired outputs (in the form of reference generations or known preferences depending on the version of PEPR evaluated). In each setting, we search for the best prompt of at most four elements.
Our baselines for these experiments come in the form of a minimally--rompteer model (the LLM with basic instructions relative to the task such as, "You are a hate speech detector" (see Appendix B.1) and prompts generated from our libraries at random. For the random prompt sampling baseline we match the evaluation budget of PEPR (see Appendix A for details). Moreover, we run each experiment repeatedly (typically around 1000 times), varying the amount of data used for prompt selection, while reporting the performance on all of the available data. Note that for PEPR-R prompt selection, our reference data are only logprobs corresponding to desired responses (as the likelihood of those responses is what aim to maximize) unlike the case of PEPR-R regression (for which we use data from all responses as the type of response does not matter).
7

---

Published as a conference paper at COLM 2024
Table 2: Subset of selection experiment results. In addition to results from PEPR and random baseline, we also include maximum and 75th percentile results from all relevant prompt combinations (as prompts of at most 4 elements are explored by our method, we report maximum accuracy and 75th percentile accuracy based on all prompts of at most 4 elements). Our selection results echo our findings from our regression experiments in that PEPR-P tends to outperform PEPR-R in most cases, and there are no clear trends between model size and PEPR effectiveness. Even SO, overall, PEPR ties or exceeds baselines and 75th percentile results in many cases, See Appendix B for standard deviations and full experimental results.
Labeled Portion Melhaa
Dataset
Model
0.25
0.5
Base 0.75
Romd
PEPR-R
PEPR-P Ramí
PEPR-R PEPR-1
Rund
PEPR-R
PEPR-P
Romi
PEPR-R
PEPR-P
0.52
0.41
0.53
0.52
0.41
0.53
0.52
0.41
0.53
0.52
0.41
0.53
0.19
0.37 0.55
Toy Dataset
13B
0.56
0.56
0.58
0.56
0.57
0.58
0.56
0.57
0.58
0.56
0.57
0.58
0.16
0.40 0.61
206
0.95
0.97
0.97
0.95
0.97
0.97
0.95
0.97
0.97
0.95
0.97
0.97
0.16
0.67 0:98
7B
0.73
0.68
0.69
0.73
0.67
0.68
0.73
0.67
0.68
0.73
0.67
0.68
0.65
0.69 0.76
HateCheck (Slur)
13B
0.50
0.74
0.80
0.80
0.75
0.81
0.80
0.75
0.81
0.80
0.25
0.82
0.80
0.75 0.83
0.90
0.90
0.90
7B
0.55
0.67
0.55
0.68
0.55
0.68
0.55
0.68
0.60
0.67 0.69
Biology
136
0.61
0.57
0.57
0.68
70B
0.49
0.68
0:48
0.68
0.68
0.68
0:47
0.68
0.60 0.68 0.69
0.58
0.58
0.54
0.58
Physics
138
0.58
0.52
0.58
0.53
0.57
0.53
0.57
0.53
0.51 0.65 0.66
706
0.50
0.66
0.51
0.66
0.50
0.66
0.50
0.66
0.61 0.63 0.66
7B
0.66
0.64
0.64
0.73
0.72
0.73
0.74
0.74
0.74
0.74
0.74
0.74
0.57
0.74 0.76
NI 020
13B
0.69
0.71
0.71
0.74
0.77
0.77
0.75
0.76
0.78
0.76
0.77
0.78
0.64
0.74
0.78
206
0.68
0.67
0.74
0.75
0.74
0.76
0.74
0.75
0.52
0.74
7B
0.66
0.60
0.55
0.72
0.65
0.56
0.73
0.67
0.56
0.74
0.67
0.56
0.56
0.57
0.85
19
136
0.58
0.64
0.65
0.62
0.56
70B
0.71
0.76
0.75
0.77
0.79
0.80
0.79
0.80
0.81
0.79
0.81
0.81
0.56
0.71 0.84
0.75
199
136
0.72
0.76
0.76
0.76
0.76
0.76
0.77
0.77
0.77
0.77
0.77
0.77
0.39
0.77 0.77
706
0.69
0.69
0.70
0.76
0.76
0.76
0.77
0.76
0.76
0.77
0,76
0.76
0.59 0,77 0.78
Results Table 2 displays results from a subset of experiments (see Appendix for full results).? Overall, PEPR prompts perform well, frequently scoring above the 75th percentile of all prompts and sometimes reaching the maximum performance possible. We observe that PEPR-P typically ties or scores better than PEPR-R and conjecture this is due to PEPR-P taking logprobs of undesired responses into account (via the logprob differences) alongside those of desired responses when performing prompt selection, in contrast to PEPR-R which responses. The better performance of PEPR-P is similar to our higher correlation finding in our prior regression experiments, and additionally we again note no clear PEPR performance trends with model size in our prompt selection experiments. In many cases, PEPR can achieve the best or worst results with Llama-2-13B- chat, evidencing that increases in scale may not always help. 8
only considers logprobs of desired responses
In addition, we find that PEPR can outperform baselines even when the amount of labeled data (i.e,, inputs where either reference or preference outputs are available) used for prompt selection is small. Note that "0.05" corresponds to about 5 labeled points in most experi- ments, i.e., a few-shot scenario where prompt selection is more appealing than fine-tuning, This low labeled data setting is arguably a more likely application of methods for prompt selection. Moreover, PEPR's prompt selection performance in this small data regime is nearly the same as its prompt selection performance with all of the data for the datasets we explore, additionally suggesting that PEPR is effective in this domain. 9 Furthermore, PEPR outperforms the random baseline by a large margin on the CAMEL Biology and Physics generation tasks in terms of BERTScore. BERTScore compares similarity of generated and
As the performance of PEPR-chosen prompts is stable, the standard deviations are small, so we omit them here but report them in Appendix with our full results.
$Even so, the 7B parameter model never obtains results strictly greater than the 13B and 70B parameter ones, suggesting that scale may help sometimes.
Recall that the prompt regression part of PEPR still requires all data available, but it is unsuper- vised, i.e. does not require reference generations or knowledge of preferred generation.
8

---

Published as a conference paper at COLM 2024
reference text, and is thus more sensitive to the prompt, while accuracy only takes into account predictions over a single next token, which we observed to be less sensitive to prompts in our experiments.
In the Appendix Table 5, we additionally compare to TRIPLE-SH (Shi et al., 2024), a recent algorithm for prompt selection adapting the best arm identification strategy, i.e., it evaluates a few samples with each prompt recursively, eliminating the worst performing prompts at every stage. This algorithm requires evaluating at least single sample per prompt candidate, which is infeasible in some settings (assuming an evaluation budget comparable to PEPR) since the number of prompt candidates is of the order of 2 for prompt library of K elements. PEPR bypasses this limitation due to its ability to model the effects of prompt element combinations via prompt regression without the need to evaluate all of the possible prompts. Given higher evaluation budget, TRIPLE-SH and PEPR perform comparably.
Discussion
Key takeaways PEPR illustrates that given a fixed set of prompt elements, it is not neces- sary to evaluate every possible prompt to learn its effect. Namely, related prompts (such as ones that share prompt elements) have related effects (in terms of changes in LLM performance and output) that can be measured and predicted efficiently, eliminating the need to try every prompt. This is supported by the low error and high correlation of our prompt selection experiment results in Section 3.2 and in Appendix B. These findings in turn alleviate the need for a brute-force search that grows exponentially with library size.
Limitations Upon closer scrutiny of our prompt selection results in Table 2 and Ap- pendix B, one may observe that our random baseline ties or outperforms both versions of PEPR in nontrivial number of instances. We acknowledge this, but we also note that many of the cases in which our methods outperform random combinations are ones in which the prompt library has some unhelpful elements (egg, Toy Dataset, Biology) and in contrast, at least a few cases in which our methods do not outperform random combinations are ones in which the prompt library has many helpful elements (e.g., HateCheck). This suggests that PEPR is effective at filtering out bad prompt elements (such that it can easily find good prompt) and not as effective at optimally combining good prompt elements (meaning it cannot always find the best prompt). We also argue that the regression and optimization framework provided by PEPR more interpretable in addition to being more robust than trying prompt combinations at random. Even so, we emphasize that the prompt selection part of PEPR can benefit from future research and that our prompt regression results are interesting standalone contribution. In addition, both parts of PEPR are straightforward yet effective solutions to this prompt library search problem.
Regarding the prompt library, another potential limitation of our work is our findings are only limited to the prompt libraries we tried for each dataset. Given the formatting issues discovered by Sclar et al. (2023) and Zhao et al. (2021) and prompt iteration operations employed by Prasad et al. (2023) to find effective prompts, one could argue that we may not have found optimal prompts for each dataset because our library was missing (more) useful elements and formatting. In response, we note that our experiments sought to test PEPR against each dataset and use-case, not to achieve state-of-the-art results. As such, we argue that the prompt libraries we utilize here are sufficient as our baselines also leveraged them where appropriate to enable comparisons with PEPR. Furthermore, these libraries were obtained through preliminary experiments and manual refinement not detailed in this work, and the high performance in general of all methods that used them (baselines and PEPR alike) suggests that the libraries at least contain helpful instructions.
Future Work We encourage the community to conduct further research and experiments to address the limitations above and explore related approaches. For instance, we note that future experiments could examine the effects of other prompt selection approaches (i.e., alternative bog-orobayylityy maximization strategies), more complex prompts (with elements like "ignore previous instructions" or similar), and more sophisticated prompt regression models (e.g,, ones with richer features beyond probbbbaasisities and differences and/on or ones

---

Published as a conference paper at COLM 2024
that are nonlinear). Testing whether prompt element ordering has any impacts on final outcomes and observing whether PEPR easily scales with larger prompt libraries would be other useful experiments. Moreover, while we only explore system prompt configurations in this work, experimenting with PEPR using other prompt components (e.g,, in-context learning examples, text not part of the system prompt) would be yet another interesting follow-up work. Lastly, while we selected datasets to cover a range of topics pertaining to model safety, algorithmic bias, and everyday usage of LMs, future work could examine more datasets related to these and other topics.
6 Ethics Statement
While our work does not directly deal with ethical issues and usage of LLMs, we acknowl- edge that PEPR can be utilized for harmful purposes. Namely, as our experiments illustrate how PEPR can discover effective prompts for variety of harmless and beneficial tasks (such as pirate speech generation, hate speech detection, and knowledge retrieval), they also suggest that PEPR can be used to derive prompts for malicious uses (such as hate speech or misinformation generation). This said, PEPR would require both prompt libraries and data corresponding to such malicious uses and therefore does not come out-of-the-box with harmful capabilities, but we nevertheless highlight the potential risks here and stress that researchers and practitioners should promote responsible usage of our work.
References
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellittó, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy
Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodss, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional AI: Harmlessness from AI Feedback, December 2022.
Ralph A. Bradley. 14 paired comparisons: Some basic procedures and examples. In Nonparametric Methods, volume 4 of Handbook of Statistics, pp. 299-326. Elsevier, 1984. doi: https:/ / http://oo..rgggg otreioog// 10.1016 /tt///00111110006711100000000000 606166671111114400016666 URL https: //www. sciencedirect.c.. com/
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. ar xrXiv:2005.14 14165 [cs], June 2020.
Abraham Charnes and William W Cooper. Programming with linear fractional functionals. Naval Research Logistics Quarterly, (3344):111-186 1962.
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's 's first truly open instruction-tuned 11m, 2023. URL https: //www, www.wdaaabbrkkk.c 04/12//6lll--iirst-ppp--- wwwp:pppeeoooomeeaaaaaaaaaalaaaa weeeeeiinsttuttiinoeee tuned- 11m.
http://ooo///222
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023.
10

---

Published as a conference paper at COLM 2024
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics (TACL), 8:423-438 2020.
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Communicative agents for "mind" exploration of large language model society. In Advances in Neural Information Processing Systems, 2023.
Ghanem. Camel:
Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-114, 2022.
R Duncan Luce. Individual choice behavior. John Wiley, 1959.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL, 2022.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models follow instructions with human feedback. In Advances in Neural Information Processing Systems, October 2022.
R. L.. Plackett. The Analysis of Permutations. Journal of the Royal Statistical Society: Series (Applied Statistics), 24(2):193-202 1975. ISSN 1467-9876. doi: 10.2307 00660077234667..
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient-free, edit- based instruction search for prompting large language models. In European Chapter of the Association for Computational Linguistics (EACL), pp. 3827-3846, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model, May 2023.
Paramesh Ray. Independence of irrelevant alternatives. Econometrica: Journal of the Econo- metric Society, pp. 987-991, 1973.
Paul Röttger, Bertram Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. Hatecheck: Functional tests for hate speech detection models. arxiv preprint ar xix:20022.15600 2020.
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How learned to start worrying about prompt formatting. arXiv preprint ar arXio:2310.1133 2023.
Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen. Efficient prompt optimization through the lens of best arm identification. ar preprint ar 2rXio:2402.09723, 2024.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Au- toprompt: Eliciting knowledge from language models with automatically generated prompts. In Empirical Methods in Natural Language Processing (EMNLP), pp. 2020.
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language models from scratch with minimal human supervision. arXiv preprint ar arXiv:2305 03047, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama Open foundation and fine-tuned chat models. Xrv preprint arXi::230 (tto23077.0988, 09288, 2023.
11

---

Published as a conference paper at COLM 2024
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations (ICLR), 2020. URL https:///ppnnreview net/Forummnd--SeetcVVEr
Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, and Meng Jiang. Auto-instruct: Automatic instruction generation and ranking for black-box language models. In Empirical Methods in Natural Language Processing (EMNLP), pp. 9850-9867, 2023.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance language models. In International Conference on Machine Learning (ICML), pp. 12697-12706. PMLR, 2021.
12

---

Published as a conference paper at COLM 2024
A Prompt Selection Program Details
Based on (4.1), our prompt selection binary integer program has the following optimization function:
Re Ix AA
max
where 4 E {0,1} and - In our setup, we specify an additional constraint on 4 namely that we force of them to be equal to 1. Failing to do so yields an algorithm that greedily chooses the single prompt element with best performance over the given data, but such an approach overlooks prompts with more than one element that may generalize better to unseen test data. Thus, by including this constraint while relaxing the binary integer constraint so that I E [0, 1] and turning the summations into vector dot products, one yields the following linear-fractional program:
max (A ROR)
(A.1)
x²x x
s.t. x-1 = S 1,
(A.2)
where A =< AI...., >, R R R Rk >, x< I Ik >, A is the KxK identity matrix, @ is the element-wise vector product operator, and 0, are vectors of zeroes and ones, respectively. This is a linear-fractional program due to the denominator term in the optimization function, but applying a Charnes- Carnes-Cooper transformation yields a traditional linear program (Charnes & Cooper, 1962). This is done by first applying change of variables through multiplying both parts of the fraction in the optimization function and both sides of each constraint by a nonnegative optimization (scalar) variable f. The next and final part of the transformation involves focusing on maximizing the expression corresponding to the numerator while fixing the one for the denominator to More specifically, for this problem instance, the transformation produces the following linear program:
max (A @ R) y,
(A.3)
s.t. ATy - =x==jt, 0 S Aly 1t, 5
(A.4)
where y is a new optimization (vector) variable equivalent to Ix as a result of the change of variables. Therefore, after finding optimal values for vector y and variable f under the given constraints, the 4 can be obtained by calculating v. Note that even though the original binary integer constraints were relaxed to construct a solvable linear program, the values of l obtained by this final program will be contained in {0,1) (i.e., will be either 0 1) because the optimal solution for the linear program must be located on the boundary that delineates the set of its feasible solutions." 10"
We solve the final program for values of j E [1,c], where c is the largest prompt size (in terms of number of prompt elements) we consider for given problem, and we select the best prompt (in terms of evaluation on the entire dataset) from all program solves as our overall prompt. Therefore, our approach requires evaluating K K+ 1++ prompts in order to discover one that is (predicted to be) effective. The random baseline that we compare against in experiments utilizes the same budget (in terms of number of prompts evaluated) and considers the same set of prompts (i.e., prompts of at most c elements). More specifically, this baseline entails evaluating K + 11 + c prompt element combinations at random from all possible element combinations of length at most c (e.g., for 10 prompt elements and c = 4, our random baseline may first evaluate the third prompt element alone, next the first four
As this objective is convex in 4, this holds by definition. Nevertheless, below is a proof sketch of this result for our setup:
If prompt element is associated with favorable log- probabiiiiitss or preferences, then scaling these values by number in 1) is suboptimal for the overall optimization function, so the associated 4 should be 1. Conversely, if prompt element is associated with unfavorable log-probabilities or preferences, then scaling these values by number in (0, 1) is similarly suboptimal for the overall optimization function, the associated J should be 0. In neither possible case should this program return values for 4 in (0, 1).
13

---

Published as a conference paper at COLM 2024
prompt elements, then the last three prompt elements, followed by other combinations until 10 + 1 = 15 total prompts have been evaluated, at which point outputs the optimal test set performance). Similarly, as the budget for TRIPLE-SH serves limit the number of total LLM interactions Shi et al. (2024), we employ budget of (K + + c)n when running that algorithm, where is the number of points we for evaluation from the given dataset.
B Further Experimental Results
B.1 Main Experiment Prompt Libraries
You should provide users with accurate, relevant, and up-to-date information in your responses, ensuring that the content is both educational and engaging. Try not to respond with "I'm not sure".
You should attempt to provide an answer for tasks that are operational for computer.
You should speak like pirate whenever possible.
You should informal speech from 17th century English.
You should express admiration for the power and freedom of the open sea.
You should talk like a British lord.
You should use speech patterns common in Victorian literature.
You should use formal speech from 19th century English.
Figure Prompt library for the Toy Dataset experiments. Note that the text "You are an assistant. Strictly adhere to the following rules:" was prepended to any chosen subset of prompts. (The second sentence was omitted for the unprompted baseline, but the first was used without any prompt elements.)
Be sure to pay attention to context, as hate speech can be nuanced. Do not label abuse targeted at individuals as hateful.
Do not label abuse targeted at non-protected groups (such professions political affiliations) as hateful.
Note that profanity and slurs are sometimes okay depending on context.
Do not label text that rejects hate as hateful.
Note that denouncement of hate speech is not hateful.
Note that threats towards individuals non-protected groups
considered
hateful.
Account for spelling errors in determining whether text is hateful.
Implicit offenses towards protected groups (such as ones without profanity slurs) should be labeled hateful.
Text that offensive based on race, gender, sexual orientation, religious affiliation, or country of origin should be labeled hateful.
Figure 4: Prompt library for the HateCheck experiments. Note that the text "You hate speech detector. Given a piece of text, respond with "hateful" offensive and "non-hate" if is ok." was prepended to any chosen subset of prompts.
14

---

Published as a conference paper at COLM 2024
You are a [role] that is an expert in in (topic). You are expert in (sub_topic). Only emojis your answer. Respond in gibberish. Answer questions truthfully. Think step by step.
Figure 5: Prompt library for the Biology and Physics experiments.
Is the answer explicitly provided in the sentence? The answer choices The answer is 'yes' the sentence explicitly contains the answer. The answer is 'no' if the sentence does not explicitly contain the answer. If the answer is implied, say 'no'
'yes' 'no'
Think step by step.
Answer questions in single word. Respond with emojis. Answer with tons of gibberish.
Figure 6: Prompt library for the NI Task 020 experiments.
this task, you are given a text from tweets. Return the sentiment of the following.
Return positive if the content is positive. Return negative if the content is negative. Think step by step.
Answer questions in single word. Respond with emojis.
Answer with tons of gibberish.
Figure 7: Prompt library for the NI Task 195 experiments.
Determine if the following sentences agree disagree with each other. Return yes if the sentences agree. Return no if the sentences disagree.
Think step by step.
Answer questions in single word. Respond emojis. Use gibberish as your answer.
Figure 8: Prompt library for the NI Task 199 experiments.
15

---

Published as a conference paper at COLM 2024
B.2 Main Experiment Prompt Regression Results
Dataset
NI Task 020
NI Task 199
PEPR-R
PEPR-P
PEPR-R
PEPR-P
FOW BIE ATE
7B
Po: Pare 140
CANPET oour (e
13B
70B
Figure 9: Predicted-versus-ttuee value plots corresponding to prompt regression experiments on two datasets, NI Task 020 and NI Task 199.
Dataset
Toy Dataset
Biology PEPR-R
Physics PEPR-R
PEPR-R
PEPR-P
SAn CORTICRRAAA CARE
DOE CRR
7B
estny i maSIC AXT 2EN0
CM
CL-AAR
13B
70B
Figure 10: Predicted-versus versus-truue value plots corresponding to prompt regression experi- ments on three datasets, our Toy Dataset, CAMEL Biology, and CAMÉL Physics. Note that as the CAMEL datasets lack preference data, we only evaluate PEPR-R on those datasets.
16

---

Published as a conference paper at COLM 2024
B.3 Main Experiment Prompt Selection Results
Table 3: Comprehensive accuracy results are reported in Table 3 with standard deviations pertaining to experiment settings. Models here are from the Llama-2-chat family. We include maximum and high percentile results from all relevant prompt combinations and baselines. Note that the high-performing prompts that PEPR suggests have small standard deviations, suggesting the stability of prompts selected as compared to the random method.
Labeled Data Portion Methed
Dataset
Model
0.1 PEPR-R
0.5 PEPRIR
1 PEPR-R
Base 0.75 Max
Read
PEPR-P
Kond
PEPR-P
Real
PEPR-P
7B
0.52
0.04 0:11 0.00 0.53
0.00
0.52
0.01 0:41 0.00 0.53 0.00
0.52 J. 0.01 0.41 1. 0.00 0.53 0.00
0.19
0.37
0.55
Toy Dalaset
13B
0.58
0.00
0.57
0.00 0.58
0.00
0.56
0.05
0.57 0.00 0.58
0.00
0.16
0.61
708
0.95
0.06 0.97 1 0.00 0.97
0.00
0.95
0.06
0.97 A: 0.00
0.97 A 0.00
0.95 J: 0.06
0.97 1: 0.00 0.97 1: 0.00
0.16
0.67
0.98
7B
0.73
0.02 0.01
0.01
0..33
0.02
0.00 0.68
0.73 + 0.02 0.67 0.00 0.68
0.00
0.65
0.76
HateCheck (Slur)
13B
0.80
0.02 0.75 # 0.00 0.80
0.02
0.00
0.02
0.75 0.00 0.81 0.01
0.08 +: 0.02 0.25 + 0.00 0.82 : 0.00
0.80
0.75 0.83
708
0.90 à: 0,03 0.90 1: 0,02 0.83 à 0.02
0.900
0.03 0.90 0.00 0.83 d: 0.01
0.90 :: 0.03 0.90 :: 0.00 0.53 4 0.00
0.71 0.84
0.95
78
0.57
0.12 0.68 0.00
0.55 + 0.12 0.68 + 0.00
0.55 + 0.12 0.68 + 0.00
0.60 0.67 0.69
Bology
138
0.57
0.10 0.65 0.07
0.58 1: 0.10
0.68 1: 0.03
0.58 1: 0.10 0.68 1: 0.00
0.58 0.67 0.70
708
0.68
0.14
0.47 0.14 0.68
0.00
0.60
7B
0.53
0.09 0.65 0.00
0.51 1 0.39 0.65 1. 0.00
0.53 1. 0.10 0.65 1 0.00
0.53 0.63 0.65
Physics
138
0.66 0.00
0.57 0.08
0.66 0.00
0.57 0.08 0.66 0.00
0.54
708
0.51
0.12 0.66 0.00
0.50 + 0.12 0.66 :: 0.00
0.50 =: 0.12 0.66
0.00
0.61
0.63 0.66
0.74
0.01
0.71 0.01 0.74 0.00
0.00
0.57
0.76
NI 020
138
0.71
0.07 0.72 0,05 0.74 0.07
0.75
0.02 0.76 0.01 0.78 0.00
0.76 J: 0.01 0.77 1: 0.00 0.78
0.00
0.64
0.74 0.78
70B
0.71 à:
0.08 0.71 à: 0.08
0.74
0,03
0.775 a: 0.03 0.74 = 0.01 0.75
0.00
0.76 4: 0.01
0.74 =: 0.00 0.25 : 0.00
0.52
0.74
0.79
7B
0.69
0.10 0.60 0.10 0.55
0.05
0.74
0.08 0.67 0.02 0.56 0.01
0.74 + 0.08 0.67 + 0.00 0.56 + 0.00
0.56
0.57
0.85
NI 195
13B
0.62
0,08 0.59 0.03 0.60
0.00
0.64 = 0.09 0.62 0.01 0.62 + 0.05
0.65 + 0.08 0.62
0.000
0.66
+ 0.00
0.36
01.6 0.83
70B
0.74
0.07
0.75
0.05
(0.99
0.04
0.80
0.03
0.79 1: 0.03 0.81
0,00
0,81
0,00
0.56
0.84
78
0.74
0.10 0.76
0,06
0.74
0.10
0.77
0.00 0.77 0.00 0.77 0.00
0.77 + 0.00 0.77 + 0.00
0.77
0.00
(2700
0.77 0.77
NI 199
13B
0.74
0.10
0.77
0.77
0.01
0.77
0.00 0.77
0.000
0.77
0.00
0.39
0.77
708
0.74
0.09
0.74
0.07
0.74
0.06
(0.77
0.06
0.76
0.00
0.76 :: 0.00
0.77 :: 0.00 0.76
:: 0.00
0.76 :: 0.00
0.59
0.77
0.78
Labeled Data Fortion Method
Dataset
Model
0.05 PEPR-R
0.25
0.75 Max
Rand
PEPR-P
Romd
PEPR-R
PEPR-P
7B
0.52
0.04
0.41
0.53
0.00
0.52
00.19
0.37 0.55
Toy Dataset
13B
0.36
0.05
0.56
0.06
0.58
0.01
0.56
0.03
0.37 0.00
0.58
0.00
0..6
0.40
0.61
70B
0.95
0.06
0.97
0.00
0.97
0.00
0:95
0.06
0.97 0.00
0.97
0.00
0.16
0.67
0.58
7B
0.73
0.02
0.68
0.01
0.69
+ 0.01
0.73 + 0.02 0.67 + 0.00 0.68 0.01
00.65
0.69
0.76
HateCheck (Slur)
13B
0
0.02
0.74
0.01
0.80
0.03
0.80
0.02
0.25 0.000 0.81 0.02
(0.00
0.75
0.83
70B
0.84
7B
0.55
0.12
0.65 0.00
0.55 0.12 0.68 0.00
0.60 0.67 0.69
Bielogy
0.67
70B
0.47
0.14 0.68 0.00
0:47 0.14 0.68 0.00
0.60 0.68 0.69
0.00
0.63 0.65
Physics
13B
0.57
0.08
0.66 0.00
0.57 0.08 0.66 0.00
0.51 0.65 0.66
70B
0.50
0.12
0.66
0.00
0.50 0.12 0.66
0.00
0.61 0.63 0.66
7B
0.74
0.01 0.74 0.01
0.74 =: 0.01
0.74
0.01 0.74 0.00
0.74 0.00
0.57
0.74 0.76
NI 020
13B
0.75
0.22 0.76
0.01
0.78
0.00
0.76
0.01 0.27
01.00
0.78
0.00
01.64 0.74 0.78
70B
0.75
0.74
0.25
0.52
0.74
0.79
78
0.74
-
0.08 0.67
0.02
0.56 + 0.01
0.74
0.08 0.67
0.00
0.36 0.00
(0.56
0.57 0.85
195
0.05
0.36
70B
0.79
0.04 0.80
0.03
0.81
0.01
0.79
0.03 0.81
0.00 0.81
0.00
0.56 0.71 0.84
0.77
NI 199
13B
0.77
0,00
0.77
0,00
0.77
0.00
0.77
0.00
0.77
0.00
0.77
0.00
0.39
0.77 0.77
706
0.77
0.06
0.76
0.00
0.76
:: 0.00
0.77 :: 0.00
0.76 : 0.00
0.76 :: 0.00
0.59
0.77 0.78
17

---

Published as a conference paper at COLM 2024
Table 4: Results on other 10 data classes from the HateCheck dataset. As with results in Table 3, standard deviations for prompts selected by PEPR are small, suggesting prompt stability. See (Röttger et al., 2020) for more details about each class.
Labeled Data Portion Methosl
Data Class
Model
0.1 PEPR-R
0.5 PEPR-R
Base 0.25 Max
Raal
PEPR-P
Rand
PEPR-P
Rand
PEPR-R
PEPR-P
7B
0.98
0.01 0.91 0.00
0.99 0.00
0.98 + 0.01 0.91 + 0.00 0.99 + 0.00
0.98 0.01 0.91 + 0.00
0.99 0.00
0.97
0:95
0.99
Derogation
138
0.94
0.03
0.79
0.00
0.96
0.00
0.94
1: 0.03 0.79 1: 0.00 0.96 1 0.00
0.94
0.03 0.79 :: 0.00 0.96
0.00
0.87
0.85
0.96
70B
0.00
1.00
1.00
1.00
1.00
1.00
7B
1.00
0.00 1.00
0.00
1.00
0.00
1.00
0.00
1.00 0.00
1.00
0.00
1.00
0.00
1.00 0.00
1.0
0.00
1.00
1,00
1.00
Threstening lan guage
0.99
0.00
0.95
0.00
1.00
0.00
1.00
0.96
1.00
70B
1.00
0.00 1.00
0.00
1.00 0.00
1.00 A: 0.00
1.00 1: 0.00 1.00 1: 0.00
1.00 4: 0.00 1.00 2: 0.00 1.00 1: 0.00
1.00
1,00
1.00
7B
0.97
0.95
0.96
0.97
0.01 0.94 0.00 0.96
0.00
0.97
0.01 0.4 0.00 0.96
0.00
[0:93
0.95
0.98
Profanity usage
13B
0.97
0.01 0.96 0.00
0.98
0.00
0.97 #: 0.01 0.96 # 0.00 0.98 + 0.00
0.97 = 0.01 0.96 = 0.00 0.98 = 0.00
0.95
0.95
0.98
70B
1,00
0.00
1,00
0.00
0.96
0.00
1.00 1: 0.00
1.00 1: 0.00 0:95
0.02
1.00 4: 0.00 1.00 A 0.00 0.93 A: 0.00
00.90
0.99
1.00
7B
0.99
0.01 0.93
0.00
1.00
0.00
0.99 + 0.01 0.95 + 0.00 1.00 + 0.00
0.99 + 0.01 0.85 + 0.00 1.00 + 0.00
00.97
0.96
1.00
Pronoun
refer-
138
0.98
0.01 0.91
0.00
0.99
0.00
0.98 + 0.01 0.91 +: 0.00 0.99 +: 0.00
0.98 = 0.01 0.91 = 0.00 0.99
0.00
00:83
0.94
1.00
70B
1.00
1.00
1.00
7B
0.96
0.01 0.95
0.01 0.96 0.00
0.96
0.01
0:95 1. 0.00 0.96 0.00
0.96
0.01
0.95 0.00 0.96
0.00
0.57
0.93
0.98
Negation
0.96
0.96
0.00
0.92
0.95
0.00
0.95
0.97
70B
0.99
0.00 0.98
0.00 0.97 4: 0.01
0.99 1: 0.00 0.98 A: 0.00 0.97 A: 0.00
0.99 2: 0.00 0.98 1: 0.00 0.97 1: 0.00
0.899
0.96
1.00
0.991
0.00
0.99
0.01
0.00 1.00
0.00
0.99
1.00
0.00
(0.98
0.97
1.00
Phrasing
13B
0.97
0.02
0.81 0.00
0.99
0.00
0.97
+ 0.02 0.81 # 0.00 0.99
+: 0.00
0.97
= 0.02
0.81 # 0.00 0.99
= 0.00
0.955
0.90
1.00
70B
1,00
0.00 1,00
0.00
1,00
0.00
1.00
0.00
1.00 1: 0.00 1.00 1: 0.00
1.00
0.00
1.00
0.00
1.00
0.00
1.00
1.00
1.00
7B
1.00
0.00 0.99 0.01
1,00
0.00
1.00 + 0.00 1.00 + 0.00 1.00 + 0.00
1.00
+ 0.00
1.00 0.00 1.00
0.00
(0.98
0.97
1.00
Non-hale group
13B
1,00
0.00
1,00 0.00
1,00
0.00
1.00
0.00
1.00 0.00
1.00
0.00
1.00
0.00
1.00
0.00
1.00
0.00
0:988
1.00
1.00
identifiers
70B
0.99
0.00
0.98
0,00
0:98
0.00
0.98
0.00
0.98
0.00
1,00
1.00
7B
0.94
0.04 0.79 + 0.04 01.00 + (.71
0..94 + 0.04 0.81 + 0.00 0.71 + 0.04
0.94 +
0.04 0.81 + 0.00 0.70 + 0.00
00.15
0.79
0.98
Counter speech
0.98
0.05
0.94
0.86
70B
0.58
0.08 0.55
0.19
0.62
0.00
0.58 + 0.08 0.28 : 0.00 0.62 +: 0.00
0.58
0.08 0.28 + 0.00 0.62 + 0.00
(0.04
0.36
0.71
0.99
0.02
0.99
0.02 0.91 0.00 0.99
0.02
0.00
[0.6]
0.90
1.00
Abuse
against
13B
0.99
0.02 0.93 0.01
0.83
0.01
0.99 1: 0.02 0.93 1. 0.00 0.92 J. 0.03
0.99 1: 0.02 0.93 1: 0.00 0,95 1: 0.00
0.74
0.93
1.00
non-poot prot. targets
70B
0.75
0.06 0.66
0.00
0.68
0.04
(0775 4: 0.06 0.66 1 0.00 0.67 +: 0.03
0.75 =: 0.06 0.66 = 0.00 0.65 = 0.00
0..69
0.62
0.87
7B
0.93
0.01 0.88
0.00
0.93
0.00
0.93 + 0.01 0.83 + 0.00 0.93 + 0.00
0.98 + 0.01 0.83 + 0.00 0.98
+ 0.00
(0:91
0:90
0.96
Spelling varia-
138
0.89
0.003 0.76
0.00
0.93
0.00
0.89 =: 0.03 0.76 #: 0.00 0.93 +: 0.00
0.89 = 0.03 0.77- = 0.00 0.93
0.00
01.44
0.81
0.54
Bons
70B
1.00
1.00
0.00
1.00
0,00
1.00 0,00
1.00
1.00
0.00 1.00
1.00
1,00
1.00
Labeled Data Portion Method
Data Class
Model
0.15 PEZR-N
0.25 PEPR-R
Bose 0.75 Max
Rend
PEPR-P
Rond
PEPR-P
0.98 0.01 0.91 0.00 0.98
0.00
0.01
0.91
0.00
0:97
0.95
0.99
Derogation
0.94 + 0.05 0.79 + 0.00 0.96
0,80
0.03
0.79
0.96
0.00
0.87
0.96
70B
1.00 1: 0.00 1.00 1: 0.00
1.00
1: 0.00
1.00
0.00
1.00
1:
0.00 1.00 1: 0.00
1.00
1.00
1.00
1,00 + 0.00 1,00 + 0.80 1,00
+ 0.00
0.00
0.00
0.00
1.00
1.00
1.00
Threatening Jan-
136
0.99 + 0.00 0.95 + 0.00 1.00
+ 01.00
0.99
0.00
0.95
0.00 1.00
+ 0.00
0:99
0.98
1.00
guage
70B
1.00 + 0.00 1.00 + 0.00
1.00 + 0.00
1.00
0.00
1.00
0.00
1.00
0.00
1,00
1.00
1.00
7B
0.97 1: 0.01 0.95 1. 0.00 0.96 J. 0.02
0.97
0.01
0.95 0.00 0.96
0.01
[0.5
0.95
0.98
Profanity usage
138
0.97
J. 0.01 0.96 1. 0.00 0.95 1. 0.00
0.97
0.01
0.96
0.00
0.98
0.00
0.95
0.95
0.98
1.00 0.00 1.00 0.00 0.96
0.900
7B
0.99 1. 0.01 0.95 1 0,00 1.00 1. 0.00
0.59
0.01 0.95 0.00 1.00 0.00
0.97
0.96 1.00
Prenoun
refor.
0.98 + 0.01 0.91 + 0.00 0.99 + 0.80
0.98
0.01
0.91
0.00
0.00
0.96
0.94
ence
70B
1.00 :: 0.00 1.00 +: 0.00 1.00 +: 0.00
1.00
::
0.00
1.00 :: 0.00
1.00
0.00
1.00
1.00
1.00
0.96 + 0.01 0.95 + 0.01 0.96 + 0.00
0.96
0.01
0.00 0.96
0.00
[.87
0.98 0.98
Negation
136
0.96 + 0.00 0.92 + 0.00 0.95 + 0.01
0.96
0.00
0.92
0.00
0.95
0.00
[.96
0.95
0.97
708
0.99 +: 0.00 0.98 +: 0.01 0.98 + 0.01
0.99
0.00
0.98
0.00
0.97
0.00
0..99
0.98
1.00
7B
0.99 + 0.01 0.86 + 0.00 1.00 + 0.00
(199
0.01
0.86
0.00 1.00
0.00
[8:98
0.97
1.00
Phrasing
138
0.97
1. 0.02 0,81 1. 0.00 0.99 1. 0.00
0.97
0.02
0.81
0.00
0.99
0.00
0.95
0.50
1.00
1.00 0.00 1.00
1.00
0.00
11.00
7B
1.00 1. 0.00 0.99 1 0.02 1.00 1. 0.00
0.00
1.00
0.01
1.00 0.00
0.98
0.97
1.00
Non-hate group
1.00 1: 0.00 1.00 1. 0.00 1.00
0.00
0.96
1.00
identifiess
70B
0.99 :: 0.00 0.98 4: 0.00 0.98 4: 0.00
0.99
0.00
0.88
0.00
0.98
0.00
1.00
1.00
1.00
0.94 + 0.04 0.7% + 0.05 0.82 + 0.11
0.94
0.04
0881
0.00
0.08
0.15
0.79
0.98
Counter speech
136
0.98 + 0.03 0.94 + 0.00 0.90 + 0.03
0.98
0.03
0.94
0.00
0.91
0.01
0.34
0.86
1.00
70B
0.58 d: 0.08 0.68 4: 0.03 0.62 4 0.00
0.58
0.08
0.28
0.00
0.62
0.00
0:04
0.36
0.71
7B
0.99 + 0.02 0.90 + 0.000 0.97 + 0.04
0.99
0.02
0.90
0.00
0.99
0.01
[B.61
0.90 1.00
Abuse
against
13B
0.99 + 0.02 0.94 + 0.01 0.88 + 0.42
0.99 0.02
(193
0.00
0,88
0.00
0.74
0.93
1.00
non-ppoot tirgets
0.75 1: 0.06 0.66 1: 0.68 4: 0.04
0.49
7B
0.93 1. 0.01 0.84 4 0.01 0.93 0.00
0.93
0.01 0.83 0.01 0.93 0.00
0.91 0.50 0.96
Spelling barss
varia-
0.89 1: 0.03 0.76 1. 0.00 0.93
0.00
0.51
70B
1.00 4: 0.00 1.00 +: 0.00
1.00 4: 0.00
1.00
:: 0.00
1.00
:: 0.00 1.00
:: 0.00
1.00
1.00 1.00
18

---

Published as a conference paper at COLM 2024
Table 5: Subset of selection experiment results with TRIPLE-SH baseline. Note that TRIPLE- SH results could not be computed for some smaller values of dataset portions due to too few samples. Evidently, though TRIPLE-SH outperforms both versions of PEPR for some models and datasets, PEPR can be used with small amounts of data (i.e,, limited evaluation budget) and is arguably more interpretable.
Labeled Data Portion Melland
Dataset
Model
0,05
0.25
0.5
[ase 0.75 Max
T8-SH
PEPR-8
PEPR.P
TR.SH
PEPR-R
PEPR.P P| TR-SH
PEPR.R
PEPR-P
FR-SH
PEPR-R
PEPR.P
7B
0.39
0.41
0.53
0.51
0.41
0.53
0.52
0.41
0.53
0.54
0.41
0.53
0..19 0.37 0.55
Dataset
13B
0:48
0.56
0.58
0.56
0.57
0.57
0.57
0.58
0.59
0.57
0.58
0.16 0.40 0.61
70B
0.97
0.97
0.97
0.97
0.16
7B
N/A
0.68
0.67
0.68
0.72
0.67
0.68
0.67
0.68
0065
0.69 0.7
HateChock (Stur)
13B
N/A
0.51
0.75
0.51
0.81
0.75
0.22
0.00 0.8
70B
N/A
0.90
0.83
0.98
0.90
0.83
0.94
0.90
0.83
0.94
0.90
0.83
0.71 0.84 0.5
7B
N/A
0.68
0.68
0.69
0.68
(a60
Biology
13B
N/AA
0.64
0.69
0.66
0.70
0.68
0.70
0.68
0.58 0.67 0.70
208
N/A
0.68
0.69
0.68
0.69
0.68
0.69
0.68
0.60 0.68 0.69
78
N/A
0.58
0.65
0.58
0.65
0.58
0.65
0.55
10.53 0.63 0.65
Physics
158
N/A
0.52
0.66
0.53
0.66
0.53
0.66
0.53
0.54 0.65 0.66
70B
N/A
0.66
0.66
0.66
(66)
0.66
7B
N/A
0.64
0.61
0.74
0.72
0.73
0.74
0.74
0.71
0.74
0.71
0.71
0.:57
0.71 0.76
N20
13B
N/A
0.73
0,75
0.75
0.74
0.64
70B
N/A
0.67
0.72
0.76
0,71
0.74
0.77
0.74
0.75
0.77
0.74
0.75
0.74 0.79 0.52
7B
N/A
0.60
0.55
0.77
0.83
0.56
0.:83
0.56
0.56
NT
13B
N/A
0.58
0.99
0.77
0.61
0.61
0.79
0.62
0.62
0.79
0.62
0.66
0.56
0.56 0.83
70B
N/A
0.76
0.75
0.81
0.79
0.80
0.81
0.80
0.81
0.91
0.81
0.81
0.56
0.71 0.84
7B
N/A
0.75
0.72
0.77
0.77
0.77
0.77
0.77
0.77
0.77
0.77
0.77
00200
0.77 0.77
N999
13B
N/A
0.76
0.76
0.77
0.76
0.26
0.77
0.77
0.77
0.77
0.77
0.77
0.39
0.77 0.77
70B
N/A
0.76
0.59
0.78
B.4 Supplemental Experiments
Mistral-7B-Insttruct-o022 We test our method with an additional model, Mistral Al's Mis- tral 7B Instruct v0.2 (Jiang et al., 2023), on a subset of datasets used in our main experiments. More specifically, we do so on all datasets except our Toy Dataset and HateCheck.
Table 6: Mistral-7B-Instrucete
results reported in Table with standard deviations per- taining to experiment settings. We include maximum and high percentile results from all relevant prompt combinations and baselines.
Labeled Data Portion Methad
Dataset
0.1 PEPR-R
0.5 PEPR.R
Base
0.75 Max
Rand
PEPR-P
Rand
PEPR-P
Raal
PEPR-R
PEPR-P
Biology
0.53 + 0.10 0.68 + 0.01
0.53 + 0.10
0.68 0.00
0.55 + 0.10 0.68 + 0.00
0.56
0.68
0.70
Physies
0.51 1: 0.08 0.66 1: 0.00
0.51 =:: 0.08 0.65 0.00
0.51 1: 0.08 0.65 1. 0.00
0,50
0.65
0.66
NI 020
0.77 = 0.03 0.78 à: 0.02 0.79 = 0U0
0.79 :: 0.01 0.79 0.01
0.79 #: 0.01
0.80 0.00 0,30 + 0.00 0.80 0.00
0.76
0.78
0.82
NI 195
0.07 0.73 + 0.03 0.7 0.03
0.80 + 0.03
0.72 + 0:03
0.80 0.03
0.77 + 0.00
0.73 0.00
0.56
0.71
0.83
0.60 0.01 0.57 0.14
0.61 1 0.09
0.09
0.39
0.00
Labeled Data Portion Method
Dataset
0.05 PEPR-R
0.25 PEPR-R
Base
0.75 Max
Read
PEPR.P
PEPR.P
Biology
0.54 0.09
0.68 0.00
0.56
0.68
0.70
Physics
0.54 0.08
0.00
0.55 0.08
0.65 1: 0.00
0.50
0.65
0.66
NI 020
0.77 + 0.04 0.78 0.02 0.78 0,22
0.78 + 0.02 0.79 + 0.01 0.79 + 0.00
0.76
0.78
0,82
NI 195
0,73 J. 0.08
0.72 1: 0.00
0,70 1 0.03
0.78 0.04
0.75 1 0.03 0.72 :: 0.00
0.56
0.71
0.83
NI 199
0.52 4: 0.13
0.40 d: 0.04
0.57 0.14
0.60 0.10 0.40 :: 0.03
0.45 0.13
0.36
0.44
0.73
19


Reviews:
[{"review_id": "0BLyzJ7djC", "reviewer": "Reviewer_t5Qb", "summary": "This work studies the prompt library search problem where the model needs to predict prompt combinations in a library that can lead to optimal performance. This work further proposes a method including three steps: (1) building a prompt library; (2) generating weight for each element; (3) prompt selection based on weights.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": true, "final_justification": "", "rebuttal": "Thank you for your feedback and your highly positive opinion of our work. Please see our responses to your comments below:\n\n>  There is a lack of comparison with existing prompt engineering methods. The existing comparison with a random baseline is not enough to show the advantage of the proposed PEPR method.\n\nWhile we concede that we primarily compare results of PEPR with a random baseline, we also report 75th percentile performance metrics (relative to results from all possible prompt element combinations) in Table 2 and similar tables in our Appendix. In doing so, we approximate the results of brute-force searches with a fixed computation budget, which in turn are explored in other parts of the prompt engineering literature (e.g., GrIPS by Prasad et al. (2023)). We can also opt to include the performance results of the bandit algorithm introduced by Shi et al. (2024) in a revision of our paper.\n\n> The performance of this method highly relies on the quality of the prompt library.\n\nOther related prompt engineering work (like that of Shi et al. (2024)) also assumes a prompt library or similar is used as a starting point for iterating on prompts. Therefore, though our method is dependent on the prompt library’s quality, this is also a constraint of other prompt engineering methods. Furthermore, other works consider relatively few prompt candidates (up to 100) whereas we are able to consider more (around 1000 due to possible combinations of prompt elements) because we account for the compositional structure of prompts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The problem formulation of prompt library search is novel and interesting but could benefit from more explanation of its motivation and practical usage.\n\n- This work provides well-formulated mathematical definitions and formulae for prompt regression and prompt selection. \n\n- This framework enables quantitative analysis of individual prompt component's impact in regression analysis. \n\n- The experiments are conducted across a wide range of language models.", "reasons_to_reject": "- There is a lack of comparison with existing prompt engineering methods. The existing comparison with a random baseline is not enough to show the advantage of the proposed PEPR method.\n\n- The performance of this method highly relies on the quality of the prompt library.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pxv8miB1VO", "reviewer": "Reviewer_cEYe", "summary": "The paper introduces a framework, Prompt Exploration with Prompt Regression (PEPR), which aims to optimize the selection of prompt elements for language models. It claims that unlike prior methods, PEPR can predict the output effects of various prompt combinations without exhaustive testing. The authors present a methodology involving building a library of prompt elements, assigning weights through regression analysis, and selecting the optimal combination based on these weights. The paper asserts that its approach, tested on various datasets and model sizes, improves prompt selection efficiency and effectiveness. It suggests that PEPR could significantly streamline the process of prompt engineering, although it also calls for further research to refine and expand the method.\n\nOverall, the paper is well written and introduces an approach for which the COLM conference is arguably the perfect venue. Hence, we believe it should be accepted.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your comments on our work and your very positive review. Please see our responses below:\n\n> Can you shed further light on what you mean by \"solving (3.3) does not require knowledge of the reference (correct) y for the inputs (e.g., when there is a finite set of meaningful generations (such as classification or multiple-choice QA), one can simply plug every possible y for every input xi when fitting the regression coefficients)\". In a quick reading it might sound like the PEPR method only supports classification usecases but that is patently not the case when considering the rest of the paper.\n\nWe mean that of a possible set of outputs, we do not need to know which one is correct in order to perform regression. This is because our regression approach models log-probabilities (and corresponding differences) of outputs themselves rather than assigning probabilities of correctness. In contrast, our proposed selection method *does* require knowledge of which outputs are correct in order to derive a performant prompt relative to correct outputs (see Table 1). Note that in using terms like “correct” and “possible set of outputs”, we do not mean to imply that our methods are only suited for classification (as you have deduced) but rather that even a given set of open-ended responses has associated qualities of correctness and desirability that are needed for selection and not for regression.\n\n> Would it be possible to quantify the level of interaction between prompt elements when their effects on language model outputs are not independent and make use of this information in the prompt selection step?\n\nThis is an interesting idea. Though we do not explore such an approach, it may be possible to detect dependence between prompt elements whenever our regression model is repeatedly or on average very inaccurate (relative to ground-truth log-probabilities and differences) for a given prompt element combination. In doing so, perhaps coefficients could be tweaked for specific combinations of elements such that selection is more accurate. More complicated models like linear regressions with interaction terms and factorization machines could account for pairwise interactions and increase effectiveness (perhaps at the risk of reducing interpretability). At any rate, this line of reasoning could be a fruitful future direction of work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- PEPR provides a method to predict the outcomes of combined prompt elements without testing each combination, addressing the inefficiency of brute-force methods by providing a more principled method of building prompts from a selection of templates.\n- The framework integrates regression analysis to assign quantitative impacts to individual prompt components, allowing for systematic optimization.\n- PEPR utilizes both reference text generations and preference data, offering flexibility in aligning language model outputs with human judgments or desired behaviors.\n- The method is validated across multiple open-source language models of the Llama2 family and four diverse tasks, showing its adaptability and broad potential for future utilization.\n- PEPR introduces a novel approach to prompt library search, a previously underexplored area in language model research.", "reasons_to_reject": "- The paper does not fully address the scalability of PEPR with very large prompt libraries or complex prompt structures, which could limit its practicality in extensive applications.\n- The methodology may not account adequately for the nuanced interactions between prompt elements beyond their individual contributions, potentially oversimplifying complex linguistic phenomena.\n- Further, the methodology relies heavily on the initial quality and composition of the prompt library, potentially limiting its effectiveness if the library is not well-constructed.\n- The experiments lack a comparison against a wider range of existing prompt engineering methods, which could help in understanding PEPR's relative performance.\n\nAll in all, these reasons are relatively superficial and do not warrant the rejection of the paper. Moreover, the authors address most of them in the manuscript to a great extent.", "questions_to_authors": "1. Can you shed further light on what you mean by \"solving (3.3) does not require knowledge of the reference (correct) y for the inputs (e.g., when there is a finite set of meaningful generations (such as classification or multiple-choice QA), one can simply plug every possible y for every input xi when fitting the regression coefficients)\". In a quick reading it might sound like the PEPR method only supports classification usecases but that is patently not the case when considering the rest of the paper.\n\n2. Would it be possible to quantify the level of interaction between prompt elements when their effects on language model outputs are not independent and make use of this information in the prompt selection step?", "extra_scores": {}}, {"review_id": "COYiasNOC4", "reviewer": "Reviewer_jCnP", "summary": "The paper tackles a specific prompt engineering problem for LLMs, aiming to optimize prompt combinations from a library of individual elements. The goal is to predict the effects of these combinations on LLM outputs and create an optimal prompt for specific tasks. The proposed solution, Prompt Exploration with Prompt Regression (PEPR), consists of three steps: building a prompt library, deriving weights for each element through prompt regression, and selecting prompt elements via these weights. This process incorporates generated texts or human preferences to align prompts with the desired outcomes. PEPR's effectiveness is validated using several open-source LLMs across different datasets and tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We appreciate your positive points and the chance to respond to your questions and concerns.\n\n> Motivation for prompt library search problem\n\nSystem prompt selection is often a process of adding phrases to correct undesired behavior. Libraries of such phrases can grow big, thus requiring a method to identify an optimal combination algorithmically.\n\n> Generalizability of the PEPR prompts\n\nPEPR allows prompt elements to depend on the input (footnote 1), thus the same prompt library can be used across different tasks of related nature, e.g., in Figure 5 we show such a prompt library used for both CAMEL Biology and Physics datasets. In comparison to prior prompt-engineering approaches which are often task-specific, this feature increases the generalizability of PEPR.\n\n> Scale of data used for parameter optimization\n\nIn Sections 3.2 and 4.2 we specify that we use little data for optimization (as many as 5 samples in several cases).\n\n> In Figure 2, regression performance deteriorates as the number of prompt elements increases, indicated by larger MAE and smaller r-values. Coupling between prompt elements.\n\nWhile the results for predicting the effects of 2 prompt elements are the best across experiments, results for 4 elements are often better than for 3, indicating no clear trend w.r.t. the number of prompt elements. Moreover, our prompt regression yields $r \\approx 0.9$ in most cases, indicating that a linear model fits data well, i.e. there is no significant effect due to coupling between the prompt elements.\n\n> \"minimally-prompted model\" and baselines\n\nIt is an LLM with basic instructions relative to the task (e.g., “You are a hate speech detector”; see Appendix B.1). We use it to see the effects of our prompt library relative to those of instruction tuning. Furthermore, we approximate methods involving constrained brute-force searches (e.g., Prasad et al. (2023)) by reporting 75th percentile and maximum possible metrics from all element combinations, but we can also report results with the method of Shi et al. (2024) in a revision.\n\n> Comparison to random baseline\n\nThe random baseline is quite strong when the prompt library is good, as can be seen by comparing results to 75\\% percentile and max performance. As noted in Section 5, our method can filter out unhelpful prompt elements and is more interpretable than trying prompts at random. We also reiterate our regression model is an interesting standalone contribution.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is commendable for its comprehensive review and summarization of up-to-date related works, effectively highlighting novelty of the paper.\n\n2. The quality of writing in this paper is excellent. It is exceptionally clear and well-structured, making it easy for readers to understand and follow the arguments and methodologies presented.\n\n3. The authors have included a detailed appendix which enriches the paper further. This appendix serves as a valuable resource for readers seeking a deeper understanding of the paper's contributions and the intricacies of the methodologies used.", "reasons_to_reject": "1. The central concern of the paper is the prompt library search problem within prompt engineering, which seems under-motivated. The quality of the final prompt is inherently limited by the quality of the constructed prompt library, particularly in terms of coverage and richness. Moreover, PEPR's approach to determining the weight of each prompt element through parameter optimization on specific data significantly constrains the generalizability of the resultant prompts. Additionally, the paper does not adequately discuss the scale of data used for parameter optimization, leaving PEPR's practical applicability unclear.\n\n2. The evaluation and rationalization of PEPR's effectiveness are incomplete. The experiments on prompt regression merely test the assumption of the independence of irrelevant alternatives, while those on prompt selection only assess PEPR's impact on task performance under various settings. Crucially, there is a lack of direct comparison between PEPR and the advanced prompt engineering methods described in the related work section. Such comparisons are vital to directly demonstrate PEPR's superiority and the relevance of analyzing prompt element combinations for producing the optimal overall prompt as outlined in the initial sections of the paper.\n\n3. More importantly, current experimental results (referenced in Table 2) show that PEPR barely outperforms or does not surpass the random baseline. As expressed in the last paragraph of section 4.2, even with performance metrics variations across different datasets, these outcomes further challenge the significance and motivation behind the paper's focus on the prompt library search problem.", "questions_to_authors": "1. In the prompt selection experiments illustrated in Figure 2, regression performance deteriorates as the number of prompt elements increases, indicated by larger MAE and smaller r-values. What could be the underlying reasons for this trend? Does this imply that an increase in the number of prompt elements leads to more coupling between them, thereby reducing their independence? Moreover, does the extent of this coupling depend on the quality of the prompt library?\n\n2. What exactly is the \"minimally-prompted model\" mentioned in the second paragraph of section 4.2? Why was this baseline chosen instead of more advanced methods summarized in the related work section? What are the implications of using such a baseline for the validity of the experimental results?", "extra_scores": {"ethics_concerns_details": "N/A"}}, {"review_id": "Kt5D9oQJi0", "reviewer": "Reviewer_X964", "summary": "This paper investigates how to select the optimal prompt from a prompt library given a specific model and a task. The authors propose a prompt regression approach, which utilizes task datasets to fit prompt weights, thereby achieving prompt selection. The authors design regression methods based on log-probability and preference for two types of task datasets, respectively. Overall, the methods proposed in the paper outperform baseline models, demonstrating their effectiveness.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your evaluation of our work and overall positive sentiment. We will work to address the edits you have suggested provided our paper is accepted.\n\n> My concern is whether this approach is task-specific.\n\nPEPR allows prompt elements to depend on the input (footnote 1), thus the same prompt library can be used across different tasks of related nature, e.g., in Figure 5 we show such a prompt library used for both CAMEL Biology and Physics datasets. This brings in a certain degree of generality, but we note that prompt engineering is typically a task-specific problem.\n\n> Are multiple datasets mixed for training?\n\nWe don’t mix HateCheck and CAMEL, but we do mix categories within them (like Biology and Physics for CAMEL) when fitting prompt regression (prompt selection is per category).\n\n> Overconfidence in LLMs\n\nWe model relative differences in log-probability, so overconfidence might not be an issue if some prompt makes the LLM less overconfident.\n\n> Results in Figure 2 validate the assumption that \"the elimination or addition of an element from the prompt library does not affect interactions between the other prompts.\"\n\nFigure 2 validates that a linear model (corresponding to independence of irrelevant alternatives) holds because of high correlation and low MAE. If interactions were to have a major impact, a linear model would not do well.\n\n> The proposed method didn't compare with other related works, comparing only with its baselines.\n\nWe report 75th percentile and maximum performance as a proxy for budget-constrained searches seen in related work, but we can also report results of the method proposed by Shi et al. (2024) in a revision. Note that prior works are typically limited to around 100 prompt variants and they need to evaluate each at least a few times. Our method models the compositional structure of prompts and we applied it to over 1000 prompt variations (a prompt library with 10 elements) without evaluating each variant even once.\n\n> In some cases, even random prompt combinations can outperform both versions of PEPR. Is it possible that certain tasks are insensitive to prompts?\n\nRandom prompt combinations is a fairly strong baseline if the prompt library is good. In such cases both “random” and PEPR always outperform the base model and 75\\% percentile in most cases, suggesting that prompts always have an effect. Our method is able to filter out unhelpful prompt elements, while prompt regression also facilitates interpretability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed methods are reasonable, and the results demonstrate their effectiveness.\n\n- The proposed method is derived from regression theory and is supported by mathematical definitions and formulations.", "reasons_to_reject": "- The effectiveness of the methods may vary depending on the nature of the task.\n\n- The proposed method didn't compare with other related works, comparing only with its baselines.\n\n- In some cases, even random prompt combinations can outperform both versions of PEPR.", "questions_to_authors": "- The objective of the proposed methods is to learn prompt weights through regression conducted on specific datasets. Once these weights are obtained, they can be applied to each data point in the test set. My concern is whether this approach is task-specific. Do we need to know which task category the input belongs to in order to apply the corresponding prompt weights? If so, does this mean that such methods primarily apply to narrow-domain tasks, while being unsuitable for open-domain dialogue scenarios?\n\n- During training, are multiple datasets mixed for training, or are they trained separately?\n\n- Table 2 should be placed on page 8.\n\n- From the results in Table 2, it seems that the effectiveness of PEPR may be related to the dataset. For some datasets, such as HateCheck and NI 199, even the \"random\" prompt achieves good results. Although the authors also provide some explanations for this phenomenon in the Discussion section, is it possible that certain tasks are insensitive to prompts?\n\n- Previous studies have shown that large language models commonly suffer from overconfidence issues, wherein even incorrect or illusory generated results are assigned a high probability. Could this potentially affect methods based on log-probability?\n\n- Please provide more experimental details for both the prompt regression and prompt selection to make it more clear and understandable.\n\n- Please explain how the experimental results in Figure 2 validate the assumption that \" the elimination or addition of an element from the prompt library does not affect interactions between the other prompts.\"\n\n- Typo\n  \n  - Introduction section, \" rigorously in the following sections,\" => \"rigorously in the following sections.\" and \"selection components of PEPR,\" => \"selection components of PEPR.\"", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper explores the problem of automatically identifying suitable prompts. This work takes advantage of similarities between prompt variations and their performance, and proposes a method called Prompt Exploration with Prompt Regression that predicts performance of prompt combinations given performance of component variations. 

Experiments cover a wide range of LMs and tasks. Reviewers note that there is not thorough comparison with existing prompt engineering methods (although authors suggest they will include more baselines in a final version of the paper), and also that the proposed method is constrained by the quality of the underlying prompt library. While the proposed approach doesn't always outperform a random baseline, the authors argue that prompt regression facilitates interpretability by showing the influence of prompt components on the overall prompt.
Decision: Accept