Prompt Exploration with Prompt Regression

Authors: michael  feffer
ronald  xu
yuekai  sun
mikhail  yurochkin
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-20 19:37:53

Abstract: In the advent of democratized usage of large language models (LLMs), there is a growing desire to systematize LLM prompt creation and selection processes beyond iterative trial-and-error. Prior works majorly focus on searching the space of prompts without accounting for relations between prompt variations. Here we propose a framework, Prompt Exploration with Prompt Regression (PEPR), to predict the effect of prompt combinations given results for individual prompt elements as well as a simple method to select an effective prompt for a given use-case. We evaluate our approach with open-source LLMs of different sizes on several different tasks.

Main paper: 
 introduction: 
 Large language models (LLMs) have captured the public's attention and imagination over the past couple of years and are poised to impact various sectors and industries going forward. By prompting and training LLMs in certain ways, researchers, practitioners, and end-users have adapted them to specific problems. This said, success has varied due to randomness quintessential to LLMs. Thus, despite growing interest in prompt engineering, processes involved in deriving prompts differ little from iterative trial-and-error.We focus on a specific type of prompt engineering problem. Namely, given an LLM, a dataset of inputs, and a prompt library composed of individual prompt elements that can be chained together, our goal is to predict how element combinations affect LLM outputs and use these predictions to derive an optimal prompt for the given task. Though this problem does not involve searching over an entire language space, the solution space still grows exponentially with the number of prompt library elements, rendering brute-force approaches practically intractable. Therefore, to address this prompt library search problem, we propose our method, Prompt Exploration with Prompt Regression (PEPR).Using PEPR involves three steps that are outlined in Figure 1 , and PEPR itself comprises two procedures. After building a prompt library for the given task, the prompt regression part of PEPR derives parameter weights for each prompt library element based on how much it affects LLM outputs. Using these weights, the next part of PEPR, prompt selection, chooses prompt elements relative to desired behavior. Finally, the overall prompt is recovered after this prompt selection step. Both the prompt regression and prompt selection parts can leverage either reference text generations or human-labeled preference information to yield prompts in line with the provided data.Figure 1 : Overview of PEPR utilization. After building a prompt library, the prompt regression step of PEPR uses the prompt library in conjunction with reference text or preference information to determine the influence of each prompt element on overall output. The second step, prompt selection, uses the parameters derived from prompt regression and data corresponding to desired behavior to select prompt elements in line with this behavior. Finally, the overall prompt is recovered from prompt selection.Overall, our contributions are as follows:• To our knowledge, we are (or are among) the first to work on this prompt library search problem, which we define more rigorously in the following sections. • We provide mathematical definitions and formulae for the prompt regression and prompt selection components of PEPR. • We validate both PEPR components using several open-source LLMs across a variety of different datasets and tasks, and • We outline clear next steps for future work in this area. 
related work: 
 Prompt engineering We draw inspiration from the work of Prasad et al. (2023) , who note that not all parts of a prompt may be helpful for obtaining good LLM output. As such, they split prompts into parts and iteratively search for the most effective prompts through substituting, adding, and removing these parts. Our approach is similar in that we search for the best way to combine elements from a prompt library given a prompt regression model, but we do not consider substitution or addition operations. We also discuss our findings with those of Sclar et al. (2023) and Zhao et al. (2021) in mind, as they discover that minute prompt formatting like spacing and punctuation can have nontrivial effects on LLM performance. While we do not explore differences at such granular levels, we are concerned with obtaining the best prompt for a given task by building it from a set of prompt elements, which we argue is a similar problem and note their takeaways accordingly. Other prior work considers prompt engineering in the form of iterating on in-context learning examples (e.g., Liu et al. (2022) ), prompt template (e.g., Jiang et al. (2020) ), keywords (e.g., Shin et al. (2020) ), and task description (e.g., Zhang et al. (2023) ). In a recent paper, Shi et al. (2024) consider the problem of general prompt iteration with limited resources by framing the issue of choosing a prompt from a given set as a special case of a multi-armed bandit problem. While we also note resource consumption and utilize a fixed set of potential prompt elements in our work, we diverge from their findings and contributions in that PEPR explores how these elements interact with each other and predicts the effects of adding or removing elements from the prompt on resulting performance without the need to evaluate every prompt variation. In this sense, we conjecture we are among the first to work on this and related problems.Language model alignment Our work builds on-top of foundational LLM research exploring reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022) , in-context learning (Brown et al., 2020) , and Constitutional AI (Bai et al., 2022) . Namely, our prompt regression and optimization processes for preference data are inspired by preference learning approaches found in the work of Ouyang et al. (2022) . Moreover, our application of the ) is similar to the application in the direct preference optimization (DPO) work of Rafailov et al. (2023) , but we note that we diverge from their work in that our score/reward function differs from theirs (as they additionally incorporate log-probabilities of the unprompted base model in their formulation) and furthermore they perform LLM training based on their BT model application whereas we perform regression and optimization over a prompt library. Regarding the work of Brown et al. (2020) with in-context learning and Bai et al. (2022) with Constitutional AI, our prompt regression and optimization methods should work with prompt libraries composed of in-context learning examples or LLM principles. Our work is therefore also similar to that of Sun et al. (2023) who introduce a pipeline for fine-tuning and aligning LLMs to principles, but we again note that we diverge from these works in that we perform no LLM training but rather build an effective prompt from a library. Thus, our work also extends that of Liu et al. (2022) who note that not all in-context learning examples are equally effective. In a similar fashion, our method aims to build an effective prompt from a prompt library whose elements may differ in both the type and magnitude of effect they have on the LLM in question. 
prompt regression: 
 Here, we detail the theory and assumptions behind prompt regression, the first of two parts in our approach to prompt optimization, and illustrate our empirical results.Consider an LLM π and a library of prompts s = (p 1 , . . . , p K ) whose prompt elements p k each steer the LLM in certain ways (e.g., in Constitutional AI (Bai et al., 2022) or via in-context learning (Brown et al., 2020) ). Our main goal is to be able to easily predict the effect of an arbitrary combination of prompts from the library on the model's behavior.Let I = {I k ∈ {0, 1}} K k=1, where I k = 1 implies that element p k is included in the prompt s(I ). We posit that the log-probability of response y given input x and prompt s(I ) is a convex combination of the log-probabilities, or logprobs, of response y given x and elements p k ∈ I:log π(y | (s(I), x)) ≈ ∑ k:I k =1 λ k (I) log π(y | (p k , x)), (3.1)where λ(I ) ∈ ∆ |s(I)|-1 is a set of weights corresponding to each element. 1 To model these weights efficiently, we assume that the elimination or addition of an element from the prompt library does not affect interactions between the other prompts. This assumption is motivated by social choice theory, where it is known as the independence of irrelevant alternatives (Ray, 1973) . Our model of the weights is thusλ k (I) = λ k ∑ m∈I λ m , (3.2)whereλ k := λ k ({I m = 1} K m=1 ), i.e., the weight of the prompt element when prompting with the complete prompt library s. We learn {λ k } K k=1 via a simple constraint regression:min λ∈∆ K-1 n ∑ i=1 log π(y i | (s, x i )) -∑ k λ k δ i k 2 , δ i k := log π(y i | (p k , x i )). (3.3)We note two key advantages of our method:1. (3.1) allows us to estimate the effect of any of the 2 K -1 prompt combinations while only evaluating K + 1 prompts for fitting λ; 2. solving (3.3) does not require knowledge of the reference (correct) y for the inputs {x i } n i=1 (e.g., when there is a finite set of meaningful generations, such as classification or multiple-choice QA, one can simply plug every possible y for every input x i when fitting the regression coefficients). 2We denote this method as PEPR-R as it utilizes reference (log-probability) data.Prompt regression for preference data Aligning LLMs to human preferences is often based on preference data, e.g., RLHF (Ouyang et al., 2022) , where annotators choose a preferred response from several options. Classification and multiple-choice QA can also be viewed as preference data where the correct answer is preferred over others. We demonstrate how our method can be used to automate prompt engineering using preference data.Let {(x i , y 1 i , y 2 i )} n i=1 be a dataset of inputs x i and potential LLM responses y 1 i and y 2 i , where y 1 i is preferred over y 2 i . We adopt a Bradley-Terry (BT) model (Bradley, 1984) :P{y 1 ⪰ y 2 | x; π, s(I )} = 1 1 + exp β log π(y 2 | (s(I), x)) -log π(y 1 | (s(I), x)) , (3.4)where ⪰ indicates preference, β > 0 is a temperature parameter, and our score/reward function is log-probability of the associated response. 3 Instead of performing constraint regression to minimize the difference between estimated and actual log-probabilities, we can substitute log π(y i | (s, x i )) with log P{y 1 ⪰ y 2 | x; π, s(I )} ≈ log π(y 1 i | (s, x i )) -log π(y 2 i | (s, x i )),the approximate4 log-likelihood of desired preferences and do the same withδ i k := log π(y i | (p k , x i )).The resulting objective function ismin λ∈∆ K-1 n ∑ i=1 log π(y 1 i | (s, x i )) -log π(y 2 i | (s, x i )) -∑ k λ k δ i k 2 , δ i k := log π(y 1 i | (p k , x i )) -log π(y 2 i | (p k , x i )). (3.5) The resulting prompt regression model is therefore log π(y 1 i | (s(I), x)) -log π(y 2 i | (s(I), x)) ≈ ∑ k:I k =1 λ k (I)[log π(y 1 i | (s(I), x)) -log π(y 2 i | (s(I), x))]. (3.6)Similarly to regressing on log-probabilities, we do not require knowledge of which of y 1 and y 2 is preferred with preference data. This allows us to use additional "unlabeled" data to learn λ. We name this method PEPR-P as it uses preference (log-probability difference) data.We conducted several experiments to test the independence of irrelevant alternatives assumption by evaluating both versions of PEPR in their ability to predict prompt effects using four different types of datasets and several open-source LLMs.We sample 100 prompts from the databricks-dolly-15k dataset (Conover et al., 2023) and embed a subset of prompt library elements in the system prompt of evaluated LLMs to generate responses like those of a pirate. We then apply PEPR with the full library to build a system prompt aligned with pirate responses.We use test set data from the HateCheck dataset (R öttger et al., 2020) to validate our method. The creators of this dataset note that while hate speech detectors typically do well with overt hate speech, they perform poorly in nuanced scenarios (e.g., reclaimed slurs, hate speech counters). To this end, we crafted a prompt library to address these limitations and applied PEPR to build system prompts to respond to examples with hateful or non-hate depending on content. Note that we run prompt selection once per data class for each of the 11 classes present in the dataset (i.e., we subset data and build a system prompt for that class accordingly, but we regress with all data regardless of class).CAMEL We leverage the Biology and Physics datasets generated from the CAMEL experiments (Li et al., 2023) to test the method on text generation. We sample 100 points from each dataset and apply PEPR to uncover the prompts that generate the best outputs from LLMs in line with those of subject matter experts.We use data corresponding to several tasks from the Natural Instructions (NI) dataset (Mishra et al., 2022) . Specifically, we randomly select 100 data points each from tasks 020 (recognize if the answer to a question is in some context), 195 (sentiment analysis), and 199 (determine if sentences agree with one another) and apply PEPR to build prompts that produce aligned outputs from LLMs.Models For all experiments, we evaluated our approach using models from the Llama-2-chat family of LLMs (Touvron et al., 2023) . As their instruction following and system prompt features lend well to our goal of building the best prompts for particular use-cases, we opt for the chat versions of these language models over their base model variants.For all models and datasets, we used prompts created from our prompt library as system prompts and obtained logprobs and logprob differences associated with desired and undesried generations based on these prompts. After obtaining results from individual prompt elements and the entire prompt library (fed as a single prompt), we performed regression and estimated the behavior of prompts of size 2, 3, and 4. We then compared the predicted results to the behavior of the ground-truth prompt combinations (of the same sizes and element compositions). Note that for datasets with open-ended text responses (such as some of the NI tasks and our Toy Dataset), we work with average logprobs across response tokens (i.e., log-probability of response divided by number of tokens) as well as their differences. , for a regression model reflecting the effects of 10 prompts, we aim to illustrate how its predictions of the prompt with elements 2 and 5 compare to ground-truth outputs). While there appear to be no clear trends across model size, PEPR-P appears to do the same or better than PEPR-R in most cases. Additionally, both versions of PEPR have low error and high correlation in general, suggesting that our assumption of the independence of irrelevant alternatives typically holds.Table 1 : Overview of differences between prompt regression and prompt selection. Prompt regression does not require knowledge of ground-truth, whereas prompt selection does to build an appropriate prompt. PEPR-R formulae are used here; replacing logprobs with differences yields an equivalent table for PEPR-P.Prompt Selection Input π, {x i }, p 1 , . . . , p K ; π, {x i }, p 1 , . . . , p k ; possible generations y 1 i , y 2 i for x i s correct generation y i for x i s Process Learn λ k s.t. for all i ∑ k λ k log π(y 1 i |x i , p k ) ≈ log π(y 1 i |x i , p 1 , . . . , p K ) ∑ k λ k log π(y 2 i |x i , p k ) ≈ log π(y 2 i |x i , p 1 , . . . , p K ) Learn I k s.t. max I ∑ i log π(y i | (s(I, x i ))Output Regression model parameters λ k Prompt element selectors I k 
prompt selection: 
 Here, we show how PEPR can be used for prompt engineering by describing the second part of our approach: prompt selection (see Table 1 for differences from regression).Just as prompt regression can leverage either reference generations or preference data to predict LLM outputs, so too can prompt selection to choose an effective subset of prompt elements from the library. We describe each version of selection accordingly.Prompt selection via log-probability data prompt regression Let {(x i , y i )} n i=1 be a dataset of inputs and corresponding reference outputs. To select the prompt maximizing the likelihood of reference output generations, we apply our model from (3.1) and solve maxI ∑ i log π(y i | (s(I, x i )) → max I 1 ∑ k λ k I k ∑ i ∑ k λ k I k (-δ i k ) = ∑ k λ k I k R k ∑ k λ k I k , (4.1)whereR k = -∑ i δ i k and δ i k := log π(y i | (p k , x i )) as defined in (3.3). This is an instance of an integer program which are typically hard to solve, but, fortunately, we can cast it as a simple linear program by relaxing I k ∈ [0, 1] and applying the Charnes-Cooper transformation for linear-fractional programming (Charnes & Cooper, 1962) . The resulting solution is guaranteed to be on the boundary of the feasible set, i.e., recovering binary I k ∈ {0, 1} prompt selection variables. See Appendix A for details.Now suppose we have a dataset {(x i , y 1 i , y 2 i )} n i=1similar to the prior one except that instead of reference outputs y i , we have preference data in the form of potential outputs y 1 i , y 2 i for each input x i such that y 1 i ⪰ y 2 i . Our goal is to find a prompt maximizing the approximate log-likelihood of desired preferences using the prompt regression model corresponding to preferences (3.6): maxI ∑ i log P{y 1 i ⪰ y 2 i | x; π, s(I )} → max I ∑ i ∑ k λ k (I)(log π(y 1 i | (p k , x i )) -log π(y 2 i | (p k , x i )).This is the same as (4.1) substitutingδ i k := log π(y 1 i | (p k , x i )) -log π(y 2 i | (p k , x i )).To evaluate our approach, we run both versions of PEPR against each dataset to select prompts from prompt libraries and compute performance metrics to assess effectiveness relevant to the given task. For our Toy Dataset, our performance metric is accuracy, i.e., whether the model is more likely to generate the desired pirate output (for each instance, 1 if so, 0 otherwise). For HateCheck sections and Natural Instructions tasks, our metric is top-1 accuracy, i.e., whether the prompted model exhibits correct behavior based on the given input. For the CAMEL datasets, we utilize BERTScore (Zhang et al., 2020) to measure generation quality. The CAMEL datasets do not have preferences, thus only PEPR-R is applicable. We utilize the same models, datasets, and logprobs described in Section 3.2 with the caveat that the optimizing data employed corresponds to desired outputs (in the form of reference generations or known preferences depending on the version of PEPR evaluated).In each setting, we search for the best prompt of at most four elements.Our baselines for these experiments come in the form of a minimally-prompted model (the LLM with basic instructions relative to the task such as, "You are a hate speech detector" (see Appendix B.1) and prompts generated from our libraries at random. For the random prompt sampling baseline we match the evaluation budget of PEPR (see Appendix A for details). Moreover, we run each experiment repeatedly (typically around 1000 times), varying the amount of data used for prompt selection, while reporting the performance on all of the available data. Note that for PEPR-R prompt selection, our reference data are only logprobs corresponding to desired responses (as the likelihood of those responses is what we aim to maximize) unlike the case of PEPR-R regression (for which we use data from all responses as the type of response does not matter). 7 Overall, PEPR prompts perform well, frequently scoring above the 75th percentile of all prompts and sometimes reaching the maximum performance possible. We observe that PEPR-P typically ties or scores better than PEPR-R and conjecture this is due to PEPR-P taking logprobs of undesired responses into account (via the logprob differences) alongside those of desired responses when performing prompt selection, in contrast to PEPR-R which only considers logprobs of desired responses. The better performance of PEPR-P is similar to our higher correlation finding in our prior regression experiments, and additionally we again note no clear PEPR performance trends with model size in our prompt selection experiments. In many cases, PEPR can achieve the best or worst results with Llama-2-13Bchat, evidencing that increases in scale may not always help.8 In addition, we find that PEPR can outperform baselines even when the amount of labeled data (i.e., inputs where either reference or preference outputs are available) used for prompt selection is small. Note that "0.05" corresponds to about 5 labeled points in most experiments, i.e., a few-shot scenario where prompt selection is more appealing than fine-tuning. This low labeled data setting is arguably a more likely application of methods for prompt selection. Moreover, PEPR's prompt selection performance in this small data regime is nearly the same as its prompt selection performance with all of the data for the datasets we explore, additionally suggesting that PEPR is effective in this domain. 9 Furthermore, PEPR outperforms the random baseline by a large margin on the CAMEL Biology and Physics generation tasks in terms of BERTScore. BERTScore compares similarity of generated and reference text, and is thus more sensitive to the prompt, while accuracy only takes into account predictions over a single next token, which we observed to be less sensitive to prompts in our experiments.In the Appendix Table 5 , we additionally compare to TRIPLE-SH (Shi et al., 2024) , a recent algorithm for prompt selection adapting the best arm identification strategy, i.e., it evaluates a few samples with each prompt recursively, eliminating the worst performing prompts at every stage. This algorithm requires evaluating at least a single sample per prompt candidate, which is infeasible in some settings (assuming an evaluation budget comparable to PEPR) since the number of prompt candidates is of the order of 2 K for a prompt library of K elements. PEPR bypasses this limitation due to its ability to model the effects of prompt element combinations via prompt regression without the need to evaluate all of the possible prompts. Given a higher evaluation budget, TRIPLE-SH and PEPR perform comparably. 
discussion: 
 Key takeaways PEPR illustrates that given a fixed set of prompt elements, it is not necessary to evaluate every possible prompt to learn its effect. Namely, related prompts (such as ones that share prompt elements) have related effects (in terms of changes in LLM performance and output) that can be measured and predicted efficiently, eliminating the need to try every prompt. This is supported by the low error and high correlation of our prompt selection experiment results in Section 3.2 and in Appendix B. These findings in turn alleviate the need for a brute-force search that grows exponentially with library size.Limitations Upon closer scrutiny of our prompt selection results in Table 2 and Appendix B, one may observe that our random baseline ties or outperforms both versions of PEPR in a nontrivial number of instances. We acknowledge this, but we also note that many of the cases in which our methods outperform random combinations are ones in which the prompt library has some unhelpful elements (e.g., Toy Dataset, Biology) and in contrast, at least a few cases in which our methods do not outperform random combinations are ones in which the prompt library has many helpful elements (e.g., HateCheck). This suggests that PEPR is effective at filtering out bad prompt elements (such that it can easily find a good prompt) and not as effective at optimally combining good prompt elements (meaning it cannot always find the best prompt). We also argue that the regression and optimization framework provided by PEPR is more interpretable in addition to being more robust than trying prompt combinations at random. Even so, we emphasize that the prompt selection part of PEPR can benefit from future research and that our prompt regression results are interesting as a standalone contribution. In addition, both parts of PEPR are straightforward yet effective solutions to this prompt library search problem.Regarding the prompt library, another potential limitation of our work is our findings are only limited to the prompt libraries we tried for each dataset. Given the formatting issues discovered by Sclar et al. (2023) and Zhao et al. (2021) and prompt iteration operations employed by Prasad et al. (2023) to find effective prompts, one could argue that we may not have found optimal prompts for each dataset because our library was missing (more) useful elements and formatting. In response, we note that our experiments sought to test PEPR against each dataset and use-case, not to achieve state-of-the-art results. As such, we argue that the prompt libraries we utilize here are sufficient as our baselines also leveraged them where appropriate to enable comparisons with PEPR. Furthermore, these libraries were obtained through preliminary experiments and manual refinement not detailed in this work, and the high performance in general of all methods that used them (baselines and PEPR alike) suggests that the libraries at least contain helpful instructions.We encourage the community to conduct further research and experiments to address the limitations above and explore related approaches. For instance, we note that future experiments could examine the effects of other prompt selection approaches (i.e., alternative log-probability maximization strategies), more complex prompts (with elements like "ignore previous instructions" or similar), and more sophisticated prompt regression models (e.g., ones with richer features beyond log-probabilities and differences and/or ones that are nonlinear). Testing whether prompt element ordering has any impacts on final outcomes and observing whether PEPR easily scales with larger prompt libraries would be other useful experiments. Moreover, while we only explore system prompt configurations in this work, experimenting with PEPR using other prompt components (e.g., in-context learning examples, text not part of the system prompt) would be yet another interesting follow-up work. Lastly, while we selected datasets to cover a range of topics pertaining to model safety, algorithmic bias, and everyday usage of LMs, future work could examine more datasets related to these and other topics. 
ethics statement: 
 While our work does not directly deal with ethical issues and usage of LLMs, we acknowledge that PEPR can be utilized for harmful purposes. Namely, as our experiments illustrate how PEPR can discover effective prompts for a variety of harmless and beneficial tasks (such as pirate speech generation, hate speech detection, and knowledge retrieval), they also suggest that PEPR can be used to derive prompts for malicious uses (such as hate speech or misinformation generation). This said, PEPR would require both prompt libraries and data corresponding to such malicious uses and therefore does not come out-of-the-box with harmful capabilities, but we nevertheless highlight the potential risks here and stress that researchers and practitioners should promote responsible usage of our work.Based on (4.1), our prompt selection binary integer program has the following optimization function:max I ∑ k λ k I k R k ∑ k λ k I k ,whereI k ∈ {0, 1} and R k = -∑ i δ i k .In our setup, we specify an additional constraint on I k , namely that we force j of them to be equal to 1. Failing to do so yields an algorithm that greedily chooses the single prompt element with best performance over the given data, but such an approach overlooks prompts with more than one element that may generalize better to unseen test data. Thus, by including this constraint while relaxing the binary integer constraint so that I k ∈ [0, 1] and turning the summations into vector dot products, one yields the following linear-fractional program:max x (λ ⊙ R) T x λ T x , (A.1) s.t. x T 1 = j, 0 ≤ A I x ≤ 1, (A.2)where λ =< λ 1 , . . . , λ K >, R =< R 1 , . . . , R K >, x =< I 1 , . . . , I K >, A I is the KxK identity matrix, ⊙ is the element-wise vector product operator, and 0, 1 are vectors of zeroes and ones, respectively. This is a linear-fractional program due to the denominator term in the optimization function, but applying a Charnes-Cooper transformation yields a traditional linear program (Charnes & Cooper, 1962) . This is done by first applying a change of variables through multiplying both parts of the fraction in the optimization function and both sides of each constraint by a nonnegative optimization (scalar) variable t. The next and final part of the transformation involves focusing on maximizing the expression corresponding to the numerator while fixing the one for the denominator to 1. More specifically, for this problem instance, the transformation produces the following linear program:max y,t (λ ⊙ R) T y, (A.3) s.t. λ T y = 1, x T 1 = jt, 0 ≤ A I y ≤ 1t, 0 ≤ t (A.4)where y is a new optimization (vector) variable equivalent to tx as a result of the change of variables. Therefore, after finding optimal values for vector y and variable t under the given constraints, the I k can be obtained by calculating y t . Note that even though the original binary integer constraints were relaxed to construct a solvable linear program, the values of I k obtained by this final program will be contained in {0, 1} (i.e., will be either 0 or 1) because the optimal solution for the linear program must be located on the boundary that delineates the set of its feasible solutions. 10We solve the final program for values of j ∈ [1, c], where c is the largest prompt size (in terms of number of prompt elements) we consider for a given problem, and we select the best prompt (in terms of evaluation on the entire dataset) from all program solves as our overall prompt. Therefore, our approach requires evaluating K + 1 + c prompts in order to discover one that is (predicted to be) effective. The random baseline that we compare against in experiments utilizes the same budget (in terms of number of prompts evaluated) and considers the same set of prompts (i.e., prompts of at most c elements). More specifically, this baseline entails evaluating K + 1 + c prompt element combinations at random from all possible element combinations of length at most c (e.g., for 10 prompt elements and c = 4, our random baseline may first evaluate the third prompt element alone, next the first four prompt elements, then the last three prompt elements, followed by other combinations until 10 + 1 + 4 = 15 total prompts have been evaluated, at which point it outputs the optimal test set performance). Similarly, as the budget for TRIPLE-SH serves as a limit on the number of total LLM interactions Shi et al. (2024) , we employ a budget of (K + 1 + c)n when running that algorithm, where n is the number of points we use for evaluation from the given dataset.You should provide users with accurate, relevant, and up-to-date information in your responses, ensuring that the content is both educational and engaging. Try not to respond with "I'm not sure". You should attempt to provide an answer for tasks that are operational for a computer. You should speak like a pirate whenever possible. Strictly adhere to the following rules:" was prepended to any chosen subset of prompts. (The second sentence was omitted for the unprompted baseline, but the first was used without any prompt elements.)Be sure to pay attention to context, as hate speech can be nuanced. Do not label abuse targeted at individuals as hateful. Do not label abuse targeted at non-protected groups (such as professions or political affiliations) as hateful. Note that profanity and slurs are sometimes okay depending on context. Do not label text that rejects hate as hateful. Note that denouncement of hate speech is not hateful. Note that threats towards individuals or non-protected groups are not considered hateful. Account for spelling errors in determining whether text is hateful. Implicit offenses towards protected groups (such as ones without profanity or slurs) should be labeled as hateful. Text that is offensive based on race, gender, sexual orientation, religious affiliation, or country of origin should be labeled hateful. Table 3 : Comprehensive accuracy results are reported in Table 3 with standard deviations pertaining to experiment settings. Models here are from the Llama-2-chat family. We include maximum and high percentile results from all relevant prompt combinations and baselines. Note that the high-performing prompts that PEPR suggests have small standard deviations, suggesting the stability of prompts selected as compared to the random method. 0.98 ± 0.01 0.91 ± 0.00 0.99 ± 0.00 0.94 ± 0.03 0.79 ± 0.00 0.96 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.98 ± 0.01 0.91 ± 0.00 0.99 ± 0.00 0.94 ± 0.03 0.79 ± 0.00 0.96 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.98 ± 0.01 0.91 ± 0.00 0.99 ± 0.00 0.94 ± 0.03 0.79 ± 0.00 0.96 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.97 0.87 1.00 0.95 0.85 1.00 0.99 0.96 1.00Threatening language 7B 13B 70B1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.95 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.95 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.95 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.01 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.98 ± 0.00 0.98 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.98 ± 0.00 0.98 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.98 ± 0.00 0.98 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.95 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.95 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.02 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.98 ± 0.00 0.98 ± 0.00 1.00 ± 0.00 1.00 ± 0.01 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 1.00 ± 0.00 0.99 ± 0.00 0.98 ± 0.00 0.98 ± 0.00 Mistral-7B-Instruct-v0.2 We test our method with an additional model, Mistral AI's Mistral 7B Instruct v0.2 (Jiang et al., 2023) , on a subset of datasets used in our main experiments. More specifically, we do so on all datasets except our Toy Dataset and HateCheck. Note that prompt elements can also be defined to depend on the input provided to the LLM (i.e., p k (x)). Our approaches remain the same mathematically, and we consider such prompts in some of our experiments (e.g., with CAMEL Biology and Physics datasets) to show technical feasibility.The total number of terms in (3.3) is nC, where C is the number of possible outputs for an input.The generalized version of the BT model, the Plackett-Luce (PL) model(Plackett, 1975;Luce, 1959), can be used to model preference data with more than two responses.The approximation is accurate assuming larger β.Though PEPR-P induces larger MAE than PEPR-R, we neither recommend nor discuss this comparison as error is semantically different across the two methods (i.e., error in predicting a logprobability is categorically different from error in predicting a log-probability difference).6 If interactions or coupling between prompt elements were to have a major impact, a linear model would not do well here.As the performance of PEPR-chosen prompts is stable, the standard deviations are small, so we omit them here but report them in Appendix B with our full results.Even so, the 7B parameter model never obtains results strictly greater than the 13B and 70B parameter ones, suggesting that scale may help sometimes.Recall that the prompt regression part of PEPR still requires all data available, but it is unsupervised, i.e. does not require reference generations or knowledge of preferred generation.As this objective is convex in I k , this holds by definition. Nevertheless, below is a proof sketch of this result for our setup:If a prompt element is associated with favorable log-probabilities or preferences, then scaling these values by a number in [0, 1) is suboptimal for the overall optimization function, so the associated I k should be 1. Conversely, if a prompt element is associated with unfavorable log-probabilities or preferences, then scaling these values by a number in (0, 1] is similarly suboptimal for the overall optimization function, so the associated I k should be 0. In neither possible case should this program return values for I k in (0, 1). 
 
 Appendix:  


Decision: This paper explores the problem of automatically identifying suitable prompts. This work takes advantage of similarities between prompt variations and their performance, and proposes a method called Prompt Exploration with Prompt Regression that predicts performance of prompt combinations given performance of component variations. 

Experiments cover a wide range of LMs and tasks. Reviewers note that there is not thorough comparison with existing prompt engineering methods (although authors suggest they will include more baselines in a final version of the paper), and also that the proposed method is constrained by the quality of the underlying prompt library. While the proposed approach doesn't always outperform a random baseline, the authors argue that prompt regression facilitates interpretability by showing the influence of prompt components on the overall prompt.

Reviews: review: paper_summary: paper_summary: This work studies the prompt library search problem where the model needs to predict prompt combinations in a library that can lead to optimal performance. This work further proposes a method including three steps: (1) building a prompt library; (2) generating weight for each element; (3) prompt selection based on weights.
strength_weakness: strength_weakness: - There is a lack of comparison with existing prompt engineering methods. The existing comparison with a random baseline is not enough to show the advantage of the proposed PEPR method.

- The performance of this method highly relies on the quality of the prompt library.
paper_summary: paper_summary: The paper introduces a framework, Prompt Exploration with Prompt Regression (PEPR), which aims to optimize the selection of prompt elements for language models. It claims that unlike prior methods, PEPR can predict the output effects of various prompt combinations without exhaustive testing. The authors present a methodology involving building a library of prompt elements, assigning weights through regression analysis, and selecting the optimal combination based on these weights. The paper asserts that its approach, tested on various datasets and model sizes, improves prompt selection efficiency and effectiveness. It suggests that PEPR could significantly streamline the process of prompt engineering, although it also calls for further research to refine and expand the method.

Overall, the paper is well written and introduces an approach for which the COLM conference is arguably the perfect venue. Hence, we believe it should be accepted.
strength_weakness: strength_weakness: - The paper does not fully address the scalability of PEPR with very large prompt libraries or complex prompt structures, which could limit its practicality in extensive applications.
- The methodology may not account adequately for the nuanced interactions between prompt elements beyond their individual contributions, potentially oversimplifying complex linguistic phenomena.
- Further, the methodology relies heavily on the initial quality and composition of the prompt library, potentially limiting its effectiveness if the library is not well-constructed.
- The experiments lack a comparison against a wider range of existing prompt engineering methods, which could help in understanding PEPR's relative performance.

All in all, these reasons are relatively superficial and do not warrant the rejection of the paper. Moreover, the authors address most of them in the manuscript to a great extent.
questions: questions: 1. Can you shed further light on what you mean by "solving (3.3) does not require knowledge of the reference (correct) y for the inputs (e.g., when there is a finite set of meaningful generations (such as classification or multiple-choice QA), one can simply plug every possible y for every input xi when fitting the regression coefficients)". In a quick reading it might sound like the PEPR method only supports classification usecases but that is patently not the case when considering the rest of the paper.

2. Would it be possible to quantify the level of interaction between prompt elements when their effects on language model outputs are not independent and make use of this information in the prompt selection step?
paper_summary: paper_summary: The paper tackles a specific prompt engineering problem for LLMs, aiming to optimize prompt combinations from a library of individual elements. The goal is to predict the effects of these combinations on LLM outputs and create an optimal prompt for specific tasks. The proposed solution, Prompt Exploration with Prompt Regression (PEPR), consists of three steps: building a prompt library, deriving weights for each element through prompt regression, and selecting prompt elements via these weights. This process incorporates generated texts or human preferences to align prompts with the desired outcomes. PEPR's effectiveness is validated using several open-source LLMs across different datasets and tasks.
strength_weakness: strength_weakness: 1. The central concern of the paper is the prompt library search problem within prompt engineering, which seems under-motivated. The quality of the final prompt is inherently limited by the quality of the constructed prompt library, particularly in terms of coverage and richness. Moreover, PEPR's approach to determining the weight of each prompt element through parameter optimization on specific data significantly constrains the generalizability of the resultant prompts. Additionally, the paper does not adequately discuss the scale of data used for parameter optimization, leaving PEPR's practical applicability unclear.

2. The evaluation and rationalization of PEPR's effectiveness are incomplete. The experiments on prompt regression merely test the assumption of the independence of irrelevant alternatives, while those on prompt selection only assess PEPR's impact on task performance under various settings. Crucially, there is a lack of direct comparison between PEPR and the advanced prompt engineering methods described in the related work section. Such comparisons are vital to directly demonstrate PEPR's superiority and the relevance of analyzing prompt element combinations for producing the optimal overall prompt as outlined in the initial sections of the paper.

3. More importantly, current experimental results (referenced in Table 2) show that PEPR barely outperforms or does not surpass the random baseline. As expressed in the last paragraph of section 4.2, even with performance metrics variations across different datasets, these outcomes further challenge the significance and motivation behind the paper's focus on the prompt library search problem.
questions: questions: 1. In the prompt selection experiments illustrated in Figure 2, regression performance deteriorates as the number of prompt elements increases, indicated by larger MAE and smaller r-values. What could be the underlying reasons for this trend? Does this imply that an increase in the number of prompt elements leads to more coupling between them, thereby reducing their independence? Moreover, does the extent of this coupling depend on the quality of the prompt library?

2. What exactly is the "minimally-prompted model" mentioned in the second paragraph of section 4.2? Why was this baseline chosen instead of more advanced methods summarized in the related work section? What are the implications of using such a baseline for the validity of the experimental results?
paper_summary: paper_summary: This paper investigates how to select the optimal prompt from a prompt library given a specific model and a task. The authors propose a prompt regression approach, which utilizes task datasets to fit prompt weights, thereby achieving prompt selection. The authors design regression methods based on log-probability and preference for two types of task datasets, respectively. Overall, the methods proposed in the paper outperform baseline models, demonstrating their effectiveness.
strength_weakness: strength_weakness: - The effectiveness of the methods may vary depending on the nature of the task.

- The proposed method didn't compare with other related works, comparing only with its baselines.

- In some cases, even random prompt combinations can outperform both versions of PEPR.
questions: questions: - The objective of the proposed methods is to learn prompt weights through regression conducted on specific datasets. Once these weights are obtained, they can be applied to each data point in the test set. My concern is whether this approach is task-specific. Do we need to know which task category the input belongs to in order to apply the corresponding prompt weights? If so, does this mean that such methods primarily apply to narrow-domain tasks, while being unsuitable for open-domain dialogue scenarios?

- During training, are multiple datasets mixed for training, or are they trained separately?

- Table 2 should be placed on page 8.

- From the results in Table 2, it seems that the effectiveness of PEPR may be related to the dataset. For some datasets, such as HateCheck and NI 199, even the "random" prompt achieves good results. Although the authors also provide some explanations for this phenomenon in the Discussion section, is it possible that certain tasks are insensitive to prompts?

- Previous studies have shown that large language models commonly suffer from overconfidence issues, wherein even incorrect or illusory generated results are assigned a high probability. Could this potentially affect methods based on log-probability?

- Please provide more experimental details for both the prompt regression and prompt selection to make it more clear and understandable.

- Please explain how the experimental results in Figure 2 validate the assumption that " the elimination or addition of an element from the prompt library does not affect interactions between the other prompts."

- Typo
  
  - Introduction section, " rigorously in the following sections," => "rigorously in the following sections." and "selection components of PEPR," => "selection components of PEPR."

Comments: title: Response to Authors' Rebuttal
Discussion period is now open
comment: Thank you for your responses. I will keep my original scores.
Thank you for your responses. I would like to keep my original scores.
Thank you for your feedback and for considering the rebuttal.
Thank you for your feedback and for increasing the score. We will incorporate the clarifications into the revised draft. We will also add a comparison to Shi et al. (2024) (a recent method studying a related problem setting, but limited to smaller prompt libraries) to strengthen the empirical validation as you suggested.
hi, thank you for your response. I'd like to see this submission accepted, though I stick to my rating score.
Thank you for addressing my concerns. Your responses clarified the motivation for the prompt library search problem and demonstrated the generalizability of PEPR prompts across related tasks. You also provided useful information on the scale of data used for optimization and the rationale behind the "minimally-prompted model." However, some issues remain unresolved, particularly regarding the completeness of comparisons to advanced methods. While I appreciate the effort and the improvements in understanding, the responses still leave some questions about methodological details. Therefore, I have decided to increase the score from 4 (reject) to a borderline accept, reflecting the clarifications provided and the potential of your approach. Further rigorous validation would enhance the confidence in the findings.
Hi reviewers, please take a look at the author's rebuttals and the other reviews for this paper!

If the rebuttals addressed your concerns, please let the authors know about this and update your review. If not, please continue to engage with the authors and the other reviews in the discussion forum.

There is significant disagreement among the reviewers here. 

jCnP, you are the most negative about this paper, particularly with concerns about the limitations of using a prompt library constructed a priori, and a lack of experiments comparing with prior methods, plus a lack of significant improvement over baseline methods. Did the authors' response address these concerns?

t5Qb and cEye, you are both quite positive about the paper. Would you like to argue for its acceptance?