Efficient multi-prompt evaluation of LLMs
Authors: Felipe Maia Polo
Ronald Xu
Lucas Weber
Mírian Silva
Onkar Bhardwaj
Leshem Choshen
Allysson Flavio Melo de Oliveira
Yuekai Sun
Mikhail Yurochkin
Venue: NeurIPS 2024 poster (2024)
Keywords: llm
multi-prompt evaluation
efficient evaluation
evaluation
Primary area: evaluation
TL;DR: We propose an efficient method for multi-prompt evaluation of LLMs.

Abstract: Most popular benchmarks for comparing LLMs rely on a limited set of prompt templates, which may not fully capture the LLMs’ abilities and can affect the reproducibility of results on leaderboards. Many recent works empirically verify prompt sensitivity and advocate for changes in LLM evaluation. In this paper, we consider the problem of estimating the performance distribution across many prompt variants instead of finding a single prompt to evaluate with. We introduce PromptEval, a method for estimating performance across a large set of prompts borrowing strength across prompts and examples to produce accurate estimates under practical evaluation budgets. The resulting distribution can be used to obtain performance quantiles to construct various robust performance metrics (e.g., top 95% quantile or median). We prove that PromptEval consistently estimates the performance distribution and demonstrate its efficacy empirically on three prominent LLM benchmarks: MMLU, BIG-bench Hard, and LMentry; for example, PromptEval can accurately estimate performance quantiles across 100 prompt templates on MMLU with a budget equivalent to two single-prompt evaluations. Moreover, we show how PromptEval can be useful in LLM-as-a-judge and best prompt identification applications.

Efficient multi-prompe evaluation of LLMs
Felipe Maia Ronald Xu Lucas Weber*, Mírian S56 Onkar Bhardwaj Leshem Choshen Allysson Flavio Melo de Oliveira Yuekai Sun Mikhail Yurochkin University of Michigan, MIT, University Pompeu Fabra, *Federal University of Minas Gerais SIBM Research, MIT-IBM Watson Al Lab
Abstract
Most popular benchmarks for comparing LLMs rely limited set of prompt templates, which may not fully capture the LLMs' abilities and can affect the reproducibility of results leaderboards. Many recent works empirically verify prompt sensitivity and advocate for changes in LLM evaluation. In this paper, consider the problem of estimating the performance distribution across many prompt variants instead of finding a single prompt to evaluate with. We introduce Prom a method for estimating performance across large set of prompts borrowing strength across prompts and examples to produce accurate estimates under practical evaluation budgets. The resulting distribution can be used obtain performance quantiles to construct various robust performance metrics (e.g., top 95% quantile or median). We prove that PromptEval consistently estimates the performance distribution and demonstrate its efficacy empirically on three prominent LLM benchmarks: MMLUU BIG-bench Hard, and L.Mentr; for example, PromptEval can accurately estimate performance quantiles across 100 prompt templates on MMLU with a budget equivalent to two single-pun pongl-promppt evaluations. Moreover, we show how PromptEval can be useful in LLM-as-a- -LM-as-a-judge and best prompt identification applications.
Introduction
In recent years, the rapid progress of large language models (LLMs) has significantly influenced various fields by enhancing au- tomated text generation and comprehen- sion. As these models advance in com- plexity and functionality, key challenge that arises is their robust evaluation [Perlitz al., 2023]. Common evaluation meth- ods, which often rely on single or limited number of prompt templates, may not ad- equately reflect the typical model's capa- bilities [Weber et al., 2023b]. Furthermore, this approach can lead to unreliable and in- consistent rankings on LLM leaderboards, as different models may perform better worse depending on the specific prompt
MMLU
0.08
Estimation error
Budget
0.06
x1
x2
0.04
x3
X4
0.02
0.00
25
50
75
95
Quantile
Figure 1: Average estimation error for performance quantiles across 100 templates given limited budget (in multiples one-template MMLU evaluations).
* "Corresponding author: E-mail:
"Our code can be found in aattps://githie can be found in https: / http///gggggaceee.e// co/PromptEval. PromptEvvll is also integrated into PromptBench [Zhu et al., 2024]: please check https:/ http:::///gthbbbb
htt///fllpemaapppoo hwwpoooopppoppttaaaaaommoo and the MMLU data
/githtb.t.. wtt.aa...ooooiicoooottoooi omptbencch.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

template used. An ideal evaluation framework should minimize dependence on any single prompt template and instead provide holistic summary of performance across a broad set of templates. Mizrahi et al. [2023], for example, suggests using summary statistics, such as the average performance across many templates, as a way to compare the abilities of different L.M.s. However, the main drawback of this method is the high computational cost when dealing with numerous templates and examples.
We introduce PromptEval, a method for efficient multi-prompt evaluation of LLMs. With small number of evaluations, PromptEval estimates performance across a large and given pool of different prompt templates. Our approach is grounded in robust theoretical foundations and utilizes well- established models from the fields of educational assessment and psychometries, such as Item Response Theory (IRT) [Cai et al.. 2016, Van der Linden, 2018, Brzezińska, 2020, Lord et al., 1968]. Our method based on an IRT model that allows borrowing strength across examples and prompt templates to produce accurate estimates of all considered prompts with an evaluation budget comparable to evaluating single prompt. In Figure 1, we demonstrate the ability of our method to jointly estimate various performance quantiles across 100 prompt templates with an evaluation budget ranging from one to four times of a conventional single-promp evaluation on MMLU [Hendrycks et al., 2020].
Performance distribution across prompts can be used to accommodate various contexts when compar- ing LLMs [Choshen et al., 2024]. For example, it can be used to compute the mean performance as suggested by Mizrahi et al. [2023]. One can also use performance distributions directly to compare L.MS via various notions of stochastic dominance for risk-sensitive scenarios [Nitsure et al., 2023]. Here we primarily focus on the full distribution and its quantiles as they provide a flexible statistic that can inform decisions in varying contexts. For instance, a typical model performance corresponds to median (50% quantile), 95% quantile can be interpreted as performance achievable by an expert prompt engineer, while 5% quantile is of interest in consumer-facing applications to quantify low-end performance for user not familiar with prompt engineering. We also demonstrate ($6) how our method can be used to account for prompt sensitivity in the LLMdae-a-judge framework [Li et al., 2023] and to do best prompt identification [Shi et al., 2024].
Our main contributions are:
We propose ($3) a novel method called PromptEval which permits efficient multi multi-prompt evaluation of LLMs for a given pool of prompt templates with limited number of evaluations. Moreover, we theoretically show ($4) that PromptEval has desirable statistical properties such as consistency in estimating performance distribution and its quantiles.
We practically demonstrate ($5) efficacy of PromptEval in estimating performance across 100+ prompts and finding the best-performing prompt for various LLMs using data derived from three popular benchmarks: MMLU [Hendrycks et al., 2020], BIG-bench Hard (BBH) [Suzgun et al., 2022], and LMentry [Efrat al., 2022].
We show ($6) how PromptEval can be applied account for prompt sensitivity in the LLM-as-a- judge framework and to identify the best prompt in a large pool of options.
We conduct the first large-scale study of prompt sensitivity of 15 popular open-source LLMs on MMLU. We present our findings based on evaluating 100 prompt templates in Section 7 and release the evaluation data.
1.1 Related work
LLMs' sensitivity to prompt templates The sensitivity of Large Language Models (LLMs) to the prompts is well-documented. For example. Sclar et al. [2023] revealed that subtle variations in prompt templates in few-shot settings can lead to significant performance discrepancies among several open-source LLMs, with differences as large as 76 accuracy points tasks from the SuperNat- uralInstructions dataset [Wang et al., 2022]. Additionally, they report that the performance of different prompt templates tends to correlate weakly between models. This finding challenges the reliability of evaluation methods that depend on a single prompt template. To measure LLMs sensitivity, the researchers suggested calculating a "performance spread," which represents the difference between the best and worst performances observed. Mizrahi et al. [2023] conducted complementary analysis using -tate-of-the-art models and subsets of BigBench and L.Mentry [Srivastava et al., 2022, Efrat et al., 2022]. The authors arrive at similar conclusions with respect LLMs' sensitivity to the used

---

prompt templates and empirically showed that the LLM ranking considering different formats are usually weakly or intermediately correlated with each other. As a solution to the lack of robustness in LLM evaluation, the authors propose the use of summary statistics, as the average performance, for LLM evaluation, Some other works, e.g., Voronov et al. [2024], Weber et al. [2023b,a], show that even when in-context examples are given to the models, the prompt templates can have a big impact on the final numbers, sometimes reducing the performance of the strongest model in their analyses to a random guess level [Voronov et al., 2024]. In different direction, Shi et al. [2024] acknowledges that different prompt templates have different performances and proposes using best-arm-identificate efficiently select the best template for application at hand. One major bottleneck is still on how to efficiently compute the performance distribution for LLMs over many prompt templates; we tackle this problem.
to
Efficient evaluation of LLMs The escalating size of models and datasets has led to increased evaluation costs. To streamline evaluations, Ye et al. [2023b] considered minimizing the number of tasks within Big-bench [Srivastava et al., 2022]. Additionally, Perlitz et al. [2023] observed that evaluations on HELM [Liang et al., 2022] rely on diversity across datasets, though the quantity of examples currently utilized is unnecessarily large. Perlitz et al. [2023] also highlighted the problems in evaluating with insufficient prompts and called to evaluate on more, suggesting evaluating the typical behavior by sampling prompts and examples together by employing stratified sampling, where subscenarios give the strata; in our work, we also apply stratification but consider prompt templates and examples to give the strata. To accelerate evaluations for classification tasks, Vivek et al. [2023] suggested clustering evaluation examples based model confidence in the correct class. More recently, Maia Polo et al. [2024] empirically showed that it is possible to shrink the size of modern LLM benchmarks and still retain good estimates for L.Mss performances. Similarly (and in parallel to this work) Ashury- Tahan et al. [2024] recognized unlabeled examples that better distinguish between models or prompts, by analyzing model outputs on them, hence saving costly annotation for them. Despite these advancements in streamlining LLM evaluations, there are no other works that propose general and efficient method to estimate the benchmark performance of LLMs across prompt templates to the best of our knowledge.
Item response theory (IRT) IRT [Cai et al., 2016, Van der Linden, 2018, Brzeziniska, 2020, Lord et al., 1968] is a collection of statistical models initially developed in psychometrics to assess individuals" latent abilities through standardized tests but with increasing importance in the fields of artificial intelligence and natural language processing (NLP). For example, Lalor et al. [2016] used IRT's latent variables to measure language model abilities, Vania et al. [2021] applied IRT to benchmark language models and examine the saturation of benchmarks, and Rodriguez et al. [2021] explored various uses of IRT with language models, including predicting responses to unseen items, categorizing items by difficulty, and ranking models. Recently, Maia Polo et al. [2024], Shabtay al. [2024] suggested using IRT for efficient LLM performance evaluation; both works used the Performance-IFe IRT (pIRT) estimator to evaluate LLMs. PromptEval is built upon pIRT.
2 Problem statement
In this section, we describe the setup work and what our objectives are. Consider that we want to evaluate a large language model (LLM) in a certain dataset composed of J examples (also known as questions or items the literature) and each one of the examples is responded to by the LLM through prompting; we assume that there exists I different prompt templates that can be used to evaluate the LLM. After the prompt template i E I A [4] and example j E J A [J] are channelled through the LLM, some grading system generates correctness score YI (0, 1}. which assumes 1 when the prompt template and example jointly yield correct response and otherwise?. For each one of the prompt templates i E I, we can define its performance score as
S و Ejer Yij-
The performance scores S.'s can have a big variability, making the LLM evaluation reliant on the prompt choice. To have a comprehensive evaluation of the LLM,, we propose computing the full distribution of performances and its corresponding quantile function, i.e..
F(x) 1 Etez (z) and ((p) - inf E IR F(x) 2 p).
(2.1)
In some cases, the correctness score may be a bounded number instead of binary see Appendix B.
3

---

The main challenge in obtaining this distribution is that it can be very expensive since the exact values for the performance scores S, require I- J evaluations. In this paper, we assume that only a small fraction of evaluations is available, e.g., < 5% of the total number of possible / evaluations, but we still aim to accurately estimate the performance distribution and its quantiles. More concretely, we assume the correctness scores Yiz² are only evaluated for small set of indices & _ I x T; in compact notation, we define YE - {Y,j ((J))E- Here, the letter & stands for evaluations. Using the observed data Yg, our main objective is to estimate the performance scores distribution F (resp.
quantile function Q), ie., computing a function F (resp. Q) that is close to F (resp. Q).
3 Performance distribution and quantiles estimation
We propose borrowing strength across prompt templates and examples to produce accurate estimates for the performance distribution and its quantile function. To achieve that, we need a model for the conrectness scores Yaj 's that allows leveraging patterns in the observed data and estimators for individual S,'s. We start this section by first introducing a general model for Yij 's and then we introduce estimators for the performance distribution and quantile functions.
3.1 The correctness model
We assume the observations Yuj are independently sampled from a Bernoulli model parameterized by promp//cxample-spciine parameters. That is. we assume
Yi Bernoull((uss).
(3.1)
where May denotes the mean of the Bernoulli distribution specific to prompt format and example j. We can write j4j- = u00;,, 3,), where 0,'s are prompt-specific parameters, 8y's are example-specific parameters and j function that maps those parameters the Bernoulli mean. This probabilistic model is very general and comprehends factor models such as the large class of Item Response Theory (IRT) models [Cai et al., 2016, Van der Linden, 2018, Brzeziñska, 2020. Lord et al., 1968]; as we will see, our model can be seen as a general version of an IRT model. For generality purposes, we assume that the parameters 0, 's and B 's can be written as functions of prompt-specific (x, 's) and example-specific (zj s) vectors of covariates. That is, we assume 0; f (2x) or 13 g, (2,), where se and ? are global parameters that can be estimated. These covariates can be, for example, embeddings of prompt templates in the case of 2; 's and some categorization or content of each of the examples in the case of 2j's ss this work, we adopt u(0,,, = a (0, 3,) = (z)( - 9r (2,)). where denotes the standard logistic function and the functions S and g, have their image in IR. That is, our model assumes that
P(Y., =
-
(3.2)
expl- -x;)())))))()))))))))
The functions fu and 9y can be represented with neural networks. On the simpler side, one could just assume fø and 9-, are linear, that is, 0, = +6 x, or Bg = 7 Tzy linear logistic test model in psychometrics [Fischer, 1973, De Boeck, 2004]. We consider that, in some cases, a constant can be embedded in x; in order to include an intercept in the model. When x; and 2j one-hot encoded vectors, i.e., vector of zeros but with I's on the entries i and j. the model in (3.2) reverts to a popular IRT model known as the Rasch model [Georg, 1960, Chen et al., 2023], which is widely used in fields such as recommendation systems [Starke al., 2017] and educational testing [Clements et al., 2008]. One major limitation of the basic Rasch model is that the number of parameters is large, compromising the quality of the estimates for 1/ and 7 when either the number of prompt formats / or the number of examples J is large and |&| is small, i.e., only a few evaluations are carried out. This degradation in the quality of the estimates can directly affect the quality of the performance distribution estimates. Finally, we fit the parameters z and 7 obtaining the estimates ei and f, by maximizing the likeliilddood of the observed data (negative cross cross-entropy loss), i.e.,
2j; this formulation is known as the
(vb,5) E arg max 4.77
Σ (K.))E
Yo log PVy,; = 1; x) 7) + (1 - Y.,) log (1 - IP,,,, 1;4b,7) (3.3)
Realize that fitting the model with linear/affine fo and g.y, including the Rasch model c reduces to fitting logistic regression model with 2; and zj as the covariates. This observation highlights that
For detailed fitting procedure in the Rasch model case, please check Chen et al. [2023].

---

Algorithm 1: PromptEval
Algorithm 2: Two-way balanced sampling
Input: Ye, (ii) covariates and 2j 's.
Input: (i) sets Z and J, (ii) budget B.
Output: Estimates for the performances distribution and its quantile function (2.1).
Output: Observed indices
Initialize 0
Fit and y using observed scores Ye and covariates z4's ss's and zj (3.3).
for b = 0 to B 1-11 do
Among E I with the least number of evaluations, randomly pick one of them and
For cach i E Z, compute S; B[S, Yel (3.4).
call
Compute estimates
Among J E J such that (i, & &, randomly pick from the ones with the least number of evaluations.
F() - 7 (1)20)())
Q(- = inffa F(x)
Update + 3 ((i
return F and Q
return E.
the fitting process expected to be very cheap in practice. For example. in our experiments, we fit logistic regression models in datasets with less than 2k samples and couple of hundred (or a few thousand) columns, which performed few seconds by a modern laptop. We include some more comments on the computational complexity of our method in Appendix C.
3.2 Performance distribution and quantiles estimation using the correctness model
The model in (3.1) can be naturally used for performance estimation. That is, after observing Yg, the best approximation (in the rean-squared-error sense) for the performance of prompt format E I, S, is given by the following conditional expectation
E|S;
Yol
Yu
Σ JEJ.
Pij
JJ:
JEJ.
where J; A A (i.)) E 8) and A JT/JJ. In practice, computing ESS; Ye] is impossible because the parameters 0,'s and By's are unknown. We can, however, use a plug-in estimator for the conditional expectation using their maximum likelihood estimators, changing Pij for (z)( (x())))
A
1- JAJA
E Yel
Yu JEJ.
Σ jEJ,
(3.4)
The basic version of this estimator, when no elaborate covariates (e.g., embeddings) are included, is known as the Performance-IRT (pIRT) estimator [Maia Polo et al., 2024]. We apply extended version of pIRT, which we call X-pIRT, to estimate the performance distribution across prompt templates. After observing Yg and fitting (ú, f), we can compute S, É(S, Ye] for all i I. Then, we define our estimators for the distribution of performances and its corresponding quantile function 2.1 as
f(x) A EiEr ((),,))(a) and @(p) - inf R. F(x) 2).
(3.5)
We name the procedure of obtaining f' and Q PromptEval and summarize it in Algorithm 1.
Sampling Ys We have assumed Ys is given so far. In practice, however, we need to choose E, with |8|5 S B where B E N is the budget, and then sample the entries Y,, for all (i.j) E E. One possible option is sampling (i,j) without replacement from I x J giving the same sampling probability to all entries. This option is, however, suboptimal because its high instability: with high chance, there will be some prompt formats (or examples) with a very low number of evaluations while others will have many. more stable solution is given by Algorithm which balances the number of times each prompt format and examples are evaluated. Algorithm 2 can be seen as woo-ayy stratified random sampling which the number of examples observed for each prompt format is (roughly) the same and the number of prompt formats that observe each one of the examples is (roughly) the same.
5

---

Theoretical guarantees
In this section, we claim the consistency of the distribution and quantile estimators detailed in Algorithm I as I - oc. We prove a result for the case in which fy and g, are linear/aftine functions. Before we introduce our results we need introduce some basic conditions. As an extra result, in Appendix I.1 we also show that our extended version of pIRT (3.4), X-pIRT, is uniformly consistent over all E Z. which can be useful beyond this work. We start by assuming that the covariates are uniformly bounded.
Condition 4.1. There is universal constant c > O such that supsez s | <
The next condition requires the number of unseen examples to increase sufficiently fast I, oc. which a realistic condition under the low-budget setup. Weaker versions of this condition are possible; adopt this one because it makes our proof simpler.
Condition 4.2. Assume (i) m IJ Til is the same for all i's and grows to infinity and (ii) exp(8m)/I 30 as
oo for any >
The third condition requires the model we work with to be correctly specified and the maximum likelihood estimator defined in (3.3) to be consistent I,J oo, i.e., approach the true value. Evidently, |&| needs to grow to infinity as I,J + nevertheless, it could be the case that [E)((I-)) + O. When Jap and gy are linear/affine, the maximum likelihood procedure (3.3) equivalent to fitting logistic regression model and, in that case, the convergence of (&) 9) is well- studied and holds under mild conditions when the dimensions of the covariates are fixed; see, for example, Fahrmeir and Kaufmann [1985].
Condition 4.3. The data point Yıg is sampled from a Bernoulli distribution with mean o (v Iq To zz) for some true global parameter values too and "io- Moreover, we assume that () - (No,, 70) probability as J + OC.
We now introduce the main result in Theorem 4.4, which shows the consistency of the distribution and quantile functions estimators introduced in Algorithm See Appendix I for the proof.
Theorem 4.4. Under conditions 4.1, 4.2, and 4.3, it true that
(Qz(0) (Qx(0)-Q((p)) + in probability as I, J for any p E [0,1]. 1],
and that
WI P) + 0 in probability as 1,J -
where W1FF,F) is the Wasserstein I-distance between the distributions F and F.
5 Assessing multi-prompt evaluation strategies
General assessment We assess the performance distribution and quantile function estimation methodology introduced in $3 in estimating the performance of LLMs and different prompt formats on data from three popular benchmarks. For given LLM and dataset, we consider two evaluation steps. First, we compare the full performance distribution with the estimated distribution, iee. in this case, all quantiles are considered. To compare the full performance distribution F and its estimate F. both defined in $3, we use the Wasserstein I-distance which is equivalent to the average quantile estimation error in this case, i.e.
(F, Jo' -())) = () S
where S(g) (resp. 5(0)) is the i-th smallest value in {S.)ieT (resp. {É[S; Yelher). Second, we estimate some quantiles of interest (e.g., 1///50/75//90 for the performance distribution across prompt formats and compare them with the true quantiles, that is, for some p E [0, we use [Q(p) - Q()) to measure the quality of our estimations.
Data We use data derived from three popular benchmarks: MMLU [Hendrycks et al., 2020], BIG-bench Hard (BBH) [Suzgun al., 2022], and .Menrry [Efrat et al., 2022]. In the following, we give more details about each one of the used datasets.

---

MMLU is multiple choice QA benchmark consisting of 57 subjects (tasks) comprising approx- imately 14k examples. We ran 15 different open-source LLMs (including different versions of Ilama-3 [Meta, 2024], Mistral [Jiang et al., 2023], and Gemma [Gemma et al., 2024J) combined with 100 different prompt variations for each one of the MMLU tasks. We found that, within each one of the MMLU tasks, prompt templates can have great variability in their performances, making within-task analysis most suitable for assessing our method. More details and analysis of the collected data can be found in $7 and Appendix J.
BIG-bench Hard (BBH) is a curated subset of UGG-bench [Srivastava et al., 2022], containing challenging tasks which LLMs underperform the average human score. For BBH, evaluation scores released by Mizrahi et al. [2023]. The evaluation data includes 11 open-source LLMs combined with a different number of prompt variations, ranging from 136 188 formats, for 15 tasks containing 100 examples each.
the
LMentry consists of simple linguistic tasks designed capture explainable and controllable linguistic phenomena. Like BBH, we use data generated by Mizrahi et al. [2023]. The authors made available the full evaluation data from 16 open-source LLMs combined with different number of prompt variations, ranging from 226 to 259 formats, for 10 tasks containing from 26 to 100 examples each.
Methods and baselines We consider different variations of the model presented in (3.2) coupled with Algorithm I; for all variations,
linear fe and 97- The most basic version of the model in (3.2) assumes æ; and 2j are one-hot encoded vectors, Le., vector of zeros with I's on the entries and j. reverting the model to Rasch model [Georg, 1960, Chen et al., 2023]. Despite its simplicity, we show that it can perform well in some cases. A more advanced instance of (3.2) assumes x; are either obtained using a sentence transformer [Reimers and Gurevych, 2019] embed prompt templates or by extracting discrete covariates from the text, e.g., as the presence of line breaks, colons etc. (see Appendix Table 2). An example of a prompt template for LMentry used by Mizrahi et al. [2023] is "Can /category/ be used to classify all the /words/ provided? Respond with either "yes" or "no". Our method also allows using example covariates 2j, however, upon preliminary tests with sentence transformer we didn't 't observe improvements and chose to use one-hot-encode vectors as in the basic Rasch model to represent examples. Next we detail the methods for obtaining the prompt covariates:
* Prompt embeddings. We embed prompt templates using pre-trained sentence transformer variant [Karpukhin et al., 2020] and reduce their dimensionality to d 25 using PCA. This is the most general solution that also works well in practice. We call it EmbPT.
Fine-tuned prompt embeddings. Sentence transformers in general might not be most suitable for embedding prompt templates, thus we also consider fine-tuning BERT [Devlin et al., 2019] as an embedder. To do so, we use evaluation data for all examples and prompt formats from a subset of LLMs (these LLMs are excluded when assessing the quality of our estimators) and fine-tune pert-base -art-bbas-unccseed to predict Yú, as in (3.3). We call this variation EmbFT and provide additional details in Appendix L. We acknowledge that obtaining such evaluation data for fine-tuning teningunng might be expensive, however, it might be justified in some applications if these embeddings provide sufficient savings for future LLM evaluations.
Discrete prompt covariates. For BBH and LMentry, we coded a heuristic function that captures frequently occurring differences in common prompting templates. Examples of such covariates are the number of line breaks or the count of certain special characters (e.g., dashes or colons). Each one of these covariates is encoded in x; for each one of the prompt templates E Z. A full list of the used heuristics is detailed in Appendix M. For MMLU, we adopted approach of [Sclar et al., 2023] to generate prompt variations via templates (see Algorithm 3). which also provides a natural way to construct the covariates, e.g., the presence of dashes or colons.
To the best of our knowledge, the methods introduced in $3 are the first ones handling the problem of efficient evaluation of performance distribution of LLMs across multiple prompts. Thus, we compare different variations of our method with one natural baseline ("avg") which estimates S, by simply averaging Y.j. that is, using the estimates S. = ग Σjer, Vij. The estimates for the distribution and quantile function are then obtained by computing the function in (3.5) using Save instead of š, To make comparisons fair, we sample the data using Algorithm 2 for all methods and the baseline.
7

---

MMMLU
LHentry
BBH
2.16
2.22-
0.12
2.00-
002
2.64-
0:51
2.00
600
12:00
1920
0:03
100
1250
6003
600 Dudge
1900
Bisigel
Budgee
40g
Pewthorete
PE-Embrr
PE-InerT
Figure 2: Performance distribution estimation errors measured with Wasserstein- distance three benchmarks.
5-th Quantile
25-th Quantile
50-th Quantile
75-th Quanttle
95-th Quantile
Extimation MI1E
2.1.6-
0.16-
0.16
3.16-
0.36-
MALLI
0.00
0.00
0.00
3.00
0.00
2.00
062
1200
0.00
300
200
12:00
1552
0.00
032
603
1200
3.00
300
000
60n
1250
1598
0.00
330
060
900
1250
1552
Extmalion 4r00
0.1.6
0.16
0.16
3.16
0.36-
L.antry
2.00
0.00
0.68
3.08
0.68
2.00
683
1200
0.00
310
640
12:0
1902
0.00
602
903
1200
3.00
300
1250
1998
0.00
350
sto
1259
1995
0.20
0.20
Esinetton crrr
0.38
0.26
0.35
88H
0.00
0.08
3.08
0.08
0.00
0.00
1201
0.00
920
0.00
avg
PE Rasch
PE: discrete
PE EmaPT
PE EmbFT
Figure 3: Performance quantile estimation errors for varying quantilex (columns) and benchmarks (rows).
Key results We investigate the effectiveness of the different variations of PromptEval (PE) against the "avg" baseline strategy in quantile estimation and overall performance distribution estimation across prompt templates. In total, we consider five variations of PromptEval: (i) PE-Rasch (model in (3.2) is Rach model), (ii) PE-discrete (discrete covariates are used for prompt templates), (iii) PE-EmbPT (pre-trained LLM embeddings (fine-tuned LLM embeddings are used for prompt templates). Within each one of the benchmarks, conduct different experiment for each one of the tasks, LLMs, and 5 random seeds used when sampling Yg. We report the average estimation етrоr across tasks, LLMs, and seeds, while the error bars for the average estimation eтrоrs across LLMs. We collect results for four different numbers of total evaluations, where |3] E {200, 400, 800, 1600}. To make our results more tangible, 200 evaluations equivalent, on average, to 15% to the total number of evaluations BBH, 0.88% to the total number of evaluations on ..Menrr, and 0.81% to the total number of evaluations on MML.U.
used for prompt templates), and (iv) PE-EmbFT
Distribution estimation. Our results for distribution estimation can be seen in Figure 2. We see that, in general, all variations of PromptEval, including its simplest version (PE-Rasch), can do much better distribution estimation when compared to the baseline. Among our methods, the ones that use covariates are the best ones.
Quantile estimation. Our results for quantile estimation are presented in Figure 3. As before, even the simplest version of our method (PE-Rasch) does much better than the considered baseline. For all the other variations of PromptEval, estimating extreme quantiles is usually hard and needs more evaluations, while more central ones (e.g., median) can be accurately estimated with 200 evaluations, providing more than 100x compute saving in most cases. Regarding the different variations PromptEval, we found that the pre-trained embeddings are robust across benchmarks, while the discrete covariates could not do well on LMentry data. Using covariates obtained via fine-tuning the BERT model provides some further improvements, for example, for extreme quantiles and small evaluation budget settings on MMLU. However, fine-tuning requires collecting
8

---

Cohere Command
Doven15-78-Ches
1istral-7--rsttruu--0..22
LLMMa-2-7BB-Chat
0.5
truch
gicund bush
ghound bruhh deg (9,-2.053
ground truc
WW,-0.0550
(8) (9,-000000
2xg (11-00.07))
TE saach 10:
Kasch (W6, 1aa--mm..00 0.723)
PLitanch 1.0.8
7E-damch (.0-0.039)) PE EmbIT -0,-0.0111
EmbFT w0.013)
0.0
P5 EmoFT (9.-0.004))
PE-EMAP 10:-0.014)
0.0
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
0.3 0.4
AlparaEval2.00 rate
AlpacaEval 2.0 win rate
AbacaEval 2.0 win rate
Alpa capaalEvall 2.0 win rate
Figure 4: Estimating LLM-as aLM-as-a-judge distribution of scores for 100 prompt variations given to the judge.
large amounts of evaluation data and in most cases, we anticipate that it would be more practical to use PromptEval with pre-trained embedder and moderate evaluation budget instead.
Further applications of PromptEval
6.1 Estimating the distribution of scores for the LLM-aas--judge framework
In this subsection, we explore the concept of LLM-aas---jdde using the AlpacaEval 2.0 [Li et al., 2023] benchmark. Specifically, we generate 100 prompt templates' to present to the judge, GPT-4o- mini [OpenAI, 2024], allowing us to assess how sensitive model evaluation is to different evaluation prompts. We evaluated the performance of four LLMs with similar capabilities (Cohere Command Qwenl.5-7B-Chel [Team, 2024],
[Jiang et al., 2023], LLaMa-2-70BBCCH [Touvron et al., 2023]) using only s 2% of the total evaluations (1.6k/80.5k). In contrast to the previous experiments, we do not make changes in the prompt templates given to the evaluated LLMs when giving an instruction. To fit PromptEval, a threshold at 1/2, but we do not binarize the responses at test time. Figure 4 shows that the different variations of PromptEval can obtain a much lower Wasserstein loss (W1) when compared with the baseline "avg". In Appendix G, we provide additional plots for this experiment. Specifically, Figure 11 presents the score distribution histograms for the four models under consideration, while Figure 12 illustrates how certain prompt templates consistently lead the judge to assign higher (or lower) scores across models. Despite this pattern, we observe that the ranking of the four LLLMs changes in 36% of the prompt templates.
binarize AlpacaEval 2.0 instance scores imposing
6.2 Best-prompt identification
The best- pest-prommt identification task [Shi et al., 2024] is to find the best prompt from a set of fixed templates, i.e., the one that gives the best performance for a task at hand, Shi et al. [2024] propose framing this problem as bandit problem and using linear model or an MLP to predict the performance of each prompt template. To apply PromptE- val in this setting we use our model (3.2) and X-pIRT to estimate how good each template is coupled with sequen- tial climination algorithm [Azizi et al., 2021] (as in Shi et al. [2024]) to select prompt-example pairs for evalua- tion in each round. In Figure 5 we compare our PE to the baseline TRIPLE-GSE [Shi et al., 2024] with logis- tic regression performance predictor and the same three types of covariates (PE-OneHot corresponds PE-Rasch in previous experiments). For all covariate choices, we show that using PromptEvvl for best-prompt identification results in lower regret, i.e., the performance of the best template minus the performance of the chosen template. We include the full results for other benchmarks and also apply TRIPLE-GSE with an MLP in Appendix H.
PE-Dneltot
0.04
PE-Sacreet
Embrm
TRIPL D-ct--Dmette
Regret
0.03
TRIPLE GSE-EnePT
0.02
0.01
500
1000
1500
Budget
Figure 5: Best-prompt identification.
'e generate 10k variations using ChatGPT and undersample to 100 deleting prompts that are too similar to each other. You can see our code here. 'https:/ docs. cohere. com/ v2/ docs/ http///cc/coo//// cowmand-monnooooeeooooooo

---

7 Analysis of prompt sensitivity on MMLU
Prior work reports strong sensitivity LLMs spurious prompt template changes (see Section 1.1). For example, Sclar et al. [2023] observe performance changes of up to 80% for Natural Instructions tasks [Wang et al., 2022] due to template changes. Despite its popularity, no such analysis exists for the MMLU dataset to date. We here provide an in-depth analysis of MMLU prompt sensitivity.
Performance spread When averaged across subjects, we observe relatively small perfor- mance spreads per LLM compared to other datasets in the literature (see Figure 16 in the Appendix K). For example, we can consistently identify 1aama-3-70B-nma
tan-El
20.0
tan-th-ol
17.5
fan-tb-ul
lama-3-700
15.0
Densily
12.5
bicon-1800
as the best
10.0
performing model, independent of the prompt template. On the other hand, scores within in- dividual subjects are highly inconsistent. Fig- ure 6 shows the distribution of prompt spreads (max-min acc.) across subjects per LLM. Most LLMs demonstrate significant average spread of around 10% the subject level.
germe-7h
5.0
2.5
0.0
0.3
Max Min Accuracy
Figure Accuracy spread across 57 subjects.
Template consistency In practice, having consistently performing templates is highly relevant within a single LLM or across LLMs for the same subject. To evaluate the template consistency, we rank template performances either across subjects across LLMs to then calculate the agreement across those rankings using Kendall's W [Kendall and Smith, 1939, inspired by Mizrahi et al. 2023).
Within LLMs, we observe that Gemma-7B-i- has notably higher Kendall's W of 0.45 than any other model, meaning a fixed set of prompts performs best across many subjects (for full results, see Table 1 in the Appendix). Across LLMs, we do not observe high correlations within any of subjects (see Figure in Appendix K). Hence, similar to previous findings [e.g. Sclar al., 2023], we do not identify any coherent template preferences across LLMs (for detailed results, Appendix K).
Conclusion
PromptEval enables a more comprehensive evaluation of LLMs. We hope that comparing distributions or quantiles across many prompt variants will enable more robust leaderboards and address the common concern of comparing LLMs with a single pre-defined prompt. Prior our work, major limitation of such evaluation was its cost. We demonstrated empirically across several popular benchmarks that our method can produce accurate performance distribution and quantile estimates at the cost of 2-4 single- Lingle-prompt evaluations, out of hundreds possible. However, several questions remain: how to decide on the set of prompts for evaluation and how to best utilize our distribution estimates for comparison in various contexts. For the former, we utilized suggestions from prior work [Mizrahi et al., 2023, Sclar et al., 2023] and for the latter, we primarily focused on quantiles as well-established robust performance measures.
Besides evaluation, another common problem in practice finding the best prompt for given task. Our method can be applied in this setting when there is a pre-defined set of candidate prompts (Figure 5). However, several recent works [Prasad al., 2023, Yang et al., 2023, Li and Wu, 2023, Ye et al., 2023a] demonstrate the benefits of dynamically generating new prompt candidates. For example, Prasad et [2023] propose an evolutionary algorithm that creates new prompts based on the ones that performed well at an earlier iteration. Extending PromptEval to accommodate an evolving set of prompt candidates is an interesting future work direction.
We comment on the limitations of our work in Appendix A.
Acknowledgements
This paper is based upon work supported by the National Science Foundation (NSF) under grants no. 2027737 and 21
10

---

References
Shir Ashury-Tahan, Benjamin Sznajder, Leshem Choshen, Liat Ein-Dor, Eyal Shnarch, and Ariel Gera. Label-efficient model selection for text generation. arXiv preprint arXiv:2402.0789,, 2024.
Mohammad Javad Azizi, Branislav Kveton, and Mohammad Ghavamzadeh. Fixed-budget best-arm identification in structured bandits. arXiv preprint arXiv:2106.04763. 2021.
Elron Bandel, Yotam Perlitz, Elad Venezian, Roni Creedman-Melameed Ofir Arviv, Matan Orbach, Shachar Don- Yehyia, Dafna Sheinwald, Ariel Gera, Leshem Choshen, Michal Shmueli-Scheuer, and Yoav Katz. Unitxt: Flexible, shareable and reusable data preparation and evaluation for generative ai, 2024.
Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. Open Ilm leaderboard. https:
//huggingfaach
haw.giinnnace..osppcceeeee. http://aacce//uggggggaaeeeee// /oppe/.////////e//eeeeeeeeeee papen_llaa 11m leaderboard, 2023.
Justyna Brzezińska, Item response theory models in the measurement theory. Communications in and Computation, 49(12):3299-3313 2020.
Li Cai, Kilchan Choi, Mark Hansen, and Lauren Harrell. Item response theory. Annual Review of Statistics and Its Application, 3:297-321,
Yunxiao Chen, Chengcheng Li, Jing Ouyang, and Gongjun Xu. Statistical inference for noisy incomplete binary matrix. Journal of Machine Learning Research, 24(95):1 2023.
Leshem Choshen, Ariel Gera, Yotam Perlitz. Michal Shmueli-Scheuer and Gabriel Stanovsky. Navigating the modern evaluation landscape: Considerations benchmarks and frameworks for large language models (Ims). In International Conference on Language Resources and Evaluation, 2024. URL https:tt////. / /api...... swmanticscholar org/ www.oorgggouui CorpusID :229p0022222 269804253.
Douglas Clements, Julie Sarama, and Xiufeng Liu. Development of measure of early mathematics achievement using the rasch model: The research-based early maths assessment. Educational Psychology, 28(4)4457-482, 2008.
Paul De Boeck. Explanatory item response models: A generalized linear and nonlinear approach. Springer Science & Business Media, 2004.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chupter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186, 2019.
Avia Efrat, Or Honovich, and Omer Levy. Lmentry: A language model benchmark of elementary language tasks. arXiv preprint arXiv:2211.02065 2022.
Ludwig Fahrmeir and Heinz Kaufmann. Consistency and asymptotic normality of the maximum likelihood estimator generalized linear models. The Annals of Statistics, 13(1):342-368, 1985.
Gerhard H Fischer. The linear logistic test model as an instrument in educational research. Acta psychologica, (7(6))359-3744 1973.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 12 2023. URL https://zenoooo wrt.eorgg//.ooo.eeeoeeee rrcoord//12566ssee
Team Gemma, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju. Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08299 2024.
Rasch Georg. Probabilistic models for some intelligence and attainment tests. Copenhagen: Institute of Education Research, 1960.

---

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03. 03300, 2020.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online, November 2020. Association for Computational Linguistics. doi: 10. 18853//1/2202..cco 0etp2200..emmmmmmmm.55.... URL https: : /twp///wwww///// aclweb.. org/ httg//athologyy20000 /2020. emnlp-man halppmmmi..5555.
Maurice G Kendall and B Babington Smith. The problem of rankings. The annals of mathematical statistics, 10(3):275-287, 1939.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 1412.6980 2014.
John P Lalor, Hao Wu, and Hong Yu. Building evaluation scale using item response theory. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference Empirical Methods in Natural Language Processing, volume 2016, page 648. NIH Public Access, 2016.
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. Alpacaeval: An automatic evaluator of instructiion-folloiin models. attps:///gt thub. cow/tatsu- hatpaattsuuaaaaaaaaaeaaeeeeeeeee _eval, 2023.
Yujian Betterest Li and Kai Wu. Spell: Semantic prompt evolution based Ilm. arXiv preprint arXiv:2310.01260, 2023.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXhv:2211.09110 2022.
FM Lord, MR Novick, and Allan Birnbaum. Statistical theories of mental test scores. 1968.
Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuckai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating iims with fewer examples. In Forty-first International Conference Machine Learning, 2024.
Meta. Introducing meta Ilama 3: The most capable openly available Ilm to date. https://ai.mett meta. com/blog/neta- -lama- 3, 2024.
Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of what art? call for multi-prompt Ilm evaluation. arXiv preprint arXiv:2401.00594 2023.
Apoorva Nitsure, Youssef Mrouch, Mattia Rigotti, Kristjan Greenewald, Brian Belgodere, Mikhail Yurochkin, Jiri Navratil, Igor Melnyk, and Jerret Ross. Risk assessment and statistical significance in the age of foundation models. arXiv preprint arXiv:2310.07132, 2023.
OpenAl. Gpt-4o mini: advancing cost-efficient intelligence. attps://openns com/index/ 2024. Accessed: 2024-09-29.
go 5ot-4o-mins mwiimmiiiiaa www--avvvnnnnncooaae cost
Yotam Perlitz, Elron Bandel, Ariel Gera, Ofir Arviv, Liat Ein-Dor, Eyal Shnarch, Noam Slonim, Michal Shmueli-Scheuee and Leshem Choshen. Efficient benchmarking (of language models). arXiv preprint arXiv:2308. /696, 2023.
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: instruction search for prompting large language models. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3827-3844 2023.
Gradient-free. edit-based
12

---

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URI. http//axxv....aaaaee.//. httpg//b////00 10084.
Sidney Resnick. probability path. Springer, 2019.
Pedro Rodriguez. Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd- Graber. Evaluation examples are not equally informative: How should that change NLP leader- boards? In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 1Ith International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pages 4486-4503, On- line, August 2021. Association for Computational Linguistics. doi: 10. 00.18653/v1/2022 wawp://////2....oooog.3.33. URL https:://///. htt////ccaaatoooggyy. org/202 walll.cclllnnn.. 346.
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design How learned start worrying about prompt formatting arXiv preprint arXiv:2310.1111. 11322444,, 2023.
Nimrod Shabtay. Felipe Maia Polo, Sivan Doveh, Wei Lin, M Jehanzeb Mirza, Leshem Chosen, Mikhail Yurochkin, Yuekai Sun, Assaf Arbelle, Leonid Karlinsky, et Livexiv multi-modal live benchmark based on arxiv papers content. arXiv preprint 11X1:22410.10783 2024.
Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen. Best arm identification for prompt learning under a limited budget. arXiv preprint urXiv:2402.09723 2024.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro. Aditya Gupta, Adrià Garriga-Alonss et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.044 04615, 2022.
Alain Starke, Martijn Willemsen, and Chris Snijders. Effective interface designs increase energy-efficient behavior in a rasch-based energy recommender system. In Proceedings of the eleventh ACM conference on recommender systems, pages 65-73, 2017.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le. Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether tfain-of-thouuht can solve them. arXiv preprint 2rXiv:2210.09261, 2022.
Qwen Team. Introducing qwenl. 5 qwenlm. github. io, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.00 092888 2023.
Wim J Van der Linden. Handbook of item response theory: Three volume set. CRC Press, 2018.
Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, and Samuel R Bowman. Comparing test sets with item response theory. arXiv preprint arXiv:2106.008 2021.
Rajan Vivek, Kawin Ethayarajh, Diyi Yang, and Douwe Kiela. Anchor points: Benchmarking models with much fewer examples. arXiv preprint arXiv:2309.08633 2023.
Anton Voronov, Lena Wolf, and Max Ryabinin. Mind your format: Towards consistent evaluation of in-context learning improvements. arXiv preprint arXiv:2401.0777 2024.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaci, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, al. Suppernaturallnsttuccios::: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint Xr0::22004. 07705, 2022.
Lucas Weber, Elia Bruni, and Dieuwke Hupkes. The iel consistency test. arXiv preprint arXiv:2312.00445, 2023a.
13

---

Lucas Weber, Elia Bruni, and Dieuwke Hupkes. Mind the instructions: holistic evaluation of consistency and interactions in prompt-based learning. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLI.), pages 294-313, 2023b.
Chengrun Yang, Xuezhi Wang. Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. arXiv preprint arXiv:2309.034009 2023.
Qinyuan Ye, Maxamed Axmed, Reid Pryzant, and Fereshte Khani. Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661 2023a.
Qinyuan Ye, Harvey Yiyun Fu, Xiang Ren, and Robin Jia. How predictable large language model capabilities? case study on big-bench. arXiv preprint arXiv:2305. 2023b.
Kaijie Zhu, Qinlin Zhao, Hao Chen, Jindong Wang, and Xing Xie. Promptbench: A unified library for evaluation of large language models. Journal of Machine Learning Research, 25(254): 1-22, 2024.
14

---

A Limitations
While our method offers a more reliable and flexible evaluation, it relies on using multiple prompts. As result, if selecting a single prompt was challenge in earlier benchmarks, determining the appropriate set of prompt templates now becomes the key challenge. Although methods have been proposed for generating and diversifying multiple prompts [Mizrahi et al., 2023], and the selection of individual prompts becomes less critical when many are used, this remains limitation for future work to address.
Another limitation is that we do not focus on prompt engineering or attempt to solve this issuc. While addressing prompt engineering would be significant contribution to the field, approach assumes a predefined set of prompts and focuses solely on evaluation or optimization within that pool of prompts. This is practical and widely used setting but does not address the broader prompt engineering challenge.
B Adapting the correctness model for bounded Yi
There might be situations in LLM evaluation in which Yı, & {0, 13 but Yu E [0, 1]. For example. in AlpacaEval 2.0 [Li et al., 2023], the response variable is bounded and can be translated to the interval [0, 1]. Also, some scenarios of HELM [Liang al., 2022] and the Open LLM Leaderboard [Beeching et al., 2023] have scores in [0. 1]. One possible fix is changing the model for YúJ. For example, if Yú are continuous, the Beta model would be appropriate. Another possibility that offers a more immediate fix is binarizing Yıy as proposed by Maia Polo et al. [2024]. That is, using a training set containing correctness data from LLMs, we could find constant such that Eng, Yijt 23 Li.y, [Yar 2 c). where the index / represents each LLM in the training set. Then, we define Yi A 1[2 2 cl and work with this newly created variable.
C Comments on the computational complexity of PromptEval
Consider the case of our experiments in which prompts represented by embeddings of fixed size, examples are represented by one-hot encodings, and our model is given by logistic regression. Because the dimension of the embeddings does not depend on the number of prompt variations, the number of samples and variables used to fit our model does not vary with the number of prompt variations. Then, computational costs are constant with respect the number of prompt templates. On the other hand, the number of variables (and consequently samples, to make estimation possible) should increase linearly with the number of examples, which are usually hundreds or few thousand. Thus, this should not be problem in most practical cases.
D Computing resources
All experiments were conducted using virtual machine with 32 cores. The results for each benchmark separately can be obtained within 3-6 hours.
For fine-tuning BERT embeddings, employ multiple NVIDIA A30 GPUs with 24 GB vRAM, requiring 70 hours of training and an additional approximately 350 hours for hyperparameter search. Fine-tuning can be conducted on GPUs with smaller capacities.
E Estimation errors by task
In Figures 7, 8, and 9, we analyze the Wasserstein 1-distance per task for each benchmark when using the method PE-EmbPT, robust and versatile variation of PromptEval. The results show that for BBH and LMentry, the estimation errors (Wasserstein l-distance) are more uniform across tasks compared MMLU, where some tasks exhibit higher estimation errors. This discrepancy occurs because all tasks in BBH and LMentry have the same number of examples, whereas tasks in MMLU, particularly those with higher estimation errors, have a significantly larger number of examples when compared to the others. In those cases, a larger number of evaluations is recommended.
15

---

Bw200
8e400
B=800
B=1600
0.15
Wasserstein 1-dist.
BBH tasks
10:
0.05
LLMs
Figure 7: Estimation enror for the BBH tasks.
B=200
Bm400
B=800
B=1600
Wasserstein 1-dist.
LMentry tasks
0.10
0.05
LLMs
Figure 8: Estimation error LMentry tasks.
F The influence of the number of prompts in PromptEval performance
We repeat the main experiment in the paper (randomly) cutting the number of prompt templates by factor of 5. This means we use only 20 prompt variations in MMLU, for example. In summary, PromptEval still does well, beating the baseline. However, the gap between PromptEval and the baseline has shortened due to fewer variations. This fact highlights that the bigger the number of templates, the more useful PromptEval can relative to the baseline. The results are depicted in Figure 10.
G Extra plots for the LLM-as-a-judge experiment
In Figure 1I, we show the performance distribution histograms for the four considered models across prompt templates. Figure 12 shows that prompt templates consistently lead the judge assign higher (or lower) scores across models; in this plot, we normalize the scores within rows so is assigned to the lowest score and is assigned to the maximum score (brighter colors denote higher scores).
H Extra results for best- identification
In Figures 13, 14, and 15, we can see the full results for MMLU, BBH, and LMentry. For all benchmarks, we can see that within each triple "PE", "TRIPLE-GSE", "TRIPLE-MLPPGSE" the "PE" version always has some advantage with a lower regret.
The tuning and fitting process of the Multi-Layer Perceptron (MLP) classifier involves setting up a pipeline that includes feature scaling and the MLP classifier itself, which has 30 neurons in its hidden layer. This process begins by defining a range of values for critical hyperparameters: the 12 regularization strength is tested over the range from 0.001 to 10, and the initial learning rate is tested over the range from 0.001 to 0. These values are systematically tested through cross- cross-valices malssdaiioo to determine the optimal combination. During this phase, cross- validdaaloo ensures that the model evaluated on different subsets of the data to prevent overfitting and ensure robust performance. Once the best hyperparameters are identified, the final model is trained on the entire dataset using these optimal settings, resulting in a well-tuned MI.P classifier ready for deployment.
16

---

Be-200
Bw400
B=80p
B-1600
0.30
10-
0.25
Wasserstein 1-dist.
0.20
MMLU tacks
0.10
0.05
LLMs
Figure 9: Estimation for the MMLU tasks.
MMLU
LMeenryy
BBH
Wassensrein 1distancc
0.36
0.12
0.12
0.12
0.04
0.04
0.04
0.00
0.00
503
Budget
Budget
Budget
avg
Rasch
PE discrete
PE-EmbPT
PE-EmbFT
Figure 10: Performance estimation quality when we keep only 20% of the prompts variations for each task, In summary, PromptEval still does beat the baseline. However, the gap between PromptEval and the baseline has shortened due to fewer variations.
I Theoretical results
L1 Consistency of X-pIRT
In Theorem L.I, claim that the X-pIRT estimator is uniformly consistent over all I.
Theorem I.1. Under conditions 4.1, 4.2, and 4.3, it is true that
suptez
B3 S
Ys
S - in probability as I, J + xc.
A direct consequence of Theorem I.I is that
| Eiez ÉsS, Ysl - Eiez S S E
BIS, Ysl - S - supiez EsS, Ysl S + 0
in probability J co. This means that the mean of predicted performances is also consistent if a practitioner wants to use it as summary statistic.
The proof of Theorem I.I is embedded in the proof of Theorem 4.4.
1.2 Proof of Theorem 4.4
For the following results, we denote s) x; as 0; and TT,, 2, as By, and TT,, r, as and 4T 2; as 3,- Moreover, if sequence random variables (X,) converge to in distribution, we denote X, op(1).
17

---

Cohere Command
Qwen1.3-78-CM
Mistral-0-instruud-000
LLaMa-2-70B-hhat
35
30-
30
20
25
20-
20
15
10
10
10
0.20
0.13
0.20
0,23
0.50
0.10
0:20
0.25
0.30
0.19
0.25
0.20
0.25
0.30
0.30
0.13
0.20
0.23
9,50
AlpacaEval 2.0 Win Fate
AlpacEEval 2.0 Win Rate
AbacaEvall 2.0 Win Rate
AlpacaEval 2.0 Win Rate
Figure 11: Performance distribution histograms for the four considered models across prompt templates.
Normalized AlpacaEval 2.0 win rate
0.0
2.5
20
40
80
prompt templates
Figure 12: Prompt templates consistently lead the judge to assign higher (or lower) scores across models..
Lemma 1.2. Under Conditions 4. and 4.3, we have that sup 0, op(1) and supjed 13j
8y1 (1) as I,
Proof. We prove that supiez (0, - 0,] (0())). The second statement is obtained in the same way.
See that
8 0,1 sup ( +b) x3 S sup 2 |a55 s c/ 26 op(1)
sup 1E
2;
J 20. Where the first inequality is obtained using the Cauchy-Schwand inequality, the second obtained using Condition 4.1, and the last equality is a consequence of Condition 4.3 and the continuous mapping theorem [Resnick, 2019].
Lemma L.3. Under Conditions
4.1 and 4.3, it is true that
sup RET
S
Yel E[S;
YE] = op(1) as I,J + oc.
Proof. See that
1 - A
sup ieI
S
Yg]
E(S
Ye]
sup iEZ
Σ 00, B,) -0) 3,)
J J7
jRJ,
1 - A
Σ
a B,) a 8,)
sup 1ET J [J.
3KJ;
1 X;
S sup
0, 3, + +
sEz 4(J - (J.1))
infe Ax
ssup 10, 0,1 + sup 13, 834
4
5 sup Ô. 0: + sup 133 3₂1
(1(
where the third step is justified by the fact that or is 1 1/4 4-iishchhi and the last step justified by Lemma 1.2.
Lemma 1.4. Under Condition 4.2, it is true that
sup [ Yel
Sil = (1(
(1) as 1, co.
18

---

0.05
TRULL GAL--G2-- Cratton
0.04
TRYLL
Regret
CLLGGEL--OOO 1r m oit diome Pa et imber
0.03
0.02
0.01
500
1000
1500
Budget
Figure 13: Best pest-prompt identification for MMLU
12 Orefts
0.12
0.10
TREPL DOE dicido
TRIPL OHEPUT
Regret
0.08
TAPLE GIE-ERI TRULL-- G2E-2NART
0.02
500
1000
1500
Budget
Figure 14: Best-prompt identification for BBH
Proof. For an arbitrary > 0, see that
P sup E[S; YE]
St
=
U Ysl - 5 2
iET
Vie
(LEiS;
Ysl - -
KE
P
A; Σ
Yis +
1- A
Σ
0(0 3B))
-3))-
Σ Yu
26
[T+ jET
| T.
j2T
JET
Σp P
(1 Ax)
Σ
Zig 2
J Tsl
IET
jEZ.
Z 2
VEI
JæJ.
where Zij A Yú (0, B,). Consequently, |Zyl S 1 and B[Z,y] inequality, we obtain
Applying Hoeffding's
IP
sup E[S: Yel - -S 2 < 2lexp (-2me²) iET
2exp (log I 222mm
2exp (log(exp(2e2m)//1
Lemma 1.5. Let .I.... On and by.... bn be two lists of real numbers and og and by be the p-lower quantiles of those lists. Admit that there my a ''s lower than ag m2 a 'ss equal to as (besides
19

---

0.3
TIBUD CSE Oveirsl
TRP E-GE dismn
TAP-E-64E EmrPT
Regret
0.2
TAPLD WLP------------ S5af-
0.1
0.0
500
1000
1500
Budget
Figure 15: Best-prompt identification for I.Mentry
a; itself), and my a's greater than 0g Then, there are at least m, + 1 b's lower equal to by and at least mi b's greater or equal to by.
Proof. If a, the p-lower quantile of A {111'' .. anJ. then by definition a; is the lowest value in A such that
Ha E A: : a aa) + Ha E A: : ai > a} 2 p.n
m2+1
Because is the lowest value in A to achieve that, then m, < p n. This implies that there are at least my 1 values in B {b1. bn lower or equal to by as the p-lower quantile of B.
Finally, because mi + m2 + 2 p n, we know that cannot have more than m2 + m2 values strictly lower than bj. otherwise the p-lower quantile of B could not be bj but some of those values. Therefore, B have at least my values greater or equal to bj-
Lemma L6. Let and i* be indices in I such that Q(p) = E.S. Ysl and Q(p) = S for an arbitrary fixed p [0, Under supter [B[S; Yg] - Syl for arbitrary c 0, if [[S,,
Ye] - S > 2e, for some i' E I, then + Consequently, if i' then [ Yg] - S I| s 2e
under sup B[S: Ye] S.1 S.
Proof. Define A L {Si..... . and assume that there are M1 values in A lower than S M2 values equal to St- (besides S,. itself), and M, values greater than Sp,. If [Sp S > 2e, for certain index i', there are two possibilities: (i) Sy + 2e < St- or (ii) S. + 2 < Sy. Under the event
sup,ez [BSS YE] S.l S we have:
If (i) holds, then there is at least M2 + M3 + values of É[S Yel's such that É[S,, Ye] < [SS Ye] (including SS YgD. This implics that at most M1 values of E[S, Ygl's will be less or equal E Yz]. By Lemma I.5, we know that j' +
If (ii) holds, then there is at least M, - M2+ + values of É.S. YEl's such that B[S, Ye] > [5 Ye] (including S YeD) This implies that at most M3 values of [E[S; Yel's 's will be greater or equal S.S. Yg]. By Lemma I.5, we know that j }..
This means that under supjez EiS, for some E I, then i + i
Yg] Sy S e for an arbitrary > 0, 0f Yel -Sp1 2c,
Proof of Theorem 4.4 (Part 1). Let and be indices in I such that Q(p) E|S Ys] and Q(p) = Sc. Notice that Lemma 1.6 guarantees that sup:ez [8[S, Ys] - S, S implies [[[S;
Ysl S 2c, for arbitrary 0. Consequently.
TP (1a[)) Ys) S. s 2x 2 P sup |8s Ys) S.1s + o(1)
20

---

because
sup iE7
Eis, Yg - 5:
- sup iEZ
E[s: Yg] E[S; Yg
+sup iEZ
[B[;; Yg] Sil or(1) as I,J -
holds by lemmas 1.3 and 1.4. Because > 0 is arbitrary, we have that (Ô(p) Q(p) op(1).
Proof of Theorem 4.4 (Part 2). We start this proof by showing that
IQ(U) Q(U) or(()
with U Unif(O, 1] independent of Q and Q.
For an arbitrary c > 0, see that
lim F(QQU)-Q(U))) Q(U)) 1,J
lim E ((((())-(U))3>>) 1,0-00
E lim P(I((U) Q(U)) T.Jo
where the second equality justified by the Dominated Convergence Theorem [Resnick, 2019) and last one is justified by (Q(p) Q(p) = op(1). Now, we see that
J. (Q(t) - (())dtt
lim [FW = lim 1,
1,
[(Q())
= lim
1
where the last step is justified by Fubini's Theorem [Resnick, 2019], (Q(U) Q(U) op)(), and the Lebesgue Dominated Convergence Theorem [Resnick, 2019], For arbitrary and applying Markov's 's inequality, we get
lim ((W 1,J-000
> 2 lim EF1(F,,)) 1,J
J Details MMLU data
Algorithm for automatically generating templates can be seen as a graph traversal of template graph, whose nodes are defined by which features they have: separator SEP, space SPA, and an operator OP. By traversing this graph, we can collect unique templates that can used in the evaluation of LLMs tasks.
Algorithm 3: TemplateGeneration
Input: Base prompt template features: Separator SEP, Space SPA, Operator OP.
Output: Prompt templates.
From template agenda, pop template. Swap SEP with another SEP, add to templates. Swap SPA A with another SPA., A, add to templates. Swap OP with another OP. add to templates. Add the generated templates to the agenda.
return generated templates.
Next, we utilize the unitxt [Bandel et al., 2024] preprocessing library to build custom datasets with the generated templates. Standardized and accurate evaluation then carried out via the LM-Eval-Harnes. [Gao et al., 2023] evaluation library.
21

---

K
Details MMLU spread analysis
Figure 16 depicts the performance of LLMs on the whole MMLU,
flan-ul2
flan-t5-xxl
200
flan-t5-xl
llama-3-70biinsruuct
iilxrrl-Bx7bbiisstuuttto0
150
Density
llama--8b-iinstruct
falcon-180b
codellima-34b-instuuut
100
falcon-40b
gemma-7b
merlinite-7b
50
1istrrl-7o-insttrut----22
mistral-7b-v0-1
llama-3-8b
0
0.5 0.6
0.7
0.8
gemma-7b-it
MMLU Accuracies
Figure 16: MMLLU accuracy (all 57 subjects).
To correlate the ranks from different judges, we can use Kendall's W. Kendall's W [Kendall and Smith, 1939] ranges from (no agreement) to 1 (perfect agreement) and is calculated as W 125 mn=}== where S is the sum of squared deviations of the total ranks from the mean rank, m the number of rankers, and is the number of objects ranked. In case, first have MMLU subjects ranking prompt templates, and then we have LLMs ranking prompt templates.
In Figure 17, the distribution of Kendall's W for subjects ranking templates. The correlation not significant, with the highest W around 0.25. This suggests that there is no "best" prompt for subject.
Frequency
4
0.075
0.100
0.125
0.150
0.175
0.200
0.225 0.250
Kendall's W
Figure 17: Kendall's W per MMLU subject: Here we see a distribution of Kendall's W over the 57 subjects of MMLU.
22

---

In Table 1, we see the values of Kendall's W for each model. For most models, the W value is not high, but for gemma-7b and miittall-7-vvO. the value of W 0.45 and 0.35, respectively. Curiously, both of the top-ranked prompt templates have lots of commas. 'hee best-ranked prompt is "The, following, are, multiple, choice, questions, (with, answers), about, topic], question], Answers), choices], Answer]". Interestingly, the comma separation of each word or phrase in this prompt template may aid the model in parsing and effectively understanding the different components the prompt structure.
Table Kendall's W per LLM
Model
Kendall's W
metelllama/llame-388-insssttt
0.126027
meta-llama//lama--444
0.252835
meta-llana/llaan-----biinnnttlte
0.101895
mistrali/mmistral-----nsstttttoo
0.219841
mistralai/miittra--b--v--
0.345592
mistraai/miixrrall-777---nttttt-
0.131487
codellamma/cooelllaaa---------n-
0.287066
lbm-miistrala/mmerliittt--
0.146411
google/gemmn--Tbiit
0.445478
google/gemma-7b
0.179373
google/fian-t5-x
0.066501
google/flan-t5xxxl google/flan-ul2
0.056257
0.109076
tiluae/falcon-180b
0.165600
tiluae/falcon-40b
0.100173
Figure illustrates sensitivity for Ilama-3-8b, gemma-7b, and merlinite-7b. respectively. On the template graph, distance means templates differ by only 1 feature, distance means templates differ by 2 features, etc. We see that there is no significant correlation between template distance and the accuracy spread. In the cases of gemma-7b and merlinite-7b. 7b, the accuracy spread for templates with smaller distance seems to be smaller, possibly implying that the template graph for these models is smooth.
50
Distance
100
Distance 1
80
Distance 1
40
Distance
Distance Distance 3
Distance
Density
Distance
Density
Density
Distance 3
30
40
20:
0
0.00
0.02
0.04
0.06
0.08
0.00
0.01
0.02 0.03
0.04
0.00
0.01
0.02
0.03
0.04
Accuracy Spread
Accuracy Spread
Accuracy Spread
Figure 18: Model sensitivity for llama-3-8b, gemma-7b, and merlinite-7b.
L
BERT fine-tuning details
L.1
Model
augment the BERT model by extending its input embeddings by |J] [Example ID] tokens which use to feed information about the example identity to the model. Additionally, add linear
23

---

downward projection (d = 25) on top of the final BERT layer to reduce the dimensionality of the resulting covariates.
L.2 Training data
To obtain training examples, we concatenate all prompting templates with all [Example ID] tokens giving us [/] [J] model inputs (giving the following dataset sizes for the respective benchmarks: BBH 209,280 LMentry 175,776; and MMLU 1,121,568). Labels consist of vectors of correctness scores Biy from the LLMs in the training set, making the training task multi-label binary classification problem. We train on an iid split of half of the LLMs at time and test on the other half. Additionally, the training data are split along the example axis into an 80% training and 20% validation set.
L.3 Hyperparameters
We run a small grid search over different plausible hyperparameter settings and settle on the following setup: We employ the Adam optimizer [Kingma and Ba, 2014] with an initial learning rate of 2e-5 and a weight decay of le-5. The learning rate undergoes a linear warm-up over 200 steps, followed by exponential decay using the formula Ireurnnt -² Insts where is the number of steps after the warmup phase and the decay factor is set to 0.99995. We train with a batch size of 96.
M Heuristics for discrete features
For the BBH and LMMenrr benchmarks, we use the following heuristics to construct feature represen- tations of prompt templates.
Table 2: Overview of Discrete Features
Category
Feature Name
Description
All Caps Words
Count of all uppercase words
Casing Features
Lowercase Words
Count all lowercase words
Capitalized Words
Count of words with the first letter capitalized
Line Breaks
Count of line breaks
Formatting Features
Framing Words
Count of capitalized or numeric words before colon
Colon (:)
Count ":
Dash (-)
Count -_
Double Bar (II)
Count of 'II'
Separator token
Count of "ssep>'
Special Characters Features
Double Colon (::)
Count of :::"
Parenthesis Left (O
Count of '('
Parenthesis Right ())
Count ')'
Quotation (")
Count of ...
Question Mark (?)
Count of "
Length Feature
Space Count
Count spaces
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and Introduction section, summarize the contributions and the scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have included section discuss limitations in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

Justification:
The assumptions are discussed in the statements of the theorems and the proofs are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide general setting the experiments the paper and details are provided in the code itself submitted as supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
26

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide simulation settings that are accessible and reproducible through the submitted zip file.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
ieiaanissssooooiiioiiiiiiiiooiii wwwsssoooliioooooooooooooooooooo olycy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/
http//guiieeooooooee Sudeee/mmdmSbiiic SaleeSuumissss wesssssssoosoooooooo innoolccy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We do specify those parameters in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We provide errorbars for the plots.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
27

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include section in the appendix about this.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp://neut httppp//////u/s.....//// wwtpeerwwp.pppppppiipeepppe httpppllic/thhhcctudeellllllllll
Answer: [Yes]
Justification: Yes, we followed the NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work may have potential societal consequences, none of which we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
28

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We make citations when needed.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi twwode.hooo..... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
29

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "gvOWtpOQvH", "reviewer": "Reviewer_PhKA", "summary": "This paper introduces PromptEval, an efficient multi-prompt evaluation method for LLMs, showing its statistical consistency and effectiveness across benchmarks (MMLU, BBH, LMentry) and studying prompt sensitivity in 15 open-source LLMs.", "questions": "Please refer to my comments in \"Weaknesses\".", "limitations": "The author pointed out the limitations in the Appendix.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors conducted a comprehensive theoretical analysis to verify that PromptEval possesses ideal statistical properties, and carried out extensive experiments using three benchmarks (MMLU, BBH, LMentry) and fifteen open-source large language models (flan series, llama series, mistral series, etc.) \n- They introduces five variations, comparing the baseline under different budgets (i.e., different sampling rates) and various quantiles, demonstrating the significant effectiveness of PromptEval.", "weaknesses": "- The current experiments have only been conducted on open-source models. I look forward to experiments on some mainstream closed-source models (such as the GPT series or the Claude series) and explore corresponding variant strategies.\n- This method PromptEval requires that a large number of active prompt temples exist in its benchmarks, but this is often not satisfied. Does this method work if there are only a few temples in benchmarks? This requires authors to carry out some experiments to prove.\n- The construction of the baseline is relatively simple, and authors need to compare it with some relevant methods (like TRIPLE[1]) to highlight the advantages of PromptEval.\n\n[1] Chengshuai Shi, Kun Yang, Jing Yang, and Cong Shen.  Efficient Prompt Optimization Through the Lens of Best Arm Identification, arXiv preprint arXiv: http://arxiv.org/abs/2402.09723.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, thank you for your work on our paper. We addressed the issues you raised below. Please let us know if you have any questions.\n\n- **New experiment with closed-source models:** We have conducted a new experiment using closed-source models. Due to the high costs associated with running these models (such as GPT-4), we carried out a small-scale experiment. Moreover, to enhance the diversity of experiments and interest of our paper, we explored the concept of LLM-as-a-judge. In this setup, we generated various prompt templates to present to the judge, which in this case is a closed-source model (GPT-4o-mini). This approach allows us to assess how sensitive evaluated models' performance is to different evaluation prompts. Specifically, we used AlpacaEval 2.0 [1] as the benchmark, generated 100 prompt variations using ChatGPT, and presented these to the judge. We evaluated the performance of four LLMs with similar capabilities (cohere, Qwen1.5-7B-Chat, Mistral-7B-Instruct-v0.2, llama-2-70b-chat-hf) using only 2% of the total evaluations (1.6k/80k). The new results have been included in the additional PDF submitted through OpenReview. In summary, we show that PromptEval offers much superior performance in terms of estimation error (Wasserstein distance $W_1$) when compared with the baseline. Please feel free to reach out if you have any questions.\n\n- **New experiment with fewer prompt variations:** We repeat the main experiment in the paper (randomly) cutting the number of prompt templates by a factor of 5. This means we use only 20 prompt variations in MMLU, for example. In summary, PromptEval still does pretty well, beating the baseline. However, the gap between PromptEval and the baseline has shortened due to fewer variations. This fact highlights that the bigger the number of templates, the more useful PromptEval can be relative to the baseline. We include the new version of Figure 2 in the extra pdf submitted through OpenReview, which summarizes well the new results. \n\n- **Baselines:** We compared against TRIPLE-GSE (the best version of TRIPLE, designed to run when many prompt templates are available) in our “best prompt identification” experiment. Please check the end of Section 5, where we show that PromptEval outperforms TRIPLE in **best prompt identification** in MMLU. We do not compare against TRIPLE in the **performance distribution** estimation experiment because that method is not designed for that purpose. Currently, as far as we know, PromptEval is the only method designed for efficient estimation of performance distribution across prompt templates (besides the naive averaging 'avg', included in our experiments).\n\n**References**\n\n[1] Li, Xuechen, et al. \"Alpacaeval: An automatic evaluator of instruction-following models.\" (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZviOh2URmI", "reviewer": "Reviewer_hMGK", "summary": "This paper introduces PromptEval, a novel method for efficiently evaluating the performance of large language models (LLMs) across multiple prompt templates. They propose a statistical framework based on Item Response Theory (IRT) for estimating LLM performance distribution across many prompts using limited evaluations and theoretical guarantees on the consistency of the proposed estimators.", "questions": "1. How sensitive is PromptEval to the choice of initial prompt set? How might this impact its use in practice?\n2. How does the computational complexity of the estimation process scale with the number of prompts and examples?", "limitations": "The authors have provided a comprehensive discussion of limitations in the appendix, which is commendable.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper addresses an important and timely problem in LLM evaluation, recognizing the limitations of single-prompt evaluations and proposing a novel solution.\n2. The theoretical foundation is solid, with clear assumptions and complete proofs provided in the appendix.", "weaknesses": "1. While the authors compare different variations of their method, they don't explore alternative modeling approaches beyond IRT. It would be interesting to see comparisons with other potential frameworks.\n2. While the method saves evaluations, the paper doesn't discuss the computational requirements of the estimation process itself, which could be relevant for very large prompt sets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, thank you for your dedication to our paper. We addressed the issues you raised below. Please let us know if you have any other questions.\n\n- **Use of IRT and alternatives:** We use (a generalization of) IRT because it is the model most suited to our data. Although there are other psychometric models (e.g. classical test theory based on Gaussian factor analysis), they are not as suitable as IRT. Is there a specific alternative model that you wish us to comment on?\n\n- **Computational cost of estimation for the experiments in the paper:** The estimation process is pretty cheap. For example, in our experiments, which reflect very well real use cases, we fitted logistic regression models in datasets with less than 2k samples and a couple of hundred (or a few thousand) columns, which is performed in a few seconds by a laptop. We will make this observation clear in the paper.\n\n- **More details on the computational complexity with growing dataset dimensions:** Consider the case of our experiment in which prompts are represented by embeddings of fixed size and examples are represented by one-hot encodings. Because the dimension of the embeddings does not depend on the number of prompt variations, the number of samples and variables used to fit our model does not vary with the number of prompt variations. Then, *computational costs are constant with respect to the number of prompt templates*. On the other hand, the number of variables (and consequently samples, to make estimation possible) should increase linearly with the number of examples, which are usually hundreds or a few thousand. Thus, this should not be a problem in most practical cases. We will add a comment about this point in the paper. Thank you for bringing this up!\n\n- **Sensitivity of PromptEval:** Our method aims to estimate performance distribution across a **given prompt set**, which can be obtained using methods from [1,2], for example. We anticipate the method to work well with any initial prompt set in the sense of accurately estimating performance distribution across the given prompt set, as supported by experiments considering different ways to generate the initial prompts. We note that if the initial prompt set is too limited or biased in some sense, the resulting estimate of performance distribution might not accurately reflect sensitivity to **all possible prompts**. We make this point clearer in our \"Limitations\" section. Thank you!\n\n\n**References**\n\n[1] Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of what art? a call for multi-prompt LLM evaluation. arXiv preprint arXiv:2401.00595, 2023.\n\n[2] Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tHwCEghH9o", "reviewer": "Reviewer_emjz", "summary": "The authors introduce a novel method called PromptEval which permits efficient multi-prompt evaluation of LLMs across different prompt templates with a limited number of evaluations.", "questions": "N/A", "limitations": "N/A", "rating": 8, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Theoretical guarantees that PromptEval has desirable statistical properties such as consistency in estimating performance distribution and its quantiles.\n- Large-scale study of prompt sensitivity of 15 popular open-source LLMs on MMLU.", "weaknesses": "N/A", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, thank you for your comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers,

Thank you for your time reviewing our paper. We include new experiments suggested by reviewer PhKA in the extra pdf. 
1. In the first experiment, we explore the concept of LLM-as-a judge. We used PromptEval to estimate the distribution of performances given by a closed-source LLM (GPT-4o-mini, the "judge"), across 100 evaluation prompt templates, when evaluating other LLMs responses on AlpacaEval 2.0. Realize that, contrasting with previous experiments, we do not rephrase prompts given to the models being evaluated but only to the evaluator. This approach allows us to assess how sensitive evaluated models' performance is to different evaluation prompts. As a result, we show that PromptEval is also valuable for more robust LLM-as-a-judge benchmarks. 
2. In the second experiment, we wanted to check how PromptEval performs when fewer prompt variations are available. Then, we repeat the main experiment of the paper only keeping 20% (randomly chosen) of the total number of prompts. In summary, PromptEval still does pretty well, beating the baseline. However, the gap between PromptEval and the baseline has shortened due to fewer variations. This fact highlights that the bigger the number of templates, the more useful PromptEval can be relative to the baseline.

Meta-review: The paper argues that a robust evaluation of LLM performance should be conducted on a distribution of prompts and discusses a sample-efficient way to obtain such an evaluation. For the proposed problem, the paper makes a solid contribution by (1) proposing an efficient estimation of such a measurement with theoretical guarantees under certain assumptions, and (2) conducting comprehensive experiments on open-source LLMs, which leads to some interesting findings. Reviewers also like the proposed direction and only had some minor comments. 

The AC, however, is worried that the proposed distribution-based measurement may not be the best measurement to use because: 

(1) The AC thinks it's better to measure both "robustness" (how an LLM performs under various different prompts for the same task) and "performance" (the performance of an LLM with the best prompt). There are many cases where only the latter (peak performance) is important since we want to deploy a model with good performance. The former may be used in Q/A-typed queries by general users, where users may not try to find the best prompt. The paper will provide a good measurement for the former but not the latter. 

(2) As also pointed out by a reviewer, it is unclear whether the provided set of templates is complete. So even though the paper measures performance under a distribution of templates, it can still be biased due to the template family used in the evaluation. 

Given that the paper is making a solid contribution to measuring the "robustness" perspective, the AC would still recommend acceptance but suggests the authors revise the paper to state this limitation of the work clearly in the introduction.
Decision: Accept (poster)