Prediction-Powered Ranking of Large Language Models

Authors: ivi  chatzi
eleni  straitouri
suhas  thejaswi
manuel  gomez rodriguez
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 15:35:52

Abstract: Large language models are often ranked according to their level of alignment with human preferences---a model is better than other models if its outputs are more frequently preferred by humans. One of the popular ways to elicit human preferences utilizes pairwise comparisons between the outputs provided by different models to the same inputs. However, since gathering pairwise comparisons by humans is costly and time-consuming, it has become a common practice to gather pairwise comparisons by a strong large language model---a model strongly aligned with human preferences. Surprisingly, practitioners cannot currently measure the uncertainty that any mismatch between human and model preferences may introduce in the constructed rankings. In this work, we develop a statistical framework to bridge this gap. Given a (small) set of pairwise comparisons by humans and a large set of pairwise comparisons by a model, our framework provides a rank-set---a set of possible ranking positions---for each of the models under comparison. Moreover, it guarantees that, with a probability greater than or equal to a user-specified value, the rank-sets cover the true ranking consistent with the distribution of human pairwise preferences asymptotically. Using pairwise comparisons made by humans in the LMSYS Chatbot Arena platform and pairwise comparisons made by three strong large language models, we empirically demonstrate the effectivity of our framework and show that the rank-sets constructed using only pairwise comparisons by the strong large language models are often inconsistent with (the distribution of) human pairwise preferences.

Main paper: 
 introduction: 
 During the last years, large language models (LLMs) have shown a remarkable ability to generate and understand general-purpose language [1] . As a result, there has been an increasing excitement in their potential to help humans solve a variety of open-ended, complex tasks across many application domains such as coding [2] , healthcare [3] and scientific discovery [4] , to name a few. However, evaluating and comparing the performance of different LLMs has become very challenging [5] . The main reason is that, in contrast to traditional machine learning models, LLMs can solve a large number of different tasks and, in many of these tasks, there is not a unique, structured solution. As a consequence, there has been a paradigm shift towards evaluating their performance according to their level of alignment with human preferences-a model is better than other models if its outputs are more frequently preferred by humans [6] [7] [8] [9] [10] .One of the most popular paradigms to rank a set of LLMs according to their level of alignment with human preferences utilizes pairwise comparisons [10] [11] [12] [13] [14] [15] [16] [17] . Under this paradigm, each pairwise comparison comprises the outputs of two different models picked uniformly at random to an input sampled from a given distribution of inputs. Moreover, the pairwise comparisons are used to rank the models with a variety of methods such as the Elo rating [18] [19] [20] [21] [22] , the Bradley-Terry model [10, 17, 23] or the win-rate [12, 17, 23] . While it is widely agreed that, given a sufficiently large set of pairwise comparisons, higher (lower) ranking under this paradigm corresponds to better (worse) human alignment, there have also been increasing concerns that this paradigm is too costly and timeconsuming to be practical, especially given the pace at which models are updated and new models are developed.To lower the cost and increase the efficiency of ranking from pairwise comparisons, it has become a common practice to ask a strong LLM-a model known to strongly align with human preferences-to perform pairwise comparisons [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] . The rationale is that, if a model strongly aligns with human preferences, then, the distributions of pairwise comparisons by the model and by the human should in principle match [24, 27, 34] . Worryingly, there are multiple lines of evidence, including our experimental findings in Figure 3 , showing that the rankings constructed using pairwise comparisons made by a strong LLM are sometimes different to those constructed using pairwise comparisons by humans [12, 14-16, 19, 35, 36] , questioning the rationale above. In this work, we introduce a statistical framework to measure the uncertainty in the rankings constructed using pairwise comparisons made by a model, which may be introduced by a mismatch between human and model preferences or by the fact that we use a finite number of pairwise comparisons.Our contributions. Our framework measures uncertainty using rank-sets-sets of possible ranking positions that each model can take. If the rank-set of a model is large (small), it means that there is high (low) uncertainty in the ranking position of the model. To construct the rank-sets, our framework first leverages a (small) set of pairwise comparisons by humans and a large set of pairwise comparisons by a strong LLM to create a confidence ellipsoid. By using prediction-powered inference [37] [38] [39] , this confidence ellipsoid is guaranteed to contain the vector of (true) probabilities that each model is preferred over others by humans-the win-rates-with a user-specified coverage probability 1α. Then, it uses the distance between this ellipsoid and the hyperplanes under which pairs of models have the same probability values of being preferred over others to efficiently construct the rank-sets. Importantly, we can show that, with probability greater than or equal to 1α, the constructed rank-sets are guaranteed to cover the ranking consistent with the (true) probability that each model is preferred over others by humans asymptotically. Moreover, our framework does not make any assumptions on the distribution of human preferences nor about the degree of alignment between pairwise preferences of humans and the strong LLM. Experiments on pairwise comparisons made by humans in the LMSYS Chatbot Arena platform [28] and pairwise comparisons made by three strong LLMs, namely GPT 3.5, Claude 3 and GPT 4, empirically demonstrate that the rank-sets constructed using our framework are more likely to cover the true ranking consistent with (the distribution of) human pairwise preferences than the rank-sets constructed using only pairwise comparisons made by the strong LLMs. An open-source implementation of our methodology as well as the data on pairwise preferences of strong LLMs used in our experiments are available at https://github.com/Networks-Learning/prediction-powered-ranking.Further related work. Our work builds upon recent work on prediction-powered inference, ranking under uncertainty, and ranking of LLMs.Prediction-powered inference [37] [38] [39] is a recently introduced statistical framework to obtain valid p-values and confidence intervals about a population-level quantity such as the mean outcome or a regression coefficient using a small labeled dataset and a large unlabeled dataset, whose labels are imputed using a black-box machine learning model. However, our work is the first to use prediction-powered inference (as a subroutine) to construct rank-sets with coverage guarantees. In this context, it is worth acknowledging that a very recent work by Saad-Falcon et al. [40] has used prediction-powered inference to construct (single) rankings, rather than rank-sets. However, their rankings do not enjoy coverage guarantees with respect to the true ranking consistent with (the distribution of) the human preferences. Moreover, an independent, concurrent work by Boyeau et al. [23] has also used prediction-powered inference to construct (single) rankings based on the estimated coefficients of a Bradley-Terry model. However, the estimated coefficients come with large, overlapping confidence intervals, which would have led to uninformative rank-sets, had the authors used them to construct rank-sets.The vast majority of the literature on ranking under uncertainty has focused on confidence intervals for individual ranking positions [41] [42] [43] [44] [45] [46] [47] . Only recently, a paucity of work has focused on joint measures of uncertainty for rankings [48] [49] [50] [51] . Similarly as in our work, this line of work also seeks to construct rank-sets with coverage guarantees. However, in contrast to our work, it estimates the quality metric (in our work, the probability that an LLM is preferred over others) and the confidence intervals separately for each of the items (in our work, LLMs) using independent samples. As a consequence, it needs to perform multiple comparison correction to create the rank-sets.In recent years, there has also been a flurry of work on ranking LLMs using benchmark datasets with manually hand-crafted inputs and ground-truth outputs [52] [53] [54] [55] [56] [57] [58] . However, it has become increasingly clear that oftentimes rankings derived from benchmark datasets do not correlate well with rankings derived from human preferences-an improved ranking position in the former does not lead to an improved ranking position in the latter [12-14, 17, 26] . Within the literature on ranking LLMs from pairwise comparisons, most studies use the Elo rating system [18] [19] [20] [21] [22] , originally introduced for chess tournaments [59] . However, Elo-based rankings are sensitive to the order of pairwise comparisons, as newer comparisons have more weight than older ones, which leads to unstable rankings [15] . To address this limitation, several studies have instead used the Bradley-Terry model [10, 17, 23] , which weighs pairwise comparisons equally regardless of their order. Nevertheless, both the Elo rating system and the Bradley-Terry model have faced criticism, as pairwise comparisons often fail to satisfy the fundamental axiom of transitivity, upon which both approaches rely [15, 60] , Recently, several studies have used the win-rate [12, 17, 23] , which weighs comparisons equally regardless of their order and does not require the transitivity assumption, but requires humans to make pairwise comparisons between every pair of models. In our work, we build upon the win-rate and lift the above requirement by using pairwise comparisons made by a strong LLM. 
llm ranking under uncertainty: 
 Let M be a set of k large language models (LLMs) and P (Q) be a distribution of inputs on a discrete set of inputs Q. Moreover, assume that, for each input q ∼ P (Q),1 each model m ∈ M may provide an output r ∼ P m (R | Q = q) from a discrete set of outputs R. Further, given two outputs r, r ′ ∈ R from two different models, the (binary) variables w, w ′ ∼ P (W,W ′ | Q = q, R = r, R ′ = r ′ )indicate whether a human prefers r over r ′ (w = 1, w ′ = 0) or viceversa (w = 0, w ′ = 1). In the case of a tie, then w = w ′ = 0. In what follows, we use m(r) and m(r ′ ) to denote the models that provide outputs r and r ′ respectively, and without loss of generality, we assume that the output r is shown first. Then, our goal is to rank all models according to the (empirical) probability θ m that their outputs are preferred over the outputs of any other model picked uniformly at random.To this end, we start by writing the probability θ m as an expectation over the distribution of inputs, outputs and pairwise preferences:EQUATIONwhere note that the order of the pairs of outputs is picked at random. Next, following previous work [48, 50] , we formally characterize the ranking position of each model m ∈ M in the ranking induced by the probabilities θ m using a rank-set [l(m), u(m)], whereEQUATIONare the lower and upper ranking position respectively and smaller ranking position indicates better alignment with human preferences. Here, note that it often holds that θ m ̸ = θ m for all m ∈ M \ {m} and then the rank-set reduces to a singleton, i.e., l(m) = u(m).In general, we cannot directly construct the rank-sets as defined above because the probabilities θ m are unknown. Consequently, the typical strategy reduces to first gathering pairwise comparisons by humans to compute unbiased estimates of the above probabilities using sample averages and then construct estimates [ l(m), û(m)] of the rank-sets using Eq. 2 with θm rather than θ m . Under this strategy, if the amount of pairwise comparisons we gather is sufficiently large, the estimates of the rank-sets will closely match the true rank-sets. However, since gathering pairwise comparisons from humans is costly and time-consuming, it has become a very common practice to gather pairwise comparisons ŵ, ŵ′ by a strong LLM, rather than pairwise comparisons w, w ′ by humans [12-14, 28, 29, 31, 61-64] , and then utilize them to compute unbiased estimates of the probabilities θm that the outputs provided by each model is preferred over others by the strong LLM, which can be written in terms of expectations as follows:EQUATIONIn general, one can only draw valid conclusions about θ using (an estimate of) θ if the distribution of the pairwise comparisons by the strong LLM P ( Ŵ , Ŵ ′ | Q = q, R = r, R ′ = r ′ ) closely matches the distribution of pairwise comparisons by the humans P (W, W ′ | Q = q, R = r, R ′ = r ′ ) for any q ∈ Q and r, r ′ ∈ R. However, there are multiple lines of evidence showing that there is a mismatch between the distributions, questioning the validity of the conclusions drawn by a myriad of papers. In what follows, we introduce a statistical framework that, by complementing a (large) set of N + n pairwise comparisons ŵ, ŵ′ by a strong LLM with a small set of n pairwise comparisons w, w ′ by humans, is able to construct estimates [ l(m), û(m)] of the rank-sets with provable coverage guarantees. More formally, given a user-specified value α ∈ (0, 1), the estimates of the rank-sets satisfy thatEQUATIONTo this end, we will first use prediction-powered inference [37, 38] to construct a confidence ellipsoid that, with probability 1α, is guaranteed to contain the (column) vector of (true) probabilities θ = (θ m ) m∈M . Then, we will use the distance between this ellipsoid and the hyperplanes under which each pair of models m, m ∈ M have the same probability values of being preferred over others, to efficiently construct the estimates [ l(m), û(m)] of the rank-sets. 
constructing confidence regions with prediction-powered inference: 
 Let the setD N = {(q i , r i , r ′ i , m(r i ), m(r ′ i ), ŵi , ŵ′ i )} N i=1comprise pairwise comparisons by a strong LLM to N inputs and the setD n = {(q i , r i , r ′ i , m(r i ), m(r ′ i ), w i , w ′ i , ŵi , ŵ′ i )} n i=1comprise pairwise comparisons by the same strong LLM and by humans to n inputs, with n ≪ N . In what follows, for each pairwise comparison, we will refer to the models m(r) and m(r ′ ) that provided the first and second output using one-hot (column) vectors m and m ′ , respectively. Moreover, to summarize the pairwise comparisons2 in D N and D n , we will stack the one-hot vectors m and m ′ into four matrices, M N and M ′ N for D N and M n and M ′ n for D n , where each column corresponds to a one-hot vector, and the indicators w and ŵ into six (column) vectors, ŵN and ŵ′ N for D N and ŵn , ŵ′ n , w n and w ′ n for D n .Algorithm 1: It estimates θ and Σ using prediction-powered inference.Input: k, DN , Dn Output: θ, Σ ŵN , ŵ′ N , MN , M ′ N ← SUMMARIZE(DN , k) wn, w ′ n , ŵn, ŵ′ n , Mn, M ′ n ←SUMMARIZE(Dn, k) λ ← LAMBDA(k, ŵN , ŵ′ N , MN , M ′ N , wn, w ′ n , ŵn, ŵ′ n , Mn, M ′ n ) // Algorithm 4 a ← 1 k ((MN + M ′ N ) 1N ) ⊤ ⊙ I k -1 (MN • λ ŵN + M ′ N • λ ŵ′ N ) b ← 1 k ((Mn + M ′ n ) 1n) ⊤ ⊙ I k -1 (Mn(λ ŵn -wn) + M ′ n (λ ŵ′ n -w ′ n ))θ ← ab // prediction-powered estimate (Eq. 5)A ← 1 k (λ ŵN -M ⊤ N a) ⊤ ⊙ MN + 1 k (λ ŵ′ N -M ′⊤ N a) ⊤ ⊙ M ′ N B ← 1 k (λ ŵn -wn -M ⊤ n b) ⊤ ⊙ Mn + 1 k (λ ŵ′ n -w ′ n -M ′⊤ n b) ⊤ ⊙ M ′ n Σ ← 1 N 2 AA ⊤ + 1 n 2 BB ⊤ // estimate of covariance (Eq. 7)return θ, ΣThen, building upon the recent framework of prediction-powered inference [37] , we compute an unbiased estimate θ of the vector of (true) probabilities θ:EQUATIONwhere 1 d denotes a d-dimensional column vector where each dimension has value 1 and I k denotes a k-dimensional identity matrix. Here, note that the first term a utilizes the pairwise comparisons by the strong LLM from D N to compute an unbiased estimate of the vector of probabilities θ defined in Eq. 3 using sample averages, and the second term b utilizes the pairwise comparisons by the strong LLM and by humans from D n to compute an unbiased estimate of the difference of probabilities θ -θ defined in Eqs. 1 and 3, also using sample averages. The parameter λ ∈ [0, 1] weighs the comparisons ŵ, ŵ′ differently than the comparisons w, w ′ . Details on why this can be useful and on the selection of λ are in Appendix B.2.Further, as shown in Angelopoulos et al. [38] , the difference of probabilities θθ converges in distribution to a k-dimensional normal N k (0, Σ), whereΣ = E[( θ -θ)( θ -θ) ⊤ ],and thus the confidence regionEQUATIONwhere Σ is an empirical estimate of the covariance matrix Σ using pairwise comparisons from D N and D n , i.e.,EQUATIONwith  and χ 2 k,1-α is the 1α quantile of the χ 2 distribution with k degrees of freedom, satisfies thatA = 1 k (λ ŵN -M ⊤ N a) ⊤ ⊙ M N + 1 k (λ ŵ′ N -M ′⊤ N a) ⊤ ⊙ M ′ N , B = 1 k (λ ŵn -w n -M ⊤ n b) ⊤ ⊙ M n + 1 k (λ ŵ′ n -w ′ n -M ′⊤ n b) ⊤ ⊙ M ′ n ,EQUATIONAlgorithm 1 summarizes the overall procedure to compute θ and Σ, which runs in O(k 2 (N + n)) time.Algorithm 2: It constructs [ l(m), û(m)] for all m ∈ M Input: M, DN , Dn, α Output: {[ l(m), û(m)]}m∈M k ← |M| θ, Σ ← CONFIDENCE-ELLIPSOID(k, DN , Dn) // Algorithm 1 for m ∈ M do l(m) ← 1, û(m) ← k for m ∈ M \ {m} do d ← | θm-θ m| √ 2 -1 2 ( Σm,m + Σ m, m -2 Σm, m)χ 2k,1-α // Eq. 10if d > 0 and θm < θ m then 8 l(m) ← l(m) + 1 else if d > 0 and θm > θ m then 10 û(m) ← û(m) -1 return {[ l(m), û(m)]}m∈M4 Constructing Rank-Sets with Coverage GuaranteesFor each pair of models m, m ∈ M such that m ̸ = m, we first define a hyperplane H m, m ⊆ R k as follows:EQUATIONThen, for each of these hyperplanes H m, m, we calculate the distance d(C α , H m, m) between H m, m and the confidence region C α defined by Eq. 6, i.e.,EQUATION)where Σ is the empirical covariance matrix defined by Eq. 7.Now, for each pair of models m, m ′ ∈ M, we can readily conclude that, if the distance d(C α , H m, m) > 0, then, the confidence region C α either lies in the half-space of R k where x m > x m if θm > θ m or it lies in the half space of R k where x m < x m if θm < θ m. Building upon this observation, for each model m ∈ M, we construct the following estimates [ l(m), û(m)] of the rank-sets [l(m), u(m)]:EQUATIONImportantly, using a similar proof technique as in Lemma 1 in Neuhof and Benjamini [48] , we can show that the above rank-sets estimates enjoy provable coverage guarantees with respect to the rank-sets [l(m), u(m)] induced by the probabilities θ that the outputs of each model is preferred over any other model by humans (proven in Appendix A):Theorem 4.1The estimates [ l(m), û(m)] of the rank-sets defined by Eq. 11 satisfy thatEQUATION)Algorithm 2 summarizes the overall procedure to construct the rank-sets [ l(m), û(m)] for all m ∈ M, which runs in O(k 2 (N + n)). 
experiments: 
 We apply our framework to construct rank-sets for 12 popular LLMs using pairwise comparisons made by humans in the LMSYS Chatbot Arena platform 3 and pairwise comparisons made by three strong LLMs. We show that the rank-sets constructed using our framework are significantly more likely to cover the true ranking consistent with (the distribution of) human pairwise preferences than the rank-sets constructed using only pairwise comparisons made by the strong LLMs.Experimental setup. Our starting point is the Chatbot Arena dataset [12] , which comprises 33,481 pairwise comparisons made by 13,383 humans about the responses given by 20 different LLMs to 26,968 unique queries. In what follows, we refer to each pair of responses to a query by two different LLMs and the query itself as an instance. As an initial pre-processing, we filter out any instance whose corresponding query is flagged as toxic or multiturn. Then, we gather pairwise comparisons made by three strong LLMs, namely GPT-3.5-turbo-0125 (GPT3.5), GPT-4-0125-preview (GPT4) and Claude-3-Opus-20240229 (CL3), about all the (pre-processed) instances from the Chatbot Arena dataset. To this end, we use (almost) the same prompt as in Zheng et al. [12] , which instructs each strong LLM to output option 'A' ('B') if it prefers the response of first (second) LLM, or option 'C' if it declares a tie. Further, we filter out any instances for which at least one strong LLM provides a verbose output instead of 'A', 'B', or 'C', and focus on a set of LLMs with at least 96 pairwise comparisons between every pair of LLMs in the set. After these pre-processing steps, we have 14,947 instances comprising 13,697 unique queries and 12 different LLMs and, for each instance, we have one pairwise comparison made by a human and three pairwise comparisons by the three strong LLMs. Refer to Appendix C for more information regarding the 12 LLMs, the number of pairwise comparisons between every pair of LLMs, and the prompt used to gather pairwise comparisons made by the three strong LLMs.To draw reliable conclusions, in each experiment, we construct rank-sets 1,000 times and, each time, we use a random set of N + n = 6,336 instances with an equal number of instances per pair of models, out of the 14,947 instances. The values of N and n vary across experiments and they define two random subsets, also with an equal number of instances per pair of models.In our experiments, we construct rank-sets using the following methods: a) BASELINE: it constructs (unbiased) rank-sets using the pairwise comparisons made by humans corresponding to the random set of N + n instances via Algorithms 2 and 3, shown in Appendix B.1. The constructed rank-sets are presumably likely to cover the true rank-sets. b) LLM GPT4, LLM GPT3.5 and LLM CL3: they construct (possibly biased) rank-sets using the pairwise comparisons made by one of the three strong LLMs corresponding to the random set of N + n instances via Algorithms 2 and 3. c) PPR GPT4, PPR GPT3.5 and PPR CL3: they construct (unbiased) rank-sets using pairwise comparisons made by one of the three strong LLMs corresponding to the random set of N + n instances and pairwise comparisons made by humans corresponding to the random subset of n instances via Algorithms 1 and 2. d) HUMAN ONLY: it constructs (unbiased) rank-sets using the pairwise comparisons made by humans corresponding to the random subset of n instances via Algorithms 2 and 3.In the above, note that a), b) and d) use linear regression to construct a confidence region C α using only pairwise comparisons by either humans or a strong LLM via Algorithm 3, which runs inO(k 2 (N + n)) in a)-b) and in O(k 2 n) in d), and then use this confidence region to construct the rank-sets via Algorithm 2.Quality metrics. Since the true probabilities θ are unknown, we cannot compute the true rank-sets of the 12 LLMs under comparison, which presumably may be singletons. As a result, we cannot estimate the (empirical) coverage probability-the probability that the rank-sets constructed using the above methods cover the true rank-sets, which Theorem 4.1 refers to. To overcome this, we assess the quality of the rank-sets using two alternative metrics: rank-set size and baseline intersection probability.Here, smaller (larger) rank-set sizes and larger (smaller) intersection probabilities are better (worse).The baseline intersection probability is just the (empirical) probability that the rank-sets [ l(m), û(m)] . Intuitively, we expect that the larger the baseline intersection probability, the larger the coverage probability since the BASELINE method uses a large(r) number of pairwise comparisons by humans to construct (unbiased) rank-sets and thus it is expected to approximate well the true rank-sets. Further, note that the baseline intersection probability tells us how frequently there exists at least one single ranking covered by both one of the above methods and the BASELINE method. In Appendix D.2, we experiment with an alternative metric, namely baseline coverage probability, which is the (empirical) probability that the rank-sets constructed using one of the above methods covers the rank-sets constructed using the BASELINE method. In Appendix E, we additionally evaluate our framework in a synthetic setting where the true rank-sets are known, allowing us to compute the coverage probability and rank-biased overlap (RBO) [65] .Quality of the rank-sets. Figure 1 shows the average rank-set size against the baseline intersection probability for rank-sets constructed using all methods 4 except BASELINE for different α values 5and n = 990. The results show several interesting insights. First, we find that rank-sets constructed using only pairwise comparisons by a strong LLM (LLM GPT4, LLM GPT3.5 and LLM CL3) achieve much lower baseline intersection probability, even orders of magnitude lower, than those constructed using only pairwise comparisons by humans (HUMAN ONLY) or using both pairwise comparisons by a strong LLM and humans (PPR GPT4, PPR GPT3.5 and PPR CL3). This suggests that the distributions of pairwise comparisons by strong LLMs and humans are actually different, questioning the rationale used by an extensive line of work that proposed using only pairwise comparisons by strong LLMs to rank LLMs [12, [25] [26] [27] [28] [29] 31] . Second, we find that rank-sets constructed using both pairwise comparisons by two of the strong LLMs and humans (PPR GPT4 and PPR CL3) achieve a better trade-off between average rank-set size and baseline intersection probability than those constructed using only pairwise comparisons by humans (HUMAN ONLY). This suggests that pairwise comparisons by strong LLMs are valuable if they are complemented with (a few) pairwise comparisons by humans. Third, we find that, among the three strong LLMs, GPT 4 stands out as the best performer.Figure 2 shows the average rank-set size against the baseline intersection probability for rank-sets constructed using PPR GPT4, PPR GPT3.5 and PPR CL3 for different values of n and α (the same values as in Figure 1 ). 6 The results show that the trade-off between rank-sets and baseline intersection probabilities improves rapidly as the number of pairwise comparisons by humans n increases but with diminishing returns.Structure of the rank-sets. In this section, we take a closer look to the structure of the rank-sets constructed using BASELINE, LLM GPT4 and PPR GPT4. In Appendix D.3, we include additional results for all other methods.First, we compute the empirical probability that each ranking position is included in the rank-sets constructed by BASELINE, LLM GPT4 and PPR GPT4 of each of the LLMs under comparison. Figure 3 summarizes the results for n = 990 and α = 0.05, which reveal several interesting insights. We find that there is lower uncertainty regarding the ranking position of each model for LLM GPT4 than for PPR GPT4. However, for LLM GPT4, the ranking position with the highest probability mass differs from BASELINE in 7 out of 12 LLMs, including the top-2 performers. In contrast, for PPR GPT4, it only differs from BASELINE in 3 out of 12 LLMs. This questions once more the status quo, which proposed using only pairwise comparisons by strong LLMs to rank LLMs [12, [25] [26] [27] [28] [29] 31] .Next, we compute the empirical probability of each rank-set constructed by BASELINE, LLM GPT4 and PPR GPT4 for each of the LLMs under comparison. Figure 4 summarizes the results for GPT 4, Claude 1, Vicuna and PaLM 2 for n = 990 and α = 0.05. In agreement with the findings derived from Figure 3 , we observe that the distribution of rank-sets constructed by LLM GPT4 is more concentrated than the distribution of rank sets constructed by PPR GPT4. However, the rank-sets with the highest probability mass constructed by LLM GPT4 coincide with those constructed by BASELINE much less frequently than those constructed by PPR GPT4. Refer to Appendix D.3 for qualitatively similar results for other LLMs. 
discussion and limitations: 
 In this section, we highlight several limitations of our work, discuss its broader impact, and propose avenues for future work.Data. Our framework assumes that the queries and the pairwise comparisons made by humans and the strong LLMs are drawn i.i.d. from fixed distributions. In future work, it would be very interesting to lift these assumptions and allow for distribution shift. Moreover, our framework assumes that the pairwise comparisons made by humans are truthful. However, an adversary could have an economic incentive to make pairwise comparisons strategically in order to favor a specific model over others. In this context, it would be interesting to extend our framework so that it is robust to strategic behavior.Methodology. Our framework utilizes rank-sets as a measure of uncertainty in rankings. However, in case of limited pairwise comparison data, rank-sets may be large and overlapping, reducing their value. In such situations, it may be worthwhile to explore other measures of uncertainty for rankings beyond rank-sets. Further, to measure the level of alignment with human preferences, our framework utilizes the win-rate-the probability that the outputs of each model are preferred over the outputs of any other model picked uniformly at random. However, if we need to rank k LLMs and k is large, win-rate may be impractical since, to obtain reliable estimates, we need to gather O(k 2 ) pairwise comparisons made by humans. Finally, our framework constructs rank-sets with asymptotic coverage guarantees, however, it would be interesting to derive PAC-style, finite-sample coverage guarantees.Evaluation. We have showcased our framework using pairwise comparisons made by humans in a single platform, namely LMSYS Chatbot Arena, and pairwise comparisons made by just three strong LLMs. As a result, one may question the generalizability of the conclusion derived from the rank-sets estimated using our framework. In this context, it is also important to acknowledge that, in LMSYS Chatbot Arena, the queries are chosen by the humans who make pairwise comparisons and this may introduce a variety of biases. Therefore, it would be interesting to apply our framework to human data from other platforms.Broader Impact. Our framework rank LLMs according to their level of alignment with human preferences-a LLM is ranked higher than others if its outputs are more frequently preferred by humans. However, in many application domains, especially in high-stakes scenarios, it may be important to account for other important factors such as accuracy, fairness, bias and toxicity [57] . 
conclusions: 
 We have introduced a statistical framework to construct a ranking of a collection of LLMs consistent with their level of alignment with human preferences using a small set of pairwise comparisons by humans and a large set of pairwise comparisons by a strong LLM. Our framework quantifies uncertainty in the ranking by providing a rank-set-a set of possible ranking positions-for each of the models under comparison. Moreover, it guarantees that, with a probability greater than or equal to a user-specific value, the rank-sets cover the ranking consistent with the (true) probability that each model is preferred over others by humans asymptotically. Finally, we have empirically demonstrated that the rank-sets constructed using our framework are more likely to cover the true ranking consistent with (the distribution of) human pairwise preferences than the rank-sets constructed using only pairwise comparisons made by the strong LLMs.A Proof of Theorem 4.1Note that Eq. 12 holds if and only if:EQUATIONTherefore, to prove the theorem, it is sufficient to prove that Eq. 13 holds. Now, to prove that Eq. 13, we first show that the probability on the left hand side of the above equation is smaller than or equal to the probability P(θ / ∈ C α ).To this end, first note that, if for at least one model m ∈ M, we have that l(m) > l(m) or û(m) < u(m), then it holds thatm∈M [l(m), u(m)] ̸ ⊆ [ l(m), û(m)].Next, without loss of generality, assume that, for model m, we have that l(m) > l(m). In this case, from Eqs. 11 and 2 we get:m∈M\{m} 1{d(C α , H m, m) > 0} • 1{ θm < θ m} > m∈M\{m} 1{θ m < θ m},which means that there must be at least one model m ∈ M such that x m < x m ∀x ∈ C α and θ m > θ m, which implies that θ / ∈ C α . As a result, we can immediately conclude that,lim n P ∃m ∈ M : [l(m), u(m)] ̸ ⊆ [ l(m), û(m)] ≤ lim n P(θ / ∈ C α ) = α.This concludes the proof.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We report error bars in Figure 6 in Appendix D.1, and Figure 12 in Appendix E.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We discuss both potential positive societal impacts and negative social impacts of our work in Section 6.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We do not release any data or models that have a high risk of misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The creators or original owners of assets used in the paper are properly credited, and the license and terms of use are explicitly mentioned in Appendix C and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this section, we present a pseudocode implementation of the algorithm to construct confidence regions using linear regression.Algorithm 3: It estimates θ and Σ using linear regression. Input: k, D Output: θ, Σ w, w′ , M , M ′ ← SUMMARIZE(D, k) θ ← 1 k (M + M ′ ) 1 |D| ⊤ ⊙ I k -1 (M • w + M ′ • w′ ) // linear reg. estimate A ← 1 k ( w -M ⊤ θ) ⊤ ⊙ M + 1 k ( w′ -M ′⊤ θ) ⊤ ⊙ M ′ Σ ← 1 |D| 2 AA ⊤ //The parameter λ ∈ [0, 1] weighs the comparisons ŵ, ŵ′ by the strong LLM differently than the comparisons w, w ′ by humans. This way, if the strong LLM's preferences are strongly aligned with human preferences, the pairwise comparisons by the strong LLM can be weighed close to or equally with the pairwise comparisons by humans. Conversely, if the strong LLM's preferences are not well aligned with human preferences, the pairwise comparisons ŵ, ŵ′ can be weighed lower than the pairwise comparisons w, w ′ . Following Angelopoulous et al. [38] , we select the λ that minimizes tr(Σ), whereEQUATIONwhere Σ N is the sample covariance matrix of the estimate of θ computed from Algorithm 3 using the pairwise comparisons by the strong LLM D N , and Σ n is the sample covariance matrix of the estimates of θ and θ computed via Algorithm 3 using the pairwise comparisons by the strong LLM and by humans respectively in dataset D n . Detailed computation of λ is shown in Algorithm 4, which runs in O(k 2 (N + n)).Algorithm 4: It computes λ Input: k, DN , Dn Output: λ wn, w ′ n , ŵn, ŵ′ n , Mn, M ′ n ←SUMMARIZE(Dn, k) ân ← 1 k ((Mn + M ′ n ) 1n) ⊤ ⊙ I k -1 (Mn • ŵn + M ′ n • ŵ′ n ) an ← 1 k ((Mn + M ′ n ) 1n) ⊤ ⊙ I k -1 (Mn • wn + M ′ n • w ′ n ) Ân ← 1 k ( ŵn -M ⊤ n ân) ⊤ ⊙ Mn + 1 k ( ŵ′ n -M ′⊤ n ân) ⊤ ⊙ M ′ n An ← 1 k (wn -M ⊤ n an) ⊤ ⊙ Mn + 1 k (w ′ n -M ′⊤ n an) ⊤ ⊙ M ′ n Σn ← 1 n 2 ÂnA -1 n -, ΣN ← CONFIDENCE-ELLIPSOID(k, DN ) //In this section, we provide the implementation details and computational resources used to execute the experiments discussed in Section 5. Our algorithms are implemented in Python 3.11.2 programming language using NumPy and SciPy open-source libraries for efficient matrix operations. Further, we use the matplotlib package to facilitate visualizations of our results. The complete details of the software requirements can be found in the source code provided as part of the supplementary materials. Our experiments are executed on a compute server equipped with 2 × AMD EPYC 7702 processor with 64 cores per processor and 2 TB of main memory. It is worth noting that, our experiments are not resource intensive and can be executed on a standard desktop computer or a laptop.Pairwise comparisons and preprocessing. Using the dataset from LMSYS Chatbot Arena We preprocess the dataset by filtering out instances-an instance is a pair of responses to a query by two different models and the query itself-for which at least one strong LLM returned a verbose output instead of 'A', 'B' or 'C', and choose a set of 12 popular large language models for our experiments. The chosen models, along with their specific versions, are listed in Table 1 . In Figure 5 , we show the number of pairwise comparisons per each pair of these chosen models after completing all preprocessing steps. In this section, we discuss additional experimental results that were omitted from the main paper due to space limitations.In Figure 6 , we show a more detailed analysis of the average rank-set size plotted against the baseline intersection probability, with individual plots for each strong LLM: GPT4 (top), CL3 (middle) and GPT3.5 (bottom), for n = 990 and different α values. 8 All observations made in Figure 1 apply to Figure 6 as well. Moreover, it is evident that among the three strong LLMs, GPT 4 clearly outperforms the others, achieving higher baseline intersection probability and returning rank-sets with smaller average size. Specifically, in Figure 6 (a), the difference between average rank-set size between HUMAN ONLY and PPR GPT4 is significantly pronounced, while the gap gradually narrows in subsequent plots for PPR CL3 and PPR GPT3.5.In this subsection, we investigate a more conservative quality metric, namely baseline coverage probability, which is the (empirical) probability that the rank-sets [ l(m), û(m)] constructed by any method cover the rank-sets [ l(m), ũ(m)] constructed using the BASELINE method, i.e.,EQUATIONThe baseline intersection probability, which we discussed in Section 5 and illustrated in Figure 1 , is a less conservative metric compared to the baseline coverage probability. While the latter represents the probability that all rank-sets are covered by the BASELINE rank-sets, the former only considers the probability that the rank-sets intersect. Thus, achieving high baseline coverage probability is difficult, particularly when the BASELINE rank-sets are larger.Quality of the rank-sets when considering the baseline coverage probability. In Figure 7 , we show the average rank-set size against the baseline coverage probability for rank-sets constructed using all methods (except BASELINE) for n = 990 and different values of α (the same values as in Figure 6 ). Immediately, we notice that the baseline coverage probability of all methods is very low. For rank-sets constructed using pairwise comparisons only by a strong LLM (LLM GPT4, LLM CL3 and LLM GPT3.5), the baseline coverage probability is close to (or exactly) zero. Rank-sets constructed using only pairwise comparisons by humans (HUMAN ONLY) or prediction-powered ranking using a strong LLM (PPR GPT4, PPR CL3 and PPR GPT3.5) achieve better baseline coverage probability. However, the difference in performance among these methods is not clear to distinguish, which motivated us to consider the baseline intersection probability metric for our experimental results in Section 5.In Figure 8 , we show the average rank-set size against the baseline coverage probability for rank-sets constructed using PPR GPT4, PPR GPT3.5 and PPR CL3 for different values 9 of n and α (the same α values as in Figure 6 ). Similarly as in Figure 2 , results show that there is a trade-off between average rank-set size and the baseline coverage probability, which improves rapidly as the number of pairwise comparisons by humans n increases, but with diminishing returns. In this subsection, we take closer look at the structure of the rank-sets constructed by all methods. We compute the empirical probability that each ranking position is included in the rank-sets constructed by all methods, of each of the LLMs under comparison. The results are summarized in Figure 9 for α = 0.05 and n = 990.Empirical probability of ranking positions. Our first observation is that, the ranking positions of each model in LLM GPT4 and LLM CL3 have lower uncertainty compared to PPR GPT3.5 and PPR CL3, respectively. However, LLM GPT3.5 exhibits higher uncertainty compared to PPR GPT3.5. Nonetheless, the ranking positions with the highest probability mass in LLM GPT4, LLM CL3 and LLM GPT3.5 significantly deviate from the BASELINE. Specifically, the ranking position with highest probability mass differs for 7 out of 12 models. In contrast, for PPR GPT4, PPR CL3 and PPR GPT3.5 it only differs from the BASELINE for 3 out of 12 LLMs. These findings once again question the rationale of relying solely on pairwise comparisons by strong LLMs to rank LLMs [12, [25] [26] [27] [28] [29] 31] .Our second observation is that there is no significant difference in the uncertainty in ranking positions across HUMAN ONLY, PPR GPT4, PPR CL3 and PPR GPT4. However, the distribution of probability mass across different ranking positions differs slightly among these methods. This observation is clearly seen in PPR GPT4, where the Alpaca model has zero probability mass for position 6.Empirical probability of rank-sets. Next, we compute the empirical probability of each rank-set constructed by all methods and for each of the LLMs under comparison with n = 990 and α = 0.05.The results are summarized in Figure 10 . Consistent with the observations from Figure 9 , we note that the distribution of rank-sets constructed by LLM GPT4 is more concentrated than those constructed by other methods. Conversely, LLM GPT3.5 exhibits a more spread-out distribution of rank-sets, indicating higher uncertainty in its ranking positions. This observation is consistent across all LLMs considered for ranking. Despite the more concentrated distributions of rank-sets for LLM GPT4 and LLM CL3, we observed that the ranking positions with the highest probability mass often differed from those of the BASELINE, with discrepancies observed in 7 out of 12 models. On the contrary, the rank-sets constructed by PPR GPT4, PPR CL3 and PPR GPT3.5 exhibit distributions that are neither excessively spread out nor highly concentrated. But the rank-sets with the highest probability mass constructed by these methods coincide with those constructed by BASELINE more frequently than their LLM only counterparts. Once again, these findings underscore our argument that rank-sets obtained using only pairwise comparisons of strong LLMs are not very reliable.In this section, we evaluate our framework in a synthetic setting where the true rank-sets of the models are known. This allows us to validate the theoretical coverage guarantee of Theorem 4.1 without relying on a proxy metric, and also compute the rank-biased overlap (RBO) [65] .Experimental setup. We consider k = 8 models and, in each experiment, we generate a random vector of true win probabilities θ (Eq. 1), which induces the true rank-sets of the models. 10 To obtain win probabilities θ (Eq. 3), we add random noise to the vector and then re-normalize θ. We sample the noise from a Uniform(-u, u) distribution, where u a parameter we manually set to simulate different alignment levels between strong LLMs and human preferences. A larger u indicates a greater difference between θ and θ, meaning the LLM is less aligned with human preference, and vice versa. In our experiments, we set u ∈ {0.05, 0.1, 0.3} to simulate three different strong LLMs.To draw reliable conclusions, for each experiment, we create rank-sets 300 times, each time using a different set of n + N = 50,000 simulated pairwise comparisons by humans and the three strong LLMs, with an equal number of pairwise comparisons per pair of models. We ensure that each model provides the first and second response to an equal number of pairwise comparisons. Let m a and m b be the two models in a pairwise comparison, with m a giving the first response. For each pairwise comparison, first, we generate a number x ∈ (0, 1) uniformly at random. For the human outcome, if x < 2 θ ma , then the response of model m a is preferred (w = 1). Similarly, for the strong LLM outcome, if x < 2 θma , then the response of model m a is preferred ( ŵ = 1). In every comparison we set w ′ = ŵ′ = 0.Coverage probability. Using the generated pairwise comparisons, we compute rank-sets in a similar way as described in Section 5, with α = 0.1. Since the true rank-sets are known, we can compute the (empirical) coverage probability, shown in Figure 11 . The results show that the coverage probability increases with n, consistent with Theorem 4.1. Further, the coverage probability is greater when u is smaller, indicating that our method achieves better results when the strong LLM is more aligned to human preference.For each method, we obtain a ranking T by sorting the models in descending order of their θ values. We then compute the RBO metric relative to their true ranking T as follows:RBO(T, T , p) = (1 -p) i∈[k] p i-1 |T :i ∩ T:i | iwhere T :i and T:i represents the top i models in ranking T and T , respectively, and p ∈ [0, 1] is a chosen parameter. When p = 1, all models are weighed equally. As we decrease p, more emphasis is given to the top-ranked models, and at p = 0, only the top-ranked model is considered. In Figure 12 , we compare RBO values as we increase the number of human pairwise comparisons n, for p = 0.6.The results show that increasing n improves RBO across all methods. Additionally, combining human pairwise preferences with a strong LLM further improves RBO values, demonstrating that our method performs better than those solely relying on strong LLM preferences. We repeated our experiments with multiple values of p ∈ [0, 1] and observed no significant variation in the results. Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material.We denote random variables with capital letters and realizations of random variables with lower case letters.We assume that each model m ∈ M participates in at least one pairwise comparison in both DN and Dn.https://chat.lmsys.org/In Appendix D.1, we include a version of this figure with three panels, where each panel contains the results for one strong LLM and confidence regions for the rank-set sizes.α ∈ {0.4, 0.3, 0.25, 0.2, 0.15, 0.1, 0.075, 0.05, 0.025, 0.01}.n ∈ {66,132, 198, 462, 990, 1452, 1980, 2442, 2970}.The user prompts are licensed under CC-BY-4.0, while the model outputs are licensed under CC-BY-NC-4.0.α ∈ {0.4, 0.3, 0.25, 0.2, 0.15, 0.1, 0.075, 0.05, 0.025, 0.01}n ∈ {66,132, 198, 462, 990, 1452, 1980, 2442, 2970}.In our experiments, we generate true win probabilities θ with unique values, so the rank-sets are always singletons, resulting in a unique true rank for each model. 


Decision: This paper studies how to rank Large Language Models (LLMs) automatically, using a strong LLM as an alternative to human preferences. It achieves this by focusing on the uncertainty that arises in such rankings compared to the distribution of human preference rankings. 

The reviewers identified the importance of the problem, the statistically-grounded framework, extensive evaluation, broad applicability, clarity of presentation, and the novelty of the methodology all as strengths of the paper.

To further improve the paper, the authors should consider testing their approach on a wider range of benchmarks, tightening up their theoretical justifications, and so on. The reviewers left a number of potentially interesting suggestions that the authors should carefully consider.

Reviews: review: paper_summary: paper_summary: The paper tackles an interesting problem of evaluating ranking large language models automatically using a strong LLM as alternative to human preference estimates. The work primarily focuses on modelling uncertainty in such a ranking generated when compared to the distribution of human preference rankings. Since, pairwise comparison by humans are cumbersome and pairwise comparisons by strong LLMs are not completely consistent with human preferences, the authors propose a framework that improves upon pairwise ranking by strong LLMs. The authors propose a prediction power inference based framework to construct rank sets that provide coverage guarantees with respect to the true ranking consistent with human preferences.
strength_weakness: strength_weakness: 1.	While the authors perform extensive evaluation, it might be a good idea to also test on other benchmarks like MT-bench or AlpacaEval related to approximation of human judgements. While authors already discuss the generalization aspect in limitations with regards to this, I would like to add it would also help address the concern regarding selection bias of test set and evaluators. Additionally, due to input limitations the benchmark may also not be representative of tasks that require reasoning over long form inputs and specifically complex reasoning tasks. Hence the leaderboard may only weakly correlate with real-world performance on these tasks. Additionally a minor point is that the metrics considered for rating response “relevance , helpfulness , accuracy , creativity and level” as shown in prompt might also change depending on the task: For instance when evaluating on a benchmark akin to QA tasks where precise information is needed creativity may not be a valid metric anymore. Hence evaluating on more benchmarks would help give a clearer picture on usefulness of the proposed framework.

2.	While it is appreciated that the work provides the proof for theoretical coverage guarantees, the claim regarding the coverage guarantees made in Introduction and beginning of section 4 should be revisited as true rank-sets (true probabilities unknown) cannot be computed for LLMs. The Baseline intersection probability is a weak approximation for coverage guarantees. Though the authors argue that baseline method approximates well the true rank sets due to being constructed from large number of human pairwise comparisons this might not necessarily hold due to selection bias, distribution shift and various other factors.  Without further evidence the claim that  Baseline intersection probability is a good approximation for true coverage measure is not well supported.
questions: questions: Did the authors also try few-shot prompting the strong LLMs by showing few examples on how to judge the responses ? Would be interesting to see if this leads to any change in final observation and insights.

For a small sample set would it be possible to empirically test coverage guarantees without the other metric being the proxy assumption?
limitations: limitations: Due to the inherent limitations of the benchmark, this work may also not be representative of tasks that require reasoning over long form inputs and specifically complex reasoning tasks.
paper_summary: paper_summary: - Focuses on uncertainty in rankings using a small set of human pairwise comparisons and a large set of model estimated comparisons using a concept of rank sets. A rank set is a set of ranks a specific model can take. A large rank set indicates high uncertainty in ranking position and vice-versa a small set implies a confident rank assessment. The method works by constructing a confidence ellipsoid which in turn using methods of prediction powered inferences (methods to estimate confidence intervals when you have a small set of labeled gold standard data and a large set of machine labeled data). The paper employs the methods to rank 12 LLMs using data from LMSYS Chatbot Arena.
strength_weakness: strength_weakness: - The main weakness is on evaluation. The paper proposes two metrics: rank-set size and baseline intersection probability. Small rank-set sizes are better - presumably as the confidence in ranks is better, and baseline intersection probability - large is better as it supposes the baseline method is closer to the true ranking. First: The paper should really report more standard ranking metrics such as precision/recall @ k, RBO, MAP or Normalized Discounted Cumulative Gain (NDCG). To factor in rank sets, you could use most probable ranking from your rank sets to compute these. Second: the absence of true rankings makes the empirical aspects less convincing. A synthetic experiment where true rankings are known perhaps would make sense to empirically demonstrate the main claims.

- Use of machine labels just appear to widen the confidence bounds (e.g. Figure 3). The base conclusions on ranking appears to remain the same. In this instance, therefore, it remains unclear what the value of machine labeled information is. 

- Some additional insights into rank sets would add to the paper. For instance, how stable are rank sets to minor changes in $\mathcal{M}$? Uncertainty in rank for one model depends the overall set of models being considered, so this would be interesting to study. There are possibly other facets, such as how rank sets can be used in practice or the quality of the machine labeled data.
questions: questions: NA
limitations: limitations: Yes.
paper_summary: paper_summary: This paper proposes a statistical framework to rank a collection of LLMs according to how well their output aligns with human preferences. This framework does this using a small set of human-obtained pairwise comparisons from LMSYS Chatbot Arena platform and a larger set of pairwise comparisons by a "strong" LLM and additionally provides an uncertainty estimate by giving a set of rankings for each LLM being compared. This study shows that, with at least probability threshold the user can set, the predicted rankings will eventually become increasingly likely to match the true order in which humans would prefer the models. The authors perform several experiments to empirically demonstrate the valitdity of the proposed framework.
strength_weakness: strength_weakness: - unless i misunderstood something, the small set of human pairwise comparisons has length = 1. Although, the potential bias and truthfulness of the human pairwise comparisons has been discussed in the limitation section, i think that it could be interesting to explore the potential error propagation in the rank sets from erroneous human comparisons.
- it is not clear to me, how the self-recognition [1] problem can be tackled with this framework. It has been shown that LLMs have non trivial capability of recognizing their own generation. Would this not be the case for one of the strong LLMs? wouldn't they tend to rank their generation higher? This fact combined with the previous question might affect the generazalization ability of this framework.


[1]: Panickssery, Arjun, Samuel R. Bowman, and Shi Feng. "Llm evaluators recognize and favor their own generations." arXiv preprint arXiv:2404.13076 (2024).
questions: questions: see weaknesses.
limitations: limitations: limitations have been discussed.
paper_summary: paper_summary: The paper studies uncertainty estimate in the LLM ranking problem, where the task is to rank LLMs based on their response quality. Ideally the labels should be from humans, but due to the cose, people use models such as GPT-4 as auto raters. There lacks a good study of uncertainty estimation in the problem. The paper applies prediction powered inference (PPI) to construct a rank set for each candidate LLMs. Experiments are conducted on the Chatbot Arena data. It is shown that the PPI approach work as intended to produce reasonable rank set size vs accuracy (when comparing with an oracle method using only human data) trade-offs.
strength_weakness: strength_weakness: Overall I am positive of the paper as I feel the problem is important and applying PPI is a good proposal. However, the reviewer is not enthusiastic enough to give a higher rating due to the following concerns:

The paper is a more or less straightforward application of PPI to rank set construction. The theoretical properties shown in this paper mostly strictly follow those of PPI. Thus, the depth of this work is not substantial enough to warrant a higher rating in terms of novelty and technical depth.

The rank set is at dataset level, which may not be very useful in practice. For example, compared to a standard usage of PPI on a numeric metric, which will provide a concrete internal. For the paper, people end up with a discrete rank set for each candidate model. While it provides some uncertainty by looking at the set size, one may still end up wondering how useful that is. For example, if model 1 has rank 2,3,4, and model 2 has rank 3,4,5. It provides some information that model 1 seems better, but the reviewer is not sure how useful it really is (e.g., by how much? - as the guarantee is at the set level). 

There are several points about the experimentation that the reviewer is not certain about
- As the authors acknowledged, only one dataset is used so the generalization is less clear. F
- The baseline / ground truth still needs some processing, such as a regression fit. This is different from standard tasks where human ground truth are given without the need to process anything. Thus the reviewer is not fully convinced how solid the conclusions are, e.g., “questioning the rationals used an extensive line of work that … (use LLM rankings for evaluation)”
- Figure 1 is not entirely convincing - at least the PRP methods do not achieve pareto frontier here. The reviewer understands the argument about the x axis, still, for a pareto problem, one may not really argue one method is better than the other if it is not pareto optimal.

The flow of the paper may be improved. The reviewer was puzzled about the methods and algorithm 1,2,3 when reaching to the experiments. Figure 1 is not very easy to interpret.

Some other minor limitations / future work, some are discussed in the paper: iid assumption - in practice, there could be a bias, e.g. using active learning to send data to human.
questions: questions: See above.
limitations: limitations: The authors list several concrete limitations. Most are treated as future work.

Comments: title: Response to rebuttal
comment: I appreciate the authors' responses to my questions and additional clarifications. Overall, I would like to keep the current score rating.