UniCBE: An Uniformity-driven Comparing Based Evaluation Framework with Unified Multi-Objective Optimization
Authors: Peiwen Yuan
Shaoxiong Feng
Yiwei Li
Xinglin Wang
Yueqi Zhang
Jiayi Shi
Chuyi Tan
Boyuan Pan
Yao Hu
Kan Li
Venue: ICLR 2025 Spotlight (2025)
Keywords: evaluation
efficient
scalability
accuracy
convergence
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose UniCBE, a comparing-based evaluation framework with better scalability, accuracy and convergence.

Abstract: Human preference plays a significant role in measuring large language models and guiding them to align with human values. Unfortunately, current comparing-based evaluation (CBE) methods typically focus on a single optimization objective, failing to effectively utilize scarce yet valuable preference signals. To address this, we delve into key factors that can enhance the accuracy, convergence, and scalability of CBE: suppressing sampling bias, balancing descending process of uncertainty, and mitigating updating uncertainty.
Following the derived guidelines, we propose UniCBE, a unified uniformity-driven CBE framework which simultaneously optimize these core objectives by constructing and integrating three decoupled sampling probability matrices, each designed to ensure uniformity in specific aspects. We further ablate the optimal tuple sampling and preference aggregation strategies to achieve efficient CBE.
On the AlpacaEval benchmark, UniCBE saves over 17% of evaluation budgets while achieving a Pearson correlation with ground truth exceeding 0.995, demonstrating excellent accuracy and convergence. In scenarios where new models are continuously introduced, UniCBE can even save over 50% of evaluation costs, highlighting its improved scalability.

Published as a conference paper at ICLR 2025
UNICBE: AN UNIFORMITY-DRIVEN COMPARING BASED EVALUATION FRAMEWORK WITH UNIFIED MULTI-OBIECTIVE OPTIMIZATION
Peiwen Yuan',
Shaoxiong Feng", Yiwei Li',
Xinglin Wang',
Yueqi Zhang',
Jiayi Shi', Chuyi Tan',
Boyuan Pan', Yao Hu², Kan Li
School of Computer Science, Beijing Institute of Technology
2 Xiaohongshu Inc
(peiwenyuan, wwwp:wwwwwwa.iggin...nn 2bangyq/ebit.coo
(shijiayi, tanchuyi, likan)abit.edee edu..du.c.c cn [ 30mmammmmml.....mmmmmm com (panboyuan, xiahou seaaboppuan.iilooeeeeeeeoooonon hetahooueeiiachooshh..uu.
com
ABSTRACT
Human preference plays significant role in measuring large language models and guiding them to align with human values. Unfortunately, current comparing based evaluation (CBE) methods typically focus on a single optimization objective, fail- ing effectively utilize scarce yet valuable preference signals. To address this, we delve into key factors that can enhance the accuracy, convergence, and scala- bility of CBE: suppressing sampling bias, balancing descending process of uncer- tainty, and mitigating updating uncertainty. Following the derived guidelines, we propose UNICBE, unified uniformity-driven CBE framework which simultane- ously optimize these core objectives by constructing and integrating three decou- pled sampling probability matrices, each designed to ensure uniformity in specific aspects. We further ablate the optimal tuple sampling and preference aggrega- tion strategies to achieve efficient CBE. On the AlpacaEval benchmark, UNICBE saves over 17% of evaluation budgets while achieving Pearson correlation with ground truth exceeding 0.995, demonstrating excellent accuracy and convergence. In scenarios where new models are continuously introduced, UNICBE can even save over 50% of evaluation costs, highlighting its improved scalability.
INTRODUCTION
The ongoing evolution of large language models (LLMs) has made it increasingly important to assess their alignment with human preferences (Dubois et al. 2024 Zheng et al. 2023). The pref- erence signals provided by humans erucial for accurately assessing and guiding models toward safe and reliable AGI (Ji et al.] 2023 Jiang et al.) 2024). However, the rapid iteration of LLMs in training and application scenarios has created substantial demand for evaluation. complicating the acquisition of sufficient labor-intensive human preferences (Chiang et al. 2024 Cui et al.) 2023). Therefore, exploring the use of precious preference signals for efficient model alignment evaluation is of great significance and requires long-term research.
Current mainstream model evaluation paradigms include scoring- evaluation (SBE) (Liu al. Cai et al., 2023) and comparing-basee evaluation (CBE) (Chiang et al. 2024 Dubois et al.
2023
2024
The former requires the judge to offer preference scores for individual responses, while the
latter needs the judge to establish preference order among multiple candidate model responses. By directly comparing the responses of different models, Zheng et al. (2023); Liu et al. (2024) confirm that CBE can more accurately assess model performance. However, the O(NM²) evaluation over- head limits the practicality of CBE when there are M models to evaluate on N samples (Qin et al.) 2024). To achieve efficient CBE, various methods have been explored Chiang et al. (2024); Zhou et al. (204)] Dubois et al. (024).. As shown in Figure [ based on existing observational results, these methods iteratively allocate preference budget to the next (models, sample) tuple according to
*Corresponding author.

---

Published as a conference paper at ICLR 2025
* T times (preference budget)
Models
score
Judge
0.3
Prefe- rence
Alloc- ating
oggre- gatton
0.1
Samples
Results
0.7
objective
(
tuple
preference order
0.5
Accuracy)
Figure I: Flowchart of the process for comparing-base evaluation.
respective optimization objectives. Specific preference aggregation methods (e.g., ELO rating (Elo & Sloan 1978)) then applied to predict the model capability scores based these preference results. Nevertheless, as shown in Table [ the optimization objectives of these methods are often singular, failing to simultaneously achieve the accuracy, convergence, and scalability well. We will discuss this in detail in $2 and conduct experimental validation in $5.2
Table Optimization Objectives widely applied CBE Methods. The number of '+' indicates the degree of optimization for the objective, which is discussed in a|nd measured in Table
Qin al. (2024)
Chiang et 2024)
Dubois et al. 2024
Ours UNICBE
Methods
RANDOM
ARENA
ALPACAEVAL EVA
Accuracy
Convergence
++
Scalability
To develop a method that can accurately assess model performance, quickly converge evaluation results, and ensure good scalability when models are introduced, theoretically analyze and summarize the following guidelines:
Improving the accuracy of evaluation results relies on completely uniform sampling of tuple combinations, so as to mitigate sampling bias.
Accelerating the convergence process involves ensuring the uniformity of the win rate uncer- tainty matrix during its descending process to reduce observation variance.
Enhancing scalability requires sufficient budgets being allocated to new added models to ensure the uniform allocation among models, which helps reduce updating uncertainty.
Based on these insights, we propose UNICBE, a unified uniformity-driven framework that can achieve CBE with better accuracy, convergence and scalability. In each iteration of the evalua- tion process, we first establish sampling probability matrices under different optimization objectives respectively based on real-time preference results. Afterwards, we integrate these matrices obtain global sampling probability matrix. Furthermore, we explore various tuple sampling strategies and preference aggregation methods to achieve optimal evaluation results.
To comprehensively validate the effectiveness and generalizability of UNICBE, we conduct exper- iments involving various types judges (LLMMs and humans), different benchmarks, varied model sets to be evaluated, diverse scenarios (static and dynamic), and multiple evaluation metrics. The main results indicate that, compared to random sampling bascline, UNICBE saves over 17% of evaluation budgets when achieving the same assessment accuracy (with Pearson coefficient ex- ceeding 0.995 with the ground truth), demonstrating significantly better convergence and accuracy than baselines. Furthermore, in scenarios where models continuously introduced, UNICBE save over 50% of evaluation costs compared to random sampling, showcasing excellent scalability.
RELATED WORK
Comparative preference signals have long been used for model training (Ouyang et al., 2022 Touvron et al. 2023) and evaluation (Chiang et al. 2024 Yuan al. 2024). Centered around comparing-baa based evaluation, we will discuss existing budget allocation strategies and preference aggregation methods below.
2

---

Published as a conference paper at ICLR 2025
Budget Allocation Many efforts have been made to explore preference budget allocation ap- proaches. The most naive method is to randomly select (models, sample) tuple for judging each time until the preference budget is reached (Qin et al.) 2024). This method ensures relatively uniform sampling across tuple combinations in expectation, thereby guaranteeing the accuracy of evaluation results according derivation in $3.2 Arena (Chiang et a 2024) aims sample model pairs proportionally to the variance gradient of win rate at each step, seeking to accelerating the convergence of evaluation by reducing the uncertainty of the observed win rate matrix in a greedy manner. AlpacaEval (Dubois et al. 2024) measures model performance by comparing the models under evaluation with fixed reference model. When new models are introduced, preference budget is prioritized for them to stabilize the estimation of their capabilities, thereby achieving good scal- ability. Despite these methods performing well in their intended objectives, they cannot achieve a balance among accuracy, convergence, and scalability. This makes it imperative to explore better preference budget allocation strategy that can effectively reconcile all these attributes.
Preference Aggregation Due to the possibility that the same group of models may exhibit dif- ferent ranking relationships across different samples, it is essential to estimate the global model capability scores better fit these non-transitive preference results. Dubois et al. (2024): Zheng let al. (2023) directly use the average pair-wise win rate of each model as a measure of its capabil-
ity. Feng et al. ) Wu & Ajil (2023) apply the classical Elo rating system (Elo & Sloan 1978)
(see the Appendix B.I for detailed introduction) by treating the evaluation process as a sequence
of model battles in order derive model scores. Fageot al. (2024); Chiang et al. (2024) employ
the Bradley-Terry model (Bradley & Terry 1952) (see Appendix B.2 for detailed introduction) to
estimate model scores by maximizing the likelihood of the comparison results between models. We
will systematically compare the effectiveness of these preference aggregation methods in $5.3
3 PRELIMINARY
In this section, we start by symbolically introducing the working process of CBE. Afterwards, we introduce the key objectives for achieving efficient CBE: accuracy, convergence, and scalability, and analyze the factors that influence them. We mainly discuss the pair-wise evaluation scenario (where the judge provides preference between two models per time) for its wide applications (Tashu & Horváth, 2018, Qin et al. 2024). Actually, list-wise preferences can be easily converted into pair-wise ones, as demonstrated $5.4. so the discussions below are general for CBE.
3.1 PROCESS OF CBE
Generally, CBE method f can be divided into three parts: budget allocation strategy fou tuple sampling strategy frs, and preference aggregation strategy f Given benchmark 2 : $1.N and models under evaluation M m1:M. sampling matrix p at iteration I, where Pl..A denotes the probability to select tuple (m;, my. 8k)
iterate the following steps: step 1. applying foa to attain
for judging; step 2. applying f to sample certain tuple (m" m²² s') based Pl: step 3. attaining preference result r¹ from the judge, where r' E [0, denotes the degree m wins over m² (0.5 means tie). We stop this iterative process when the preset preference budget T is achieved and then apply f on preference results {(m") m² " r)) to attain estimated model scores U1:M.
3.2 ACCURACY
Theoretically. if we have budget of NM(M-1) we can explore all tuples to obtain the ground truth estimation for the model scores ûI.M. However, typically T is much smaller than π in reality considering the preciousness of preference signals. Previous studies (Vabalas al. 2019: Kossen et al., 2021 have discussed the risks of introducing sampling bias in incomplete sampling scenarios, which we believe could similarly lead to potential risks CBE. Considering that the content of each sample is (m" ml s we think the sample bias exists across both samples and models.
Bias across Samples. Since different models may excel at answering different types of queries, the model scores can vary depending on the sampled data:
Ut (0 mj, 8k, Tij.k Ne.: l:M) -i 17
for 4l,K (1)
3

---

Published as a conference paper at ICLR 2025
biss across samples: Rayms
600m
biss acroso modelss: -y
Proportion
1.15
011-
1.0
12
fit.
canaza
ese
(a) Sampling bias with different preference aggrega- tion strategies across samples and models.
(b) Interval distribution of bias across samples with
pba as preference aggregation strategy.
Ramara 2amar-2220 sib-cbat-o charoa
mirra--anee 2002
mist mettan
4- creas
guarees bo
posnaru glorace dal
teme
numbuae 12t- 1it-
Gaodor
tuste cr01
2613
apaca-hh
mittrl--ediin
07-440
02
-02
daads
daade
-04
gat-3 5-arto-0513
(c) Bias across models with fer as preference aggregation strategy.
Figure 2: Analyses of potential sampling bias risks in CBE.
where -m,.. represents the bias between the observed model score uy of m, and the ground truth ãn when sorely assessing sample Sg. To verify this, conduct experiments
the AlpacaEval benchmark (Dubois et al. 2024) using GPT-4o (OpenAl, (024) as the judge across randomly se- lected 20 LLMs (listed in Figure 2 We first traversed all model pairs for samples S1:N obtain corresponding N sets of preference results and then calculate the respective Mmm...al for E M (model scores normalized
and k 1 N according calculate the average value of Im,,...s across models and samples using different preference ag- gregation strategies f discussed in $2 As shown in Figure 2(a) with all kinds of fPa, the average difference between the model scores estimated on single sample and the ground truth values exceeds 0.25, indicating a sigmificant bias across samples. We further analyze the proportion of samples with different biases using JP in Figure 2(b) and find that they overall follow Gaussian distribution, showing the wide existence of sample bias in CBE.
equation [
average of 1). We
Bias across Models. Just humans may perform differently when facing different opponents, models may also have varying scores when competing against different models:
(((m, 8k, 8k, ri.j,))
û + Am;mmss
for ViJj (2)
We validate this from two perspectives: (I) We calculate the average I1m,,,, according to equa- tion 2 like the process above and show the results in Figure 2(a) Overall, although the bias across models significantly lower than the bias across samples, it still exists at scale around 0.05. We further visualize the pair-wise model score bias in Figure 2(c to validate its wide existence. (2) We obtain over 1.7 million pairwise preference results across 129 LLMs collected by Chatbot Arena ] After excluding pairs with fewer than 50 comparisons, we calculate the pairwise win rates and find non-transitivity in 81 model triplets (win rate: > B, > C, C A), which also verifies the existence of bias across models.
httpss:/ googl eapis com/ arenaa httpm//rea__eeeeerraeeeeee external .ataa/ppbbbb ic/cae batt1e_20220814_goo son

---

Published as a conference paper at ICLR 2025
Uniform Allocation Brings the Least Bias. Based on the discussions above, analyze the bud- get allocation strategy that can introduce the least bias. Considering the presence sampling bias, the estimation error of u, with T evaluation budget can be expressed as follows:
1 m X Tmm mm?
(3)
Considering that vr á when all the tuples are traversed, we have the following equation:
M
N
0 = u; 04
for A
(4)
The goal of obtaining the minimum estimation eтrоr for u; is transformed into sampling T numbers ( equutionn 3) from N numbers that sum zero equation 4. such that the absolute value of the sum of these T numbers is minimized. We have provided detailed proof in Appendix A that the best strategy completely uniform sampling. This denotes that the score estimation error can be minimized when the preference budgets are uniformly distributed across models and samples to bring the least sampling bias.
3.3 CONVERGENCE
During the evaluation process, preference results continuously observed, the estimated values of the models win rate matrix and model scores also change constantly. To accelerate the convergence process, analyze the uncertainty of the win rate matrix as follows. Defining that:
/' 1mm mm; & milmmy + X = Pl,
Pj (1 - (1-r')1mi my & miamm
(5)
The unbiased estimated win rate matrix & at iteration L can be calculated as follows:
sL
X
(6)
We further estimate the variance matrix θ as:
@ 1 - (x' 2 (x Φ4)
Denoting if the model pair (m,, my) has been compared on sample 8 after iterations as C k.j.k the uncertainty (standard deviation) of each element in the win matrix is follows:
(8)
k==
Allocating the next preference budget on (m, my) can reduce the uncertainty of their win rate by:
O
O
ΣN
Cly.k
V EN Cl +1
Considering that our core objective to conduct accurate capability assessments for all models and estimate their ranking relationship. we should globally ensure the uniformity of the win rate uncertainty matrix during its descending process to achieve smooth evaluation convergence.
3.4 SCALABILITY
Due to the continuous emergence of new LLMs, the demand for scalability in evaluation method is becoming increasingly prominent (Chern et al. (004).. Considering that we have evaluated mi.M with 7' budgets, when model mas+1 is introduced for assessment, well-scalable CBE method
5

---

Published as a conference paper at ICLR 2025
should be able to quickly calibrate the capability estimates of m1:441 with minimal additional preference budget. this scenario, at the beginning stage when MM+1 1 is introduced, avg(CM+1.- much smaller than avg(Cy (C+,.),...) According equation 8 the uncertainty at this point mainly arises from <M+1, which is also intuitively easy to understand. Therefore, the key to improving scalability lies in allocating sufficient evaluation budgets to the newly added models ensure the uniform allocation among models, reducing the updating uncertainty.
4
UNICBE
The discussions above reveal guidelines for strengthening scalability, accuracy, and convergence in CBE. Based this, we propose UNICBE, unified uniformity-driven framework that can simulta- neously enhance these objectives well.
4.1 BUDGET ALLOCATION
To ensure the uniformity of tuple combination sampling for minimizing the introduction of sampling bias according to $3.2 we construct pace- at iteration as follows:
paceo =a Σm= Cham x a E Clasa x a ET Cla. fjik
(10)
where Ex=1 f. denotes the times model pair (m, has been compared, Σ 1i=1 t.j.k and
En= C² C.j.A denote the times model m, and my has been tested on respectively. If certain model- model combination or model-sample combination have been sampled multiple times, equation TO will reduce the probability of such combinations being selected again, thereby achieving sufficient uniformity to minimize the introduction of bias between models and samples, respectively.
To accelerate the convergence of evaluation results, construct poon-l according 33.3 follows:
peon-l = - i.j.k
(11)
Sampling specific model pair helps reduce the uncertainty of their win rate estimation according equation 9 By sampling proportionally the win rate uncertainty matrix, we can uniformly decrease the uncertainty for each model pair, thereby facilitating convergence.
We construct paca-l allocate more preference budget to the newly introduced model so as to improving the scalability according to 33.4as as follows:
P.j.k = a E EM Cl x xa EN EN Ci...
(12)
Finally, we integrate the matrices mentioned above to obtain pl, ensuring that sampling according to pl can simultaneously balance the accuracy, convergence, and scalability of evaluation results:
paced peon-l psco-
pl
(13)
Σ Paset Pcon-l o psea-l)
4.2 TUPLE SAMPLING
After obtaining pl, we need to sample tuples for judging based on it. Two tuple sampling strategies are considered:
probabilistic sampling f means sampling tuple directly according to P!. greedy sampling f means selecting the tuple with the maximum probability in p!.
The default tple sampling strategy of UNICBE f which can avoid the suboptimal achievement of objectives due to uncertainty in the sampling process.
4.3 PREFERENCE AGGREGATION
As discussed $2. mainstream preference aggregation strategies include averaging win rate fpu legg
ppa Elo rating system Eto and Gradley-Teryy model ppa In our preliminary experiment (Figure 2(c)
observe that / better alleviate sampling bias, for which we choose it our default setting.

---

Published as a conference paper at ICLR 2025
5 EXPERIMENTS
Centered around UNICBE, we will empirically compare its performance with baselines and validate its scalability in $5.2, explore the optimal variants in 5.3 and demonstrate its generalizability under different settings in $.4
5.1 EXPERIMENTAL SETTINGS
Benchmarks. We choose AlpacaEval (Dubois et al. 2024) and MT-Bench (Zheng et al. 2023 benchmarks for experiments. For AlpacaEval, use its default version which includes 805 high-quality human annotated instructions and corresponding responses from multiple LLMs. We randomly choose 20 LLMs (listed in Figure 2 for experiments. with GPT-4o and GPT-b.5-turbo as judges (see Appendix D for the prompt). For MT-Bench, we use the released responses from the all LLMs and corresponding human preferences for experiments.
Baselines. We choose widely applied methods RANDOM, ARENA and ALPACALEVAL base- lines, which have been discussed in 2 and listed Table
Metrics. To assess the effectiveness of the CBE methods, we evaluate the accuracy of both the estimated model pair-wise win rates and the model scores. We calculate the average absolute error between the estimated win rates and corresponding ground truth (the estimates when T T)- We calculate the Spearman correlation coefficient r's between the predicted model scores and the corresponding ground truth to evaluate the accuracy of the model's 's rank-order relationship, and the Pearson correlation coefficient Tp to assess the accuracy the linear relationship.
Details. To ensure the reliability of the experimental results, for each setting, we randomly select M (default to 15 for AlpacaEval and 5 for MT--encc) models and N (default to 805 for AlpacaEval and 700 for MT-Bench) samples, and report the average results across 10,000 random seeds. We don't observe obvious performance difference in preliminary experiments when varying a within the range of [1.5,3] 3] (we conduct detailed discussion about this Appendix F.1. thus we set the default value of 2 in experiments.
UNICEE
33325
64
05927
0595)
Perce enagg Undder
Aucertagge Under
3.35
233
$130
6300
10
Figure 3: Results of compared CBE methods with GPT-4o the judge on AlpacaEval benchmark. The X-axis (applicable to all plots below) represents the preference budget (k). A denotes the mean absolute error of the estimated win rate. rs and rp denote the Spearman and Pearson correlations between the the estimated model scores and the ground truth respectively.
5.2 MAIN RESULTS
Accuracy and Convergence. The results of compared CBE methods on AlpacaEval benchmark with GPT4-turbo as the judge are shown in Figure To better illustrate the results, we also calculate the percentage of preference budget saved by each method compared to RANDOM baseline when
https://tatsu- ab. thub. ic/a 1aaaapaca_ess
https: / 1ma rena.

---

Published as a conference paper at ICLR 2025
achieving the same performance. In terms of performance, ALPACAEVAL << RANDOM ARENA < UNICBE. To understand the differences in the performance of each method, we quantitatively analyze them based the guidelines summarized in $ 3 To achieve accuracy, convergence, and way that ensures uniformity across
scalability, it is supposed to allocate the preference budget tuples, uniformity across model pairs in win-rate uncertainty, and uniformity across models. We calculate the cosine similarity between the allocation results of these methods and the correspond- ing expected uniform vectors for each objective a measure, denoted Saces Beons and Beca. respectively (see Appendix [ for calculating process). As shown in Table 2 the fixed inclusion of the reference model the tuple selection of ALPACAEVAL compromises uniformity across multi- ple aspects, thereby resulting in lower 3 values and significantly poorer performance. ARENA and RANDOM respectively improve the balance uncertainty and suppression of sampling bias, result- ing in higher Beon and face values. Following our guidelines, UNICBE improves Peons Buccs and Peco simultancously and save over 17% the preference budget compared to RANDOM with a A close to 0.01, showcasing improved accuracy and convergence.
Table The measurement results of the achievement of objectives in $ for the compared methods.
Methods
RANDOM
ARENA
ALPACAEVAL
UNICBE
Bace
.5803
.5255
.0925
.7364
Boon
.9081
.9722
.3515
.9228
Bsca
.9972
.9945
.4987
.9997
Scalability. To analyze scalability, we establish a scenario where we initially have models awaiting assessment, and new models sequentially added every 2000 samplings. As shown in Figure Whenever a new model introduced, UNICBE can rapidly stabilize the performance through adaptive preference allocation skewing for the new model, saving over 50% of the budget compared to the RANDOM baseline. In contrast, ARENA and RANDOM exhibit poorer scalability since they do not consider scalability optimization objective. Although the budget allocated to the reference model is significantly more than that for other models, resulting in lower Ssca for ALPACAEVAL, the strategy of automatically allocating the budget the new introduced models also provides it with good scalability.
0.00
0.990
310
4112
0.00
20.000
115
0.60
0.99
Percentzge Under
lave Percentage Under
lave Percentage Under
1076 2053 3500
4235
E250 7:t0 2000
2000
2002
3030
1950
5312
1307
2032
3050 4003 5297 wobe roto 2502
Figure 4: Results of compared CBE methods the scenario where new model are consistently introduced every 2000 iterations.
5.3 VARIANTS ABLATIONS
Budget Allocation Objectives. We test the impact different optimization objectives by remov- ing pase, ppon, and paca from equation 13 separately. As shown in Figure the significant perfor- mance degradation observed when removing pace from UNICBE indicates that mitigating sampling bias improve accuracy is the most critical factor in achieving efficient CBE. Furthermore, that prom has a considerable impact on т., We hypothesize that this is because balancing the un- certainty among different models helps prevent any model from having significant ranking
find
8

---

Published as a conference paper at ICLR 2025
UAAS
2107
0597
1.25
C110)
0370
2225
UNISE
Save Percemtape Under
aasconaagp Underr
3322
Figure Ablation studies of UNICBE with GPT-4o as the judge on AlpacaEval benchmark.
bias due to its larger uncertainty. The performance drop when removing pace also suggests that en- suring uniformity in sampling across models not only enhances scalability but also further reduces sampling bias, thereby improving accuracy.
Tuple Sampling and Preference Aggregation Strategies. As shown in Figure 5 replacing greedy sampling with probabilistic sampling f. results in a significant performance drop. This is likely because the randomness introduced by ft hinders the achievement of multiple optimiza- tion objectives. In terms of preference aggregation strategies, the Elo rating system f shows slight performance decline compared to the BT model due its higher instability (Boubdir al.) 2023). Moreover, the strategy of directly using the average win rate Sag may introduce additional bias, as fails to consider varying strengths of the opponents faced by different models, leading to performance decrease.
5.4 GENERALIZABILITY UNDER DIFFERENT SETTINGS
Different Judges. Apart from GPT-40, also experiment with GPT-3.5-turbe and Qwen-Plut judge on AlpacaEval, and human as judge on MT-Bench. As shown the above part of Figure 6 the overall conclusion with GPT-3.5-turbo is similar GPT-4o, except for: (1) The performance of
ARENA no longer shows advantage over RANDOM. (2) There is certain decline in the performance of all methods, which is likely due to the increased noise in the preferences provided by GPT-3.5- turbo, leading to slower convergence. Similar trends are observed with Qwen-Plus (See Figure [4. Results on MT-Bench are shown in the below part of Figure 6 where UNICBE also demonstrates better performance compared to other methods. However, due to the limited preference data in- cluded in MT-Bench, the experimental results show relatively larger fluctuations. The results above demonstrate the good generalizability of UNICBE across different, judges and the data domain.
UniCEc
20.6
0359
2.0
0675
UHCEE
Lawe Percetage Undet
Save Purcaitagge Under
1.4
0.00
312
Figure 6: Results of compared CBE methods with GPT-3.5-turbo as the judge on AlpacaEval (abovee)) (above) and human as the judge on MT-Bench (below).

---

Published as a conference paper at ICLR 2025
Varied Number of Models and Samples. Finally, shown in Figure we conducte experiments by varying the number of models M and samples N. It can be observed that UNICBE achieves significantly better results compared to all the baselines under these settings, especially when M and are large.
List-wise Preference. UNICBE can also be applied list-wise preference. Suppose the judge provides a preference ranking for K models each time. We can compute a K + dimensional P. similar equation equation 13 and sample to obtain a tuple (m ..... m s'). From the judge's ranking of this tuple, we derive K(K-1) pair-wise preferences. Figure shows the results for the case where K 3.
2
be seen that UNICBE achieves savings compared RANDOM of
30% this setting. This may due to the fact that list-wise preference results in K(K-] pair- wise preferences concentrated among the K models, exacerbating the sampling bias. Therefore, UNICBE even more needed to suppress this effect with list-wise preference.
2
UniCE
0.31
0.32
0.00
0.11
(a) 15, 805
(b) .M 12, 805
(c) M 15, 600
Undera
Undes
0.12
$0.000
0.000
6.00
(d) M 15, 805
(e) M = 12, N 805
(1) M 15, N = 600
Figure 7: Effects of number of models under evaluation M and number of samples N. The results are obtained with GPT-4o as the judge on AlpacaEval.
1,00
0.90
3557
2150
0000
Save Purcertagg Undder
0.25
6:30
Figure 8: Performance of CBE methods with list-wise preference of GPT-4o on AlpacaEval.
CONCLUSIONS
The existing comparing-based evaluation methods are ineffective in fully utilizing valuable prefer- ence signals due to their constrained optimization objectives. Our in-depth analysis reveals that the key to enhancing CBE lies in mitigating sampling bias, balancing the descent process of uncertainty, and suppressing the updating uncertainty. Based on this, we propose the UNICBE framework that simultaneously optimizes the aforementioned objectives by promoting uniformity in corresponding aspects to enhance accuracy, convergence, and scalability. Comprehensive experiments and analyses confirm the strong effectiveness, improved scalability, and good generalizability of UNICBE.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work is supported by the Beijing Natural Science Foundation, China (Nos. 4222037, L181010).

---

Published as a conference paper at ICLR 2025
REFERENCES
Meriem Boubdir, Edward Kim, Beyza Ermis, Sara Hooker, and Marzieh Fadace. Elo uncovered: Ro- bustness and best practices in language model evaluation. CoRR, abw/2311. 2023. doi: 10. 08550/ARXIV.2311.7222
http:: org/00 wttpp/////......... 2311 17295
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-345, 1952.
Tianchi Cai, Xierui Song, Jiyan Jiang, Fei Teng, Jinjie Gu, and Guannan Zhang. ULMA: uni- fied language model alignment with demonstration and point-wise human preference. CoRR, abs/2312.02554, 2023. doi: 1048550/ARRXIV.312.0255
URL htt.hhtp:://// //dtss://do... org/10...
18850/arxiv.23144 02554
Steffi Chern, Ethan Chern Graham Neubig, and Pengfei Liu. Can large language models be trusted for evaluation? scalable meta-evaluation of Ilms as evaluators via agent debate. CoRR.
10s/24011.16888, 2024. doi: 1.4850WARXIV.2441.60777 48550/arXiv.2444 20....24111166....
httlhttps//// //ooi. org/10.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang. Michael Jordan, Joseph E. Gonzalez, and lon Sto- ica. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. net http://rmmmmm--MMMCYVV/
OpenReview.net, 2024. URL httpt:tttpppeeeeeeeeees
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with high-quality feedback. CoRR, abs/2310.01377, 2023. doi: 10.445500AAXXI.22100011 48550/arXiv. 20t...3311173777 01377
URL https: /tw.//doioooo/////// http//ii.ogg/..../
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. CoRR, 0bbs/2404.04455. 2024. doi: 10. URL 48550/ARXIV.2444044777
0001//10..850000 48550/ arXiv. 208500/axl...244...o. 04475
Arpad E Elo and Sam Sloan. The rating of chessplayers: Past and present. (No Title), 1978.
Julien Fageot, Sadegh Farhadkhani, Lē-Nguyên Hoang, and Oscar Villemaud. Generalized bradley- terry models for score estimation from paired comparisons.
Thirty-Eighth AAAI Confer- on Artificial Intelligence, AAAI 2024. Thirty-Sixth Conference Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Arti- ficial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 20379-200333 AAAI Press, 2024. doi: 10. 11109//AALV33300 8.30020. URL /htttss://di...... wtw.ii..r//ii......e.eee 1609/
aaai v38118 30020
Sebastian Farquhar, Yarin Gal, and Tom Rainforth. On statistical bias in active learning: How and when to fix it. In 9th International Conference Learning Representations, ICLR 2021, Virtall Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL httpppss///peeeeeeiee net./
Kehua Feng, Keyan Ding, Kede Ma, Zhihua Wang, Qiang Zhang, and Huajun Chen. Sample- efficient human evaluation of large language models via maximum discrepancy competition. CoRR, abs/2404.08008 2024. doi: 10.45550ARXXI..444.00000 URL/attps://do..oom
10. 08.48850/arxi11 .204 2004.44400 08008
Jaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of LLM via a human-preference dataset. In Advances in Neural Information Processing Systems 36: Annual Conference cessing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL httt::://paers..ni... wtw.iisssc.ppaeee cc/paper. files/paper/200 Datasets and
Neural Information Pro-
Abstract-Datasets
Benchma rks. ht.l
12

---

Published as a conference paper at ICLR 2025
Ruili Jiang, Kehai Chen, Xuefeng Bai, Zhixuan He, Juntao Li, Muyun Yang. Tiejun Zhao, Liqiang Nic, and Min Zhang. A survey on human preference learning for large language models. CoRR, (bss22006.11999 2024. doi: 10.48550/ARXI.24466611 11191. URL htt.http:: //tp:://doii org/10.
48550/arXiv 2406..406..111 11191
Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-efficient model evaluation. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 3753-5763 PMIR, 2021. htttlhhtt://poooo htt.a/e..........//1///133 kossen2las htm
wan.nngs...... ml press/v199)
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Con- ference Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, De- cember 6-10, 2023, pp. 2511 Association for Computational Linguistics, 2023. doi: WEW.:/22...MLL-MMAI.I.UU
10.18653/V1/2023..6 10.1855/V12033.MM..PPPM
emr.pp main. 153
Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulic, Anna Korhonen, and Nigel Collier. Aligning with human judgement: The role of pairwise preference in large language model evaluators. CORR, abs/2403. 16950, 2024. doi: 40448550/ARXIV.2433.00 16950. URL https:
aidei.org//0) 48 48550/arXiyy 2403. 1493.
Frederic M Lord and Melvin Novick. Statistical theories of mental test scores. IAP, 2008.
OpenAL. Hello gpt-4o. 2024-05-26.
2024. Accessed:
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- fion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL http: 5lefde53bee366 Tww.:36447991141888888 TaeecaeeDee..441111E88000003-3-- A wtw.::/00...aatttttttooooooeeeee Conference html
/ttpp::/pappp..ppppppt htwppsaappspaaaaaaaeeeeaaaaeeeaa nips. ct/papeseppppppppppppee
filles/paper/200
Felipe Maia Polo, Lucas Weher, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating lims with fewer examples. In Forty-first International Conference on Machine Learming, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenRe- net forum? id qmml 1atmm33FPPM. 3FpENG
view.net, 2024. URL
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. Large language models are effective text rankers with pairwise ranking prompting. In Findings of the Association for Compu- tational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pp. 1504-1518, As- sociation for Computational Linguistics, 2024. doi: 12665/////224FFIIIIGGS NAACL999 18..311111613111
URL
Findings-naacl. 97
Tsegaye Misikir Tashu and Tomás Horváth. Pair-wise: Automatic essay evaluation using word mover's distance. In Proceedings of the 10th International Conference Computer Supported
Education, CSEDU 2018, Funchal, Madeira, Portugal, March 15-17, 2018, Volume 1, pp. 59-66.
SciTePress, 2018. doi: 10.52200006669220500066 URL https: / /oii wwwp/ol........... /220/
70066779000590066
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshom, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar
13

---

Published as a conference paper at ICLR 2025
Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta. Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen 7hang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307.09288. 2023. doi: 1044855/ARXIV.3377.000 petts::/doi..orge
URL
08550/555330 2307 09288
Andrius Vabalas, Emma Gowen, Ellen Poliakoff, and Alexander Casson. Machine learning algo- rithm validation with limited sample size. PloS one, 14(11)xe0224355, 2019.
Rajan Vivek, Kawin Ethayarajh, Diyi Yang, and Douwe Kiela. Anchor points:
Benchmarking mod- els with much fewer examples. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguisties, EACI. 2024 Volume 1: Long Pupers, St. Ju- lian's s, Malta, March 17-22, 2024, pp. 1576-1601. Association for Computational Linguistics, 2024. URL httllttss//alllattlllllo hology wttp:oy...gg.22....... 220 eac.eaaclllooo -ooaccl-oonn..... 95
Minghao Wu and Alham Fikri Aji. Style over substance: Evaluation biases for large language models. CoRR, http/2277000025.. 2023. doi: (0-4550/ARXIV.330..00000 URL https: //
doi
2307. 00000.03023
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Boyuan Pan, Heda Wang. Yao Hu, and Kan Li. Batcheval: Towards human-like text evaluation. In Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. 15940-15958. Association for Computational Lin- guistics, 2024. URL AtRLhttps:// / ac antaanatthoooo org/2024....2....oo acl www.-lloon.....o 846
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Informa- tion Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 httpc//pppprr..eeee/e/e/////e22 and
16, 2023, 2023. URL http:://paeer..nss.cco cC wttpp/ppppppepcpppeappppp.epp.ee
Benchmarks.. html
Jin Peng Zhou, Christian K. Belardi, Ruihan Wu, Travis Zhang, Carla P. Gomes, Wen Sun, and Kilian Q. Weinberger. On speeding up language model evaluation. CoRR, 0abs/2407.006722. 2024. doi: 1044850/AARXIV.247..007777 URL httph::t/://// //doi. wwt.://///..////.111...2.222222 06172]
14

---

Published as a conference paper at ICLR 2025
A PROOF OF THEOREM IN
3.2
Given that Σ X; O, we want to attain sampling set S that satisfies |S] V
and:
= argming!
Σ
X =
argmins! X2)
(14)
EES
iES
Firstly, it is easy to know that for any sampling set S:
E
X.J
=
(15)
IES
Thus,
x.)
E[(
X;
-
X,
E
x,
= B[Var]
Σ
X
(16)
iES
iES
iES
iES
iES
Considering that:
Var
x
Σ
Var(X,,
(17)
(Eset(S)
where o; denotes the number of X; S. On this basis, we derive that:
E
x,) [[Var(X)]
Σ & (Eset(S)
IES
2 [[Var(X)]( Σ
4i (S))
(18)
(Eset(S)
V'E[Var X)) (stt (S))
2 V3E[Var (effaa(X)mmma (m)) V)
The equality condition of the first inequality is: the number of samples taken from each category is equal. The equality condition of the second inequality is: the number of sampled categories equals to min V). These two conditions imply that completely uniform sampling strategy is optimal.
B INTRODUCTION OF ELO RATING SYSTEM AND BRADLEY-TERE MODEL
ELO RATING SYSTEM
The Elo rating system (Elo & Sloan
1978) is widely used to rank participants based on their relative performance in competitive settings. Given two models, and B, with initial ratings RA and Rg, the expected score of model in pairwise comparison calculated as:
E
+ 10 - RA0/400
Similarly, the expected score for model B is:
FB + 10(R. Rn)/400
After the comparison, the actual results used to update the ratings. If model A wins, its new rating H', is updated as:
R = RA+
+ K(SA- -A)
where SA is the actual result of the match (1 for win, O for loss, and 0.5 for a draw), and K is a constant that controls the sensitivity of the rating adjustment. Model B's rating is updated similar way:
R3 Ra + K(S En)
where Sp is the actual result for model B.
15

---

Published as a conference paper at ICLR 2025
When extending the Elo rating system to multiple models, consider set of n models. Pairwise comparisons between the models are conducted, resulting in (+)
2 unique pairs:
n 2
(1 1)
2
Each pair is evaluated using the Elo score update rules, and the results are iteratively applied to adjust the ratings, ensuring that each model's rating reflects its relative performance within the set.
The extension to multiple models leverages the transitive property. For any three models i, l: (1, .,.....,... n). if R3 > R2 and R2 Rs, the transitivity implies:
R R and R, R
R Rs
This property ensures consistency in the rankings, when individual match outcomes vary. By iterating over all (2) comparisons, the Elo scores converge to reflect the overall capabilities of the models, with higher scores indicating stronger performance.
B.2 BRADLEY-TERR MODEL.
The Bradley-Tery model (Bradley & Terry 1952) estimates the probability that one model outper- forms another in pairwise comparisons. For two models M; and M, with strength parameters & and Ej, the probability that model beats model j is modeled as:
P(M > MG) = 1+ c
(19)
where E is an | |IInghhh vector of Bradley-Terry coefficients. Given set of comparing results = {(M), Mj. 1 where R' represents the degree M; wins over Mý. We set mean(() 0.
After attaining the BT scores using f we calculate the estimated win rate matrix with equation I9
C MORE EXPERIMENTAL ANALYSES
650
657
6.00
260
0.00
650
lave Percentage Under
lave Percentage Under
2.3
Randen
Artte
Save Percentage
Apaz
2310
Figure 9: Results of compared CBE methods in scenario where models and samples dynami- cally added or removed at a random frequency.
C.I PERFORMANCE IN SCENARIOS CLOSE TO REALITY
We think that conducting experiments in settings that are closer to real-world scenarios (highly dynamic and requiring real-time evaluation) can help us more comprehensively assess UNICBE and the baseline methods. To this end, we perform the following experiments: Starting with sample size of N 600 and model number of M 12, execute random operation at each time step. The operations included: adding one model to be evaluated with a probability 0.01, removing one model with probability of 0.01, adding one potential sample with probability of 0.01, randomly deleting one sample with probability of 0.01, and taking no action with probability of 0.96. Based on the experimental results shown in Figure have the following observations:
16

---

Published as a conference paper at ICLR 2025
The convergence speed of all baseline methods significantly slowed down compared to Fig- ure None the baseline methods achieve a Spearman correlation coefficient of 0.96 or a Pearson correlation coefficient of 0.97 by T 2000, highlighting the difficulty of model eval- uation in this setting. In contrast, UNICBE achieve rapid convergence, reaching a Spearman coefficient of approximately 0.97 and Pearson coefficient exceeding 0.98 by T 2000.
Over the long term, as 'I' increases, UNICBE consistently demonstrates over 10% savings in preference budget across all metrics, even under this challenging setting, showcasing its strong practicality.
An interesting observation that ALPACAEVAL exhibits better convergence the early stages compared to RANDOM and ARENA, supporting our previous conclusions in Table However, as T increases, ALPACAEVAL'' lack of accuracy optimization objective leads to its perfor- mance being surpassed by RANDOM and ARENA.
C.2 ABLATION STUDY OF UNIFORMITY CONSTRAINTS
Based on our analysis in | the degree to which uniformity is achieved is positively correlated with performance in terms of accuracy, convergence, and scalability. To explore the empirical relationship between the degree of uniformity constraints and the final outcomes, we draw inspiration from the concept of lemperaturee-asset control in random sampling. By adjusting the temperature 7 in the following formula for sampling f/" we regulate the extent of uniformity constraints according to pl in equation 13
f (i,j,k) (P! (Pi E(P!)-
(20)
As increases, the uniformity constraints become more relaxed. When T 0, it corresponds greedy sampling fg which imposes the strictest uniformity constraints. When T 1, it corresponds probabilistic sampling f/= which imposes general uniformity constraints. When T = +00, it corresponds to random sampling, where no uniformity constraints are applied. Our experimental results are shown in Figure [1]] T' increases from 0 to T +o0, the evaluation re- sults progressively deteriorate. This indicates that adopting greedy sampling to impose the strictest uniformity constraints yields the optimal evaluation performance. This observation also validates the correctness of our conclusions in 3
Two MN330
(397
1.1.0
Pardani
5110
3320
1120
6320
3375
4120
122
Save Percentagg
6300
Figure 10: Results of UNICBE with different sampling temperatures.
C.3
ADJUSTING THE WEIGHTS OF OPTIMIZATION OBJECTIVES
we integrate sampling matrices targeting different optimization objectives with equal weights. In practice. when faced with varying requirements, it is straightforward to prioritize a specific objective by adjusting the weights faces Ocons and 8sca for these matrices, as shown in equation 21
In equation
13
(pace-t)ouce o (Pron.l)0ccn (psca-l tyoca
pl
(21)
[(Pacce)00m. (Pcon-t)0om (psca-l (00on)
As demonstrated in Table 3 we set different settings and calculate the degree of achievement for each optimization objective following the procedure described in $ Compared equal-weight
17

---

Published as a conference paper at ICLR 2025
integration, users increase the corresponding B (e.g., Sace) by assigning larger weight specific optimization objective (face), thereby better meeting their practical needs (accuracy). We also observe that enhancing specific optimization objective often comes with slight decrease in the achievement of other objectives. In Figure [1 we illustrate an example of improving accuracy, where face increased from to 2. We find that the increased focus accuracy objective slightly slows down the convergence speed. As a result, when I is relatively small, the performance of Dace = 2 lags behind that of lace 1. However, the later stages, after convergence, the enhanced accuracy objective enables face 2 to outperform 8ace 1, resulting in greater savings in the preference budget.
Table 3: The measurement results of the achievement objectives 3 for UNICBE with varied hyperparameters.
ace
face =
Bace = 1
8 Bace
Settings
cco =
Ocon
i ccon
O oon =
Bsca
Osca
O2ca 2
Osca
Bace
.7380
.7355
.735 ((-00))
.7364
Bcon
:9221c.00m)
:9235++.0007
:9217(-0011
.9288
Booa
.9996.0001)
(9997.000))
(9998(.000))
.9977
3353
0.0
Lindes
Percenane
612
230
630
660
2300
600
Figure 11: Results of UNICBE with different Oace-
D PROMPT FOR HAVING AN LLM ACT AS JUDGE
We follow AlpacaEva to instruct the LLMs act as judge with the following prompt:
You are helpful assistant, that ranks models by the quality of their answers.
I want you create leaderboard of different of large-language models. To do so, I will give you the instructions (prompts) given to the models, and the responses of models. Please give the winner model based which responses would be preferred by humans. All imputs and outputs should be python dictionaries.
Here is the prompt:
{
"instruction": """ """[insruuction)"""
}
Here are the outputs of the models:
model": "model," ",
"answer:: u." (outputt)"
model"; "model2".
httppss//gittibb.cmmrtttttt- pacaeeaa
18

---

Published as a conference paper at ICLR 2025
"answer": """(outputz)""
Now please give the winner model according to the quality of their an- swers, that the winner model has the best output. Then return the winner model in the following format if modely is the winner: winner: models
You need to strictly follow the format above. Please provide the ranking that the majority of humans would give.
E THE CALCULATION PROCESS OF 3 IN TABLE 2
We calculate the 3 values for each CBE method to measure how well they align with the optimization objectives analyzed in ] ensuring uniformity across tuples, uniformity across model pairs in win-rate uncertainty, and uniformity across models. Specifically, we first construct Uscel U Uron and Usea as follows:
0, if i=j
U
M(M -1j else
(22)
U
MN
0, if i
U
(23)
clsc
M
U
(24)
M
On this basis, we calculate Baccs Boon and Byea as follows:
Been = CosineSim (Uacel, C.C.aa (dim -)) x CosineSim C.mean(dim = 0)) (25)
Boon CosineSim (U.oon
(26)
Boca CosineSim U C. C.mean = - mennmenn = -1)
(27)
F FURTHER DISCUSSIONS
F.1 AFFINITY OF THE THREE OPTIMIZATION OBJECTIVES
We have discussed $ that the keys strengthen the accuracy, convergence and scalability of CBE are: ensuring uniformity across tuples, uniformity across model pairs in win-rate uncertainty, and uniformity across models. Below we discuss their compatibility. Firstly, ensuring uniformity across models can be seen as sub-goal of ensuring uniformity across tuples, therefore they exhibit a strong affinity. The former can be considered a refinement of the latter, specifically focusing on model uniformity. Moreover, as shown inequation g the uncertainty in win-rate between models Et is inversely proportional to the number of comparisons between models. Therefore, improv- ing uniformity across model pairs in win-rate uncertainty will also contribute to more uniform distribution of comparisons between models. This further implics that ensuring uniformity across model pairs in win-rate uncertainty and ensuring uniformity across tuples compatible goals with a strong affinity. Furthermore, as shown in Table 2 pared to the baselines, experimentally validating the fact that optimizing the three objectives can be mutually beneficial. Based on this analysis, we can infer that changing essentially adjusts the optimization emphasis on different objectives. However, since the three objectives are mutually reinforcing, the effect of changing a will be relatively small.
all 3 values of UNICBE are improved com-
F.2 DISCUSSION ON SAMPLING BIAS IN INCOMPLETE SAMPLING SCENARIOS
Previous studies have discussed the risks introducing sampling bias in incomplete sampling sce- narios. Specifically, Vabalas et al. (2019) demonstrated through simulation experiments that K-fold
19

---

Published as a conference paper at ICLR 2025
cross- cross-validation (K-fold CV) produce significant performance estimation bias when dealing with small sample sizes. This bias persists even when the sample size reaches 1000. contrast, meth- ods like nested cross-validation (Nested CV) and train/test split have been shown to provide robust and unbiased performance estimates regardless sample size. Kossen et al. (202T) introduced a weighting scheme, as described in (Farquhar et al. (02T), mitigate sampling bias in active test- ing scenarios, Vivek et al. (2024) proposed leveraging information obtained from source models to select representative samples from the test set, thereby reducing sampling bias. Additionally, Poloo et al. (2024) employed Item Response Theory (Lord & Novick 2008) to correct sample bias in addressing this issue.
These studies inspired us to investigate the bias problem in the CBE scenario. Unlike the afore- mentioned studies, we found that CBE scenario, not only does sample bias exist, but model bias also plays role, and the two are coupled. This coupling poses greater challenges for analyzing and mitigating these biases. To address this, based on the analyses outlined in $ we propose the UNICBE method, which effectively alleviates biases in this scenario.
UNICEC
0.90
11.00
0.90
0.905
0.9%
0.34
0.00
20016
+037%
Porcentige Uned
0333
0.14
Perrentern saraenagee
0.00
Figure 12: Results of compared CBE methods with GPT-335-turbo the judge on AlpacaEval.
Farcom
0.85
0.60
0.81
0.00
0.75
01
0.25
0.00
Percentagg
0.00
0.00
0.56
0.00
0.00
Figure 13: Results of compared CBE methods with human as the judge on MT-Bench.
20

---

Published as a conference paper at ICLR 2025
Barcon Apea
21380
1300
0300
0.20
Savve Hercertie cereaagee Under
3337
6130
Figure 14: Results of compared CBE methods with Qwen-Plus the judge AlpacaEval.
21


Reviews:
[{"review_id": "cbOUi11REu", "reviewer": "Reviewer_D5Wd", "summary": "This paper introduces UNICBE, a uniformity-driven framework for comparing-based evaluation (CBE) that simultaneously optimizes multiple objectives in language model assessment. Current CBE methods suffer from limitations related to bias, slow convergence, and lack of scalability.", "questions": "1. How does UNICBE perform when preference signals are less reliable, as is often the case with models lower than GPT-4 or inconsistent human annotations?\n\n2. Could the authors elaborate on how UNICBE would handle scenarios with dynamic preference priorities, where, for example, accuracy is weighted more heavily than convergence?\n\n\n3. To what extent could the uniformity constraints in the sampling matrices be relaxed while maintaining cost-effectiveness?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Efficient evaluation methods are essential for increasingly large-scale LLMs, and this work addresses relevant limitations in conventional CBE.\n\nThe integration of accuracy, convergence, and scalability in a uniformity-driven approach is a valuable contribution, setting UNICBE apart from single-objective methods.", "weaknesses": "The framework assumes preference signals (particularly from automated judges like GPT-4) are consistent with human judgment, a potentially risky simplification given known limitations in automated preference evaluations.\n\nThe formulation of multi-dimensional sampling matrices and their interaction in optimizing accuracy, convergence, and scalability may be overly complex for practical implementations and difficult to interpret for further tuning or adjustment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3tRNgPvQVQ", "reviewer": "Reviewer_k9DA", "summary": "The paper presents UniCBE, a uniformity-driven comparing-based evaluation (CBE) framework designed to improve the efficiency and accuracy of evaluating large language models (LLMs) based on human preferences. UniCBE optimizes three main objectives simultaneously: reducing sampling bias, enhancing convergence by managing uncertainty, and ensuring scalability when new models are introduced. Compared to baselines like random, Arena, and AlpacaEval, UniCBE demonstrates the lowest error and highest correlation between the ground truth evaluation results.", "questions": "See Weakness.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Comparing-based evaluation is an important problem in LLM evaluation. The proposed method solved a non-trivial problem.\n- This paper improves the comparing-based evaluation from three perspectives (accuracy, convergence, and stability). It provides a solid theory foundation, and the experiment results also demonstrate the efficiency of its proposed method.\n- Their experiments cover three dimensions well in their proposed method. They also provide ablation studies on each variant. In general, most of the variants play their roles well.", "weaknesses": "- The motivation is not enough clear. Why the previous method cannot perform well in both accuracy, convergence, and scalability?\n- The runtime of the previous CBE method ($O(NM^2)$) is one of the major limitations, and the author starts from this limitation as one of the motivations for the proposed method. However, they lack the runtime analysis for the UniCBE but only an approximate number for saving time when compared to the previous method.\n- While UniCBE shows promising results for scenarios with periodically introduced new models, it may be less efficient in highly dynamic, real-time evaluation settings where new models or samples are constantly introduced at high frequencies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7fzzBnWuyk", "reviewer": "Reviewer_PBe8", "summary": "This paper studies to explore the use of preference signals with comparing-based evaluation (CBE). It proposes a unified uniformity-driven framework that can achieve CBE with better accuracy. Experiments show the proposed method saves over 17% of evaluation budgets compared to the random sampling baseline.", "questions": "None.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The study of using preference signals with lower evaluation budgets is important research direction with huge potentials for benefiting future research and development\n\n2. This paper proposes a framework to achieve CBE with better accuracy, convergence and scalability.", "weaknesses": "1. The paper can be improved with more performance comparison with existing work and state-of-the-art performance.\n\n2. Expect more statistical and experimental conclusions with the proposed CBE method for the scenario of large-scale preference learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h01skG95dm", "reviewer": "Reviewer_8LCX", "summary": "The paper presents UNICBE, a uniformity-driven comparing-based evaluation (CBE) framework designed to optimize model evaluation across three primary objectives: accuracy, convergence, and scalability. Existing CBE method such as random allocation, method used in ARENA and ALPACAEVAL fails to maximize these aspects simultaneously. \nUNICBE addresses this by simple yet effective approach to promote uniformity across sample and models. The proposed method incorporates three decoupled sampling probability matrices, each of them derived to ensure uniformity in accuracy, convergence, scalability to newly added models. Comprehensive experiments and ablation studies across multiple datasets highlight UNICBE's cost-efficient performance in model evaluation.", "questions": "As the UniCBE is based on three matrix, $P^{acc-l}, P^{con-l}, P^{sca-l}$, each targeting different goal of accuracy, convergence, scaliability, can user steer between those by adding hyperparameter for each matrix? Would it be also possible to quantify it through experiment?\n\nWhile scalability is addressed by sequentially adding models, the paper could enhance this section by incorporating real-world scenarios, where models enter and exit dynamically, further proving UNICBE’s robustness in evolving benchmarks.\n\nThe given choice of greedy sampling over probabilistic sampling and Bradley-Terry model over Elo rating system appears significant to the framework’s success. Could the authors conduct a small experiment to demonstrate that UniCBE maintains its effectiveness across different sampling and aggregation settings?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "UNICBE introduces a novel approach to CBE by balancing three critical objectives, accuracy, convergence, and scalability, representing a straightforward yet impactful optimization techniques to advance model evaluation.\nWhile the framework’s calculations and derivations to promote uniformity are relatively simple, its contributions are substantial given the current reliance on labor-intensive evaluation processes. \nThe quality of the work is supported by extensive empirical analysis across diverse models, benchmarks, and settings, with extensive testing to support the assumptions in Section 3 and performance validation across multiple configurations in Section 5.4. \nClarity is evident in the structured presentation of key concepts and experimental results, with details in experiment or notations clearly addressed. This paper's significance lies in its contribution to large language model evaluation, providing a scalable and efficient methodology that aligns closely with human preference signals, a crucial advancement for iterative model assessments.", "weaknesses": "The novelty of balancing accuracy, convergence, and scalability needs further justification, as similar uniform sampling strategies have been discussed in prior works that highlight the uniformity, such as Vabalas et al. (2019) for sampling biases, which could diminish its uniqueness. \n\nAlthough the experiment of MT-Bench is based on human evaluator, larger portion of the evaluation is relied on AlpacaEval, as larger number of models and samples are used for the evaluation with AlpacaEval. The reliance on GPT-4 and GPT-3.5-turbo as evaluators, while useful, could benefit from validation against human judgments or additional LLMs, such as Claude, to establish greater reliability and generalizability across evaluator types. \n\nMinor details, but the readability of all figures could be enhanced by widening the lines in each plot, which would improve clarity and interpretation for readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z93ubo9FIn", "reviewer": "Reviewer_8sRK", "summary": "This paper presents UNICBE, a new framework for comparing-based evaluation (CBE) to better align large language models with human preferences. Unlike traditional CBE methods that focus on single objectives, UNICBE addresses sampling bias, manages uncertainty, and optimizes preference signals through three specialized sampling probability matrices. Tested on the AlpacaEval benchmark, UNICBE achieves high accuracy with a Pearson correlation over 0.995 and reduces evaluation costs by 17%, with savings exceeding 50% when evaluating new models, highlighting its efficiency and scalability.", "questions": "The authors argue that to avoid bias, the budget should be allocated uniformly, if so how could this method be more sample efficient than random? I guess the reason is if model A is much better than B and model B is much better than C, then it's not necessary to compare A and C a lot. But if thats the reason why this method is more sample-efficient, it would be contradictory to the uniform assumption. Could the authors provide more insight into this?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "This paper tackle an important question in LLM evaluation and provide a sound solution to it. The experiments are extensive and convincing with strong results showing the advantage of the proposed methods.", "weaknesses": "I don't see any major weakness of the paper, just the presentation can be improved, especially lack in explaining why the method is better than others in an intuitive and easy to follow way.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)