Rethinking LLM Evaluation: Can We Evaluate LLMs with 200× Less Data?
Authors: Shaobo Wang
Cong Wang
Wenjie Fu
Yue Min
Mingquan Feng
Isabel Guan
Xuming Hu
Conghui He
Cunxiang Wang
Kexin Yang
Xingzhang Ren
Fei Huang
Dayiheng Liu
Linfeng Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Data Selection
Data Pruning
Large Language Model
Benchmark Compression
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose a benchmark compression method that efficiently accelerates the evaluation of large language models (LLMs).

Abstract: Benchmark suites for large language models are growing faster than our ability to pay for them. Even when training is already expensive, many use cases require repeated evaluation across many checkpoints, variants, and competing systems, and the steady expansion of benchmark suites increasingly turns evaluation into a bottleneck in tokens and compute. This scale changes what ``useful data'' means. Instead of asking whether an instance is good for training one model, we ask **which instances are necessary to keep the collective ordering of many models stable.** We analyze redundancy at the instance level and find repetition in both the text and the ranking patterns induced across models. Based on this observation, we formulate benchmark compression as a subset optimization problem that targets accurate score reconstruction and ranking preservation at the same time. We propose EssenceBench, a coarse-to-fine framework with three stages: redundancy-aware filtering with text and ranking signals, fitness-driven subset search with an iterative genetic algorithm and a fixed surrogate predictor, and attribution-guided refinement for better coverage under tight budgets. Across multiple leaderboards, EssenceBench achieves lower reconstruction error and stronger ranking preservation than prior approaches while reducing selection time. On HellaSwag with 10K instances, EssenceBench preserves 95\% of model rankings within a 5\% shift using only 50 instances, a 200$\times$ compression.  The source code will be made available upon acceptance of the paper.

Published as a conference paper at ICLR 2026
RETHINKING LLM EVALUATION: CAN WE EVALUATE LLMs WITH 2 LESS DATA?
Shaobo Wang-
Cong Wang"*
Wenjie Fu
Yue Min 1,4
Mingquan Feng"
Tsabel Guan"
Xuming Hu*
Conghui He
Cunxiang Wang"
Kexin Yang"
Xingzhang Ren"
Fei Huang"
Dayiheng Liu?
Linfeng Zhang
"Shanghai Jiao Tong University
Qwen Team, Alibaba Group
Fudan University
"The Hong Kong University of Science and Technology
Shanghai Al Lab
ZhipuAl
Z
n
Code: hccehh/bw.pr//ggghht.ooooo////// http://hww...mooggee///////eeeen
ABSTRACT
Benchmark suites for large language models are growing faster than ability pay for them. Even when training is already expensive, many use cases require repeated evaluation across many checkpoints, variants, and competing systems, and the steady expansion of benchmark suites increasingly turns evaluation into a bottleneck in tokens and compute. This scale changes what "useful data" means. Instead of asking whether an instance is good for training one model, we ask which instances are necessary to keep the collective ordering of many models stable. We analyze redundancy at the instance level and find repetition in both the text and the ranking patterns induced across models. Based on this observation, we formulate benchmark compression gets accurate score reconstruction and ranking preservation at the same time. We propose EssenceBench, coarse-to- fine framework with three stages: redundancy- aware filtering with text and ranking signals, fitness-driven subset search with an iterative genetic algorithm and fixed surrogate predictor, and attribution -ugiddd refinement for better coverage under tight budgets. Across multiple leaderboards, EssenceBench achieves lower reconstruction error and stronger ranking preserva- tion than prior approaches while reducing selection time. On HellaSwag with 10K instances, EssenceBench preserves 95% of model rankings within a 5% shift using only 50 instances, x compression.
subset optimization problem that tar-
INTRODUCTION
Large language models (LLMs) have advanced rapidly, exemplified by GPT-4 (Achiam et al., 2023). Benchmark design has expanded accordingly, moving from single-task NLP tests to broad suites spanning multilingual understanding (Zhao et al., 2024), long-context reasoning (Kuratov al., 2024), instruction following (Yin et al., 2023), mathematical reasoning (Shao et al., 2024), context learning (Dou et al., 2026), code generation (Nam et al., 2024), multidisciplinary knowledge (Zhang al., 2025), and tool use (Chen et al., 2024). Aggregation platforms such as OpenCompass (Contrib- utors, 2023) make these suites easy to run at scale, but they also make evaluation a recurring expense. In practice, evaluation is repeated across checkpoints, ablations, and competing systems, and steadily growing benchmark suites become a growing burden tokens and compute.
This paper asks how to reduce benchmark size without changing what leaderboard would conclude. When a benchmark used to compare many models, as the Open LLM Leaderboard' (Fourrier al., 2024), the main output is often the ordering of models. Benchmark compression should therefore answer a different question: which instances are necessary to keep the collective ordering stable. The goal is to keep the induced ranking structure intact while reducing evaluation cost.
Equal contribution. x Corresponding author. t Project Leader.
http::/htugiggffiscooottttttttpp https://uuggingffcc...m www.datassess/oppnnnnlitee hawp://///eeommmmmeeeeerrrrrddss

---

Published as a conference paper at ICLR 2026
GSM8K
HellaSwag
ARC
WinoGrande
Redundancy
0.0
0.2
(a) Textual Embedding, Similarity Score
1.0
GSM8K
HellaSwag
ARC
WinoGrande
(b) Model Ranking Correlation, Correlation Score
Figure 1: Prevalent redundancy across widely used benchmark datasets. Based on 10 randomly sampled instances per dataset, panel (a) depicts the text embedding similarity (Definition 3) re- flecting semantic overlap among instances, and panel (b) presents the ranking embedding similarity (Definition 4), measured through consistency of model performance rankings across sampled subsets.
HellaSwag
EssenceBench
MetaBench
0.01
Compresiion Time (h)
WinoGrande
Langer Dutasets Bring Creater Accelentio
21% Average Redundancy
GSMSK
.0011
0000
Lower Better
2.4x
1.kk
ARC
1.2
.0.04
GSMs
WinoGmmnde
HelloSwag
Average Ranking Redundancy
(b) Average Text Redundancy
Rfficiency
Figure 2: Comparison of existing benchmark compression approaches and our EssenceBench. (a) ranking and (b) text redundancy comparison and (c) compression time comparison.
We view benchmark compression subset selection problem with coupled goals: reconstructing full-benchmark scores and preserving the model ranking. The main challenge is interaction among instances. Redundancy arises not only in the surface text, where prompts are semantically similar, but also in model behavior, where different items induce nearly the same ordering across models. These observations call for objectives and selection signals that reflect both instance-level content and leaderboard-Ler ranking- level redundancy (Definition 4) filter low-value items, and then search for compact subset that preserves the overall ordering under fixed budget.
ordering. Concretely, we quantify text-level redundancy (Definition 3) and
To address this problem, we propose EssenceBench, a coarse-to- to-ffe- compression framework with three stages. We first apply redundancy-avee the search space. We then run an iterative genetic algorithm to search for compact subsets, scoring each candidate with fixed surrogate model that reconstructs full-benchmark performance from subset accuracy. Finally, we estimate per-instance attributions from elite subsets and use this signal to guide group-wiss refinement, improving coverage and avoiding local optima. Figure 3 provides an overview of the framework, and Figure highlights the empirical gains both fidelity and efficiency. Our contributions are listed follows:
filter that uses text and ranking signals to shrink
2

---

Published as a conference paper at ICLR 2026
Step 1: Coarse Filtering
Step 2: Subset Selection
Merapge Sco Uuipul
6A
Open LLM Leaderboard
Dipul
Ajjuhment
Extract
Filtered dataset
Subset pooling
Random Subsets
Matellon
Ques Questions
fudti-roond selection
X
Grocseeer
Q:
Q:
Q
Datasets
Tournament
Models
M
M
0
0
HelloSwog WineGrande
GSMAK
ARC
GAM
Acc whole dahase!
Selecticn
MMLUU
Update Suboet Pooling
Ma
0
Step 3: Sample Selection
Text Redundancy
Ranking Redundoncy
Top
Repeal Step2 and Step3
Subset
EBM
Q) (frvic))) Rowdaa
Ra
Tnan
(IR
threshuld
Filtered datasets
Higheet
threaheld Tz
+T C
Sdection Erpet
GAM
Best Group
Duplication Flltering
M
frerage Sore Whole Delovet
Aftribulion liotiion Score
Randem Group
M4
Top Samplex
The pipeline of EssenceBench. (I) Coarse Filtering. By extracting the binary score matrix for each benchmark and computing both text-level and ranking-level redundancies, samples that exceed thresholds are removed. (II) Subset Selection. A genetic algorithm (GA) searches over subsets. Fitness is evaluated by the error of predicted performance, and subsets are optimized via tournament selection, crossover, mutation, and adjustment. (III) Sample Selection. Sample attributions are estimated from top-perforning subsets and used to build candidate groups. GA is then reapplied within each group to identify the most representative and informative subset.
Figure 3:
We define two complementary notions of redundancy for benchmark compression: text redun- dancy (Definition 3) and ranking redundancy (Definition 4), capturing semantic overlap and similarity in induced model-ordering patterns, respectively.
Based on these definitions, we conduct sample-level analysis and find that modern benchmarks exhibit substantial redundancy in both text and ranking behavior, suggesting that many instances contribute limited marginal information for distinguishing models.
We propose EssenceBench, a coarse-to-fine benchmark compression framework that combines redundancy-aware filtering, iterative GA-based subset search with a fixed surrogate, and attribution- guided refinement for robust subset selection under tight budgets.
Empirically, across multiple leaderboards, EssenceBench achieves lower reconstruction error and stronger ranking preservation than prior approaches. On HellaSwag with 10K instances, it keeps 95% of model rankings within a 5% shift using only 50 instances, achieving 200 compression. On GSM8K with subset size of 500, EsseneeBench reduces RMSE by 60.7 relative to MetaBench. Moreover, on GSM8K and WinoGrande, EssenceBench already surpasses MetaBench with only 200 examples, showing higher data efficiency under tight budgets.
2 RELATED WORKS
Large Language Model Evaluation. Benchmarks are indispensable for measuring LLM capabilities and catalyzing research. Standard NLU and reading comprehension tasks include GLUE (Wang et al., 2018) and SQuAD (Rajpurkar et al., 2016): mathematical reasoning is tested by GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021b), GSM-Plus (Li al., 2024a), and MathQA (Amini et al., 2019); multimodal mathematical reasoning by Math Vista (Lu et al., 2023); advanced STEM and graduate-level question answering by GPQA (Rein et al., 2024); complex reasoning by BBH (Suzgun et al., 2023) and MuSR (Sprague et al., 2023); coding ability via HvmanEval (Chen et al., 2021) and MBPP (Austin et al., 2021); instruction following by IFEval (Zhou et al., 2023b); agentic tool use by LiveMCPBench (Mo et al., 2025): and multiple-choice and commonsense reasoning by MMLU (Hendrycks et al., 2021a), -ha-Chalen (Clark et al., 2018), Winogrande (Sakaguchi et al., 2021) and HellaSwag (Zellers et al., 2019). Platforms such Open LLM Leaderboard (Fourrier et al.,
3

---

Published as a conference paper at ICLR 2026
2024) and OpenCompass (Contributors, 2023) provide unified pipelines and live leaderboards, but they rarely consider evaluation cost and do not address how to reduce benchmarks without changing the leaderboard conclusions.
Benchmark Compression for Evaluation. Benchmark compression operates on evaluation sets and aims reduce test-time cost while preserving reliable comparisons across models (Li et al., 2024b). Representative methods include Tiny Benchmark (Polo et al., 2024), MetaBench (Kipnis et al., 2025), Reliable evaluation based on amortized difficulty estimation (Truong al., 2025), and SMART filtering (Gupta et al., 2025). While these methods improve efficiency, many do not explicitly model instance interactions induced by redundancy in both text and ranking behavior, and scalable subset search under strict budgets remains challenging.
Data Selection for Training. Data selection for training focuses on constructing smaller or cleaner pretraining and finetuning corpora. Common approaches include large-scale filtering to remove low-quality, toxic, or duplicated samples (Raffel et al., 2020; Marion et al., 2023; Zhang et al., 2022; Abbas et al., 2023), well as subset selection strategies that aim to retain downstream performance with fewer examples, often using heuristics or IRT-based scoring (Zhou et al., 2023a; Wang et al., 2023; Ivison et al., 2023). More recent studies explore using LLMs themselves to score or rank data (Xia et al., 2024; Liu et al., 2023). However, these methods are primarily optimized for learning efficiency and typically do not target benchmark-level score reconstruction ranking consistency.
3 METHODOLOOGY
This section describes how we compress benchmark into a small subset that remains predictive for many models. The core challenge is that evaluation redundancy appears at two levels: prompts can be semantically similar, and different prompts can still induce nearly identical cross-model behavior. Our solution, EssenceBench, follows a coarse-to-fine storyline: we first remove near-duplicate instances (cheap pruming), then search for a score-reconstruucive coresct (global optimization), and finally refine for better coverage under tight budgets (local correction), as illustrated in Figure
3.1 PRELIMINARIES: BENCHMARK COMPRESSION
Benchmark compression can be framed selecting k-element subset (a coreset) from a benchmark of size N under a strict evaluation budget. In the leaderboard setting, the subset should remain predictive for many models: it should accurately reconstruct benchmark scores and, crucially, preserve the induced relative ordering among models. Searching over all k-subseets is a classic NP-hard combinatorial optimization problem. We formalize benchmark compression as follows.
Definition 1 (Benchmark Compression). Let D (I
.NN } be a benchmark dataset, and let 2D + R be an aggregate scoring function that assigns a performance score to any subset of D. Given a budget < N, the compression problem seeks a k-element subset D C C that best reconstructs the full-dataset score:
D" arg min
bCD,ID=k
where -(..)) represents suitable error measure.
Directly optimizing Definition is intractable due to the exponential search space. We therefore leverage public leaderboards (Fourrier et al., 2024), which provide per-instance correctness signals for multiple models. By organizing these into score matrix, benchmark compression reduces to selecting informative columns whose aggregate statistics reconstruct the full belehmarkk evaluation.
Let Nmodel denote the number of evaluated models and let N denote the number of benchmark instances. We denote the score matrix by S {0, Naodel XN where Sky is the binary outcome of model i on instance j (1 for correct and otherwise). Let E R.Nddd be the full-benchmark accuracy vector, with y/4 = N Σ St.j. The resulting concrete formulation is given below.
Definition 2 (Concrete Formulation of Benchmark Compression). Let binary mask represent the selection of a k-element subset: m {0, 1J s.t. Σÿ=1 my k. By indexing the mask, obtain the corresponding submatrix of S, denoted by Sm- The aggregate scoring function in Definition I
4

---

Published as a conference paper at ICLR 2026
can then be expressed as a function of the selected columns, i.e., g(D) =g(Sm). The compression
objective becomes:
min m0
C(y, g(S)) s.t. m k.
(2)
3.2 A CLOSER LOOK AT THE SAMPLE REDUNDANCY PHENOMENON IN LLM BENCHMARKS
As illustrated in Figure 1, many LLM benchmarks exhibit high overlap both in the text of their prompts and in the models' performance rankings in our quantification of sample redundancy, i.e. text redundancy (Definition 3) and ranking redundancy (Definition 4). Intuitively, when two examples share similar wording or contain nearly identical behavior across some models, retaining both adds little new information while doubling evaluation costs.
Definition 3 (Sample Redundancy from Text Perspective). Let D #2.......... } be bench- mark dataset where each Ii denotes the textual input. Let Emb x + R Rdamb denote an embedding mapping of the input. The redundancy of specific sample pair is defined as:
Rrext = (Emb((x) Emb(a;)). The redundancy of a sample then defined as:
Rrexa (i) Σ Ejtt (Emb(a;; Emb(x,))
N
The overall redundancy of the benchmark is defined as the average redundancy across all samples:
Riext (D) EN Riex(()
(5)
N
Definition& 4 (Sample Redundancy from Ranking Perspective). Let D {21, 02,........... ..N....,N denote the benchmark dataset containing N instances, and lei Nmodel denwle the number of evaluated models. For each instance zz we define its performance vector across models as s; R (Raoodd, where the i-th element is the outcome (or score) of model on zz. The ranking redundancy between two instances is quantified by the Pearson correlation between their performance vectors:
Reante (i. = pp{;, Cov(ss, 8q)
Oz. Ozg
Here E 1-1, A high correlation indicates that models exhibit highly similar behavior on both instances, suggesting redundant signal for distinguishing models.
These two definitions are intuitively derived from both human and LLM perspectives. Specifically, textual redundancy quantifies semantic similarity, while ranking redundancy assesses behavioral similarity. When combined, they reveal complementary overlaps that each alone would overlook. facilitating more systematic redundancy elimination and benchmark pruning.
3.3 ESSENCEBENCH
Overview. EssenceBench compresses a benchmark in three steps under a budget k. Step I removes near-duplicate instances using text and ranking redundancy signals. Step 2 searches for k-instance subset that best reconstructs full- benchmark scores using genetic algorithm with a fixed surrogate fitness. Step 3 refines the subset using ceabbbutooguuidee grouping improve coverage and reduce local-optinum bias. Figure 3 summarizes the pipeline.
Step 1: Coarse filtering (remove near-duplicates). We prune the benchmark using two complemen- tary redundancy signals: text redundancy (Definition 3) and ranking redundancy (Definition 4). Given thresholds Ttext and Tranking scan instances in their original order and keep the first occurrence among any highly redundant pair. Concretely, instance 2; is kept if it is not overly similar to any carlier instance under both signals. Let e, {0, 1} denote whether x; kept:
6 I 1( RReexe (1.0) S Trext A Rranking (1. i) S Tranking)
(7)
5

---

Published as a conference paper at ICLR 2026
where 1() denotes the indicator function. Therefore, the filtered benchmark is Dîitered [2 & = 13 with size M (we reuse M to denote the number remaining instances after filtering). This step cheaply removes the most obvious duplication, later searches focus more latent redundancy.
Step 2: Fitness-based subset selection (genetic algorithm; GA). To shrink the filtered benchmark while preserving its predictive power, search for a k-instance subset using genetic algorithm (GA) (Lambora et al., 2019; Mirjalili, 2018; Mathew, 2012; Mitchell, 1998). A genetic algorithm is a population-basee evolutionary search method that iteratively refines candidate solutions through se- lection and randomized variation. Concretely, GA maintains a set of candidate subsets (a population), scores each candidate by fitness function, and improves candidates by recombining and perturbing high-fitnes: ones.
What happens inside one GA generation. Each generation consists of four intuitive operations: (i) evaluate each candidate subset with the fitmess function (Equation 9); (ii) select two high-fitness parents via tournament selection; (iii) mix parents with erossover and perturb with mutation to create child; and (iv) repair the child to satisfy the budget constraint ||m|| = & (flip bits if needed). Repeated over many generations, GA concentrates probability mass on subsets that best reconstruct full-benchmark scores.
Individual and population. An individual binary mask m E (0, 1 with Σド= my = k,
representing k-instance subset. The population is a set of individuals: p {m ..... m ((r)) }. where Np is the number of the individuals in the population. The final top-Ng best subsets are also (mf))..... (Ne)).
denoted as a set:
Fitness evaluation (fixed surrogate). Fitness corresponds to minimizing the reconstruction loss in Equation 2. Direct evaluation g for every candidate subset expensive, so we train a lightweight surrogate once per GA round: Generalized Additive Model (GAM) (Hastie, 2017) that maps model's subset accuracy to its hull-benchmark accuracy. The GAM is then kept fixed during the evolutionary search, ensuring both efficiency and evaluation consistency.
Let the score matrix and full-benchmark accuracies on the filtered benchmark be Süttered E (0, 1) X and Yfltered Noodel For subset mask {0, 1] 1JM with ||m]]] = k, the subset accuracy of model is
M Σ
s, s,(m)
[t
my.
(8)
k j=1
We then predict O (ms)) Using RMSE on a held-out model set V, we define:
fitness
m)
Σ (U. v))²
I
iEV
This objective is stable across candidates because g is fixed within each GA round.
Tournament selection. In this process, we aim to choose an individual from P as parent according to fitness (Equation 9). Let (a), (b)) denote the two parents chosen in this process.
Crossover. In this process, the goal is to get the two parents. Let & denote a flag which parent the new individual should follow, the crossover process generates a new individual (() by:
individual by combining the information of
m (c)
(m,)) (a) A A V (m² (b) A {,).
(10)
where E, Bernoulli(0.5), j M].
Mutation. In this process, the aim is to introduce randomness into the new individual m()), let A be a flag which sample of an individual should mutate:
my
(c)
my (c)
e Xj1
(11)
where Xy Bernoulli (4) : j M..
Adjustment. To guarantee (c) has k-ones, the adjusiment process is implemented by randomly setting superfluous ones to zeros, vice versa.

---

Published as a conference paper at ICLR 2026
Step 3: Attribution- based Sample Selection. To maximize reconstruction fidelity while preserving representational diversity during dataset compression, we train an Explainable Boosting Machine (EBM) (Nori et al., 2019) on the elite mask set (m ..... m [Ne)) from Step 2, assigning each sample in Piltered= = {II,... (.1...... #MM J data-specific attribution score; these scores are used to stratify samples into groups, upon which genetic algorithm (GA) performs optimized selection to balance signal strength and coverage. The result is compressed dataset Deompressedd of size P < M, which retains high-impact instances while preserving underrepresented patterns pattens-achieving efficient, robust, and generalizable compression without compromising reconstruction performance.
To quantify how much each sample contributes to reconstruction accuracy in the predictor model, first define the attribution of each sample within mask. For mask E E, define the selected index set Z(m) = {i mm = 1}. An EBM Im (Piiterea) = _=)F((m) f (2t) learns the training data T' Let Siitered,, denote the i-th row of Siitered, then the form of T' is: (Sittered,s, y/ NET', which constructs a map from sample score matrix to whole dataset accuracy. The component norm [lf[" [|2 in Im is defined as the attribution of sample I in mask. Aggregating over all attributions of each mask in E yields the global attribution of sample 1, denoted as Ap:
A Emes 1 [(m) 11/7" l12
(12)
Emer 1 E [(m)
where I E (iiiterr))) According to the attributions, a tri- tri-partition of the samples can be imple- mented. With retention ratio a E (0, 1), set q = aoM
P < M and create three groups of equal size q: Gnigh. Glow: Grand.Where where Ghigh contains the g samples with the largest At, Glow the smallest ones, and Grand the random ones. The grouping operation deliberately forces subsequent GAs to search in regions of different attributions so that information neglected by the current top-Ng subsets can be rediscovered, while the information that is really significant can be boosted. For every group G' E (Grigh Glow Grand J. GA is used to judge the best group to get Deompresed-
To increase diversity of pruned dataset, we iteratively repeat Step 2 and Step 3, at each round the globally best mask m² is updated whenever a lower error is observed. The details are in Appendix A.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We constructed our evaluation using data from the Open LLM Leaderboard (Fourrier et al., 2024). We use 5 standard benchmarks: GSM8K (IK samples), ARC (400 samples), HellaSwag (10K samples), WinoGrande (44K samples), and MMLU (15K samples). To validate the generalization of our approach, extended experiments to 8 modern benchmarks: LiveMCPBench (agentic tool use), Math (multimodal math), GPQA (graduate STEM), BBH (complex reasoning). MATH (advanced math), MUSR (multistep reasoning). IFEval (instruction following), and GSM-Plus (robustness). Data preprocessing followed protocols from MetaBench (Kipnis et al., 2025), utilizing a strict 9:1 train-test split based on stratified sampling of model performance strata.
Baselines. We compared EssenceBench against state-of-the-art compression methods MetaBench (Kipnis et al., 2025) and SMART (Gupta al., 2025). Additionally, we included classic selection strategies: Random Selection, GraNd (Paul et al., 2021), Perplexity (PPL) (Bengio al., 2003), K-Means Clustering, and MLP-based Selection.
4.2 MAIN RESULTS ON STANDARD BENCHMARKS
Score Reconstruction Performance.
Tables and 2 report the leaderboard score reconstruction error (RMSE) on 13 benchmarks. Across all datasets and subset sizes, EssenceBench consistently achieves the lowest RMSE. Notably, on GSM8K, EssenceBench with only &o = 200 samples achieves an RMSE of 0.86, outperforming MetaBench at A 500 (RMSE 0.96), corresponding to 2.5x improvement in data efficiency. Similarly, on WinoGrande, EssenceBench with & = 200 (RMSE 0.78) matches MetaBench with k 500 (RMSE O. 79). Beyond these standard benchmarks, EssenceBench maintains the same trend on more challenging suites (Table 2). e.g., it achieves near near-perfect reconstruction on Math Vista (RMSE 0.001 at A 300) and remains highly robust GSM-Plus (RMSE 0.010 at k 300,
7

---

Published as a conference paper at ICLR 2026
Table 1: Prediction error (RMSE 4) on 5 standard benchmarks using subset size N
(50, 100,..., 500).
5000.. EssenceBench achieves lower error than all baselines across subset sizes.
Subset Size (k)
Dataset
Method
50
100
150
200
2550
300
350
400
450
500
Random
3.689
2.853
2.293
1.945
1.627
1.401
1.343
1.228
1.149
1.040
PPL
4.194
2.699
2.301
1.922
1.719
1.515
1.359
1.300
1.199
1.130
GSMSK
GraNd
3.852
2.893
2.373
2.096
1.826
1.599
1.444
1.272
1.157
1.069
MetaBench
3.528
2.434
2.067
1.760
1.553
1.387
1.263
1.130
1.033
0.958
EssenceBench
2.769
1.667
1.152
0.864
0.718
0.610
0.559
0.504
0.456
0.377
Random
3.153
2,446
2.165
1.731
1.450
1.319
1.153
1.098
0.934
0.902
PPL
5.334
3.119
2.117
1.819
1.546
1.416
1.217
1.088
1.005
0.917
ARC
GraNd
5.334
2.947
2.148
1.840
1.602
1.350
1.157
1.044
1.000
0.943
MetaBench
2.741
2.077
1.684
1,447
1.248
1.110
0,977
0.949
0.863
0.749
EssenceBench
2.399
1.429
1.165
0.802
0.719
0.605
0.505
0.480
0.433
0.370
Random
2.574
1.807
1.466
1.198
1.157
1.020
0.975
0.893
0.797
0.834
PPL
2.975
1.995
1.643
1.298
1.080
0.931
0.916
0.833
0.794
0.747
HellaSwag
GraNd
2.922
2.010
1.589
1.257
1.043
0.960
0.884
0.824
0.842
0.783
MetaBench
2.434
1.694
1.468
1.314
1,168
1.067
0.993
0.912
0.871
0.822
EssenceBench
2,264
1.572
1.232
1.064
0.891
0.748
0.668
0.615
0.533
0.511
Random
3.500
2.871
2.148
1.949
1.594
1.531
1,277
1.154
1.128
0.985
PPI.
4.269
2.748
2.340
1.935
1.791
1.775
1.571
1.471
1.399
1.318
WinoGrande
GraNd
4.269
2.656
2.305
2.014
1.778
1.767
1.616
1.505
1.404
1.273
MetaBench
2.783
2.122
1.752
1.530
1.289
1.203
1.072
0.958
0.866
0.785
EssenceBench
2.509
1.399
0.979
0.777
0.631
0.558
0,510
0.452
0.413
0.391
Random
3.505
2.288
2.104
1,910
1.578
1.590
1.498
1.336
1.336
1.170
PPL
8,029
9.763
10.500
8.505
8.015
7.882
7.678
7.275
7.206
6.781
MMLU
GraNd
9.000
10.091
10.475
8.733
8.156
7.903
7.745
7.223
7.578
6.851
MetaBench
2.427
2.093
1.738
1.529
1.362
1.287
1.199
1.140
1.063
0.994
EssenceBench
2.412
1.829
1.395
1.113
1.022
0.846
0.767
0.691
0.641
0.597
Table 2: Prediction error (RMSE 1) on more challenging benchmarks using subset size k {50, 150, 300) EssenceBench yields the lowest eтrог across diverse evaluation settings.
Dataset
Method
k=50
k=150
k=300
Dataset
Method
k=50
k=150
k=300
Math (Multimodal)
Random
8.381
8.068
8.811
MATH
Random
3.132
1.879
1.166
MetaBench
2.951
2.779
4.389
MetaBench
2.389
1.472
0.990
(Adv. Math)
Ours
0.017
0.009
0.001
Ours
1.895
0.668
0.475
GPQA
Random
2.739
2.028
1.441
BBH
Random
5.022
2.383
1.842
MetaBench
2.117
1.660
1.207
MetaBench
3.279
2.036
1.523
(Grad. STEM)
(Reasoning)
Ours
1.841
0.833
0.470
Ours
2.442
1.063
0.733
IFEval
Random
4.309
2.248
1.317
GSM-Plus (Robustness)
Random 6,749
4.692
3.016
MetaBench
3.607
1.985
1.139
MetaBench
0.575
0.375
0.195
(Instruction)
Ours
3.603
1.239
0.555
Ours
0.171
0.030
0.010
MUSR
Random 3.100
2. 102
1.414
LiveMCP (Agentic)
Random
10.23
10.61
10.87
MetaBench
2.927
1.697
1.106
MetaBench
8.242
8.165
8,210
(Multistep)
Ours
2.538
0.949
0.412
Ours
0.015
0.003
0.001
vs. MetaBench 0.195). These results demonstrate that our strategy effectively identifies the most informative samples for accurate leaderboard reconstruction.
Ranking Fidelity. Beyond RMSE, evaluated ranking preservation using suite of metrics (Pearson, Kendall, Rank Stability, Top-50 Acc). As shown in Table 3 (and other tables in Appendix B), EssenceBench consistently achieves higher ranking correlations. For instance, on GSM8K with only 200 samples, our method attains Kendall correlation of 0 0,968, surpassing MetaBench's 's performance
2 "LveMCPBench contains only 93 instances, therefore its three columns correspond k E {50, 90).
8

---

Published as a conference paper at ICLR 2026
Ranking Changing Ratio (Percentage)
MetaBench
EssenceBench
Still Out of 5%
Within 10%
Within 5%
-10
10
-10
-5
5
(a) k 200
(b) k 400
(c) k = 500
Figure 4: Comparison of ranking change distributions hetween MetaBench and EssenceBench on the HellaSwag dataset. Our method shows significantly tighter error bounds.
Table 3: Ranking Fidelity on GSMSK. EssenceBench achieves superior correlation by fewer samples.
Metric
Method
50
100
150
200
250
300
350
400
450
500
MetaBench
0.991
0.995
0.997
0.998
0.998
0.999
0.999
0.999
0.999
0.999
Pearson f
EssenceBench
0.994
0.998
0.999
0.999
1.000
1.000
1.000
1.000
1.000
1.000
MetaBench
0.888
0.912
0.934
0.937
0,949
0,950
0,956
0,960
0.964
0.967
Kendall f
EssenceBench
0.905
0.943
0.955
0.968
0.973
0.976
0.978
0.980
0.982
0.985
Rank Err
MetaBench
0.953
0.959
0.987
0.984
0.997
0.993
0.998
0.998
1.000
1.000
(within 10%)
f
EssenceBench
0.956
0.989
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
EacssBBacch wote Coare Tlsiig
Eserrctiencc
EaceseReendi
Predication Error (RMSE)
Highes caly
Lowes only
25
Randon only
Large imgrut
20.
nalnning RMSS when k sttdll
Ammrrusion crocil
2.0
EssenceBench consistently achieves the lowest RMSE
maistining at smaller's
15-
3.5
0.5
160
430
500
100
200
300
450
100
350
(a) Effect of Coarse Filtering
(b) Effeect of Attribution
(c) Effect of grouping
Figure 5: Ablation results on GSM8K, evaluating the effect of (a) coarse filtering, (b) attribution-
based selection, and (c) grouping strategies.
450 samples (0.964). Figure 4 visually confirms this: with k 200, EssenceBench constrains ranking shifts strictly within 10%, whereas MetaBench exhibits significantly larger deviations.
Ablation Study. We analyzed the impact of Coarse Filtering, Attribution-baser selection, and Grouping strategies. As illustrated in Figure 5, results indicate that increasing refinement rounds consistently reduces RMSE (e.g., from 2.77 to 2.47 on GSM8K). Coarse filtering is crucial for small subset sizes, removing redundancy that degrades quality. Attribution-guide grouping effectively outperforms random selection when the data budget is tight (k- < 400).
Comparison with SMART and other Optimization Baselines. We compared our approach against K-means, MLP-based selection, and the concurrent work SMART (Gupta et al., 2025). As shown in Table 4 (Left), EssenceBench achieves 2.4-6.8x lower error than K-means and 1.7-3.5x improvement over MLP on GSM8K. Comparing to SMART on ARC-Challenge (Table 4, Right), we achieve better RMSE (1.104 vs 1.19)) while using significantly fewer samples (8.53% 39.25% of the dataset). As illustrated in Table 5, using SMART as our coarse filtering step yields marginal gains, suggesting coarse filtering is already highly effective because it needs less information and computation.

---

Published as a conference paper at ICLR 2026
Table 4: Left: Comparison against optimization baselines on GSM8K (RMSE L). Right: Comparison against SMART on ARC-Challenge.
Optimization Baselines (GSM8K)
SMART (ARC-Challenge))
Method
k=50
k=100
k=200
k=400
Metric
SMART
Ours
K-means
4.133
4.131
3,300
2.757
Subset Size (k)
460 (39%)
100 (8%
MLP-Sclect
4.637
3.354
2.417
1.573
RMSE +
1.193
1.104
EssenceBench
2.900
1.525
0.857
0.454
Pearson
0.997
0.998
Table 5: Comparison of EssenceBench and replacing coarse filtering step with SMART on MMLU
(RMSE ))
Subset Size (k)
100
150
200
250
300
EssenceBench
2.41
1.83
1.40
1.11
1.02
0.85
SMART coarse filtering
2.59
1.72
1.41
1.10
0.98
0.82
4.3 ROBUSTNESS, GENERALIZABILITY, AND EFFICIENCY
Robustness to Hyperparameters, We vary key hyperparameters in coarse filtering and optimization, including (Text, Tr GA population size Np. elite size Ne, and the duplication filtering strategy. Across GSM8K and ARC, RMSE changes remain within #0.15, indicating robust performance without careful tuning. Details are reported in Appendix C, Tables 8-12.
Generalization to Unseen Models. We generalization using hold-out split of 600 unseen leaderboard models (Appendix D). On GSM8K, EssenceBench maintains strong agreement with full-leaderboard scores, with Pearson correlation 2 0.992 across subset sizes (Table 13).
Diversity, Bias, and Efficiency. EssenceBench preserves diversity (change - 2.3%; Table 14) and introduces negligible MMLU distribution shift (typically 2%; Table 15). also achieves up to 49x speedup on HellaSwag (Table 16). Details are reported in Appendix E.
4.4 QUALITATIVE ANALYSIS: REDUNDANCY
To demonstrate the effectiveness our coarse filtering, we present representative cases of redundancy in Appendix F. We identify samples with high textual similarity (differing only by proper nouns or value placement) and ranking similarity (different narratives but identical reasoning structures). By filtering these effectively, EssenceBench maximizes the information density of the selected subset.
CONCLUSION
In this paper, identified sample redundancy in LLM benchmark evaluation. To address this problem, we introduced EssenceBench, coarse-to-fine benchmark compression framework that combines redundancy-aveen filtering with iterative genetic algorithm optimized for accurate reconstruction. Extensive experiments on five standard benchmarks show that EssenceBench achieves 200x reduction in benchmark size while preserving ranking fidelity.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S Morcos. Semdedup: Data- efficient learning web-scale through semantic deduplication. arXiv preprint arXiv:2303.099: 0093000 2023.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.087 087774 2023.
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation- formalisms. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies. volume (long and short papers), pp.
2019.
Jacob Austin, Augustus Odena, Maxwell Nye. Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, al. Program synthesis with large language models. arXiv preprint arXiv:2108.077: 008.0773,, 2021.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3(Feb): 137-1155, 2003.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.0333 003343 2021.
Sijia Chen, Yibo Wang, Yi-Feng Wu, Qingguo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Lijun Zhang. Advancing aogl-augeetted large language models: Integrating insights from errors in inference trees. Advances in Neural Information Processing Systems, 37: 17:106555-10658, 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: 11X3v:8803004555 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models, 2023.
Shihan Dou, Ming Zhang, Zhangyue Yin, Chenhao Huang. Yujiong Shen, Junzhe Wang, Jiayi Chen, Yuchen Ni, Junjie Ye, Cheng Zhang, Huaibing Xie, Jianglu Hu, Shaolei Wang. Weichao Wang, Yanling Xiao, Yiting Liu, Zenan Xu, Zhen Guo, Pluto Zhou, Tao Gui, Zuxuan Wu, Xipeng Qiu, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Di Wang, and Shunyu Yao. Cl-bench: A benchmark for context learning, 2026.
//trxaaavvvvv///////a//eaeeaaavv org/ / abs / 202 03587.
Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. Open Ilm leaderboard v2. https::/p/// / htt.//uggnngggg www.inngfacc... co/ spaces open- 11m leaderboard/ open 11m www.eeeeeeerbbrdrooo 2024.
Vipul Gupta, Candace Ross, David Pantoja, Rebecca Passonneau, Megan Ung, and Adina Williams. Improving model evaluation using smart filtering of benchmark datasets. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 4595-4615, 2025.
Trevor Hastie. Generalized additive models. Statistical models S. pp. 249-307 2017.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference Learning Representations (ICLR), 2021a.
11

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXhv:2103.03874 2021b.
Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, et al. Camels in a changing climate: Enhancing Im adaptation with tulu 2. arXiv preprint arXiv:2311.10700 2023.
Alex Kipnis, Konstantinos Voudouris, Luca M. Schulze Buschoff, and Eric Schulz. metabench sparse benchmark of reasoning and knowledge in large language models, 2025. URL https:
//aaxivv. wrt.vvorg//bb.. 2407. 12844.
Yury Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. Babilong: Testing the limits of Ilms with long context -naaainngnn----ystcckk.. Advances in Neural Information Processing Systems, 37:106519- 19-1655599 2024.
Annu Lambora, Kunal Gupta, and Kriti Chopra. Genetic algorithm-a literature review. In 2019 inter- national conference on machine learning, big data, cloud and parallel computing (COMITCon), pp. 380-384. IEEE, 2019.
Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehensive benchmark for evaluating the robustness of Ilms as mathematical problem solvers. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long 2024a.
Papers). pp.
Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data high-quality benchmarks: Arena-hand hard and benchbuilder pipeline. arXiv preprint arXiv:2406.1 1939, 2024b.
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312. 15685, 2023.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-We Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXhv:2310.02255, 2023.
Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining Ilms at scale. arXiv preprint arXiv:2309.04564, 2023.
Tom V Mathew. Genetic algorithm. Report submitted at IIT Bombay, 53:18-19, 2012.
Seyedali Mirjalili. Genetic algorithm. Evolutionary algorithms and neural networks: theory and applications, pp. 43-55. Springer, 2018.
Melanie Mitchell. An introduction to genetic algorithms. MIT press, 1998.
Guozhao Mo, Wenliang Zhong, Jiawei Chen, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, and Le Sun. Livemepbench: Can agents navigate an ocean mep tools? arXiv preprint arXiv:2508.0177 2025.
Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. Using an Ilm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pp. 1-13, 2024.
Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint (PXiv:190.00922. 2019.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning data diet: Finding important examples early in training. Advances in neural information processing systems, 34: 90596-20607 2021.
12

---

Published as a conference paper at ICLR 2026
Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating Ilms with fewer examples. arXiv preprint arXiv:2402.14992, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140))1-67, 2020.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprchension of text. arXiv preprint arXiv:1606. 05250, 2016.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petry, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First conference on language modeling, 2024.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9)999-106, 2021.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.0333 2024.
Zayne Sprague, Xi Ye, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett. Musr: Testing the limits of chain-of-thougee with multistep soft reasoning. arXiv preprint xxXivv2310. 16049, 2023.
Mirac Suzgun, Nathan Scales, Nathanacl Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-offthouuge can solve them. In Findings of the Association for Computational Linguistics:: ACL 2023, pp. 13003-13051, 2023.
Sang T Truong, Yuheng Tu, Percy Liang, Bo Li, and Sanmi Koyejo. Reliable and efficient amortized model-based evaluation. Forty-second International Conference on Machine Learning. 2025.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP workshop BlackboxNLP. Analyzing and interpreting neural networks for NLP, pp. 353-355, 2018.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information Processing Systems, 36:74764-74786 2023.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.04333, 2024.
Wenpeng Yin, Qinyuan Ye, Pengfei Liu, Xiang Ren, and Hinrich Schütze. Lim-driven instruction following: Progresses and concerns. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts, pp. 19-25, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 479444998880 2019.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-tra pre-trained transformer language models. arXiv preprint arXiv:2205. 01068, 2022.
Zicheng Zhang. Xiangyu Zhao, Xinyu Fang, Chunyi Li, Xiaohong Liu, Xiongkuo Min, Haodong Duan, Kai Chen, and Guangtao Zhai. Redundancy principles for milms benchmarks, arXiv preprint arXiv:2501. 13953, 2025.
13

---

Published as a conference paper at ICLR 2026
Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. How do large language models handle multilingualism? Advances in Neural Information Processing Systems, 153199 2024.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 06:55006-55022 2023a.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Inslrwction-followingg evaluation for large language models. arXiv preprint arX1v:2311.07911, 2023b.
14

---

Published as a conference paper at ICLR 2026
A PSEUDO-CODE OF ESSENCEBENCH
We provide the detailed pseudo-code for the two core components of EssenceBench: the Genetic Algorithm (GA) for subset selection and the coarse-to-fine Iterative Refinement framework.
Algorithm 1: Genetic Algorithm (Subset Selection)
Require: Filtered Score Matrix S {0, 1) NLLMXM, Ground Truth y E RRLLUMM Target Size Population Np, Elites Ng. Generations Nc.
Ensure: Best mask m", Lowest error e*, Top elites
Ny 1: Initialize population P {m(() J f==l with random binary k-masks
2: m² <- 0, e* + +o0
3: for = to NG do
4: for each m P do
5: Estimate accuracy: y = gS m/k) {Using GAM predictor g(-))
6:
Compute RMSE: {m)) = V Naa E y1)²
7: Fitness F(m) - emm
8:
end for
9:
E+ + Top Ng masks sorted by fitness
10: P' <- & {Elitism}
11:
while [P' < Np do
12:
ma. m, <- TournamenSSllction())
13:
Mchild & Crossover(ma. my)
14:
Mehild Mutate(m.sata
15:
Mehild <- aajustSize(maaaaa. k) (Ensure exactly k ones]
P' p {mehald)
17:
end while
19:
if min (PP) < then
Update m" and
21:
end if
22:
end for
23:
return m"
Algorithm 2: Iterative Refinement with Attribution Guidance
Require: Full Scores S, Ground Truth y, Target k, Max Rounds Rmax Attr. Ratio Ensure: Optimal subset mask m"
1: Candidate Indices I + (1,..., M) (All samples)
2: E* + 00
3: for to Rmaxx do
4: (Step 1: Run GA on current pool]
5:
(m, RUNGA(S),,
6:
if's < E* then
7:
m² m, t-
8:
end if
9:
if [ < or Rmax then
10: break
end if
12:
(Step 2: Attribution-based Grouping)
13:
Compute Attribution A, for each sample j E I using Elites Partition I into groups based on A₁: GHigh: GLow, GRand (Step 3: Select best group for next round)
14:
15:
16:
Evaluate candidate groups via fast GA search
17:
I +- arg ming e eG)
18:
end for
19:
return m
15

---

Published as a conference paper at ICLR 2026
B FULL BENCHMARK RESULTS
We provide the complete evaluation tables for HellaSwag and ARC. These results complement the GSM8K results provided in the main text. EssenceBench demonstrates consistent superiority in both Score Reconstruction (RMSE) and Ranking Fidelity.
Table Full Results on HellaSwag. Comparison of MetaBench vs, EssenceBench across all metrics. Highlighted rows indicate our method.
Method
Metric
50
100
150
200
250
300
350
400
450
500
RMSE 4
2.397
1.734
1.495
1.318
1.166
1.076
0.961
0,929
0.880
0.836
Pearson
0,992
0.996
0.997
0.998
0.998
0.998
0,999
0,999
0,999
0.999
MetaBench
Kendall
0.879
0.915
0.928
0.937
0,942
0,946
0,955
0.954
0.957
0.959
Top-50 Acc f
0.920
0.940
0.960
0.940
0,900
0.960
0,900
0.960
0.940
0,980
Rank Eir (within 10%) f
0.902
0.961
0.983
0.989
0.991
1.000
0.998
1.000
0.998
1.000
RMSE
1.453
1.038
0.706
0.635
0.504
0.409
0.419
Peanson
0.994
0.997
0,999
0.999
0.999
0.999
1.000
1.000
1.000
1.000
EssenceBench
Kendall f
0.899
0.931
0.951
0.959
0.966
0.967
0.973
0.974
0.978
0.978
Top-50 Acc
0.960
0.920
0.940
0.940
0.940
0.980
0.960
0.960
0.960
0.980
Rank (within 10%)
0.946
0.988
1.000
1.000
1.000
1.000
1.000
1.000
Table 7: Full Results on ARC. Comparison of MetaBench vs. EssenceBench across all metrics.
Highlighted rows indicate our method.
Method
Metric
50
100
150
200
250
300
350
400
450
500
RMSE $
2.968
2.082
1.690
1.511
1.332
1.186
1.077
0.961
0.890
0.836
Pearson
0,986
0.993
0.995
0.996
0.997
0.998
0.998
0.998
0.999
0.999
MetaBench
Kendall
0.870
0.898
0.922
0.928
0,936
0.942
0.949
0.955
0.959
0.961
Top-50 Ace
0.900
0.880
0.900
0.880
0.920
0.920
0.900
0.960
Rank Er (within 10%)
0.881
0.929
0.973
0.985
0.985
0.992
0,998
1.000
1,000
0.998
RMSE
2.045
1.104
0.841
0.703
0.612
0.529
0.501
0.422
0.368
0.347
Pearson
0.993
0.998
0,999
0.999
0,999
1.000
1.000
1.000
1.000
1.000
EsseneeBench
Kendall f
0.907
0.947
0.958
0.964
0.969
0.973
0.974
0.978
0.981
0.983
Top-50 Ace
0.900
0.940
0,960
0.920
0.960
1.000
0.960
0.980
0.960
0.940
Rank (within 10%)
0.949
0.997
1.000
1.000
1,000
1.000
1.000
1.000
1.000
C DETAILED ANALYSIS OF SENSITIVITY
C.I HYPERPARAMETE SENSITIVITY (COARSE FILTERING)
We investigate the sensitivity of thresholds Trext and Tranking used in the Coarse Filtering stage. Tables 8 and 9 show RMSE performance on GSM8K and ARC 50). The variations are minimal (+ ~ 0.15), indicating robustness to threshold selection.
Table 8: Sensitivity of Ttext and Tranking on GSM8K.
Table 9; Sensitivity of Teext and Tranking on ARC.
Text Trank
0.80
0.85
0.90
0.95
Tix Trank
0.80
0.85
0.90
0.95
0.80
2.65
2.56
2.64
2.80
0.80
2.04
2.14
2.00
2.06
0.85
2.68
2.64
2.69
2.61
0.85
2.10
2.03
2.11
2,03
0.90
2.74
2.70
2.76
2.77
0.90
2.09
2.18
2.05
2.16
0.95
2.61
2.75
2.65
2.69
0.95
2.05
2.13
2.06
2.06
16

---

Published as a conference paper at ICLR 2026
C.2 GENETIC ALGORITHM HYPERPARAMETERS
Tables and 11 display the sensitivity to Population Size (Np) and Elite Size (Ne). The method remains stable across wide range of configurations.
Table 10: Sensitivity to GA Params on GSM8K.
Table 11: Sensitivity to GA Params on ARC.
N \N
20
30
40
N₂ \N
2.74
2.78
2.72
2.64
100
200
2.73
2.67
2.63
2.71
200
2.08
2.05
2.13
2.06
300
2.69
2.64
2.66
2.66
300
2.05
2.21
2.07
2.09
400
2.60
2.66
2.70
2.62
400
2.14
2.13
2.08
2.04
C.3 ROBUSTNESS TO DUPLICATE FILTERING STRATEGY
We compared default sequential filtering against Random and Conservative (2 duplicates) strategies. As shown in Table 12, the choice of filtering strategy has negligible impact on final performance.
Table 12: Robustness of Duplication Filtering Strategies.
GSM8K
ARC
Filtering Strategy
k = 50
h 100
k 50
h 100
Original (Sequential)
2.758
1.565
2.017
1.265
Random (Stochastic)
2.760
1.469
2.101
1.312
Conservative (2 2 duplicates)
2.651
1.558
2.099
1.300
D DETAILED ANALYSIS OF GENERALIZABILITY
In original experimental setup, have already implemented rigorous data split protocol to evaluate generalizability. Specifically, for both the training and testing sets, the dataset is first ranked by score and then partitioned into ten equipotent strata. Within each stratum, 10% of the instances are stratifiedly sampled and subsequently pooled to constitute the test set; the remaining 90% are retained as the training set.
To further explore the generalizability on train-val-test partition, we conducted a rigorous hold-out experiment where randomly selected 600 models among total 6736 models from the leaderboard results as the test set, while using the remaining models for training and validation.
Our comprehensive results in Table demonstrate that the compressed benchmarks exhibit strong generalization to unseen models. The consistently high correlation coefficients (>> 0.99), ranking metrics, and error reduction patterns across all k-values prove the generalizability of EssenceBench.
Table 13: Generalizability on Unseen Models (GSM8K). Full results across metrics and subset sizes.
Metric
50
100
150
200
250
300
350
400
RMSE 1
3.463
2.323
1.937
1.703
1.499
1.323
1.327
1.122
Pearson
0,992
0.996
0.998
0.998
0.999
0.999
0.999
0.999
Kendall f
0.881
0.914
0.932
0.942
0.950
0.957
0.962
Top-50 Acc
0.360
0.600
0.740
0.820
0.940
0.880
0.940
0.920
Rank (within 10%)
0.923
0.957
0.982
0.990
0.995
0.993
0.995
0.998
17

---

Published as a conference paper at ICLR 2026
E DETAILED ANALYSIS OF DIVERSITY, BIAS AND EFFICIENCY
Diversity. We evaluate whether compression reduces data diversity on GSM8K, ARC, and Hel- laSwag. Following prior work, embed each text sample 2ς into a semantic vector v; using BGE-M3, and measure pairwise cosine similarity sim(v,, Vz). We define the diversity score as the complement of the average pairwise similarity:
N Σ J#
D = 1 N(N
sim (V: Yy),
(13)
where N is the number of samples. Higher D indicates greater diversity. As shown in Table 14, EssenceBench preserves diversity well, with relative changes within 2.3% across all three benchmarks.
Table 14: Diversity scores 'D (complement of average cosine similarity). Higher is more diverse.
Method
GSM8K
ARC
HellaSwag
Full Dataset
0.397
0.367
0.438
EssenceBench
0.393
0.366
0.428
Relative Change
-1.01%
-0.27%
-2.28%
Bias (Distributional Shift). We further analyze potential selection bias MMLU by comparing the subject distribution between the full dataset "Daww and the selected subset Dee with fixed subset size N 2000. Let Praw and (t) denote the percentage of subject i in Draw and D respectively. We quantify distributional shift using the deviation ratio:
P (1) Prow (1)
Aratio (2)
(14)
Praw (1)
Lower Aratio indicates better preservation of the original distribution. Table 15 reports six representa- tive subjects and shows consistently small deviations (typically 2%). suggesting that EssenceBench introduces negligible selection bias.
Table 15: Subject distributional shift on MMLU (N = 2000). Aatio measures relative deviation.
Subject
Raw (%)
EssenceBench (%)
Aratio
Elementary Mathematics
2.722
2.725
0.0010
College Medicine
1.260
1.247
0.0101
International Law
0.857
0.872
0,0176
Management
0.756
0,743
0,0182
Econometrics College Chemistry
0.807
0.822
0.0187
0.706
0.721
0.0212
Efficiency. We quantify the efficiency by adding all the time together. Let Tiull denote the time of full-benchmark evaluation, Teomp the compression cost, and Teval the evaluation time on the compressed subset. The evaluation then costs T Toomp Tewal- Table 16 reports the speedups.
Table 16: Computational Cost Analysis and Acceleration Factors. Time is measured hours.
Full Eval.
EssenceBench Cost (Hours)
Benchmark
Size
Time (h)
Comp.
Eval.
Total
Acceleration (t)
GSM8K
1319
1.96
0.27
0.07
0.34
4.76x
ARC
1172
1.97
0.29
0.08
0.37
4.32x
HellaSwag
10,003
22.06
0.33
0.11
0.44
49.14x
WinoGrande
1267
0.78
0.28
0.03
0.31
1.52x
18

---

Published as a conference paper at ICLR 2026
F
CASE STUDY
As clearly illustrated in Table 17, the proposed filtering mechanism successfully and consistently identifies semantically equivalent items across various cases. the text redundancy case, two questions differ in phrasing but share the same arithmetic structure. In the ranking redundancy case, two problems with different narratives yield similar model scores because both problems require multi-step numerical reasoning that includes proportional calculations, intermediate variable derivation, and aggregation of weighted quantities. These representative cases provide clear evidence that EssenceBench effectively captures and distinguishes both superficial and deeper structural redundancies.
Table 17: Examples of Redundancy handled by EssenceBench. Orange and Teal highlight identical structures or reasoning patterns.
Textual Redundancy (Lexical Overlap)
Ranking Redundancy (Score Correlation)
Zack's locker half as big as Timothy's locker. Pe- ter's locker is 1/4 as big as Zack's 's locker. If Peter's s locker is cubic inches, how big is Timothy's 's locker in cubic inches?
silver pesos 80 gold pesos. He visits her friend Anna who has twice as many silver pesos as he has and 40 more gold pesos. What's 's the total number of pesos they have together?
Axel
Timothy's locker cubic inches, Zack's locker half as big as Timothy's locker. Peter's locker is 1/4 as big as Zack's 's locker. How big is Peter's 's locker in cubic inches?
Amy is taking history test. She correctly answers 80% of the multiple-choice questions, 90% the true/false questions, and 60% of the long-answer ques- tions. The multiple-choice and true/false questions are worth point each, and the long answer questions are worth 5 points each. How many points does Amy if there are 10 multiple-choice questions, 20 true/false questions, and long answer questions?
19


Reviews:
[{"review_id": "27FiTUWhZU", "reviewer": "Reviewer_hqMF", "summary": "The paper propose using genetic algorithm to compress LLM evaluation set. Experiments show ranking is preserved with a small shift.", "questions": "1. Original benchmark performance is treated as ground truth in the paper. However, there remains doubts on that. For example, suppose if a benchmark has 50% of problems as 1+1=2, this would definitely mean that the compressed benchmark will behave differently than the original benchmark, but this isn’t a necessarily bad thing. Maybe a compressed benchmark at a certain level could better reflect an LLM’s capability.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Paper is clearly written, well motivated, and generally easy to follow.\n2. Authors identify and analyze the redundancy of the widely used LLM benchmarks.\n3. Compressing LLM benchmark by 25x (lossless) to 200x (5%) shows the potential for a far smaller “validation set” or “test set”.", "weaknesses": "1. The main benchmarks being used (GSM8K, ARC, HellaSwag, WinoGrande, MMLU) are considered saturated for the frontier LLMs these days. Newer LLMs are challenged with harder benchmarks, and harder benchmarks contain fewer instances since they are more challenging to come up with. I encourage authors to address why their overall method may adapt to modern and future benchmarks.\n2. Similar to above, authors did not test newer benchmarks. For example, how would authors method extend to agentic benchmarks (SWEBench / tau2-bench) and multimodal benchmarks (MMMU etc.) I believe a compression of those benchmarks are needed at the moment. And after all benchmark compression is a one-off effort, releasing compressed benchmarks on author’s end would be a good contribution to the community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ygic24oPLt", "reviewer": "Reviewer_ykSp", "summary": "The paper argues that sample-level redundancy makes many LLM benchmarks wasteful. The authors present EssenceBench, a coarse-to-fine pipeline that first removes redundant items, then searches for a compact subset using a genetic algorithm steered by a learned accuracy predictor, and finally refines the selection with attribution-based grouping to preserve coverage. The selection goal is to use a k-item subset to reconstruct full-benchmark scores. EssenceBench reports lower reconstruction error and strong rank fidelity.", "questions": "- I think generalizability is my biggest question/concern. Will the compressed benchmarks work for new model architectures not in the train set? How often do these benchmarks need to be recomputed, etc?\n- I would like to see a computation cost trade-off? How many GPU hours to compress each benchmark? It might be helpful to add a clear table showing benchmark size, compression time, compressed size, and number of evaluations needed to break even. This will be nice for practitioners wondering if they should use this method. \n- How were τ_text and τ_ranking chosen? Is there a principled way to set these automatically?\n- Is there a theoretical justification for using GA specifically? Did you compare against other optimization approaches? Why is this particularly suitable?\n- I'm curious if this approach can be extended to eval tasks where scoring is more complex (e.g. using LLM as a judge)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- well-motivated problem. this paper addresses what I believe is an important and practical problem of expensive LLM evaluations\n- the experimental results are thorough and consistent\n- paper is written clearly with good explanation and clear pseudo-code and a good appendix. also has thorough ablations and analysis.\n- I like the formalization of LLM benchmark compression as an optimization problem and use GA + attribution refinement", "weaknesses": "- experiments focus on static, multi-choice benchmarks. I think applicability to open-ended, interactive, or multi-modal tasks may be limited or at the very least is untested\n- the GA + EBM pipeline involves several hyperparams and the thing costs should be discussed more explicitly as its likely to be expensive. Other methods like SMART filtering [1] don't require this tuning but seem to enable accurate compression. The filtering method could be adapted to your setup as a baseline or complementary approach.\n- the coarse filtering step relies on manually set thresholds τ_text and τ_ranking. How sensitive is the method to these choices? Different benchmarks may require different thresholds, limiting generalizability.\n\n[1] Gupta, Vipul, et al. \"Improving model evaluation using smart filtering of benchmark datasets.\" arXiv preprint arXiv:2410.20245 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U6QmHP6jvc", "reviewer": "Reviewer_MmMV", "summary": "For addressing the growing cost and redundancy of large-scale LLM benchmarking, the study presents a benchmark compression framework, EssenceBench. It aims to preserve ranking fidelity while substantially reducing evaluation cost for large language models. The approach combines redundancy-aware filtering with a genetic algorithm and attribution-guided refinement to identify representative subsets of benchmark data. Experimental results on five datasets show consistent improvements over prior methods. It highlights the potential of EssenceBench to make large-scale LLM evaluation more efficient and scalable.", "questions": "1. Could the evaluation be extended with objective metrics that assess the intrinsic properties of the compressed datasets (e.g., diversity [1], bias [2] and so on)\n2. An analysis of the genetic algorithm's computational complexity and a justification for the number of generations required for convergence are needed.\n3. A more comprehensive sensitivity analysis is required for the hyperparameters to verify the robustness of the proposed framework, along with a discussion on the parameter search strategy.\n4. The definition 4 lacks sufficient details about how the ranking scores $\\mathbf{r}$ are computed from several LLMs. \n5. The case study in Table 3 offers an interesting glimpse, but a broader analysis is needed to fully support the claims about model behavior. Could you include a dedicated appendix section with more examples across different datasets to better illustrate the duplication-aware filtering in EssenceBench?\n6. In Figure 2, the legends of MetaBench and EssenceBench are mislabeled.\n7. Part of notations are inconsistent. For example, Equation 12 defines $j \\in \\mathcal{I}(m)$ while the main text refers to $j \\in \\mathcal{I}(\\mathcal{D}_{filtered})$\n\n[1] On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey. 2024.\n\n[2] Bias and fairness in large language models: A survey. 2024", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method addresses an increasingly important and practical challenge in LLM evaluation concerning benchmark cost without losing ranking reliability.\n2. The integration of fitness-based subset selection with attribution-based sample selection provides an effective paradigm for benchmark compression while maintaining the intrinsic properties of datasets (e.g., diversity).\n3. Experimental results across five benchmarks and multiple baselines demonstrate the strong empirical performance of EssenceBench.", "weaknesses": "1. The sample redundancy phenomenon has been widely observed in prior studies on dataset quality assessment. The paper should more clearly differentiate EssenceBench from these existing efforts to highlight the necessity of Genetic Algorithms.\n2. The study lacks the computational or time complexity analysis of the proposed framework, which may better demonstrate the feasibility of EssenceBench for datasets with different scales.\n3. The parameter sensitivity analysis is limited, which only investigates the effects of the number of generations (gens) and refinement rounds (rounds). Other key hyperparameters (e.g. $\\tau_{text}$, $\\tau_{ranking}$ and $N_{\\mathcal{P}}$) that likely have a substantial impact on compression quality are omitted. It raises the concern regarding parameter search complexity and the overall robustness of the proposed framework.\n4. The experimental evaluation narrowly focuses on the extrinsic metric of prediction error (RMSE) and ranking consistency. The experiments lack some quantitative metric to evaluate intrinsic quality and characteristics of the compressed datasets (e.g., diversity, bias and so on).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IE3PPKWEUc", "reviewer": "Reviewer_TShU", "summary": "The paper proposes EssenceBench, a coarse‑to‑fine framework for compressing LLM benchmarks so that rankings and overall scores can be reconstructed with far fewer evaluation items. The pipeline (i) performs coarse filtering to remove redundant items using text similarity (embedding cosine) and ranking similarity (correlation of model outcomes), (ii) runs a Genetic Algorithm to select subsets that minimize a GAM‑predicted reconstruction error (RMSE) of full‑set accuracies across models, and (iii) applies attribution‑guided selection using an Explainable Boosting Machine (EBM) to score per‑item contribution, group items (high/low/random attribution), and re‑search within the best group; this is iterated to improve diversity and convergence. Experiments on five benchmarks (GSM8K, ARC, HellaSwag, WinoGrande, MMLU) report lower RMSE and better rank preservation than MetaBench and other baselines, with large reported reductions (e.g., 25×–200× fewer samples on HellaSwag) and faster compression.", "questions": "1. In Section 3.3, the fitness uses a GAM g that maps subset accuracy s_j to full‑set accuracy y_j. It is unclear whether g is retrained per candidate subset or pre‑trained on a pool of subsets. The compute/variance implications differ drastically and affect the efficiency claim.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Casting benchmark compression as minimizing an error between full‑set and subset‑based scores is crisp and aligns the objective with leaderboard reconstruction\n2. The paper quantifies both text redundancy and ranking redundancy and visually demonstrates non‑trivial redundancy on popular datasets\n3. The three‑stage design is easy to follow and practically implementable.", "weaknesses": "1. The duplication filter “keeps the earlier item and discards the later one if similarity exceeds thresholds”, which introduces order bias and potential instability across random permutations of the dataset. No robustness check to ordering is reported.\n2. The paper mentions bge‑m3 embeddings and ranking correlations, but concrete threshold values, normalization, and sensitivity are absent\n3. Fig 2C shows wall clock hours, but the hardware, eval setup is not clear. How do you compare the eval time?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)