Difficulty–Diversity Collaborative Filtering for Data-Efficient LLM Fine-Tuning
Authors: Long P. Hoang
Wenxuan Zhang
Wei Lu
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Reasoning
Data Efficiency
Supervised Fine-Tuning
Collaborative Filtering
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: A data selection approach balancing difficulty and diversity for efficient fine-tuning of LLMs with minimal annotation effort.

Abstract: The performance of fine-tuned language models is heavily influenced by the quality and quantity of their fine-tuning data. While scaling laws suggest that larger models benefit from more data during pretraining, the Less-is-More hypothesis highlights that downstream fine-tuning often requires only a small but high-quality dataset to effectively elicit a model’s pretrained knowledge. However, identifying such premium data, particularly in terms of difficulty and diversity, typically relies on human expertise, and existing methods offer limited guidance for automatic selection from large unannotated corpora. This work presents a novel quantitative framework that formalizes the interplay between question difficulty and diversity, and introduces *Difficulty–Diversity Collaborative Filtering* (DDCF): an automated approach that tailors data selection to the unique characteristics of each language model via collaborative filtering. By leveraging a small seed dataset to predict correctness across a large unannotated corpus, our method reduces the annotation cost by $100-200\times$, while maintaining downstream performance comparable to full-corpus fine-tuning.

Published as a conference paper at ICLR 2026
DIFFICULTY-DIVESSITY COLLABORATIVE FILTERING FOR DATA-EFFICIENT LLM FINE- FINE-TUNING TUNING
Long P. Hoang' Wenxuan Zhang'
Wei Lu²
'Singapore University of Technology and Design hoang@myma il sutd. edu. sg wxzhangesutd. edu .g, wei. aei.lu@ntu.edue
NNanyang Technological University
ABSTRACT
The performance of fine-tuned language models is heavily influenced by the qual- ity and quantity of their fine-tuning data. While scaling laws suggest that larger models benefit from more data during pretraining, the Less-is-More hypothesis highlights that downstream fine-tuning often requires only small but nigh-quality dataset to effectively elicit model's pretrained knowledge. However, identifying such premium data, particularly terms of difficulty and diversity, typically relies human expertise, and existing methods offer limited guidance for automatic se- lection from large unannotated corpora. This work presents a novel quantitative framework that formalizes the interplay between question difficulty and diver- sity, and introduces Difficuly--ivversity Collaborative Filtering (DDCF): an au- tomated approach that tailors data selection to the unique characteristics of each language model via collaborative filtering. By leveraging a small seed dataset predict correctness across large unannotated corpus, our method reduces the annotation cost by 100 - -00 while maintaining downstream performance com- parable foll-corpus fine-tuning
INTRODUCTION
The remarkable success of Large Language Models (LLMs) in recent years (Grattafiori et al. 2024b Yang al. 2025b) stems largely from their ability learn rich and generalizable repre- sentations from massive pretraining corpora. To further enhance capabilities these models on downstream tasks, supervised fine-tuning (SFT) has become popular approach (Wei et al.) [022]] Chung et al. 2024). However, SFT typically involves fine-tuning pretrained models on large-scale, human-annotated instruction datasets, often comprising hundreds of thousands of examples.
Despite its effectiveness, fine-tuning on such large datasets presents several challenges. First, data collection and model training incur substantial computational costs. Second, updating model new large corpus may cause catastrophic forgetting, where continual learning of new tasks degrades performance on previously acquired knowledge (Biderman et al., 2024 Wang et al. 2024). Third, scaling up the dataset often leads to vverreepreeettation of common patterns, reducing diversity and underrepresenting rare but important examples (Kim et 2022
Zhang et al. 2025a).
Recently, the Lexs-is-More hypothesis (Zhou et al. 2023 Ye et al. 2025 Dohmatob et al. 2025) has suggested that downstream task adaptation can be achieved through minimal supervision, where the model primarily learns task-specific formatting styles reveal knowledge already encoded during pretraining. Empirical studies have shown that fine-tuning on just a few carefully selected examples sometimes outperforms naively using vast annotated corpora (Zhou et al.) 2023 Ye et al., 2025 Muennighoff et al. 2025). Furthermore, theoretical analysis (Dohmatob et al. 2025) demonstrates that, when the base model is strong, selecting harder examples offers provable advantage. How- ever, such curated datasets often rely on evolving human expertise, making them labor-intensive, inflexible, and inconvenient to adapt new models tasks.
While recent efforts have explored automated methods to improve data quality (Xia et al. 2024 Yang al. 2024b), the automatic selection without annotated output responses remains open
OOur code is publicly available at
-t..:///ii..P...bb... W..P-L.b/DDCCP

---

Published as a conference paper at ICLR 2026
High similarity Undiversified
Q1
Q2
Q3
Q4
Q1
Q4
Easy
Q1 Q2
Q3
Q4
1.0
0.3
0.8
1.0
0
0.0
0.4
0.8
0.0
M
Question Embedding
0.0
0.0
Correctness Matrix
Embedding
Factorized Matrix
Hard
An illustration of our proposed Difficuly--Diverity Collaborative Filtering framework. Given a dataset, binary conrectness matrix from model predictions is factorized into model and question embeddings. Difficulty is defined by factorized scores, while diversity is measured by question similarity. These criteria jointly guide the selection of compact yet effective training subsets, providing strong learning signals while avoiding redundant, overly similar examples.
Figure 1:
challenge. For example, (Xia et al. 2024) leveraged gradient matching to target dataset, while Yang et al. (002b) instead trained the LM on the entire annotated corpus and then selected samples by clustering their loss trajectories. Other approaches (Ye et al. 2025 Muennighoff et al. [025 Marion et al. 2023) identify challenging examples based on binary correctness, reasoning length, perplexity, and group them into manually defined categories. However, such approaches not universal questions deemed difficult for one model may not be difficult for another, and rarely achieve optimal balance of difficulty and diversity.
To address these gaps, we propose Difficult-Diveersiy Collaborative Filtering (DDCF), a frame- work that reduces both annotating and fine-tuning costs by automatically selecting compact, high- quality subset of questions tailored to each target model. As illustrated in Figure DDCF measures question difficulty using collaborative filtering over predicted correctness patterns from multiple open-source LLMs, and quantifies diversity based on question similarity. By formulating data selec- tion combinatorial optimization that directly trades off these two criteria, we can efficiently ap- proximate the optimal subset using a simple k- k-greedy strategy. Starting from an empty set, k- g-geddy iteratively adds the question with the greatest marginal gain in our fiffiuulty-dversiiyy objective un- til exactly & examples are selected. Empirically, DDCF selects compact yet impactful subsets that effectively challenge the target model while maintaining broad coverage. enabling more efficient fine-tuning and improved performance in various downstream tasks.
Our key contributions are as follows:
We propose Difficulty-Divede Collaborative Filtering, novel framework that leverages multiple LMs capture unique characteristics of each target LM, enabling automatic con- struction compact, model-specific training subsets without requiring prior annotation.
To the best of our knowledge. this work is the first to systematically quantify and analyze the interplay between difficulty and diversity in data selection, and to demonstrate how their trade-off shapes downstream fine-tuning performance.
We empirically demonstrate that DDCF outperforms existing data selection baselines across multiple benchmarks, achieving higher accuracy with the same selection budget.
RELATED WORK
Numerous approaches have been proposed to curate high-quality training data, which can be grouped
into several categories. Influence-based methods estimate each example's impact on target set via gradient matching e.g. Grad-Match (Killamsetty et al. (002), LESS (Xia et al. 2024). NICE Wang et al. 2025) or by framing selection optimal control problem (Gu et al. 2005). Heuris- tie approaches frequently utilize token probability statistics as a proxy for difficulty, exemplified by strategies prioritizing medium-perplexig (Marion al. 2023] analyzing training dynamics via
2

---

Published as a conference paper at ICLR 2026
Table Comparison of DDCF with prior data selection methods. "Difficult-Awwad" and "Diversity-Aware" reflect whether these criteria are considered in selection. "No Full-Corpus Fine- Tuning" indicates whether the method avoids training on the full corpus. Siinimal-Annotttoon denotes methods that (almost) do not rely annotations, thereby reducing annotation costs. "No LLM Feedback" indicates the method does not depend on external reward models, e.g.. ChatGPT.
Difficulty- Aware
Diversity- Aware
No Full-Corpus Fine-Tuning
Minimal Annotation
LLM Feedback
Method
Perplexity Marion al.
2023
S2L
Yang et al.
2024b
x
AlpaGasus (Chen ct al.
2024
V
x
LESS
Xia ct al.
2024
V
V
DiSF
Fan et al.
2025
V
V
DDCF (ours)
V
Dataset Cartography (Swayamdipta al. 2020 Feedback-driven frameworks leverage closed-
source LLMs (such as ChatGPT) to score or prune candidates candidates-exemplifeee by AlpaGasus (Chen al. 2024) and Evol (Liu al. 2024). Diversity-aware sampling ensures broad representational coverage through embedding similarity (e.g.. D4 (Tirumala et al., (023). DiSF (Fan et al.) (025 ),
while lightweight proxy models cluster examples from loss trajectories, as in S2L (Yang et al. 2024b). Datamodels (Ilyas et al.) 2022 Chang & Jial (023) estimate how the presence of each training/few-shot example affects the target model, but its combinatorial formulation ignores exam- ple semantics and thus cannot generalize beyond the observed training questions.
Parallel data selection, recent work has explored the problem of choosing the most appropriate model for a given question, commonly referred to as LLM routing. FrugalGPT (Chen et al. 2023) adaptively queries models in sequence until a reliable answer is obtained. More recent methods embed models and questions into shared latent space and learn routing policies using matrix fac- torization (Ong et al. 2024: Zhuang et al. 2025). while Nguyen et al. (2024) frame the problem as multi- armeed bandit.
Our work bridges these two lines of research by recasting model-question interactions as a recom- mendation problem (Lee & Seung, 2000; He al. 2017). treating models users and questions as items. This perspective allows us to learn tailored relevance scores that guide data selection in a large corpus, even without full-annotation labels. Building on this, propose lightweight collab- orative filtering framework with difficulty-diveriity re-ranking to curate a small, high-quality subset from a large unannotated corpus, yielding strong performance in low-resource fine-tuning.
Table summarizes how DDCF compares to representative data selection approaches across five key dimensions. Notably, DDCF only relies on ground-trute answers from a small seeding dataset to construct the binary correctness matrix. This design enables DDCF to uniquely satisfy all five criteria: selects a compact, challenging, and diverse subset without the need for full-corpus fine- tuning, external annotations, or feedback from closed-source LLMs. As a result, DDCF offers a scalable and domain -agnosi- solution for efficient data curation across diverse model families. Given the convenience of DDCF and its potential for widespread use, the framework can readily extend beyond supervised fine-tuning to settings such as In-Context Learning (Chang & Jia 2023 Li & Qiu 2023 Pecher et al.) 2024| Purohit et al., 2024 2025). Active Learning (Zhang & Plank 2021). and even Curriculum Learning (Soviany et al. 2022).
3 DATA SELECTION WITH MINIMAL ANNOTATION
3.1 CORRECTNESS PREDICTOR
Given m language models M (MI..... Mm, and a seed dataset of n questions Q = {qu...., 9n) with corresponding ground-truth answers, construct binary correctness matrix A E {0, 10,1) Each entry Ai indicates whether model M, correctly answers question qj. This matrix captures fine-grained model-question interactions, enabling us to analyze both model capa-
3

---

Published as a conference paper at ICLR 2026
bilities and question difficulty. For instance, certain models may perform well algebra but poorly on geometry, while questions answered incorrectly by most models likely indicate higher difficulty.
Following the approach in Zhuang et al. (225)) , enrich the binary matrix A by learning low- dimensional embeddings for both models and questions. Specifically, we learn model embeddings Eas Rmxd and question embeddings Eq TRx such that
A A = Em Eq.
(I)
where d denotes the embedding dimension and Á approximates the observed correctness matrix A. This factorization is analogous to those used in collaborative filtering (Lee & Seung. 2000) He et al.) 2017). but is inherently limited to the training set and does not generalize unseen questions.
To enable generalization, we introduce a correctness predictor f M x @ + [0, 1]. which esti- mates whether given model correctly answers given question. We instantiate f using an encoder architecture, detailed below.
The encoder comprises two modules: model encoder Φas and a question encoder ΦQ; both projecting into shared latent space Idd.
Encoder
The model encoder ΦM .M + F maps model index to an initial representation.
The question encoder ΦQ Q + Fd is defined as function composition ΦQ ha 9Q. where:
9Q R dim, uses preitrained sentence transformer to encode question text into an initial question representation E
hiq Rdim, + Rd projects this representation into the shared latent space, yielding factor- ized question embeddings Eg for each E Q.
In our implementation, ha is a multilayer perceptron.
Classifier Head The classifter predicts correctness from the Hadamard product of embeddings:
v (Ems Fay):
where d + 2² is a linear classifier. The overall predictor is thus defined as f(Mi, q;) ((MM(M) o de (a;)), which can be trained using binary cross-entropy loss.
The predicted correctness score for model M; on question fj defined as:
A. o(f(MM
where (-) is the sigmoid function, and the subscript 1 denotes the logit for the "correct" class.
Notably, Equation 2 can be viewed as parameterized version of the classical matrix factorization in Equation]] T Instead of estimating single shared difficulty score per question, this formulation allows the difficulty of question to be "personalized" for each model's characteristic. This per- sonalized modeling of correctness underpins our approach in the next section, where we construct Diffealty-Diversityy Collaborative Filtering strategies tailored to individual models.
3.2 DIFICULTY-DIVERSIT COLLABORATIVE FILTERING
Given a target model M, and a large unannotated dataset DD where [D] > introduced seed dataset, our goal is to select a subset S; C D of k questions that are both (1) difficult for the model M; and (2) diverse to cover wide range of topics. This ensures that the selected examples provide strong learning signals while avoiding redundancy.
Q and is the
To estimate question difficulty, we leverage the correctness predictor f introduced earlier. For every question Øj DD the model M,'s predicted correctness score is given by A,, af 9;)1), and we aggregate these into a vector A, E R (P) Lower values of A. correspond to questions the model is more likely to get wrong, thus indicating higher difficulty.
4

---

Published as a conference paper at ICLR 2026
Initial Question Representations E?
Question Embeddingg: E
30
20
Correciess Score Ag
10
10
S- Dm
0.6
0
0.4
-10
-20
-30
-40
Full Corpus
Selerted Data
-30
Full Corpus
Selected
SE Dim
Figure 2: I-SNE visualization of questions selected by DDCF (best viewed in color). Left: the selected data in the semantic space encoded by Sentence-Transformee: Right: the same data in the factorized space learned by the correctness predictor, with point color indicating difficulty (darker means harder) and selected examples highlighted in red. Our proposed DDCF framework organizes the subset to target challenging questions while preserving diversity across semantic regions.
To encourage diversity among selected questions, we define a similarity matrix K E R ||||||P), where each entry the cosine similarity of the sentence-rransfommer embeddings of questions qu and qj:
(E (E )T qu E ll2
Enj
(3)
l|
We then formulate the selection problem as a combinatorial optimization task:
A(x A(xTA.) + (1 A) Exr), s.t. Σ xE(0,1)
min
rj k.
(4)
j=1
Here, the constraint enforces the selection of exactly & questions, forming S fgglzz 1),
ie.. the curated subset for the model M,, and À E [0, 1] balances the trade-off between difficulty (x: A,))) and diversity (x Σr). Although the objective is convex over the continuous relaxation of r, the binary constraint renders the problem NP-hard and computationally intensive due to the (
memory complexity of the similarity term E.
To overcome these limitations, propose memory- and compute-efficient k-greedy heuristic that incrementally selects questions. Starting with an empty set S,, at each step, we add the next question 9, E ( St that minimizes composite score:
q = arg min 4, ED)
XAu + (1 - A) max Eas
(5)
ES;
This approach not only relaxes the original hard problem but also significantly improves memory efficiency by computing only O(k (DI) pairwise similarities on the fly. As a result, the k-greedy strategy both fast and scalable, enabling efficient selection over large unannotated corpora while maintaining strong trade-off between difficulty and diversity.
Figure provides qualitative validation of our k-greedy sampler's dual objectives. the initial representation space encoded by Centence-Transformie (left), the chosen subset spans multiple se- mantic regions of the full corpus, confirming that the on-the-fly diversity term successfully prevents redundant sampling. After projecting into the factorized embedding space (right), smooth gradient of question difficulty emerges, and the selection concentrates almost in the most challenging ques- tions. Together, these two views demonstrate that our Difficulty -Diversity Collaborative Filtering simultaneously maintains semantic diversity and precisely targets high-difficulty examples.
Therefore, the selected subset S provides highly informative slice of the large corpus for down- stream use. In the case unannotated corpora, DDCF enables cost-effective data preparation by concentrating annotation efforts, either from human experts strong teacher models, on only the
5

---

Published as a conference paper at ICLR 2026
most impactful I examples. Here, DDCF serves front-end filter that reduces supervision costs while preserving strong learning signals. For already annotated corpora, DDCF serves as a post-hoc filter that climinates trivial or redundant examples and tailors the learning path to the strengths and weaknesses of the target model, thereby shortening the training time. In both scenarios, the resulting compact, model-aware dataset S; undergo further quality checks-such lected questions and annotations- especially in high-stakes domains like medicine or law. Overall, DDCF facilitates data-efficient tuning paradigm where LLMs can be rapidly adapted with minimal supervision, even when full-corpus annotation is impractical or prohibitively expensive.
expert review of se-
4 EXPERIMENTS WITH PRE-ANNOTATED CORPUS
4.1 EXPERIMENTAL SETUP
Training the Correctness Predictor To learn factorized model and question embeddings, we train a correctness predictor using outputs from 23 open-source LMs spanning a wide range of sizes and architectures. Each model was evaluated on the seed dataset of 19,470 questions from the training splits of GSM8K (Cobbe et al. 202I) and MATH (Hendrycks et al. 2021b). Responses were labeled correct incorrect by rule-bar rule-based verifier, resulting in binary supervision for each model-question pair. We use 10% of the questions as a held-out test set. For the initial question embeddings, we employ the sentence transformer Qwen lists all 23 LMs and reports the computational cost of DDCF relative to alternative data-sclection baselines. Appendix C provides ablation studies on the number of participating models and the size of the seed dataset within the DDCF framework.
3mmbedd inmmbedingg-555 n Appendix BB
Data Selection We conduct experiments on the OpenR1-Mathe 220K dataset (licensed un- der Apache 2.0). which contains 225,129 math problems annotated by DeepSeek-R1-8 -1BB DeepSeek-AA al. 2025). We select 1,000 training instances from this corpus using different selection strategies. Based on ablation results, we set the difficulty-diversitty trade-off in Equation to 0.2 by default.
Baselines We compare our approach, DDCF, with various baselines:
Dummy baseline: (1) Random randomly samples 1,000 examples; (2) Longest selects the 1,000 longest instruction examples; (3) Binary Hard randomly samples 1,000 examples that the targeted model incorrectly answers from the seed dataset;
(4) Least Confidence (Settles) 2009 measures the model's confi- dence as the product of probabilities of the data example. (5) Perplexity (Marion al.
Annotation dependent:
2023 selects examples around medium perplexity: (6) Cartography (Swayamdipta et al.
2020 selects easy and ambiguous examples; (7) SmallToLarge (S2L. (Yang al. (024bb
samples from clusters summarizing the loss trajectory of casy-to-hard questions.
Annotation independent: (8) DiSF (Fan al., 2025) chooses samples that maximize the diversity of the question embedding space via covariance eigenvalue maximization.
Manually selected dataset: (9) LIMO, 817 instructive examples from Ye et al. (2025): (10)
s1.1-1K, 1,000 high-quality examples curated by Muennighoff et al. (2025)
It is worth noting that, unlike Random, DISF, and our method DDCF-which can be applied prior to annotation-the remaining baselines require full-corpus annotation to compute selection criteria such as gradients, reasoning length, or perplexity. We do not compare our method with selectors like AlpaGasus (Chen et al., 2024) or LESS (Xia et al. (024), as they assume different settings, such as reliance on ChatGPT feedback or access target dataset for gradient matching.
Evaluation We evaluate on 10 popular reasoning benchmarks, grouped into three categories:
In-Distribution: MATH500 (Hendrycks et al. 2021b), OlynpiadBench (He et al. 2024). GSM8K (Cobbe et al., 2021), AIGEval-SAT-Minn (Zhong et al., 2024), and AIME24.
Out-of-Distribution: Minerva (Lekkowyzz al.) 2022), which includes undergraduate- level STEM problems; Gaokao, sourced from China's 's 2024 National College Entrance Exam; and the STEM subset of MML.U (Hendrycks al. 2021a).

---

Published as a conference paper at ICLR 2026
Table 2: Performance on In- In-Distribution and Out-of-Distribution benchmarks.
In-Distribution
Out-of-Distriuttion
Method
AIME24 MATH
OlyBen
GSM8K
SAT
Avg.
Miverva
Gaokao
STEM
Avg.
Qwen2.5-Math-
Full Dataset
64.5
80.6
42.5
92.6
98.2
75.8
46.3
72.2
79.7
66.1
Model
34.6
16.4
67.1
67.7
Random
38.6
76.4
34.8
91.0
98.2
67.8
41.2
64.6
75.7
60.5
Longest
19.7
53.8
18.4
81.1
69.1
25.7
36.7
50.7
37.7
Binary Hard
29.6
67.2
28.3
89.3
85.5
60.0
31.6
53.2
67.9
50.9
Least Confid.
12.3
42.8
11.7
61.6
58.6
37.4
21.0
20.3
47.0
29.4
Cartography
74.6
34.7
41.7
Perplexity
44.7
77.8
37.8
89.3
96.8
69.3
46.7
69.6
79.1
65.1
S2L
36.7
74.4
34.8
90.1
98.2
66.9
39.3
58.2
75.4
57.7
DISF
44.6
76.2
35.4
96.8
68.6
43.4
68.4
75.6
62.4
LIMO
41.1
76.4
35.7
89.5
94.6
67.4
35.3
58.2
74.5
56.0
s1.1-K
41.9
76.6
37.4
40.1
DDCF
49.0
77.6
35.0
91.2
98.2
70.2
45.6
74.7
75.8
65.4
Qwen3-8B-Bast
Full Dataset
88.6
91.8
60.3
95.0
99.1
87.0
64.3
84.8
92.1
80.4
Base Model
47.8
60.8
36.3
89.8
98.2
66.6
40.8
58.2
84.4
61.1
Random
80.9
89.2
53.8
94.4
99.6
83.6
62.5
83.5
90.8
79.0
Longest
81.4
90.4
54.7
94.4
99.6
84.1
64.3
84.8
90.8
80.0
Binary Hard
75.0
Least Confid.
71.6
89.8
52.6
94.8
99.6
81.7
62.5
81.0
90.6
78.0
Cartography
78.7
91.6
56.9
96.7
99.6
84.7
63.8
79.9
91.2
78.3
Perplexity
79.3
89.8
55.0
94.5
83.6
60.3
83.5
91.0
78.3
S2L
76.4
91.0
55.0
94.5
99.1
83.2
62.1
78.5
91.3
77.3
DISI
94.6
99.6
91.1
LIMO
79.8
89.4
55.3
93.7
98.6
83.4
54.8
82.3
88.7
75.2
s1.1-IK
75.5
86.2
51,9
92.0
98.2
80.7
57.7
77.2
89.3
74.7
DDCF
82.2
91.0
Development Set:
We use SVAMP (Patel et al. 2021) (elementary), and AMC23 (com- petition level) to determine hyperparameter À balancing the difficulty-diversity trade-off.
We report pass @ accuracy by default, while for AMC23 and AIME24 we report @ 32, due to their small size and high difficulty. Experiment details can be found in Appendix
4.2 MAIN RESULTS
In-Distribution Results Table 2 demonstrates that DDCF consistently produces the strongest KKeexapll subsets among all selection strategies. For 5-ath.5- DDCF attains an av- erage score of 70.2, outperforms the best baseline (Perplexity, 69.3) while staying within only -5.6 points full-data training. Notably, DDCF yields larger gains on the hardest benchmarks: boosts AIME24 performance to 49.0. +10.4 over random and +14.4 over the base model. For Qwen3-8B- Base, DDCF achieves 85.0 on average, outperforming all baselines and reducing the gap full-data training to just -2.0. Its improvement is most evident on GSM8K, where DDCF reaches 95.9, sur- passing all baselines by up to +3.9. These results indicate that DDCF maintains both breadth and depth in coverage, enabling efficient fine-tuning with limited data.
Out-of-Distribution Results Under distribution shifts, DDCF also demonstrates strong general- ization. For the 7B model, it records 65.4 average-closse the gap to the full dataset down to 0.7 and surpassing every other subset strategy by margins ranging from +0.3 to +8.0. On Gaokao, DDCF not only outperforms all baselines but also exceeds the full-data performance by +2.5 (74.7

---

Published as a conference paper at ICLR 2026
between Difficulty Diversity
80
70
70.2
69.8
5
6
71.
61
628
737
73.2
628
22.5
645
74,9
66
90-
94
92
5
91.4
90
90.8
29.3
Performacc
60
55.
53.9
569
582
58
Porformange
88
68.6 88.6
50
48
AMC23 (No SFT!
45
42
86-
SVAMP
SVAMP (No SFTI
83.6 83,9
85.2
3466
34.7
35.4
26,8
83.3
318
322
(D)
1.58 (I))
82
81.1
81.1
80.8
80
50.5
7B (OOD)
1.50 (000)
80-
120K
0.0
0.1
0.2
033
0.4
0.3
0.6
0.7
0.8
0.9
Selected Sam
Difficutty Weight
Performance of the fine-tuned models with different data sizes.
Figure 3:
Figure 4: Ifffiuulty-ivversie trade-off in data selection.
vs 72.2), suggesting that efficient fine- fine-tuning might preserve generalization in multi-lingual settings. In the meanwhile, Perplexity also achieves notable results on Minerva (46.7), and STEM (79.1), For the 8B model, DDCF achieves the highest OOD average (80.5), slightly ahead of full-data fine- tuning (80.4). This edge comes primarily from Minerva, where DDCF improves by +1.9. Together, these findings highlight that compact, model-awane subsets selected by DDCF can preserve even enhance -i-oo-ddistriutionn robustness relative to training on the full corpus.
Owing to space limitations, report the results of Falcon -0B Base in Appendix D
4.3 DATA SIZE ABLATION
To assess the effect of training set size, we vary the number of selected questions l: from 0 to 225, 12 and evaluate both in-distribution (ID) and out-of-distribution (OOD) performance averaged across the benchmarks introduced earlier (Figure 3}. We compare two settings: strong base model Owen2. 5 5-Math-77 7B and weaker base model Owen2. 5-Math-1.55.
On one hand, for Qwen2. 55-Math-755 ID accuracy improves almost monotonically, rising from 55.6 at k = 0 to 75.8 at full scale, with the sharpest gain achieved within the first 1,000 samples (70.2). OOD performance, however, exhibits a mo-moontoonic trend: peaks early at 65.4 for h: 1,000, declines to around 61 at k E [4,000, 8,000J, and then recovers steadily to 66.1 at full scale. This mid-range dip suggests that while small curated sets provide strong generalization, enlarging the pool without sufficient coverage may initially dilute transferability before larger sets restore robustness. Notably, selecting only 1,000 samples already secures over 70% of the ID improvement and nearly the full OOD benefit, highlighting the data efficiency of our DDCF framework.
On the other hand, fine-tuning on small yet highly complex datasets can degrade the performance of weaker language models- phenomenon referred to the Small Model Learnability Gap (Li jet al.] 2025) or Long CoT Degradation (Luo et al., (025). Figure 3 illustrates this effect for Qwen2. 5...5-MMthh-- 5B fine-tuned DDCF subsets size k. With only k 1,000 examples, ID accuracy drops sharply from 54.3 to 45.4 (-8.9) and OOD falls from 34.6 to 31.8 (-2.8), illustrating the known phenomenon. Increasing ie 4,000 largely mitigates this effect- ID only 0.4 below its pre-finc-tuning level while at k = 8,000 both curves recover fully and begin to climb.
8,000, performance increases steadily: by k 16,000 we attain 56.9 ID and 35.4 OOD, and by k = 128,000 the model culminates at 59.2 ID and 46.0 OOD. Notably, this represents 1.0 ID / 2.2 OOD improvements over conventional full-corpus fine-tuning on all 225, 129 available samples, demonstrating that our DDCF strategy can overcome initial degradation and ultimately yield superior results with far fewer examples.
Beyond
4.4 DIFFICULTY- DIVVERSITY TRADE-OFF
To determine the optimal difficulty weight À in Equation 5 we perform an ablation study on elementary-level SVAMP and competition-level AMC23 with selection size k 1000 using
8

---

Published as a conference paper at ICLR 2026
Qwen2. 5-Matt--Mahhhh shown in Figure 4 As À increases from 0 (pure diversity) to 0.2, AMC23 performance jumps from 81.1 to 83.3% while SVAMP remains at its pre- trained baseline of 91%. Further increasing À continues to boost AMC23, peaking at 85.2 for X 1.0, but with diminish- ing returns, SVAMP performance declines by about 2 points at À = 0.5 and points at = 1.0, indicating that excessive emphasis on difficult examples undermines proficiency on simpler tasks.
Since our ultimate goal is to elicit the model's s full problem-solving ability from a small, curated fine- tuning set without eroding its pre-trained knowledge, we adopt À = 0.2 as the default parameter for Collaborative Filtering framework, striking a balanced trade-off between difficulty and diversity. Beyond this default, DDCF enables the difficulty weight À to be adjusted on the fly, allowing users to instantly tailor data selection to their priorities without additional retraining or redesigning the framework. This adaptability makes the framework both convenient and versatile, supporting wide spectrum of selection strategies within a unified formulation. Moreover, the ablation on how the quality of the large corpus affects data selection is discussed in Appendix E
our Difficulty-Diverse
4.5 DOES DDCF FRAMEWORK LEARN MODEL CHARACTERISTICS?
Let Sa and S, be subsets selected by models Ma and MM. We quantify their overlap via the Jaccard index: J(S., St) = SS. models relative to the total unique questions. A higher J indicates greater similarity in the subsets, reflecting closer alignment in model behavior. We hypothesize that models within the same family, sharing architecture and pre-training data, will exhibit higher Jaccard similarity than those from different families. Indeed, our analysis shows an average intra-family index of 0.224 versus 0. 169 inter-family, demonstrating that our framework captures meaningful model-specific characteristics.
[S,US.] which measures the fraction of questions chosen by both
Figure 5 shows the topic-wise composition of each model's 8 selected subset alongside the full dataset distribution. Although the full corpus is domi- nated by Algebra (48.1%% 16), our framework tailors sampling to each model's behavior. In partic- ular, Owen2. 5-Math-7B and Qwen2.5-323 exhibit almost identical distributions: Combina- torics holds the largest share, while Algebra, Ge- ometry, Number Theory, and Logic & Puzzles each retain substantial and balanced proportions. By contrast, L1 ama- - 3 1a-88 8B diverges markedly, de-emphasizing Algebra and boosting Combina- torics and Logic & Puzzles. This divergence shows that DDCF tailors question selection to each model's 's specific strengths and weaknesses, target- ing areas for improvement rather than sampling uniformly. The experiment on data transferability across models is in Appendix F
0wen2.5-Math-7
Qwen2.5-325
1.95
18.4
1255
2.87
15
10
14
28
2377
1.ama-3.1-8B
Full Dataset
1,4
27.5
4
8.5
11
153
Algebra Combinatorics
Inequalitiss Logic Puzzles MAD Other
BND Calculus
Geometry m00 Number Theary
Figure 5: Topic distribution of DDCF datasets.
EXPERIMENTS WITH UNANNOTATED CORPUS AT SELECTION TIME
To illustrate capabilities of DDCF on large corpora that are initially unannotated, we evaluate on the MMLU benchmark (Hendrycks et al. 2021a), which includes 99,842 training, 1,531 validation, and 14,042 test questions. While the corpus spans many disciplines. it lacks reasoning annota- tions. We therefore train a correctness predictor on the validation split and use it with DDCF to select 1,000 high-quality training examples, which are then distilled into reasoning traces using Swvnn/wwe33332 in long-thin thinkknnn mode. By filtering before annotation, DDCF reduces distilla- tion cost nearly 100x, whereas prior methods require annotating the full corpus in advance.
Table 3 compares DDCF with annotation-free baselines (Random, DISF) across three MMLU do- mains. On Qwen2.5-7B, DDCF achieves the best accuracy in all domains, improving the average by +6.2 over the base model and +2.5 over the strongest baseline, with the largest gain in STEM (81.1; +9.6 over base, +6.4 over DiSF). For Qwen3-88 8B-Base. DDCF again excels, setting new highs in Social Science (87.3) and STEM (89.7), and raising the average by +5.2 over base with only a slight
9

---

Published as a conference paper at ICLR 2026
Table 3: Performance on the MMLU benchmark.
Qwen2.5-7B
Qwen3-SB-Bel -BB-Bes
Aaloo--0B-Besss
Method
Humanities
Social Science
STEM
Humanities
Social Science
STEM
Humanities
Social Science
STEM
Base Model
39.0
77.1
71.5
62.8
84.4
66.1
80.9
DiSP
74.7
DDCF
63.5
81.4
81.1
68.2
87.3
89.7
69.9
86.0
87.2
drop Humanities. On Falcon-10B- 10B-Base, DDCF outperforms all baselines, boosting Humanities, Social Science, and STEM by +3.8, +5.1, and +5.8, respectively.
Overall, these results show that DDCF strengthens not only quantitative reasoning (STEM) but also inferential reasoning (Humanities, Social Science), even with scarce annotations. Beyond in-domain performance, OOD validation (Appendix G) reveals that fine-tuning on just 1,000 distilled MMLJU examples transfers effectively to general tasks such commonsense, reading comprehension, and instruction following, where DDCF achieves the best average across all backbones.
CONCLUSION
Difficulty-Diver Collaborative Filtering a novel concept for curating small, high-quality fine- tuning subsets from large unannotated corpora by balancing question difficulty, via a learned correct- ness predictor, and semantic diversity in embedding space. Empirically, DDCF reduces annotation costs by 100 200 while maintaining performance comparable to the full-data baseline, and our analysis shows it naturally tailors data selection to each model's 's unique strengths and weaknesses.
ACKNOWLEDGMENTS
This research/project is supported by the National Research Foundation, Singapore under its Na- tional Large Language Models Funding Initiative (AISG Award No: AISG-NML.P-20240005) and AI Singapore Programme (AISG Award No: AISG3--PDD-2024-08-000
REFERENCES
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Con- nor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA learns less and forgets less. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: /tttps://openreviee.. net/ www.:neetffoooeeee forummmrpp 1id=
aloEru2qCG Featured Certification.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O' O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In Proceedings of International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Ting- Yun Chang and Robin Jia. Data curation alone can stabilize in-context learning. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 6lst Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8123-8144 Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.18633/1/2023. acl-long.452. URL https: httpa/aannnntoooooooooooo wwt.//gg///22 http://222.aacllooooooo. acl- 452/
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna. Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training a better alpaca with fewer data. In Proceedings of The Twelfth International Conference Learning Representations, 2024. /ttttts:://oppeeetiiit http:/eet.orrtteeeee//eeeeeoooo ftte://ooommmmdddddeeeeeeeed
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.0m 05 176, 2023.
10

---

Published as a conference paper at ICLR 2026
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling entruction-finetunee lan- guage models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano. Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 2021.
(0d::210.110 arXiv:2110.1441
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjee Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song. Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi. Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, 7. 7. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang. Zhewen Hao, Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li. Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang. Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning. 2025. URL htt.ltttts//aii..cooo http://wwwxxwwwxaaa///os/ssss hbtp://b///50000 12948
Elvis Dohmatob, Mohammad Pezeshki, and Reyhane Askari-Hemmat. Why less is more (some- times): A theory of data curation, 2025. URL htttpt://axxii...tt..t otg/ abs/ /0os//251111m 03492
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled al- pacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04447. 2024.
Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, and Yan- feng Wang. Combatting dimensional collapse in LLM pre-training data via submodular file se- lection. In Proceedings The Thirfeenth International Conference on Learning Representations, 2025. URL / ew. nwt http:///ooouumocodaaaaaoof
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Ko- renev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes. Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco
11

---

Published as a conference paper at ICLR 2026
Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, lliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Eytimov, Jack Zhang. Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- soo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani. Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathicu Rita, Maya Pavlova, Melanic Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick AIrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer. Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabusappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang. Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do. Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic. Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le. Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel. Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harri- son Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, lean-Baptiste Gaya, Jeff Mareus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kylc Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L.
12

---

Published as a conference paper at ICLR 2026
Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng. Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager. Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Ro- driguez, Raf Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay. Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar. Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe. Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Ku- mar, Vishal Mangla, Vlad Ionescu. Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang. Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiao- jian Wu, Xiaolan Wang. Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang. Yossi Adi, Youngjin Nam, Yu, Wang. Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhao- duo Wen, Zhenyu Yang. Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024a. URL org/ abs/ 241722111111111 21783
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.217888 2024b.
Yuxian Gu, Li Dong, Hongning Wang. Yaru Hao, Qingxiu Dong, Furu Wei, and Minlie Huang- Data selection via optimal control for language models. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL /ttp:////pppeeiii.e/oe net/ forum?id chAL5fy8wS
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scien- tific problems. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volune 1: Long Pa- pers), pp. 3828-3850, Bangkok, Thailand, August 2024. Association for Computational Linguss tics. doi: 10. (0.18653/v1/20244 a20n4 sal-iong.211.
URL https::/aaat http:s/////alaahhloogggo org/2024.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural col- laborative filtering. In Proceedings of the Twenty-Sixth International Conference on World Wide Web, pp. 173-182, 2017.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. In Proceedings of The Ninth International Conference on Learning Representations, 2021a.
jm 3GmQ
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Advances in Thirty-Fifth Conference on Neural Information Processing Systems, 2021b.
Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Data- models: Understanding predictions with data and data with predictions. In Kamalika Chaud- huri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Pro- ceedings of the 39th International Conference on Machine Learning, volume 162 of Proceed-
13

---

Published as a conference paper at ICLR 2026
ings of Machine Learning Research, pp. 9525 9587. PMLR, 17-23 Jul 2022. URL https:
7 /pppceeedngg...t
html
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https:
V/axxiv.org/abs/2330
Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training- In Proceedings of The Thirty-eighth International Conference on Machine Learning, pp. 5464-
5474. PMLR, 2021.
Jang- Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung- Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data parameterization. In Proceedings of the Tenth International Conference Machine Learning, pp. 11102-11118, PMLR, 2022.
Daniel Lee and H Sebastian Seung. Algorithms for non-negative matrix factorization. Advances in The Thirleenth Annual Conference on Neural Information Processing Systems, 13, 2000.
Aitor Lewkowyez, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil. Imanol Schlag, Theo Gutman- Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative rea- soning problems with language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL.
net/ Eorum 2id IFXTZERXcM
Xiaonan Li and Xipeng Qiu. Finding support examples for in-context learning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 6219-6235, Singapore, December 2023. Association for Computational Lin- (tt.5//////222..nnnnnnne-e// 2023. 1.33ffindin findings wwwndings-emnlppe em. 4117
guisties. doi:
URL http//tts:///cclnnhhhoo org/
Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ra- masubramanian, and Radha Poovendran. Small models struggle to learn from strong reasoners, 2025. URL httpttttt:/aaatt
orq/ abs wt.b///502..m 12143
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A chal- lenge dataset for machine reading comprehension with logical reasoning. In Christian Bessiere (cd.), Proceedings of the Twenty-Ninth International Joint Conference Artificial Intelligence, IJCAI-20, pp. 3622-3628. International Joint Conferences on Artificial Intelligence Organiza- tion, 7 2020. doi: 10. 00244963/jcai.222005533 URL 2020/500 Main track.
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In Pro- ceedings of The Twelfth International Conference on Learning Representations, 2024. URL net Seeet/oorumlld-BTKeee
/ettps://pperreeee
Renjie Luo, Jiaxi Li, Chen Huang, and Wei Lu. Through the valley: Path to effective long cot training for small language models, 2025. (RL/sttss://axx.
hatgg//b//2500 07712
Max Marion, Ahmet Üstünn, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining LLMs at scale. In NewrIPS Workshop on Attributing Model Behavior at Scale, 2023. URL
net/forum
ad=XUIYn31057)
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In Ellen Riloff David Chiang.
14

---

Published as a conference paper at ICLR 2026
Julia Hockenmaier, and Jun ichi Tsujii (eds.), Proceedings of the 2018 Conference Empir- ical Methods in Natural Language Processing, pp. 2381-2391, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: 10.18653//1/DI8-111 12600 URL https:// antho 1 1 1260/
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zeullemoyer Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, arXiv preprint 2XXv:2501111999 2025.
Quang H Nguyen, Thinh Dao, Duy C Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V Chawla, and Khoa D Doan. Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms. arXiv preprint ar ArXiv:2407. 10034 2024.
Isaac Ong. Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning route Ilms from preference data. In Proceedings of The Thirteenth International Conference on Learning Representations, 2024.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North Americun Chupter of the Association for Computational Linguistics: Human Language Technologies. pp. 2080- 2094, Online, June 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2021. naacl-main.166
http:///aaaaaaaaaoooooorr/////// maw.::mma.aaammm.....mmm 168
Branislav Pecher, Ivan Srba, Maria Bielikova, and Joaquin Vanschoren. Automatic combination of sample selection strategies for few-shot learning, 2024. URI. htt.tttss///axii.rrreeeee
2402. 03038
Kiran Purohit, Venktesh V, Raghuram Devalla, Krishna Mohan Yerragorla, Sourangshu Bhat- tacharya, and Avishek Anand. EXPLORA: Efficient exemplar subset selection for complex rea- soning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 5367-5388. Mi- ami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/ 1/2024.emnlp-miin.33
URL https://aclatthologg/
emn lammmii.m.....
3077]
Kiran Purohit, Venktesh V, Sourangshu Bhattacharya, and Avishek Anand. Sample efficient demon- stration selection for in-context learning. In Forty-second International Conference on Machine Learning. 2025. URL
Burr Settles. Active learning literature survey. 2009.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024. /tattps://axxi..rrrt wwaaxiv.org//aai abs/ 2402.
03300
Petru Soviany, Radu Tudor lonescu, Paolo Rota, and Nicu Sehe. Curriculum learning: A survey. International Journal of Computer Vision, 13066):526-1555, 2022.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie. Yizhong Wang. Hannanch Hajishirzi, Noah A. Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing Online, November 2020. Association for Computational Linguis- URL https:: puattps://aclantee http:///aaaaatttogo...o org/2020.
(EMNLP), pp.
tics. doi: 10.18663/1/00000mmmmppp
emn 1p- ma...iiin...mm 7
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Fer- ret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Char- line Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastier Grill, Behnam Neyshabur,
15

---

Published as a conference paper at ICLR 2026
Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchi- son, Alvin Abdagic, Amanda Carl. Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Wein- berger, Dimple Vijaykumar, Dominika Rogozińskka, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimczak-Plucifske Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson. Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kar- tikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mehran Kazemi, Michael Moyni- han, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culli- ton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Mc Carthy, Sarah Cogan, Sarah Perrin, Sébastien M. R. Amold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ron- strom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dra- gan, Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Fara- bet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models at practical abt.ogg/bb///488/ 00118
Harleen
size, 2024. URL
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. D4: Improving LLM pretraining via document de-duplication and diversification. Proceedings of Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
URL htttps://ppeereiieeee wwtperewwwwwwweeeeeeeeeeeeeeeeee http:///ggaaeeeeeeeeeeeeeeeeaaan (ow.ummiiddCoLL2PPE..
Jingtan Wang, Xiaoqiang Lin, Rui Qiao, Pang Wei Koh Chuan-Sheng Foo, and Bryan Kian Hsiang Low. NICE data selection for instruction tunig in LLMS with non-differentiable evaluation metric. In Proceedings of The Forty- International Conference on Machine Learning, 2025. URL
wttpeeereeeee.nee/ooruauaaaaa wwwp/eeeeeetteeeeee 2wt 8m5HUBs
Liyun Wang, Xingxing Zhang, Hang Su, and Jun Zhu. comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe Le. Finetuned language models are zero-shot learners. In Proceedings of the Tenth International Conference on Learning Representations, 2022. URL
htttps://openreviie http://aarrreeeeeeeeeeeeeooerrar
net /tt.://mmmdgg/eeoooooooo
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In Proceedings of The Forty-First Inter- national Conference on Machine Learning, 2024.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin. Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical ex- pert model via self-improvemmnn, 2024a.
www. abs 2409. 12122
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu,
16

---

Published as a conference paper at ICLR 2026
Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang. Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mci Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025a. URL https: / /aa.ivv.v.../////aa//aaa//aaaav. wwg/abgggaaooooooooo
abs / 2505 09388
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang. Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025b. URL https://araa wwt.ariii...ggaaa.... 2412. 15115
Yu Yang. Siddhartha Mishra, Jeffrey N Chiang, and Baharan Mirzasoleiman. Smalltolarge (s2l): Scalable data selection for fine-tuning large language models by summarizing training trajectories small models. In Advances in The Thirty- Annual Conference on Neural Information Pro- cessing Systems, 2024b. WRLLhtts://// /tLLLtttpp///ppeeeeeee...wtt net/ forum d= www.0=199911100111
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chem, Shijie Xia, and Pengfei Liu. LIMO: Less is more for reasoning. In Second Conference on Language Modeling, 2025. URL https:// openreview
Mike Zhang and Barbara Plank. Cartography active learning. In Maric-Francine Moens, Xuan- jing Huang. Lucia Specia, and Scott Wen-tau Yih (eds.). Findings of the Association for Com- putational Linguistics: EMNLP 2021, pp. 395-406, Punta Cana, Dominican Republic, Novem- ber 2021. Association for Computational Linguistics. doi: 10.18653/v1/2011. http3////222...nnnngeemmm...ppp URL htt.ttsss/acclaattoooo...oo/ ftw.g///i/.......no......... emnlp. 36/
Xin Zhang, Jiawei Du, Ping Liu, and Joey Tianyi Zhou. Breaking class barriers: Efficient dataset dis- tillation via inter-class feature compensator. In Proceedings of The Thirteenth International Con- ference on Learning Representations, 2025a. URL https: pettps://openee /wwp//eeeeeeeeeeeeeeeeeeeeeeeeee nww fformm? id XOCxfByJog
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang. Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou Qwen3 embed- ding: Advancing text embedding and reranking through foundation models, 2025b. URL
abs 25066 2506.05176666666
Liammin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica. Joseph E. Gonzalez. Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Proceedings of The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL htttp: //openeview. bopenreeiiew.net/froreee www.eeeeorrdddiiiieieee
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. AGIEvval A human-centric benchmark for evaluating foundation models. In Kevin Duh, Helena Gomez, and Steven Bethard (cds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2299-2314 14, Mexico City, Mexico, June 2024. naacl. URL
Association for Computational Linguistics. doi: 10. 10.18663/122224.fmmiim bittps://aclon www.annhhhooy..oooooooo org/ http://222........ognnnaaaaaaa 1499
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 1445500065502111 2023.
Richard Zhuang, Tianhao Wu, Zhaojin Wen, Andrew Li, Jiantao Jiao, and Kannan Ramehandran.
EmbedLLM: Learning compact representations of large language models. In Proceedings of The Thirteenth International Conference on Learning Representations, 2025. URL https:
/oopenevveww net www.eeeeeeetotttdteeeeeeee. F
17

---

Published as a conference paper at ICLR 2026
Table 4: Epochs and batch sizes used for supervised fine-tuning across dataset sizes.
Dataset size k
Hyperaarameter
1K
2K
4K
8K
16K
32K
64K
128K
220K
Epochs
4
4
4
4
3
3
3
Batch size
32
32
32
32
32
64
128
128
128
A EXPERIMENT DETAILS
A.1
CORRECTNESS PREDICTORS
MLP Block The question encoder li uses residual multilayer perceptron (MLP) block for re- finement. Given an input EU E IR, the block is defined as
MLPBlock(F7 (F = F + u,
(6)
where
W2 (Dropout (W, LN(E"))))
Here, LN denotes LayerNorm, W1 Rdxdd and W2 gd'xd are linear layers with hidden dimen- sion d' 0.1 d, and 7 is ReLU activation. Dropout with rate 0.8 is applied during training. To stabilize optimization, the final projection W2 is zero-initialized, making the block behave as the identity map at initialization.
Noise Regularization To reduce overfitting to the limited set of models and questions, we inject Gaussian noise into both model and question embeddings during training. For model embedding Em; and question embedding E,* the perturbed representations are
E
E. m. + Epp
E Eg Egg
where
Cpt Cp CV Ia).
and is scalar hyperparameter controlling the perturbation scale. This stochastic perturbation acts as embedding data augmentation, preventing the predictor from memorizing spurious correla- tions in the binary correctness matrix. During inference, noise is disabled and the embeddings are used.
Training Hyperparameters We train the correctness predictor with the Adam optimizer (weight decay x 10-5 and cosine learning rate schedule with warmup ratio of 0.03. The initial learning rate is set to 1 10-3, and training runs for 30 epochs. For the OpenRI-Mathe 220K dataset, we use batch size of 1028 and set the regularization parameter
1 x 10-2 For the MMLU experiment, we use a smaller batch size 64 and increase the regularization strength to a = x 10
A.2 SUPERVISED FINE-TUNING
We fine-tune LLMs using the TRI library with a maximum sequence length of 16,384 tokens. Training is performed in balootte 16 precision with the Adam optimizer, cosine learning rate schedule, and warmup ratio of 0.03. Table summarizes the epoch and batch size configurations across different datasizes. Experiments in this paper can be done with 2 H100 gpus.
A.3 DATA SELECTION PROCEDURES
Baseline Details. Most baselines in our SFT experiments are described in Section 4.1 but we highlight additional implementation details here. For the Binary Hard baseline, we randomly sample 1,000 questions that the target model answers incorrectly from the seed datasets (GSM8K
https::///t htt.:/giithhhcooo.om htte::///itttb..cc//ogggggooof/e
18

---

Published as a conference paper at ICLR 2026
Table 5: Models used in the DDCF framework.
deepseek aidbbeee-aiideeeeeeee--mmma -mahh -b-bab-
Qwend/Qwen2.5-Mathh
google/gemma-2-22b
Qwen/Qwen2.5-Matth-7
googl//gemma-2-9b
Qwen/Qwen3-0.68- Basc
google/gemma-22277
Qwen/(Qwen--L.7BBBBs
meta-llama/lllma-322222
QwennQQwen3-11BBBBBss
meta-llama/Llamme 3.2-3B
Qwen/Qwen3-4B-Bas
meta-llamaa/Llama-33
Qwen/QQeen3-SB-Ban
mistralai/Miitral-BB-000..
tiuae/Falcon33-11
mistralai/Mistral-Nonn
tiuac/Falcon3-333 3B Basc
Qwen/Qwen2.5-77B
Bat.rFalcon37BBBBaa
Qwen/Qwen2.551144
tiune/Falcon3-1000 10B-Base
Qwen/Qwen2.5-322
Table 6: Runtime comparisons (using 8xH100 GPUs) of data selection methods.
Method
DDCF
Perplexity
Cartography
S2L
DISF
Random
Qwen2.5-Mathe
1.4
1.3
1.3
2.0
0.1
0.0
Qwen3-8B-Basc
1.4
1.6
2.0
0.0
Palcon-108 1OB--OOBB--- Base
1.4
2.1
2.1
2.0
0.1
0.0
Total
1.4
5.0
5.0
2.0
0.1
0.0
and MATH). Since GSM8K and MATH are annotated with short-CoT rationales of lower quality compared to OpenRl-Math-2206 Maaaath-220Kwhhhh which provides long-CoT annotations with structured reason- ing and rigorous reflections re-annotate these 1,000 questions using Qwen/ /ween//we33322 in long-thinking mode.
For the S2L method, we follow Yang al. (2024b) and train Pythia-70M model (Biderman al., 2023) on the full -eenR1-Math-22000 corpus as a proxy model to record loss trajectories. Samples then clustered into 1,000 groups, from which representative examples selected to form the training subset.
Finally, DiSF requires converting samples into embedding vectors prior selection. For fair comparison with our proposed DDCF, we use Qwen/ Dwven/wwen3 3-Embedd digbbedding--455 (Zhang et al., 2025b) the sentence encoder for DiSF.
Data Cleaning. Due to computational constraints, we restrict training maximum sequence length of 16,384 tokens. Accordingly, we discard all examples exceeding this length (fewer than 1% of DpenRR-MMth-220K- To enhance diversity the selected dataset, we further remove duplicated questions, retaining only the instance with the shortest completion. After cleaning, the OpenRI SeenRI-Mon -Math-220K dataset contains 189,257 examples, which we use for all experiments involving data selection.
A.4 INFERENCE HYPERPARAMETERS
To improve efficiency, accelerate inference with the SGLang framework (Zheng et al. 2024). By default, we report pass@1 accuracy, generating a single sampled response per query with top_p=0. 95, top_k=20, min.p=0, and maximum sequence length of 16,384 tokens. For the AMC23 and AIME24 benchmarks, we sample 64 responses per query and report pass @ 32 due to their small size and high difficulty.
19

---

Published as a conference paper at ICLR 2026
Table 7: Effect of the number of participating models on the correctness predictor's accuracy.
Table 8: Effect of the number of seeding ques- tions on the correctness predictor's s accuracy.
# Models
2
4
8
16
23
Questions
1K
2K
4K
8K
16K
17.5K
Accuracy 81.5
81.7
81.8
82.2
82.5
82.7
Accuracy
80.0
80.1
80.8
81.9
82.4
82.7
Table 9: Effect of Number of Participating Mod- els on Performance.
Table 10: Effect of Number of Seeding Ques- tions on ID Performance.
Models
16
23
Questions 1K
8K
17.5K
Accuracy 69.5
70.1
69.9
69.8
69.9
70.2
Accuracy
69.3
68.5
69.5
70.4
70.0
70.2
Table 11: Effect of Number of Participating Models OOD Performance.
Table 12: Effect of Number of Seeding Ques- tions OOD Performance.
# Models
8
16
23
# Questions
IK
2K
4K
8K
16K
17.5K
Accuracy 61.7
60.6
63.5
62.2
63.8
65.4
Accuracy
63.0
62.4
61.7
63.2
64.0
65.4
B LIST OF MODELS IN DDCF AND EFFICIENT DDCF RUNTIME
Table lists the 23 models included in DDCF framework. These models from Qwen 2.5
Yang et al. 2025b), Qwen 2.5 Math (Yang et al., 2024a). Qwen3 (Yang et al. 2025a) Falcon 3,
Mistral (Jang al. 2023). Llama 3 (Grattatiori al. 2024a). Gemma 2 (Team al. 2024). and
Deepseek Math (Shao et al. 2024).
The runtime comparison between DDCF and other selection methods on pre-annotated corpora such (excluding annotation cost, where DDCF holds a clear advantage) is re-
as 1-atR1-Math-2221 ported in Table 6 DDCF requires 1.4 GPU hours total, since all participating models share the same seed-set inference (using SGLang (Zheng et al., (202)), while Correctness Predictor training and the k-greedy step incur negligible cost. In contrast, Perplexity and Cartography must process the full corpus separately for each model to compute token-level probabilities, accumulating 5.0 hours-over 3x the cost of DDCF, Model-agnostic methods such as S2L and DiSF are faster but produce single fixed subset, offering no way to tailor data to different models.
C How RELIABLE IS THE CORRECTNESS PREDICTOR?
As described Section we trained our Correctness Predictor on a seed dataset of triplets, (model, questi on, binary correctness), comprising 23 open-source lan- guage models and 19,470 questions, with 1,947 questions (10%) held out for testing.
C.I CORRECTNESS PREDICTOR'S ACCURACY DIMENSION
To evaluate Correctness Predictor's reliability, we measured the predictor's accuracy on unseen test questions for the Owen2 5-Math-72 model under two conditions: (1) fixing the number of mod- els at 23 while varying the number of training questions, and (2) fixing the number of training questions while varying the number of models. Overall, the Correctness Predictor exhibits strong sample efficiency in low-data regimes alongside steady improvements as more models or questions are added. When trained with just one model, attains 81.5% accuracy, rising to 81.7% with two models and peaking 82.7% when all 23 models are included (Table 7 Likewise, increasing the number of seeding questions boosts accuracy from 80.0% with 1,000 examples to 81.9% with 8,000 examples, and ultimately to 82.7% with 17,523 examples (Table 8). These results confirm that our predictor is reliable even with minimal data and scales effectively: most gains emerge early, with incremental benefits thereafter.
20

---

Published as a conference paper at ICLR 2026
Table 13: Performance on In In-Distribution and Out-of-Distribution benchmarks.
In-Distribution
Out-of-Distribution
Method
AIME24 MATH
OlyBen
GSM8K
SAT
Avg.
Miverva
Gaokao
STEM
Avg.
Falcon- Base
Full Dataset
83.8
90.4
56.3
95.2
99.1
85.0
64.3
82.3
91.7
79.4
Model
68.6
34.2
55.7
81.4
58.9
Random
65.5
82.2
47.0
93.1
98.6
77.3
58.1
79.8
89.2
75.7
Longest
68.3
82.0
45.8
91.1
88.6
75.2
56.6
58.2
82.8
65.9
Binary Hard
67.4
83.4
49.0
94.2
77.8
74.3
57.4
50.6
76.2
61.4
Least Confid.
49.4
79.0
40.3
94.3
97.7
72.1
54.4
43.0
89.1
62.2
Cartography
76.9
Perplexity
60.1
82.8
45.2
93.5
99.1
76.1
61.8
78.5
89.6
76.6
S2L
62.2
82.4
49.2
94.0
98.6
77.3
61.0
74.7
90.0
75.2
DISF
63.2
83.0
47.7
93.4
62.1
89.2
74.5
LIMO
66.5
81.4
48.7
93.5
57.3
69.5
$1.5
48.1
68.3
55.9
sL.1-K
80.0
46.7
DDCF
66.6
83.0
46.1
93.9
98.1
60.3
78.5
88.9
75.9
C.2 DOWNSTREAM PERFORMANCE DIMENSION
We also measure the In-Distribution and under the two above conditions, shown in Table 9 Table 10 Table 11 and Table 12
of the Qwen2 5-Math-75 model
ID performance is strikingly stable. Accuracy remains tightly concentrated (68.5-70.4) across all settings, with mean of 69.8 and standard deviation of 0.25. This shows that DDCF reliably identifies strong in-domain training samples even with substantially fewer seed models or questions.
OOD performance benefits from scale. Increasing the number of participating models improves OOD accuracy from 61.7 (1 model) to 65.4 (23 models). Similarly, expanding seeding questions im- proves OOD accuracy from 63.0 (IK) to 65.4 (17.5K). This shows that larger seed matrices provide richer difficulty signals and improve generalization.
Overall, ID performance is robust even under very small seed budgets, while OOD performance im- proves steadily with more seed models and questions. This efficiency-scaling pattern makes DDCF cost-effective for in-domain fine-tuning and scalable when stronger OOD robustness is desired.
D EXPERIMENT RESULTS ON OPENR 1 -M FOR FALCON- MALCON-1OB-BALR 1OB-BASE
In-Distribution Table 13 shows that for Melcon-1OB-Basse DDCF delivers the strongest overall subset. reaching an average of 77.6. This slightly surpasses the best persfperfonmin baselines (Random and S2L, both 77.3) and narrows the gap to the full-data upper bound (85.0) to just -7.4. Perfor- mance gains are especially visible on MATH500 (83.0) and GSM8k (93.9), where DDCF matches or exceeds competing selectors. On the most challenging benchmark, AIME24, DDCF secures 66.6-well above Perplexity (60.1) and Least Confidence (49.4), underscoring its ability to capture harder examples without sacrificing breadth.
Nut-of-Distribufin On OOD tasks, DDCF remains highly competitive. It achieves an average of 75.9, ranking just behind Perplexity (76.6) but outperforming all other baselines, including Random (75.7) and S2L (75.2). Notably, DDCF preserves strong performance across datasets: improves over Random on Gaokao (+-0.7 +13.9 over weaker baselines) and stays close to the top scorer Minerva (60.3 vs 61.8 with Perplexity). Again, DDCF consistently produces a compact subset that balances difficulty and diversity, yielding competitive results with only 1,000 examples.
21

---

Published as a conference paper at ICLR 2026
Table 14: Random selection on a synthesized "bad" corpora.
In In-Distribation
Out-of-Distriuutic
Method
AIME24
MATH
OlyBen
GSM8k
SAT
Avg.
Minerva
Gaokao
STEM
Avg.
Sween.55-Math-77
Base Model
55.4
16.4
91.6
80.0
55.6
12.9
67.7
49.2
Random Low
39.6
72.9
30.6
91.4
96.8
66.3
40.1
68.6
73.8
63.2
Random High
38.6
76.4
34.8
91.0
98.2
67.8
41.2
64.6
75.7
60.5
DDCF
49.0
77.6
35.0
91.2
98.2
70.2
45.6
74.7
75.8
65.4
Qwen3-8B-
Basc Model
47.8
60.8
36.3
89.8
98.2
66.6
40.8
58.2
84.4
61.1
Random Low
72.1
89.0
54.8
96.4
98.6
82.2
63.0
80.0
90.2
77.7
Random High
80.9
90.4
53.8
94.4
99.6
83.6
62.5
83.5
90.8
79.0
DDCF
82.2
91.0
56.0
95.9
100
85.0
66.2
84.8
90.6
80.5
Falcon-10Ff 10B-Base
Base Model
41.1
34.2
81.4
93.6
39.7
Random Low
60.2
81.4
42.1
93.8
97.3
75.0
57.2
72.1
89.6
73.0
Random High
65.5
82.2
47.0
93.1
98.6
77.3
58.1
79.8
89.2
75.7
DDCF
66.6
83.0
46.1
93.9
98.1
77.6
60.3
88.9
75.9
E IMPACT OF THE LARGE CORPUS QUALITY ON THE SUCCESS OF DATA SELECTION
The impressive performance of random selection in Table 2 and Table raises the question when it can replace more sophisticated approaches. Its success, however, is conditioned on the quality of the large corpus. If the large corpus D is dominated by easy or homogeneous examples, the lack of difficulty variation limits any selection strategy, and downstream performance inevitably plateaus.
To examine this effect, we construct a synthetic "bad" corpus D composed primarily of easy questions. From the remaining pool, we sample 19,000 "bad" items (maximizing the composite score in Equation 5 and 1,000 "cherry" items (minimizing the score), forming highly skewed dataset where only 5% of random samples are genuinely high-quality. The results across three backbones are reported in Table 14
Across Qwen2.5-Math-78 Qwen3-8B-B Base, and Falcon-10B-Hai 10m--0sBB Random Low (sampling from Dey") consistently underperforms Random High (sampling from the full OpenRI M-aath-220K) on challenging benchmarks such as AIME24, MATH, and OlympiadBenchh The gaps are sub- stantial-foe instance, -3.3 (MATH) and -4.2 (OlympiadBench) on Qwen2.5-Math-7BB 8.2 (AIME24) on Qwen3-8B- BB-aasc, and 5.3 (AIME24) on Falcon-10B (OB-Base.
These results indicate that while LLMs can learn easy tasks without curated data, high-quality and diverse examples remain crucial for strong performance on difficult reasoning benchmarks. Random sampling from easy-heavy corpus cannot approach expert-level performance, e.g., AIME24.
F DATA TRANSFERABILITY BETWEEN MODELS
Figure 6 shows the performance of Qwen2. 5-a5th--hhhh- 7B after fine-tuning on DDCF datasets cu- rated for other models. Fine-tuning on its own curated data yields the highest combined perfor- mance of 67.5%. Substituting the dataset from Owen2.5-322 incurs a modest 0.4 point drop (to 67.1%), while using Gemma-2 2-----9-9 9B and Mistral- 7B-v0. subsets leads to declines of 2.0 and 2.3 points, respectively. Beyond these, we observe gradually larger drop of 2.5 points with Qwen2. 2.ren.55Maae 5-Math-1...... 5B and Falcon3-7B-Ban all, this pattern hints that datasets drawn from models with closer architectural training kinship may transfer more effectively, although more extensive experiments would be needed to confirm the precise nature of this relationship.
and 2.7 points with Llama-33 3-18B. Over-
22

---

Published as a conference paper at ICLR 2026
Qwen2.5-Math-78 Fine-tuned on Other Models' Data
0.5 Mathh
72
0wen2:5-32
Gemma-2-9b
70
69.569.699.8
Mistral 78 v0.3
69.0
2. 5-Math-1..5t
68.68.390 688.85.5
Falcon3
Performance
68-
0
67.1
67.9
Llama-3.1-4
6.6
0.0
66.5
66, 066.0
6
65.4
64.6
64
63.363.3
63.763.5
62.6
00
62
60
ID Avg.
000 Avg.
ID+ 000 Avg.
Figure 6: Each model has its own datasets. Using other models' datasets yields suboptimal results.
G OOD PERFORMANCE ON GENERAL TASKS OF LLMS FINE-TUNED ON MMLU SUBSETS
Table 15: Performance on OOD general tasks.
Qwen2.5 7B
Qwen3 8D Base
Faleon 108 Base
Method
Arg.
LogiQ4
OpcnBrokQA
AlpacaRval2.11
OpenBookkQ
AlpacaEval2.00
OpenlookQQA
AlpacaEval2.
Basc Model
47.3
83.6
5.6
51.8
82.6
16.5
48.1
8018
7.0
47.0
Randon
50.7
89.4
33.3
61.0
93.2
99.3
53.5
90.
49.7
64.5
DICF
48.2
90.4
32.5
01.3
92.0
99.5
56.2
42.9
53.3
650
While DDCF is tailored for reasoning-centtrc MMLU tasks, Table 15 shows it also transfers effee- tively to out-of-distribution (OOD) general tasks. Fine-tuning on just 1,000 distilled MMLU amples leads to strong performance across diverse benchmarks, including commonsense reasoning (LogiQA (Liu et al. 2020)), reading comprehension (OpenBookQA (Mihaylov al. 2018 and instruction following (AlpacaEval 2.0 (Dubois et al. 2024h), without using any target-task labels.
DDCF outperforms the base models by average of +18.0 points and achieves the highest overall average (65.0) among all methods. On average, it improves commonsense reasoning by +6.2 over Base, delivers state-of-the-arr reading comprehension on Qwen2.5 and Falcon (+9.1 avg), and shows the largest gains in instruction following (+38.7), surpassing Random and DiSF on stronger back- bones. These results underscore DDCF's broad generalization ability beyond its intended domain.
23


Reviews:
[{"review_id": "ECyfRymeup", "reviewer": "Reviewer_KfMD", "summary": "This paper proposed Difficulty–Diversity Collaborative Filtering (DDCF) as an algorithm for model-specialized data filtering, which can be applied for LLM finetuning and active learning for CoT trace annotation. From the collaborative filtering, the learned parametrized model embedding and problem encoder can predict the problem's correctness; then the final data selection scores combine the difficulty and diversity of the selected subset, using the k-greedy strategy. \nEmpirically, the proposed method achieves a SOTA performance, while with 100x less annotation costs.", "questions": "1. Have you encountered any cold start issues when training the collaborative filtering framework? How do you collect the seed dataset used to train the CF system?\n\n2. According to Figure 5, the data distribution selected by different models seems quite close. How much could the collaborative filtering from multiple models improves the performance comparing to train the question encoder with a classifier head upon a single model's correctness?\n\n3. Are there any performance differences when using models from different families/scales? Would the obtained encoder also be able to transferrable between different model scales?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed DDCF algorithm is neat and effective, which provides an efficient and reliable method to quantify the model-specialized difficulty and diversity.\n\n2. The DDCF method demonstrates a significant improvements on the data efficiency on both instruction tuning on annotated and unannotated data.\n\n3. The paper is well-written and easy to follow, with clear illustration and extensive experimental results.", "weaknesses": "1. The proposed collaborative filtering algorithm might have a cold start problem with insuffient data samples or biased data distributions. The performance can also greatly depend on the seed coreset.\n\n2. Lack of ablations on the number of models and questions used to train the CF model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8PZlvGpTVK", "reviewer": "Reviewer_ksTm", "summary": "In this paper, the authors addressed the inefficiency of data selection from large scale datasets for LLM supervised fine-tuning: high annotation costs, catastrophic forgetting, and redundant examples, by proposing Difficulty-Diversity Collaborative Filtering (DDCF). DDCF is a data selection framework that curates compact, high-quality subsets from large (often unannotated corpora), it leverages collaborative filtering to balance two criteria in data selection: question difficulty and semantic diversity, to enable SFT with minimal annotation overhead.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow;\n\n2. DDCF can leverage unannotated data in SFT;\n\n3. DDCF can balance difficulty and diversity in data selection.", "weaknesses": "1. Considering that $h_m$ in the model encoder does not change the dimension of the embeddings, why should the authors use $h_m$ in the formulation of the model encoder, please provide some evidence to demonstrate the necessity of $h_m$.\n\n2. For the experimental setting, [1] has demonstrated that most data selection algorithms can not outperform random selection in the experiments on datasets with more than 1 million data points regarding both effectiveness and efficiency. Data selection algorithms are expected to be applied in real-world industry scenarios, which often need to deal with datasets containing more than 1 million data points. Therefore, empirical results only on OpenR1-Math-220K can not provide convincing support to demonstrate the effectiveness of DDCF for real-world scenarios.\n\n3. The provided empirical results are not that promising, some results can not even outperform the base model, like GSM8k in Table. 2, \n\n4. Considering that the proposed DDCF needs several LLMs to conduct annotation before conducting data selection, and it will obviously add computational cost, the authors should provide computational anaylsis in the paper to demonstrate the efficiency of the proposed method.\n\n[1] Xia, T., Yu, B., Dang, K., Yang, A., Wu, Y., Tian, Y., ... & Lin, J. (2024). Rethinking data selection at scale: Random selection is almost all you need. arXiv preprint arXiv:2410.09335.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RJcl3fthCz", "reviewer": "Reviewer_b2jG", "summary": "The paper introduces a new strategy for selecting a set of high-quality samples that can be used to improve fine-tuning of LLMs on different tasks. The method is designed to work with unlabelled and limited data by first training a simple linear model for predicting the difficulty of the samples based on the behaviour of different LLMs on the questions. Afterwards, the difficulty is balanced with diversity of the samples to select the set of samples. Through extensive experiments the paper shows such selection leads to set of samples that can result in better accuracy after fine-tuning.", "questions": "See weaknesses for more details:\n\nHow does the proposed DDCF method compared with different baslines for selecting samples in terms of performance-cost trade-off (including random selection)?\n\nWhat is the impact of dataset quality on the sucess of sample selection?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method is simple, easy to understand and builds on the previous advances or other methods that were shown to work well. As such, it is not overly complicated and is easy to use which may allow its widespread use.\n\nThe method is evaluated through multiple experiments, ablation studies and multiple datasets and models.\n\nWriting is very clean and paper is easy to read and understand.", "weaknesses": "**Missing related approaches for sample selection**\n\nAlthough the paper compares with multiple baselines, there are 2 approaches that are very similar to the DDCF method, which are completely missing from the paper. This includes the \"Datamodels\" selection [1, 2], which also trains a linear model to predict the benefit of each sample to the model (where the DDCF can be viewed as an extension of this approach that utilises multiple models at the same time when calculating the difficulty); and \"Dataset Cartography\" [3, 4] which determines the difficulty of the samples based on their training dynamics and shows that combination of easy and ambiguous samples provides benefit for training. In both cases, these methods are more related to determining the difficulty of the samples and so the comparison with them would strengthen the paper -- also including them in the related work section. I acknowledge that they are mostly used for selecting in-context examples, but they are not constrained by the typical approaches there (that select for specific test sample) and so could be used here, especially considering how similar they are to the proposed method.\n\nThere are also other methods for selecting in-context examples that balance multiple properties (including diversity) that could be included as part of the related work [5, 6, 7, 8].\n\n**Missing cost comparison across different approaches**\n\nBased on the experimental setup, for the DDCF method to work, you run ~19.5k inferences on using 23 different models. One of the main claims of the paper is that this method is less expensive than other approaches. As such, I would expect there to be a comparison in cost with different methods across different settings -- for example, if I am interested in selecting for samples for one specific model, running the extensive inferences from the DDCF method may not be the best solution. I would suggest adding a more in-depth discussion for the cost-performance trade-off. I acknowledge that there in Appendix C, there is an analysis how the DDCF difficulty evaluation performs when changing the number of models and questions that are requires, but I would expect to evaluate this not only based on the \"correctness\" of the difficulty predictor but based on the samples chosen in these data-contrained settings instead. \n\nSimilarly, based on the results, the random selection achieves strong accuracy already -- it is close to the proposed DDCF method and the best-performing approaches. I would suggest adding a discussion on this in regards to the trade-offs, as random selection is quite cheaper (and does not require labels). It might be an effect of the data, i.e., some datasets that have high-quality samples already do not benefit as much from more sophisticated selection, while on others (like AIME24) that may be more complicated or noisy, it might produce significantly better results. As such, I would suggest doing a more detailed analysis for these characteristics.\n\nA little side note: I would also suggest better references to the Appendix B and C that to a certain extent deal with this problem, but it is not as obvious from the text that it is included.\n\n\n**Results presented withouth consideration for LLM randomness**\n\nAll the results in the paper are presented from a single run, although the LLMs (and fine-tuning) are affected by different effects of randomness. This affects both the \"corectness predictor\", where repeated fine-tuning could lead to different results, but also the LLM fine-tuning on the selected samples. Although the expected differences might be small, the differences in results across different approaches are not that high so it would be interesting to so how the method is affected by randomness.\n\n\n**(minor) Writing changes**\n\nIn Table 2 (and also other tables) some numbers are bolded or underlines, but there is no explanation why. It is also done only for specific datasets. What does this mean? I would expect either bolding the best performing models (and underlining the runner-up) or removing it altogether.\n\nThere are cases on data where the DDCF method does not outperform other methods, although you claim it happends -- e.g., Qwen-2.5 on Minerva and STEM datasets is outperformed by Perplexity and/or s1.1-1K; or Qwen-3 on STEM for multiple approaches. I would suggest updating lines 356-359 as it is a bit misleading in terms of results (and not really necessary as I would not expect the method to be the best across all settings); same with Appendix D\n\n**References**\n1. Datamodels: Predicting Predictions from Training Data\n2. Data Curation Alone Can Stabilize In-context Learning\n3. Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics\n4. Cartography Active Learning\n5. Finding Support Examples for In-Context Learning\n6. Automatic Combination of Sample Selection Strategies for Few-Shot Learning\n7. EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning\n8. Sample Efficient Demonstration Selection for In-Context Learning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)