DELIFT: Data Efficient Language model Instruction Fine-Tuning
Authors: Ishika Agarwal
Krishnateja Killamsetty
Lucian Popa
Marina Danilevsky
Venue: ICLR 2025 Poster (2025)
Keywords: Data Efficient Instruction Fine-Tuning;  Data Subset Selection; Submodular Functions
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We introduce DELIFT, a novel algorithm that efficiently and systematically optimizes data selection across the three key stages of fine-tuning by using a pairwise utility metric to capture informativeness of data points.

Abstract: Fine-tuning large language models (LLMs) is crucial for task specialization but often becomes resource-intensive due to redundant or uninformative data. Existing data selection methods typically rely either on computationally expensive gradient-based metrics or static embeddings that fail to adapt dynamically to the model’s evolving state, thus limiting their practical effectiveness. To address this,
we propose DELIFT (Data Efficient Language model Instruction Fine-Tuning), leveraging a novel, computationally efficient utility metric inspired by In-Context Learning (ICL). Our ICL-based metric measures the informational value of each data sample by quantifying its effectiveness as an in-context example in improving model predictions for other samples, reflecting its actual contribution relative to the model’s current state. Integrated with tailored submodular optimization methods, DELIFT systematically selects diverse, informative subsets optimized specifically for each fine-tuning stage: instruction tuning, task-specific adaptation, and continual fine-tuning. Experimental results across multiple datasets and model scales show DELIFT reduces fine-tuning data requirements by up to 70% without compromising performance, consistently outperforming existing methods by up to 26% in effectiveness and efficiency.

Published as a conference paper at ICLR 2025
DELIFT: DATA EFFICIENT LANGUAGE MODEL IN- STRUCTION FINE-TUNING
Ishika Agarwal', Krishnateja Killamsetty", Lucinn Popa², Marina Danilevsky
University of Illinois Urbana-Champaign, 2IBM Research
'ishikaa2@ @lllinois.edu *krishnateja.K k @ibm.com, {Ipopa, mdanile } @us. ibm.immccom
ABSTRACT
Fine-tuning large language models (LLMs) is crucial for task specialization but often becomes resource- insource-intensive due to redundant or uninformative data. Ex- isting data selection methods typically rely either on computationally expensive gradient-based metries or static embeddings that fail to adapt dynamically to the model's evolving state, thus limiting their practical effectiveness. To address this, we propose DELIFT (Data Efficient Language model Instruction Fine- Sine-Tnnin)). leveraging novel, computationally efficient utility metric inspired by In-Context Learning (ICL). Our ICL-based metric measures the informational value of each data sample by quantifying its effectiveness as in-context example in improv- ing model predictions for other samples, reflecting its actual contribution relative the model's current state. Integrated with tailored submodular optimization methods, DELIFT systematically selects diverse, informative subsets optimized specifically for each fine-tuning siee-tnnig stage: instruction tuning, task-specific adapta- tion, and continual fine-tuning Experimental results across multiple datasets and model scales show DELIFT reduces fine-tuning data requirements by up to 70% without compromising performance, consistently outperforming existing methods by up to 26% in effectiveness and efficiency.
INTRODUCTION
Large Language Models (LLMs) have become indispensable for solving variety of natural lan- guage processing tasks, ranging from question answering and summarization to complex dialogue and reasoning (Brown et al., 2020 Touvron et al. 2023). Despite their remarkable adaptability, fine-tuning LLMs often requires enormous computational resources and time, especially when significant portion of the training data either redundant or uninformative (Gururangan al. 2020 Sorscher et al. 2023). This challenge grows more critical with increasing model and dataset sizes, posing key limitation to the broader deployment of L.MSs.
Existing data selection methods generally fall under two paradigms: (1) static embedling-bbusd ap- proaches that compute sample similarities without reflecting the model's evolving state (Bukharin) & Zhao 2024 Chen et al., 2024), and (2) gradient-based methods that offer more model-specific feedback but often entail prohibitive computational overhead, especially for large-scale models (Kil- lamsetty al. 20216; Xia et al. 2024). Although both paradigms
yicld initial benefits, they often fail to account for how a model's knowledge shifts over multiple fine-tuning phases: (1) In- struction Tuning (Mishra al, 2022 Wei et al. 2022 Longpre et al., 2003),, which enhances the model's ability to follow diverse instructions; (2) Task-Specific Fine- Tuning (Gururangan et al.) 2020 Cobbe al., (202T) which focuses refining domain expertise; and (3) Continual Fine- Tuning (Madotto et al.) 2021] WW et al.) 2024), which incrementally incorporates new knowledge
while mitigating catastrophic forgetting.
Thus, a natural question arises:
Can develop unified, computationally efficient data selection framework that adapts to all stages of fine-tuning and maximizes model performance while minimizing data redundancy?

---

Published as a conference paper at ICLR 2025
Use Case 1:1hhe tunz model to collowinstruccions Suboot should contain points that are diverse.
Datanet
Inctuusion
Inpui
Culput
contect, arswar
Question:
Who New Zealand's
Chrisoophr Lucon
queet on.
Contect: Chrietopent
sarveed 22rd prima
New Zecland nd since November 2023.
questi
Whiie suntence with the words.
Sun, dag.
Criae the lvent
with
Classity the given obiects Inco cotogory.
Crob, tuno, lobstor.
Seafcod.
Use case 2 improve mudel'ss performance mattarmtlical rearoring bandhhmekk. kank Subrett shuuld tain points that diverse and representative of the benchmark.
(Exo mple) Bononma Dota
Qutput
worked
240
How much dld work?
Ban paid for cineerme (520), Charias dinner (51 (0 and
$53
Dennie' dinner (515). How much did he pay?
cranges, frientx. meny cranggoo
daas wach fradddd gall?
baseboll trodoo How
Ditsssel
Output
matty
Fred had 25rroses and gave t0 Hom. How many lef??
15 roses
yycls grow many pie Mike
Nac.
the
How muph the plo is len?
(b)
following
negative? "Happiness"
Use case 3: continuat tearnin restew centiment anotysie dotssets, Subeet should comsin points that are diverse and complementary Phose data.
Imput
Output
This restaurent has good pael secapt that il scometimes tco suill.
Magetiv
impotient and rude, they rushod orcermyy
Nogotive
this restaumet
comfortsble,
Poativve
The food come quicity
Positive
Prase
Outpett
The fildd rice omazing!
Positiv
quality low, and lens focus property.
Negst
This
and mn
Posittve
(c)
The reataurent closss tno sorly.
Negstiv
Figure 1: DELIFT data selection across fine-tuning stages. (a) Instruction Tuning: Diverse instruc- tions selected; redundant samples pruned. (b) Task-Specific Fine-Tuning: Mutually informative (with benchmark data) and diverse samples are prioritized for selection. (c) Continual Fine-tuning: New samples that are novel are integrated; new samples with overlapping information are pruned.
In this paper, we introduce DELIFT (Data-Efficient Language Model Instruction Fine-Tuning)) single-stop solution designed to address data selection across all fine-tuning stages within a single framework. DELIFT is grounded in information theory yet uses the practical intuition of in-context examples to assess the 'inormation gain' of each data sample relative to the current state a model. Specifically, we propose
utility metric that captures how effectively model's 's prediction of another. By combining these pairwise utilities with submodular optimization, DELIFT generates diverse, nonredundant subsets uniquely tailored to each fine-tuning phase.
sample improves the
2

---

Published as a conference paper at ICLR 2025
We evaluated DELIFT various tasks and model scales, consistently observing that it can prune up to 70% of the training data without hurting performance and often improving it- outperforming existing methods by up to 26% in efficiency and effectiveness. In doing so, show that careful, utility-drveen data selection can be far more effective than sheer data volume, opening the door to more feseulle-friendly and targeted fine-tuning.
Our primary contributions are as follows.
1. A unified information-theorer data selection paradigm that leverages pairwise utilities grounded in conditional pointwise mutual information, making it adaptable to instruction tuning, task-specific adaptation, and continual fine-tuning.
2. A single-stop, submodular optimization framework that integrates these utilities to provide diverse, high-value subsets for each fine-tuning stage without incurring prohibitive computation.
3. Extensive empirical validation showing up to 70% data reduction with minimal (and sometimes zero) performance loss across multiple domains, demonstrating substantial gains in both efficacy and efficiency.
The remainder of this paper organized as follows. Section reviews prior work on data-efficient strategies for fine-tuning LLMs and situates our approach within the literature. Section 3 introduces our information-thhore tion to enable data selection across diverse fine-tuning stages. Section 4 presents comprehensive experiments demonstrating the effectiveness and efficiency of our framework on multiple tasks and models. Finally, Section 5 discusses the broader implications of our results, outlines limita- tions, and suggests directions for future research. Our complete code base is publicly available at https: giggitttnn thub. wwt.:ooo/cccooooa www.aggawwwlsshikk.............a seliii enabling further exploration and repli- cation.
utility metric and describes how it integrates with submodular optimiza-
2 RELATED WORK
Data Subset Selection for Deep Neural Networks. Selecting an informative subset of training samples is a longstanding strategy to reduce computational costs and enhance model generaliza- tion. Indee-ende Approaches. Traditional model-independen techniques, such clus- tering or distance metrics on pre-trained embeddings (Bukharin & Zhao 2024 Du et al., [023] Killamsetty al. 2023). capture broad semantic similarities but do not reflect the model's chang- ing state, limiting their effectiveness during iterative fine-tuning. Model-Dependentt Approaches. Model-dependent methods incorporate the model's evolving knowledge by analyzing gradients or loss values (Killamsetty et al. 2021bba a Xia et al., 2024). often outperforming static approaches. However, performing gradient or influence estimations scale becomes prohibitively expensive for large models. Techniques like LESS (Xia et al., (2024) alleviate some overhead via parameter- efficient fine-tuning (e.g., LoRA), yet still incur repeated gradient or influence calculations that scale poorly with dataset size. Subset Selection with LLM Feedback. Another emerging direction leverages LLM feedback to score or filter training samples. For instance, SelectIT (Liu et al.) 2024) employs self-reflection prompts et al. 2024) rely on external heuristics. Though these provide a form of model-aware sampling, they typically lack principled theoretical grounding. In addition, all these approaches primarily target a single fine-tuning stage, limiting their adaptability for instruction tuning, task-specific adaptation, or continual learning.
data quality, while filtering approaches using GPT-4 (Chen
Our Contribution. contrast, we present unified, information-theorett framework that oper- ates effectively across all fine-tuning stages: instruction tuning, task-specific adaptation, and contin- ual fine-tuning. Our novel utility metric quantifies how one data point aids the prediction of another, mirroring the model's evolving knowledge. Integrated within submodular selection paradigm (Fu- jishige 2005 [yer et al. 2021). this approach balances diversity, coverage, and informativeness throughout the entire tine-tunigg pipeline. As result, bridge the gap left by existing meth- ods that are either restricted to a single phase or computationally infeasible at scale, demonstrating consistent performance improvements and notable efficiency gains.
3

---

Published as a conference paper at ICLR 2025
3 METHODOLOGY
Our goal to efficiently identify a subset of data that maximizes the performance large language models across three fine-tuning stages: (1) Instruction Tuning, (2) Task-Specific Fine-Tuning, and (3) Continual Fine- Tuning. This section first introduces our novel information-theore pairwise utility metric (Section (.TT) and then explains how leverage submodular optimization achieve data-efficient selection (Section 3.2). Finally, we show how these components unify into a single-
stop solution for all fine-tuning stages (Section 3.3
3.1 PAIRWISE UTILITY METRIC
Let D {(x; vi)} be training set, where each z; is an imput sequence and y; is the corresponding output. Consider two samples (i) and (#z, Wi), and let GT; denote the ideal "ground truth" distribution that assigns probability to the token sequence y, and otherwise. We define p(y any)
as the model's predicted probability distribution of y given alone, and p( 24, Z4, as the predicted distribution of y, when (xy, My) also provided (e.g., as an in-context example).
Definition (Utility Function). We capture the information gain of (rrj, Mj) for predicting (24, vs) via:
UFi d(GT,, p(u =.)) d(GT, p(y 2z,2,,
where -(- is a distance between probability distributions.
Iioormation-Thoreiit Interpretation. Below we state a simplified version of our main theoret- ical result (see Appendix A for the full proof):
Theorem (Informal Statement) If d) is chosen to be the Kullback-Leibler (KI.) divergence, then the utility Uij coincides with the (conditional) pointwise mutual information between y/ and (Ig, Uj) given 24. Formally,
UF =log"
p(y:
Ii, 25, Ni, Mi)
Σ log
plyee
ply: x.)
Thus, Fiy captures how much (zzz Uj) informs the prediction of ys given 2,.
Practical Computation. In practice, for numerical stability, we adopt length-normalized Eu- clidean distance rather than the KL--divergence
d(GT, p(y -)) = -|11 p(u: 1.)||
where we only extract the probability assigned each ground-truth token under teacher foreing. This preserves the spirit of the PMI-based formulation while avoiding the instability issues often encountered with near-zero probabilities large vocabularies. We compute U Fi, for pairs (i. once before fine-tuning. Although this step is (n ) in the dataset size, cost is amortized because the same utility matrix can be reused for different fine-tuning stages or methods.
3.2 SUBMODULAR OPTIMIZATION FOR DATA SELECTION
Fij: 0)) and utilize in well-studied submodular functions (Fujishige) 2005 [yer al. 202T). Submodularity naturally captures diminishing returns and promotes coverage of diverse yet informative samples.
After computing U Fij. we define a kernel matrix 8iy (e.g., set 85j (U
Objectives.
We primarily adopt three variants:
Facility Location (FL.): (A) EiED maxjEA 8ij-
2. Facility Location Mutual Information (FLMI): FFLMI(A:D) EIED max +
" EyeA maXieD; 8ig-

---

Published as a conference paper at ICLR 2025
3. Facility
Location
Conditional
Gain
(FLCG): fFLCG
Dg)
EIED max max Sij
P maXkED; Sck.
FL supports coverage of D itself, ideal for instruction tuning: FLMI additionally ties the selec- tion to a task-specific dataset Dr; and FLCG incorporates previously used Dz promote continual learning of novel examples.
Algorithm Greedy Maximization for Submodular Function
Require: Dataset D, submodular function J' budget k:
1: Initialize subset .A <
2: for 1 to A do
3: Select d= arg Ma (F(AU()) - f(A))
4: Update A + {
5: end for
6: return A
Subset Selection.
To choose a subset A of size k we apply a classic greedy heuristic (Nemhauser al. 1978), each step picking d' arg ma U {d)) - f(A) (see Algorithm [ This
yields approximation factor for submodular functions, ensuring that we obtain near-optimal subsets efficiently.
3.3 A SINGLE-STER STOP SOLUTION FOR ALL FINE-TUNING STAGES
By combining the ubieity-based kernel with the above submodular objectives, we obtain DELIFT, unified framework that selects data holistically across instruction tuning, task-specific fine-tuning, and continual learning:
Instruction Tuning: Use Facility Location (FL) for diverse coverage of general instructions.
Task-Specific Fine-Tuning: Use FLMI to align samples with target benchmark domain Dy-
Continual Fine-Tuming Use FLCG to incorporate new, complementary data while avoiding redundancy with existing knowledge Dg.
Despite these distinct objectives, each stage uses the underlying pairwise utility metric to capture how one sample can enhance the model's predictions on another. This consistent, model- aware metric is key DELIFT's strong performance across diverse tasks and model sizes.
EXPERIMENTAL RESULTS
We conduct extensive experiments evaluate DELIFT in three fine-tuning scenarios: Instruc- tion Tuning, (2) Task-Specific Fine-Tuning, (3) Continual Fine-Tuuing This section outlines our experimental setup and baselines, followed by results across multiple models, datasets, and fine-tuning paradigms, along with interpretative commentary to utilize the extra space for deeper insights.
4.1 EXPERIMENTAL SETUP
Models. We evaluate DELIFT both base LLMs (Llama-3.2-3B, Mistral-7B-vO.J, opt-125m) and instruction-tumed LLMs (Qwen2-72B-Instruce Phi-3-mini- 1iniiii-22kk-mstrrccll This variety cov- ers different parameter scales (millions vs. billions) along with diverse adaptation strategies (ICL, QLoRA, and, for smaller models, full fine-tuning). This diversity in model size and training setup provides robust test-bed for assessing DELIFT across both resource-rich and more constrained environments.
Datasets. We group the datasets by the primary goal of fine-tuning, ensuring a clear mapping from the data the corresponding submodular objective. In particular: 1. Instruction Tuning:

---

Published as a conference paper at ICLR 2025
Mix-Instruct (Jiang al. 2023), P3 (Sanh et al. 2021). Both aim enhance general instruction- following behavior, featuring a variety of task prompts and user requests. 2. Task-Specific Fine- Tuning: HotpotQA Yang al.) 2018) aligned with MMLU (Hendrycks et al. 2021), Mix-Instruct aligned with M7--Bench (Zheng et al. 2023), and vix-Instruct aligned with GSM-8k (Cobbe et al.) extract only the most relevant samples from large corpus to improve performance on a specific target benchmark. 3. Continual Fine-Tunings (a) SQuAD (Ra- ipurkar et al., 2016) paired with HotpotQA inject more complex, multi-hop reasoning data after simpler QA, and (b) a proprietary IBM/Govermmen domain query rewriting dataset)
2021]. These pairings allow
In all cases, we fixed an approximate budget of 30% for subset selection unless otherwise noted, striking a balance between data efficiency and coverage.
Baselines. addition Full Data baseline, where the entire training set is used, we compare DELIFT with:
Random: Uniformly selects 30% of the training set provide a simple, model-agnostic bench- mark.
SelectIT (Liu al. 2024): Generates self-reflection prompts within the LLM to rate data quality,
filtering out lower-quality samples.
LESS (Xia et al., 2024): Employs gradient- based influence estimates, approximated via LoRA, to identify highly impactful data points for model parameter updates.
DEFT-UCS: Uses sentence embeddings to cluster the dataset for diversity; although it captures semantic variety, it lacks explicit model feedback to guide selection.
DELIFT (SE): Operates the same submodular optimization as DELIFT but replaces our utility- based kernel with static sentence embedding similarities to highlight the benefit of DELIFT's dy- namic, model-aware approach.
These baselines range from naive (Random) to sophisticated (gradient-based, reflection-based, or cmbedding- furnishing a thorough comparison against DELIFT.
Metrics. We measure model performance across different aspects:
ROUGE (Lin 2004;: Focuses on n-gram overlap summarization tasks or generative text alignment.
2. BGE (Xiao et al. 2023): Evaluates semantic similarity through the dot product of normalized sentence embeddings.
Assigns a 1-5 rating reflecting correctness, clarity, and instruction adherence; detail the scoring rubric and example prompts in Appendix C
LAJ (LLM-as-Judge) (Km et al. 2023):
Classification Accuracy: Used primarily for multiple-choice tasks like MMLU.
By combining these metries, we obtain both text-overlap measures (ROUGE), semantic evaluations (BGE), a holistic LLM-based score (LAJ), and classification perspective (accuracy), offering a comprehensive view of model improvements under each fine-tuning scenario.
4.2 USE CASE 1: INSTRUCTION TUNING
Setting. Instruction tuning aims to broaden a model's capacity to follow instructions spanning diverse domains and styles. We adopt the Facility Location (FL) objective to maximize coverage of varied instruction types.
4.2.1 RESULTS ON INSTRUCTION- MODELS.
Tables D and compare DELIFT with baselines on Mix-Instuuct and P3, using two instruction- tuned models Srren-722BIInstruee and Phi-3-mini-11 1aamiin--2k-insttucc]] Across both ICL and QLoRA,
'Query rewriting transforms follow-up up queries (e.g.. "How much is it?") into standalone forms (e.g.. "How much is the subscription for IBM Cloud?"). This is essential for real-world systems where user queries often rely on prior context.

---

Published as a conference paper at ICLR 2025
DELIFT surpasses other subset selection strategies and manages to prune up 70% of data with minimal loss. Interestingly, DELIFT approaches or occasionally exceeds the Full Data baseline, indicating strong redundancy in instruction data.
Model
Qwen2
Phi-3
Method
ICL
QLoRA
ICL
QLoRA
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAU
ROUGE
BGE
LAJ
Initial
37.87
78.92
2.98
36.36
82.55
3.02
25.76
43.34
1.42
35.50
80.46
2.58
Random
39.00
80.66
3.12
44.45
85.46
3.12
33.05
72.73
2.92
44.70
83.75
2.95
SelectTT
LESS
42.08
83.24
3.26
45.16
84.95
3.28
47.10
85.94
3.23
48.68
85.86
3,24
DELIFT (SE)
47.43
84.40
3.28
86.50
3.28
46.62
85
3.24
45.64
83,70
3,27
DELIFT
48.46
85.77
3.35
52.79
88.04
3.37
49.83
85.27
3.32
50.31
84,40
3.33
92.24
55.92
3.84
Table Use Case MixInstruct. Bold indicates best performance. DELIFT prunes 70% data yet. stays close to Full Data.
Qwen2
Method
ICL
QLoRA
QLoRA
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
L.A.
ROUGE
BGE
LAJ
Initial
18.03
59.13
1.54
20.15
58.38
1.78
20.10
48.66
1.36
20.64
49.17
1.39
Random
20.83
2.24
24.51
2.36
Select|
31,38
71.08
2.86
32.96
74.76
2.90
35.37
66.67
2.52
38.98
69.84
2.54
LESS
34.59
83.23
3.07
35.03
83.37
3,50
39.69
72.12
3.17
40.32
70.89
3.24
DELIFT (SE)
34.69
83.31
3,43
35.46
83.43
3.53
37.07
71.49
3.52
38.13
79,68
3.74
DELIFT
35.48
83.69
3.58
35.60
83.64
3.54
40.66
84.00
3.68
41.91
84.53
3.76
Full Data
36.43
84.25
3.53
35.88
76,87
3.63
42.07
85.26
3.78
.4.73
87.03
3.82
Table 2: Use Case 1: P3. DELIFT again discards most data while retaining strong performance.
4.2.2
EVALUATION ON BASE MODEL.
Table verifies these findings wenn--nsttctoo-ttuuue -tuned Alama-3.2-3B (base). Even without prior instruction aligmment, DELIFT outperforms gradient- and embedding-basee baselines by a solid margin, corroborating that the utility metric is robust model initialization.
ICL
QLoRA
Method
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
Inirial
25.88
61.97
1.90
28.51
73.14
2.48
Random
28.64
77.14
2.59
38.79
79.89
2.53
Selectff'
42.67
DEFT-UCS
41.55
80.12
2.63
41.03
81.86
2.59
2.78
DELIFT (SE)
51.19
83.54
2.72
55.32
85.92
2.79
DELIFT
54.58
88.29
2.83
58.57
90.98
2.98
Full Data
55.46
92.71
3.31
61.23
95.52
3.10
Table Use Case 1: Llama-3.2-3B (base) on Mix-Instruct (30% subset).
Takeaway (Use Case 1): The FL objective and utility- kernel consistently prune large volumes of instruction data without undermining final performance a strong indicator that many instruction examples are either repetitive or uninformative.
4.3 USE CASE 2: TASK-SPECHIS SPECIFIC FINE- TUNING
Setting. refine models for specialized domains (reasoning, QA). Using Facility Location Mu- tual Information (FLMI), select examples aligned with target dataset Dr.

---

Published as a conference paper at ICLR 2025
Method
Qwen2 (QLoRA)
Phi-3 (QLoRA)
Initial
82.10
69.10
Random
79.31
65.16
SelectTT
LESS
80.35
66.72
DELIFT (SE)
80.10
66.36
DELIFT
81.70
68.70
Full Data
78.36
64.50
Table Use Case HotpotQA and MMLU (5-shot) for Qwen2 and Phi-3 (classification accuracy). DELIFT outperforms Full Data by 3.34% for Qwen2 and by 4.20% for Phi-3.
Model
Qwen2
Phi-3
Method
ICL
QLoRA
ICL
QLoRA
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
Initial
44.32
74.86
2.48
47.65
77.92
2.72
39.57
69.43
2.31
42.89
72.76
2.53
Random
49.78
79.54
2.83
52.91
82.67
3.05
44.63
74.28
2.62
47.85
77.39
Select|T
54.92
83.71
3.12
57.86
86.59
3,31
49.75
78.64
2.91
52.68
81.52
3.13
LESS
$9.63
85.89
3.29
62.74
88.72
3.48
54.82
81.95
3.08
57.73
84.67
3,29
DELIFT (SE)
62.85
DELIFT
64.73
87.82
3.47
67.91
90.64
3.66
59.58
83.76
3.26
62.47
86.48
3.47
Full Data
65.89
88.65
3.55
69.72
91.53
3.74
60.76
84.59
3.34
64.31
87.42
3.55
Table 5: Use Case 2: Mix-Instruct and MT-Bench. Bold indicates the best performance. There is a 2.91% drop from Full Data to DELIFT after pruning 70% of the data, and a 1.14% drop from DELIFT the next best baseline.
4.3.1 HOTPOTQA + MMLU & MIXINSTRUCT + MT-BENCH.
Empirical Findings. Tables and 4 illustrate DELIFT's consistent success in bridging the gap between diverse source datasets MixxInstrrct or HotpotQA) and target benchmarks (MT-Bench, MMLU). In some cases (HotpotQA MMLU), DELIFT surpasses the Full Data bascline by prun- ing unhelpful training points.
4.3.2 FURTHER EXPERIMENTS COMPLEX REASONING TASKS.
To rigorously test DELIFT on more challenging datasets, we apply to Mistral-7B-v0.1 (base) using Mix-Instruct aligned with GSM-8k, corpus of math word problems that demand multi-step logical reasoning. Table 6 shows that DELIFT yields strong improvements under both ICL and QLoRA, surpassing all competing baselines. Even the simpler FL-only variant outperforms LESS and DEFT-UCS, highlighting the advantage of carefully selected examples when dealing with more complex reasoning tasks. Moreover, using FLMI for task alignment confers additional boost, reinforcing the importance of matching the submodular objective to the dataset's complexity and problem-solving requirements.
ICL
QLoRA
Method
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
Initial
17.06
55.13
1.35
27.95
65.92
1.36
Random
19.18
$6.03
1.38
33.85
81.74
1.85
Select|T
DEFT-UCS
30.06
76.99
1.59
41.45
87.84
2,08
LESS
31.69
77.87
2.26
43.86
88.22
2.60
DELIFT (SE)
31.84
78.62
2.44
43.04
90.53
2.54
DEI DELIFT (FL. instead FL
32.30
78.
2.54
44.62
91.04
2.63
DELIFT
33.25
79.10
46.12
92.97
2.71
Full Data
33.33
81,59
2.79
49.10
94.57
2.85
Table 6: Use Case 2: Mistral-7B-v0.1 (base): Mix-Instruct -> GSM-8k with FLMI.
Takeaway (Use Case 2): By explicitly matching data to target domain benchmark, DELIFT
discards irrelevant samples and in some cases- -iilds better results than training on the entire dataset.
8

---

Published as a conference paper at ICLR 2025
4.4
USE CASE 3: CONTINUAL TUNNNG
Setting. A model must assimilate data without forgetting old knowledge. We Facility Location Conditional Gain (FI.G)) to prioritize novel samples while avoiding overlaps.
4.4.1 EXAMPLES: IBM - GOVERNMENT, SQUAD - HOTPOTQA.
Tables and confirm that DELIFT mitigates redundancy. Notably, on IBM + Government, DELIFT sees less than 1% drop vs. Full Data, versus a 3 gap for baselines like LESS and DEFT-UCS.
Model
Qwen2
Phi-3
Method
ICL
QLoRA
ICL
QLoRA
ROUGE
BGE
LAJ
ROUGE
BGB
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
Initial
44.11
70.49
2.43
48.49
80.85
2.62
40.66
58.68
1.52
43.96
69.56
2.29
Random
55.57
85.26
2.91
55.52
85.53
2.94
45.76
76.19
2.45
58.94
82.41
2,89
SelectIT'
LESS
64.28
85.41
3.29
69.85
89.33
3.45
66.01
87.20
3.19
67.53
88.17
3,22
(SF)
92.47
3.35
DELIFT
69.49
87.94
3.60
74.19
92.23
3.65
74.11
89.41
3.57
74.38
91,55
3.57
Full Data
87.84
3.65
76.83
92.63
71.23
91.10
77.12
3.64
Table 7: Use Case 3: IBM and Government. DELIFT effectively selects novel examples, outper- forming baselines.
Model
Qwen2
Pli-3
Method
ICL
QLoRA
ICL
QLoRA
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
ROUGE
BGE
LAJ
Initial
51.51
66.97
1.77
54.18
2.50
58.23
58.12
Random
$4.38
79.12
2.57
59.23
82.02
2.66
44.29
59.45
1.33
50.29
61.52
1.60
LESS
67.16
85.76
2.94
69.72
86.63
3.26
60.97
81.41
2.84
61.56
81.53
2.88
DELIFT (SE)
73.75
88.01
3.26
74.84
88.79
3.30
64.44
83.95
3.03
66.35
84.77
DELIFT
76.94
90.41
3,33
77.56
89.99
3.34
66.55
84.65
3.25
67.09
85.17
3.32
86.06
Table 8: Use Case 3: SQuAD and HotpotQA. Bold indicates best performance. There only a 1.94% drop from Full Data to DELIFT after pruning 70% of the data, and a 2.78% drop from DELIFT next best baseline.
Takeaway (Use Case 3): By prioritizing complementary rather than redundant data, DELIFT pre- serves previous capabilities and focuses on new insights.
4.5 FURTHER COMPARISONS AND ABLATIONS
4.5.1
FULL FINE-TUNING VS. QLORA.
We also validated DELIFT under full fine-tuming on smaller opt-125m (Table 9). Results show consistent gains for DELIFT, indicating that the utility-based selection approach is independent of the particular fine-tuning methodology.
4.5.2 COMPARING SUBMODULAR OBJECTIVES.
Although FL alone can beat naive baselines in specialized or incremental settings (see Table o the specialized objectives (FLMI for domain tasks, FLCG for continual updates) yield stronger alignment. This underscores the importance of matching the submodular objective to the fine-tuning stage.

---

Published as a conference paper at ICLR 2025
QLoRA
Foll Fine-Tuming
Method
ROUGE
BGE
L.A]
ROUGE
BGF
L.A]
Random
12.55
46.99
1.25
13.07
47.91
1.30
SelectIT
14.78
49.80
1.26
15.35
50.42
1.28
DEFT-UCS
15.12
50.29
1.39
15.16
50.29
1.39
LESS
15.52
50.70
1.38
16.02
51.30
1.40
DELIFT (SE)
18.81
55.02
1.35
1.38
DELIFT
19.72
57.98
1.43
19,87
58,11
1.45
Full Data
20.39
60.39
1.95
21.64
61.70
2.05
Table 9: opt-125m Mix Mix-Instruct, comparing QLoRA vs. full fine-tuning.
4.5.3 ABLATION ON SUBSET SIZB.
Beyond consistently using 30% of the data in our main experiments, we investigated how varying the subset size influences performance. We tested budgets ranging from as little as 5% to 50% of the original training set (in increments of 10%). As shown in Figure 2 performance under all methods generally improves with larger subsets but exhibits diminishing returns beyond 30-40%. Notably, DELIFT consistently yields higher LAJ scores than other baselines at every subset size, demonstrating its robustness even under very aggressive pruning (%). These findings reinforce that substantial reductions in training data are possible without significant performance degradation, and that DELIFT identifies highly informative samples more effectively than competing methods.
4.6 KEY OBSERVATIONS
Our experiments reveal four consistent patterns across datasets, model scales, and adaptation meth- ods:
1. Utility-based kernel outperforms static or gradient-based methods, indicating that measur- ing how one sample improves predictions on another is a stronger signal than mere embeddings approximate gradients.
2. Stage-specific objectives (FL, FLMI, FLCG) are crucial for addressing the unique needs of instruction tuning (diverse coverage), task fask-specific alignment, and continual complementarity.
3. Significant pruning (up to 70%) is routinely possible, often with minimal zero performance deterioration, suggesting vast redundancy in large corpora.
4. Mathod-agnostic gains, as DELIFT consistently improves results under ICI., QLoRA, and even full fine-tuning on smaller models.
In the next section, we summarize these findings, discuss limitations such as the O(z²) cost of utility computation, and propose directions for future work, including adaptive selection during training, extensions to multimodal data, and fairness considerations.
5 CONCLUSION, LIMITATIONS, AND FUTURE WORK
In this paper, we introduced DELIFT, a novel approach to data-efficient fine-tuning of large lan- guage models by employing a versatile pairwise utility metric combined with submodular optimiza- tion techniques for optimal data selection. Empirical evaluations showed that DELIFT can reduce data and computational requirements by up 70% while achieving performance comparable to the full dataset, and outperforming existing data selection methods by to 26% in effectiveness. These results suggest that DELIFT offers a promising method for improving the accessibility of LLM adaptation, especially for
scenarios. However, our approach has limita- tions, including potential sensitivity to the quality and diversity of initial data and the risk of bias amplification inherent in the selected data. Future work will explore integrating DELIFT with data augmentation techniques to improve robustness, incorporating fairness constraints to mitigate bi- ases, and extending the approach to emerging model architectures and multimodal learning. Our ongoing efforts are directed toward ensuring that DELIFT contributes to responsible and equitable AT development while maximizing efficiency.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGEMENT
A part of this work used the Delta system at the National Center for Supercomputing Applica- tions through allocation CIS240550 from the Advanced Cyberinfrastructure Coordination Ecosys- tem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
REFERENCES
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL
http:://pppppee........eeeo..oor fi #1le:44477ccd00rr009941111111313
neurips. cappaper flles/paper/20277 ppff
Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning, 2024. URL
https::/arxiv. org/ abw. 14736
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang. Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training better alpaca with fewer data, 2024. httth:
orq abwp//ab///e /2007.200... 08701
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.
ltte/ab/22111..11
Qianlong Du, Chengqing Zong, and Jiajun Zhang. Mods: Model-oriented data selection for instruc- tion tuning, 2023. URL htttt:: //axiv org/ abs/2311 15653
Satoru Fujishige. Submodular functions and optimization. Elsevier, 2005.
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don't t stop pretraining: Adapt language models to domains and tasks. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8342-8360, Online, July 2020. Association for Computational Linguistics. doi: 10.11863//1/2020..oo acl- maiimmainn... 740. URL
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- lional Conference on Learning Representations (ICLR), 2021.
Rishabh Iyer, Ninad Khargoankar, Jeff Bilmes, and Himanshu Asanani. Submodular combinatorial information measures with applications in machine learning. In Vitaly Feldman, Katrina Ligett, and Sivan Sabato (eds.). Proceedings of the 32nd International Conference on Algorithmic Learn-
ing Theory, volume 132 of Proceedings of Machine Learning Research, pp. 722-754. PMLR, 16-19 Mar 2021. URL URL https////pppopeeee.......r.rrrrs
html
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LIm-blender: Ensembling large language models with pairwise ranking and generative fusion. (arXiv:2306.022) 022661.... June 2023. URL http://
arxiv. orw.o////aa. abs 2006. 02561 arXiv:2306.0556 [cs].
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training, 2021a. /taptps::///xiiiii org wwtp:/orggg//aaseaeeeoooo 00123

---

Published as a conference paper at ICLR 2025
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Generalization based data subset selection for efficient and robust learning, 2021b. URL https:
//arxiv. www.wvorrrggo.. abs/ 2012. 10630
Krishnateja Killamsetty, Alexandre V. Evfimievski, Tejaswini Pedapati, Kiran Kate, Lucian Popa, and Rishabh lyer. Milo: Model-agmostic subset selection framework for efficient model training and tuning, 2023.
URL httppp://arrivi.oorrrabb/////
13287
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thome, et al. Prometheus: Inducing fine-grained evalua- tion capability in language models. arXiv preprini arXiv:2310.0033
2023.
Chin-Yes Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74 81, Barcelona, Spain, July 2004. Association for Computational Linguis- tics. URL https: : / www.clanhhloogoo /ttp://aaacooooooocc.o//0//--111
10033
Liangxin Liu, Xuebo Liu, Derek F. Wong, Dongfang Li, Ziyi Wang, Baotian Hu, and Min Zhang. Se- lectit: Selective instruction tuning for large language models via uncertainty-awe self self-reflection, 2024. URL (tthttts::// /tatps://arxivv org/ abs/ 2402 16705
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoe V. Le, Barret Zoph, Jason Wei, and Adam Roberts. The flan collection: designing data and methods for effective instruction tuning. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR. org, 2023.
Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Pascale Fung, and Zhiguang Wang. Continual learning in task-oriented di- alogue systems. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen- tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing. pp. 7452-7467, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18666 11883661/2000 http://...mmmmmaia.... URI. emn wwwlemnnmmmmm...e p- n.. 590
/ttps://accaattii woeethooogy..... wwtporgg22222.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task general- ization via natural language crowdsourcing instructions. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3470-3487, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 1/2053///20222alllllloo
URL
https::/accaathholoryy www.tthhllooo..or//222222222 org/ acl-long..l.nn.... 244
George L. Nemhauser, Laurence A. Wolsey, and Marshall L. Fisher. An analysis of approximations for maximizing submodular set functions Mathematical Programming, 44:6552944, 1978.
URL https::/ http::///...ssaaaneeeeeeemma http://ccc.........oooooooorooaa
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Perey Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Pro- ceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp- 2383-2392 Austin, Texas, November 2016. Association for Computational Linguisties. doi: 10.18653/v//D16-11
hRtttt:s////aaaaiii wwwhohooo.....oo.oo.ooo... www.oo/ooo/o.1/..///o. 1264
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Techan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization, 2021.
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning, 2023. URL httpt::tt/tto.. ://ariiv../iii/aa orq/ abs/ 2206 14486
12

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL htt.t::://axxiv.og///e//
/202..2222.. 13971
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. Finetuned language models are zero-shot learners. Interna- tional Conference on Learming Representations, 2022. URL
net/
Corum?id=gue (=w.mm222dd0/F///000000
Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey, 2024. URL https://///a org/
abs/2402. 01344
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In International Conference on Machine Learning (ICML), 2024.
Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding, 2023.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov,
and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL htttps: /ttpa/awww../.bb/////33333
09990
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging iim-as-a-judge with mt-bench and chatbot arena, 2023.
13

---

Published as a conference paper at ICLR 2025
Appendix
A THEORETICAL FOUNDATIONS AND CONNECTIONS BETWEEN THE UTILITY METRIC AND INFORMATION THEORY
Theorem Let y: (yh1. Vi...... Wr) be sequence of tokens with ground truth distribution GT,, where GT, assigns probability I the sequence yt and all other sequences. Let plys x4)) be the predicted probability of y: given input Tú, and plys Fi.Ji Yi) be the predicted probability of yi given xa and an in-context example (xz. U5). Define the utility metric Fiz using general distance metric d(.,,) between probability distributions:
UFi = d(GTi, p(u xx)) d(GTi, p(1s Ti, (i,1))
Claim:
When the distance metric d(-..) -) is the Ballback-Leibler divergence DKI. the utility metric Fty is equal to the pointwise mutual information (PMI) between the sequence ys and the in-context example ( W5) conditioned on x4;
plys 24, Fg. Us)
U PMI(e 2z, x4)
log
ply: x24)
Furthermore, F.j be expressed the sum of conditional PMI over the tokens in y::
T
UF
Σ
PMI (Jiei 2j Uj 24s vi,,,),
t=1
where yi (yi1, M12,
1)) denotes the sequence of previous tokens up to position - 1.
Proof:
1. Computing KI--Divergence Between Ground Truth and Predicted Distributions:
Since GT; assigns probability to the specific sequence Yi, the KL-divergence simplifies as follows:
d(GT, p(us 1-)) DRr(GT; p(u 1.)) log -):
because the KL-divergence between one-hot distribution and any other distribution reduces to the negative probubibbiity of the assigned event.
2. Computing the Utility Metric Uig:
The utility metric becomes:
UF = log ply: .x;) + log p(y: 14,
p(u: It 23.
log
p( I
Expressing Fij as Pointwise Mutual Information: The conditional pointwise mutual information between ys and (xzz Uz) given æ; is defined as:
p(y., Fz yj x3s)
PMI( Zz. xx4) log
p(ui xx)) p( Mj xx)
Using the chain rule:
p(yi, Tyy My x3)) plye [a wz) p(a x..)
14

---

Published as a conference paper at ICLR 2025
Substituting back:
p(u Wis Bi M1 p 50 M1 xe))
PMT 2j,09 mi) = log
p(u: xz) p( Vi 24)
p(u
Ti, Bi, U5)
llog
plya | x3)
Therefore:
U.j = PMI(Us 2g. xz))
4. Expanding U Fij as Sum of Conditional PMI Terms:
We expand plys using the chain rule:
plus 1-) II
plya vi,c))s
where Vi,ct is the sequence of previous tokens up time 1. Substituting back into U Fij:
1== p(yie Wi Fj. Mis ce)
URg = log
i= P(va
at (6,
[og P(Vit
T4, Uj ca) - p(uu 24.06,ee)
Σ PMT (yas Ty, Uj Ti Vi,<t)-
This shows that Fij is the sum of the conditional PMI of each token yat with (Ir, U5) given x; and the previous tokens.
Conclusion:
When d(s..) DxI, the utility metric Ui precisely equals the conditional PMI between ya and (xgg W5) given rt.
A.1 WHY EUCLIDEAN DISTANCE IS PRAFERRED OVER KI--DIVERGENCE FOR SUBSET SELECTION
The effectiveness of subset selection algorithms, including facility location functions, depends crit- ically on the properties of the chosen distance metric d(::). Fuclidean distance offers several key advantages over KL-divergence for this purpose:
1. Mathematical Properties
Euclidean distance is non-negative, finite, and symmetric (d(a,b) d(b,a))
Ki-diveegeecce can be infinite or undefined with zero probabilities and lacks symmetry
(K Q) + (x || P)
2. Computational Advantages
Euclidean distance uses simple arithmetic operations (subtraction, squares, square roots)
KL-divergence requires more complex logarithmic calculations and division opera-
tions
15

---

Published as a conference paper at ICLR 2025
3. Robustness in Practice
Euclidean distance handles zero probabilities gracefully
-i-dieeree becomes undefined with zero probabilities, which occur frequently in real data
Impact on Subset Selection:
The facility location function requires positive, finite similarity mea- sures model coverage effects accurately. Euclidean distance satisfies these requirements, while Kl.-divergence's 8 potential negative or infinite values can disrupt optimization.
Conclusion: While KL-divergence offers theoretical connections to mutual information, Euclidean distance provides:
Guaranteed positive and finite utility metrics
Superior computational efficiency
Better numerical stability
These practical advantages make Euclidean distance the preferred choice for computing the utility metric U Fsy in subset selection algorithms.
B SUBSET SIZE COMPARISON
To assess how subset size influences the performance of DELIFT, we performed an ablation study by varying the subset size from 5% to 100% (specifically 5%, 15%, 30%, 50%, 100%) of the entire dataset across three distinct use cases. FigureZiilustrates the performance metric LAJ as a function of subset size for each fine-tuning scenario.
B. GENERAL OBSERVATIONS
Performance Increases with Subset Size: Across methods, LAJ scores generally im- prove the subset size increases. Utilizing the full dataset consistently yields the highest performance, underscoring the benefits a larger training set.
Diminishing Returns Beyond 50%: For methods such DELIFT, SelectIT, and LESS, performance gains plateau or slow down beyond 50% subset size. This suggests that additional data beyond this point offers minimal benefits and may introduce redundancy.
B.2
DETAILED ANALYSIS METHODS
B.2.1
INITIAL VS. RANDOM SELECTION
Consistently records the lowest scores across all subset sizes, indicating that models without data-informed selection struggle to generate quality responses.
Initial Baseline:
Random Selection: Slightly outperforms the Initial baseline but maintains a relatively flat performance curve. This lack of significant improvement highlights that uninformed data selection does not substantially enhance model quality.
B.2.2 SELECTIT AND LESS METHOOSS
LESS: Demonstrates a strong upward trend, particularly when subset sizes increase from 15% to 50%. This indicates that LESS effectively selects informative subsets, especially in the mid-range subset sizes, but sub-optimal with smaller subset sizes.
SelectIT:
Initially lags behind DELIFT and LESS but shows steady improvement with larger subset sizes. For subset sizes above 50%, SelectIT approaches the performance of DELIFT, suggesting its heuristic-driven selection becomes more effective with more data.
B.2.3 DELIFT VARIANTS
DELIFT DELIFT (SE): DELIFT consistently outperforms DELIFT (SE), which uses sentence embeddings, highlighting the superiority of DELIFT's baiee kemel in cap- turing data informativeness.
16

---

Published as a conference paper at ICLR 2025
DELIFT vs. Other Methods: DELIFT outperforms all other subset selection methods across all subset sizes, particularly below 50%. This effectiveness is attributed to DELIFT's strategy of identifying the most informative samples early on, making it ideal for scenarios with limited computational resources.
DELIFT vs. Full Data: At smaller subset sizes (e.g.. 15%, 30%), DELIFT achieves LAJ scores close to the Full Data baseline. In ICL fine-tuning scenarios, a 30% subset size with DELIFT nearly matches Full Data performance, demonstrating efficiency in data reduction without significant loss in performance.
B.3 IMPACT ON DIFFERENT TUNING SCENARIOS
QLoRA fine-tuning generally yields higher scores than ICL across all methods, suggesting that QLoRA benefits more from effective data selection strategies. DELIFT, in particular, shows more pronounced improvements in QLoRA settings, indicat- ing its subsets are well-suited for efficient parameter tuning.
ICL vs. QLoRA:
Use Case Comparisons: Use Case 3 (IBM and Government datasets), DELIFT achieves the highest gains relative the Initial baseline across both ICL and QLoRA scenarios. This effectiveness is likely due to the nature of query rewriting tasks, where DELIFT's informed data selection effectively eliminates redundant irrelevant examples, resulting in higher- quality training set.
C PROMETHEUS RUBRIC
The Prometheus model served as an LLM-as-a-Judge to evaluate response quality from different data selection methods. Table Tabltaan the general rubric used for the Prometheus model scoring on all use cases and settings (except for the experiments on the query-rewriting task using the IBM- proprietary data).
C.I
USAGE NOTES
Each response is evaluated independently based on the criteria above.
The cumulative score reflects the overall quality and effectiveness of the response.
Final LAJ scores obtained by averaging the scores across all criteria.
D LLM-AS-JUDGES SCORES
In Tables 11 and 12
we show the distribution of Prometheus scores on one particular setting: Use Case 1, MixInstruct training and MixInstruct validation These figures make clear that the average LGA scores computed in Tables 1-8 are true averages of distribution scores, not averages combination of just I's and 5's.
the Qwen2-72B-Instruct model.
D.1 INTERPRETATION SCORE DISTRIBUTIONS
D.1.1 OVERALL TRENDS
Score Variability: There is sigmificant variability score distributions across different methods. The Initial and Random baselines show concentration of scores between 2.5 and 3.5, indicating average to subpar performance.
Enhanced Performance with Advanced Methods: Methods like SelectIT, LESS, DELIFT (SE), and DELIFT exhibit score distributions skewed towards higher values (3.5 4.0), with DELIFT showing the highest concentration above 3.5. This highlights their effectiveness selecting informative and useful data for fine-tuning.
D.1.2 BETHOD-SPECTHE OBSERVATIONS
Initial and Random Methods: Both methods have lower mean scores (around 3.0 to 3.2) with wide spreads, suggesting inconsistent and generally lower-quality responses.
17

---

Published as a conference paper at ICLR 2025
Intal
RYta
33-
Sandom
DELIFT
DEL
PLB Data
11.
FJi Data
24
2.7
160
25
(a)
Schent 156
(b)
Suzrett Sbret 195)
3.6
352
Smeeii
3.-
Selectff
DELIT ICE
DEL (50)
2.4
DEL
Dusa
Date
20
00
300
23
1dc
(C)
Subest 754
(d)
Supuet Stoa 176)
3.1
32
Seleet
Banden
354
Furdom
SeleetT
2.4
20
40
60
200
80
20
120
(e)
Satoet
Figure 2: Graphs of LLM-A-J scores (y-axis) of Owwn2-72B-Instrrne with varying subset sizes (x- axis) of Use Case on MixInstruct for (a) ICL and (b) QLoRA, Use Case 2 on MixInstrue and MT-Bench for (c) ICI and (d) QLoRA, and Use Case 3 on IBM and Government for (e) ICL and
(f) QLoRA.
18

---

Published as a conference paper at ICLR 2025
Evaluate the model's ability follow instructions and deliver high-quality response across the following dimensions:
1. Instruction Following: How accurately and fully does the model adhere to the given instruc-
tion?
2. Accuracy: Is the information correct, reliable, and factually sound?
3. Relevance: Does the response directly address the question task without unnecessary
information?
4. Completeness: Does the response cover all essential aspects of the instruction question
5. Depth: How thoroughly does the response explore the topic? Does it demonstrate insightful
analysis where appropriate?
6. Clarity: Is the response well-organized, easy to follow, and free from ambiguity or confusion?
Creativity: Does the response offer original or innovative approaches where applicable?
8. Helpfulness: Does the response effectively meet the user's needs and provide value in solving
problem or addressing the query?
Score of 1: The response fails to meet expectations across most or all criteria. does not follow the instruction, contains significant errors misinformation, lacks relevance, is incomplete or shallow, unclear, unoriginal, and unhelpful.
Score of 2: "The response shows major deficiencies across several criteria. It partially follows the instruction but includes significant inaccuracies, is often irrelevant, incomplete, or lacks depth, clarity, creativity, and helpfulness.
Score of 3: "The response is average, meeting some but not all criteria. It follows the instruction but may fall short in terms accuracy, depth, relevance, or helpfulness. Improvements in clarity and insightfulness may be needed.
Score of The response is strong, performing well across most criteria. follows the instruc- tion closely, is mostly accurate and relevant, provides good depth, and is well-structured. Minor improvements could enhance clarity, creativity, or helpfulness.
Score of 5: "The response excels in all or nearly all criteria. It fully follows the instruction, is highly accurate, directly relevant, complete, and demonstrates depth and insight. The response is orgl-orga creative where appropriate, and very helpful in addressing the user's needs.
Table 10: General Prometheus Rubric
SelectIT and LESS Methods:
SelectIT:
Shows improved mean scores, especially in QLoRA settings, indicating its effectiveness in resource-constrained training scenarios.
- LEESS:
Demonstrates significant performance improvements, with mean scores around 3.26 to 3.28, reflecting effective gradient-based data selection.
DELIFT Variants:
DELIFT (SE):
Skews towards higher scores but not prominently DELIFT.
Achieves the highest average scores (3.35 for ICL and 3.37 for QLoRA), outperforming all other methods and indicating its superior utility-based kernel and submodular optimization.
DELIFT:
D.1.3 COMPARISON WITH FULL DATA
DELIFT vs. Full Data: DELIFT nearly matches Full Data performance with only a slight reduction in mean scores (3.35 3.37 3.45 to 3.51). This demonstrates DELIFT's capability to retain most of the model's performance while using significantly less data.
Efficcincy of Data Pruning: Full Data shows modest increase in to DELIFT, but at the cost of substantially higher computational resources. DELIFT offers efficient alternative without major sacrifices in performance.
compared
19

---

Published as a conference paper at ICLR 2025
ICL
QLoRA
1000
1002
100
602
202
2.0
2.5
4.0
3.5
3.0
3.2
355
4.5
Initial
Scome (meen
2.500
Scames (mamn
3,020
1002
1200
1202
1300
1003
892
400
1.0
1.5
2.5
2.5
3.0
3.5
4.0
4.5
1.0
3.0
3.5
2.5
2.5
3.5
3.5
4.0
4.5
5.0
Random
(mean
3.220
ScoTEs (mean 3.120
3:000
1400
3202
100
2.5
3.2
SelectIT
Table 11: LLM-as-Judges distributions for Use Case with MixInstruct training and vali- dation set on the Srcen-72B-Inntrrus model on the Initial, Random, and SelectIT baselines. The corresponding table is Table
20

---

Published as a conference paper at ICLR 2025
ICL
QLoRA
7002
1800
1403
1200
200
800
500
200
LESS
Scires 0mean 5.26)
S001e5 Incon 3.201
1200
1200
1003
Passeetyy
200
602
600
200
3.0
3.5
2.5
2.5
3,5
3.5
4.0
4.5
5.6
1.0
155
2.0
2.5
3.0
3.5
6.0
4.5
5.0
DELIFT (SE)
Scores Oriaan 3.280
Scarc Imeun 3 3.221
1200
1200
500
1.0
2.5
2.5
2.3
5.2
5.
4.0
5.5
1.0
2.9
25
3.0
4.0
4.
5.0
DELIFT
Scarcs
5.25)
Ineon
3752
1502
Preuuery
752
200
Full Data
Table 12: LLM-as- score distributions for Use Case 1 with MixInstruct training and validation set on the Qwen2-77BB-nstrree model on LESS, DELIFT with Sentence Embedding, DELIFT, and Full Data methods. The corresponding table is Table 1
21

---

Published as a conference paper at ICLR 2025
E
LIMITATIONS
Dependence on Initial Data Quality: DELIFT's effectiveness relies on the diversity and quality of the initial dataset. Biases or lack of diversity in the dataset can propagate to the selected subsets.
Scalability Constraints: While DELIFT is computationally efficient, extremely large datasets may still present challenges in terms of computation and memory.
Domain-Speciflc Performance: DELIFT's performance may vary across different do- mains, particularly those requiring specialized knowledge or handling multimodal data.
The subset selection process may unintentionally amplify ex- isting biases within the data, necessitating careful mitigation strategies.
Bias Amplification Risks:
F
FUTURE WORK
Integration with Data Augmentation: Combining DELIFT with data augmentation tech- niques could further enhance the robustness and diversity of selected subsets.
Fairness and Bias Mitigation: Incorporating fairness constraints and bias mitigation strategies into the subset selection process ensure equitable model performance across different groups
Extension to Multimodal Learning: Adapting DELIFT for multimodal data (e.g., text, images, audio) to expand its applicability beyond natural language processing.
Theoretical Analysis: Developing a deeper theoretical understanding of the utility metric and its properties further validate and refine the approach.
Enhancing Scalability: Exploring methods scale DELIFT effectively for larger datasets and more complex models without compromising efficiency.
Our ongoing efforts aim to ensure that DELIFT contributes to responsible and equitable Al devel- opment while maximizing efficiency.
G CODE AND DATA AVAILABILITY
To facilitate reproducibility and further research, we will make the DELIFT implementation and the datasets used in our experiments publicly available. Interested researchers can access these resources through the following repository: https//loooyouuusssuuuuccccchcch selection-OCI00
H HYPERPARAMETER SETTINGS
Consistent hyperparameter settings were maintained across all experiments to ensure reproducibil-
ity:
Submodular Function: Utilized Facility Location (FL), Facility Location Mutual Infor- mation (FLMI), or Facility Location Conditional Gain (FLCG) based on the use case.
Utility Metric Scaling Factor: Set = for FLMI and P for FLCG.
Budget (% of Data): Fixed 30% for all subset selection experiments.
Optimization Algorithm: Employed greedy maximization with stopping criterion based on the budget.
Distance Metric: Used length-normalized 1.2 norm.
Teacher Forcing Technique: Applied during utility metric computation to ensure reliable prediction accuracy measurement.
cOTT
Σ
p(y: (o)) -
(2)
1EB
22


Reviews:
[{"review_id": "tq4i1OA1f9", "reviewer": "Reviewer_gaDz", "summary": "This paper presents DELIFT, Data Efficient Language model Instruction Fine-Tuning, a method to select and prune fine-tuning data for three stages - initial instruction tuning, task-specific fine-tuning, and continual fine-tuning. One of the contributions of the paper is a utility-based kernel to quantify the importance of a data point in the presence of another data point used as an in-context learning (ICL) sample. The authors further use various submodular functions (Facility Location - FL, Facility Location Mutual Information - FLMI, and Facility Location Conditional Gain - FLCG) for the three fine-tuning stages defined above. The main core of DELIFT comprises of a greedy algorithm to iteratively select data points that provide the maximum marginal gain with respect to the chosen submodular function. DELIFT shows clear gains over various baselines for all three stages of fine-tuning using the Phi-3 (3.8B params) and Qwen-2 (72B params).", "questions": "Other than the questions listed in the weaknesses, here are few clarifying questions for the authors:\n\n1. For computing the metric in equation 2, are the probability vectors ($d$ dimension) from all predicted tokens (say $x$ tokens) concatenated to form a vector of dimension $x * d$ and then the equation 2 is applied?\n2. Minor typo in Figure 2 (should be LAJ scores instead of LJA scores).\n3. Can some other baselines be also included like [1] and [2].\n\n\n[1] DEFT-UCS: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection, Das et al., 2024.\n\n[2] Text Quality-Based Pruning for Efficient Training of Language Models, Sharma et al., 2024.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes a novel utility metric function to measure mutual information of data points and their relevance. The authors further highlight the benefit of this choice by comparing against sentence embedding based metrics. DELIFT performs better compared to DELIFT used with SE.\n- The authors build targeted modular functions for each stage (FL for standard IFT, FLMI for task-specific FT, and FLCG for continual FT), catering to the optimization objectives for these 3 stages.\n- DELIFT is able to retain and sometimes improve performance over the full fine-tuning by pruning out 70% of the full data.", "weaknesses": "- Since the authors use post-trained models like Phi-3-mini-128k-instruct and Qwen2-72B-Instruct as the starting point, aren't the three stages described and the followup experiments basically a part of continual fine-tuning? I would be interested to see the effect of DELIFT when starting from base models and see if the method is able to prune out fine-tuning data for IFT starting from scratch.\n- The authors present the results on ICL and QLoRA fine-tuning but not the full parameter fine-tuning. Are there diminishing gains with full model fine-tuning? I would like to see the results corresponding to those too.\n- What's the computational cost for the data selection algorithm since it involves doing forward passes over the language model and not use cheap embedding models to compute the probability distributions. How does it compare it with the fine-tuning stage involving QLoRA. \n- Weakness 1 is further enhanced when results for use case 2 are presented, where fine-tuning actually leads to drop in performance on MMLU, where it doesn't make sense to do any task-specific fine-tuning at all. And I don't trust the results on MT-Bench since it's a very small dataset and the error bars would be huge. To mitigate this weakness, I would suggest the authors to try task-specific fine-tuning on say some reasoning dataset and then track the performance on benchmarks like MATH/GSM8K/ARC-Challenge etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oM0yudUNSP", "reviewer": "Reviewer_hvNC", "summary": "This paper addresses the challenge of selecting optimal data for fine-tuning large language models (LLMs). The authors propose a pairwise utility metric to assess the value of each sample based on the model's current performance and the relevance of other samples in the dataset. They introduce three variations of a facility location (FL) function to guide data selection at different fine-tuning stages, including (i) a vanilla FL function to prioritize diversity in general instruction fine-tuning, (ii) a mutual information-based FL function to select data that is diverse and relevant to a target task for task-specific fine-tuning, and (iii) a conditional gain-based FL function to select data that complements previously used data for continual fine-tuning. Experiments on two instructional models, Phi-3-mini and Qwen2-72B, demonstrate that the proposed methods outperform existing data selection techniques across all three fine-tuning stages.", "questions": "- In equation 3-5, what is $s_ij$? Is it the utility-based kernel $UF_ij$? Is it recomputed after each selection round? \n- For Equation 1, how is the ground truth distribution determined? The description of \"a vector of ones for each token to signify perfect predictions\" needs further clarification.\n- The latex template of the submitted manuscript appears modified, as it lacks line numbers in the left margin.\n- Why are instruction-finetuned models used for the experiments rather than pre-trained models, especially given that the first use case is instruction fine-tuning?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Data selection for LLM fine-tuning is a timely research problem, particularly in light of the growing model sizes, data availability, and resource constraints.\n- The paper discusses the varying criteria for data selection across fine-tuning stages. The proposed method can be customized for different fine-tuning objectives.", "weaknesses": "- The utility metric in Equation 1 is calculated based on in-context learning (ICL) gains, which may not directly translate to improvements in fine-tuning performance. Is there supporting literature to justify the use of ICL-derived utility in this context?\n- It remains unclear how the utility gain measured through ICL reflects sample diversity in the semantic space. Since utility values are computed only once at the beginning of the data selection process, they may not capture changes in the model's performance after fine-tuning on the current selected subset.\n- The experimental design would benefit from comparisons with additional diversity-based baselines (e.g., clustering). Analysis of the selected examples across different methods would provide insights into the strengths and weaknesses of each approach.\n- A more thorough ablation analysis on the FL functions is needed. Do the additional terms for each fine-tuning stage significantly contribute to the results? Would the vanilla FL perform adequately across all stages? Evaluating this would help clarify the need for distinct data selection strategies at each fine-tuning stage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4NW9X3RtYD", "reviewer": "Reviewer_QaUn", "summary": "In this paper, the authors propose a data selection method named DELIFT. DELIFT first employs a pairwise utility metric to quantify how beneficial a data sample is for improving the model's responses to other samples. Basing on the pairwise utility metric, DELIFT leverages 3 specially designed submodular functions to conduct diverse subsets that are useful across the instruction tuning, task-specific fine-tuning and continual fine-tuning. Experiments across various tasks and model scales demonstrate that DELIFT can reduce the fine-tuning data size by up to 70% without compromising performance, offering significant computational savings and outperforming existing methods in both efficiency and efficacy.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. **Effective performance of DELIFT** In this paper, the authors design a utility-based kernel to quantify the informativeness of data. Based on this, the authors further design submodular functions for instruction data selection, targeting Instruction Tuning, Task-specific Fine-Tuning, and Continual Fine-Tuning. This method has been proven effective in all three stages and outperforms other model-dependent baselines.", "weaknesses": "1. **Related Work not Discussed** In Entropy Law [1], the authors also addressed the instruction data selection problem from the perspective of informativeness (entropy) and quantifies the relationship between data compression ratio and model performance; However, [1] is not discussed in this paper.\n\n[1] Entropy Law: The Story Behind Data Compression and LLM Performance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)