Train on Validation (ToV): Fast data selection with applications to fine-tuning
Authors: Ayush Jain
Andrea Montanari
Eren Sasoglu
Venue: ICLR 2026 Poster (2026)
Keywords: Data selection
Influence function
Instruction tuning
LLM
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Flip train/validation roles to efficiently find training examples that matter most for your target task.

Abstract: State-of-the-art machine learning often follows a two-stage process: $(i)$ pre-training on large, general-purpose datasets; $(ii)$ fine-tuning on task-specific data.  In fine-tuning, selecting training examples that closely reflect the target distribution is crucial. However, it is often the case that only a few samples are available from the target distribution. Existing data selection methods treat these target samples as a validation set and estimate the effect of adding or removing a single sample from the training pool by performing inference on the validation set.

We propose a simpler and faster alternative that inverts the usual role of train and validation: we perform inference on the training pool before and after fine-tuning on the validation set. We then select samples whose predictions change the most. Our key insight is that the training samples most affected by fine-tuning on a small validation set tend to be the most beneficial for reducing test loss on the target distribution. Experiments on instruction tuning and named entity recognition tasks show that, in most cases, our method achieves lower test log-loss than state-of-the-art approaches. We support our findings with theoretical analysis.

Published as a conference paper at ICLR 2026
TRAIN ON VALIDATION (ToV): FAST DATA SELECTION WITH APPLICATIONS TO FINE- TUNING
Ayush Jain"
Eren Sasoglu"
Google Research
Encord e eren. wwsogluggmaii..........o com
ayush. jain. 2508 @0a08@@mmall..... com
Andrea Montanari
Granica Computing Inc. (granica.ai) andrea penttaanri@graniic..m
ABSTRACT
State-of-the-art machine learning often follows two-stage process: (i) pre- training on large, general-purpose datasets; (ii) fine-tuning on task-specific data. In fine-tuning, selecting training examples that closely reflect the target distribu- tion is crucial. However, it is often the case that only a few samples available from the target distribution. Existing data selection methods treat these target samples validation set and estimate the effect of adding removing single sample from the training pool by performing inference on the validation set.
We propose a simpler and faster alternative that inverts the usual role of train and validation: we perform inference on the training pool before and after fine-tuning on the validation set. We then select samples whose predictions change the most. Our key insight that the training samples most affected by fine-tuning on small validation set tend to be the most beneficial for reducing test loss on the target distribution. Experiments on instruction tuning and named entity recognition tasks show that, in most cases, our method achieves lower test log-loss than state-of-the art approaches. We support our findings with theoretical analysis.
INTRODUCTION
While large language models (LLMs) pretrained on internet-scale datasets, their downstream performance can be heavily dependent on the instruction-tuningg stage in which they are fine-tuned on instruction/output pairs (Ouyang et al., 2022: Zhou et al., 2024; Longpre et al., 2023; Chung et al., 2024). These datasets are significantly smaller and are often gathered by using multiple het- erogeneous sources. Instruction tuning becomes even more difficult when targeting a specialized use case (Wang et al., 2023). More generally, scarcity of domain-specine data is a ubiquitous challenge when fine-tuning foundation models.
This paper presents an -osplee-m and low-complexity method for selecting training dataset of prescribed size from heterogeneous sources to maximize the test time performance on the target distribution. Our method is motivated by the theory of influence functions (van der Vaart, 2000) yet avoids the computational burden of computing influence functions. Unlike influence- function-based data selection methods, ToV avoids per-example gradients, Hessian-vecte prod- ucts, and backpropagation through training dynamics. It instead estimates example utility using only forward loss evaluations before and after small gradient step on the validation data. We validate this approach on two token-basee learning tasks, instruction tuning and named entity recog- nition (NER), and show that in most cases outperforms state-of-the-ar data selection baselines. To illustrate its broad applicability, we show that it yields interesting results even for a simple logistic regression example (see Appendix D).
Work done while at Granica Computing Inc.
I Department of Statisties and Department of Mathematics, Stanford University

---

Published as a conference paper at ICLR 2026
To formalize the problem, assume access to two datasets: small dataset from the target distribution on and a larger one from possibly heterogeneous data sources. We refer to the dataset from the mmg! where 2!ah are i..d. samples from the target
target as the 'validation set' Z : (2]
distribution P., and the larger heterogeneous dataset as 'training pool' X (2 1..... @N), where
æ; E Z. In general the distribution of the training pool differs from IP. Our goal is to minimize the test error on the target distribution with respect to the model parameters RR:
R(O) == =[)(
(1)
where RR 2 + R is a loss function. A separate target-distribution test set nd (separate from z is used to estimate R(O) after fine-tuning.
We aim achieve this by training (or fine-tuning) the model on subset S S [N] of the training pool, e.g. running stochastic gradient descent (SGD) with respect to the empirical risk:
Rs(0))
SS| Σ (()
Let Og be the outcome of rumning SGD (or any specific training algorithm) (O) We want to select the subset S (given a constraint on its size |S]) so that Og achieves a small test loss on the target distribution, i.e. as to minimize R(Os).
1.1 TRAIN ON VALIDATION: MOTIVATION AND ALGORITHM
To select the most helpful examples at model 0 might score training examples by the decrease in validation loss induced by a single gradient step with respect to that example, then select those with the highest scores. Computing these scores directly requires N+1 full evaluations the validation set. We derive an efficient approximation to these scores.
Consider single gradient step with respect training example æ::
02 = 0 ((0,
(3)
The corresponding change in loss for a validation example z, {(0,z) E(02z, z), can be approxi- mated by a first-order Taylor expansion:
f(0. 2) l(02: 2) ~ - (VE((, 02 0) ny (V((0, v((O.z)).
where the last step follows from Eq. (3). Pruthi et al. (2020) approximate the scores by computing gradients for each training and validation example and taking their dot products; Xia al. (2024) extend this to token-based learning. Our method diverges from these approaches: it requires no per-example gradients.
Note the right-hand side is symmetric in and In other words, the decrease in loss from a step on is mirrored by the decrease in loss on æ from a step on z. Our method exploits this vali-vaaidation symmetry. The change in overall validation loss for single gradient step with respect to x is:
Maal
(1(0,2))-((0))>
e((02, (()0)=2))
2,)
Myal
Miaal
On the other hand, performing a batch gradient step at 0 with respect to the validation set gives Ex Ve(A, 22). Combining this with Eq. (5), we get
Oz vaa
O 17 meal
1 (((02))) ((0,,z))((mm: E(Om: 21)) (0- 00m ve(0,z)) R 1(0,x) E(O (00 x).
(6)
Mhal
i=1
In other words, the change in average validation loss from training on x can be approximated by the change in loss on after training the validation set n
The approximation in Eq. (6) follows directly from a first-order Taylor expansion and therefore relies only the update induced by the validation step being small and the loss landscape being locally
2

---

Published as a conference paper at ICLR 2026
Algorithm ToV Scoring Algorithm: Interleaved ToV.
1: Input: Pretrained model On. validation set Z training pool X (ar; i [M]), epochs L,
2:
selected data count ni, learning-rate schedule {y7 [m=)-=1. base model count m, & E [0, [N] of size
3: Output: Set of examples
4: Sample base subset U _ [M] of size m randomly; define Xu = ( U)
5: Initialize model: 0 < Oo: set scores Φ; 0 for all E [N]]U
6: for k = 1 to L do
7: Train 0% on Xu for one epoch with learning rate Ths obtain 0
8: Train 0g for one epoch on nd with a learning rate εмYk to obtain Orall
9:
for each [N]] \ U do
(k) + F (0x) xii) E(OR;; ax)) (see Section 2.1 for the definition of F*)
10:
11: Φi 4 Oi + of (k) /I.
12:
end for
end for
14:
Return set S S [N \ VU of size on the basis of scores di (see text)
smooth. If the update is too large or the landscape is highly non-linear, the approximation may degrade. In practice, we enforce this regime by using validation learning rate that is smaller than the base learning rate. No other assumptions, such as independence between training and validation samples, are required.
Our main objective is to evaluate the left-hand side of Eq. (6) for all x the training set. The right- hand side provides a far more efficient route: (i) Compute the loss {(O, x) for all training examples; (ii) fine-tune 0 on the validation set to obtain 2 z (iti) re-evaluate the new loss { zm), x) at each training sample æ, and approximate the effect of training æ by computing the difference with the loss at point (i).
epoch of training on the validation set and two evaluations over the training pool, as opposed to N evaluations the validation loss as suggested by direct evaluation of the left-hand side of Eq. (6), and it does not require access per-example gradients.
This requires
In the next sections, we use this idea to obtain a selection algorithm that alternates training on a subset of the training and on the validation set. A specific implementation, which we refer as Interleaved ToV (Method A), is given Algorithm 1; a slightly different implementation, Parallel ToV (Method B), is given in Algorithm 2. In Method A, we start with a small random subset U U [N] of the training pool. We train on U for epochs, resulting in models @TC ... or.. For each epoch l E [T.] we fine-tune 0 for one epoch on the validation set, resulting in models 6:. For each epoch, every remaining training example æ; with i E [N]] U scored by the change in its loss between 0% and 0re.. and scores are averaged across epochs.
The names reflect the mechanics: Interleaved ToV interleaves base-set training with single valida- tion update at each checkpoint, whereas Parallel ToV runs two training trajectories in parallel, only one of which is updated on the validation set (see Section 3).
After computing scores de as in Algorithm 1, we select S using one of two strategies: (i) choose the n examples with the largest de: (1i) choose half from the highest-scoring examples and the other half uniformly at random from U to increase diversity.
Intuitively, large di means that small amount of training on the target distribution produces large change in the model output at x; Our working assumption, motivated by the heuristics above and formalized in Section 3, is that the converse also holds: training on I; will produce large change in model output on the target distribution. Hence the scores Φi can be used to select 'important' samples for the target.
An adaptation for token-based learning is described in Section 2, along with empirical results. Sec- tion 3 provides mathematical justification that formalizes the argument above.
3

---

Published as a conference paper at ICLR 2026
1.2 RELATED WORK
Our work relates data selection and data attribution. The impact of a single example on the validation error can be approximated by first-order Taylor expansion. This idea results in data selection methods based on influence functions (Wang et al., 2018; 2020; Ai al., 2021; Kolossov et al., 2024). Classical influence functions estimate the effect of single example on the empirical risk minimizer. Most closely related to our work are Pruthi et al. (2020); Bac et al. (2024); Xia et al. (2024), which instead estimate the influence of example on the training dynamics. In particular, Bae et al. (2024) shows how to approximately propagate gradient changes at k-th epoch through all subsequent epochs. In contrast, Pruthi et al. (2020); Xia et al. (2024) make a crude approximation for this propagation. Limitations of influence-
methods are discussed in Schioppa et al. (2023).
Related ideas also were investigated in Liu et al. (2018), which however computes hypergradients for architecture search, rather than deriving per-example data-selection scores as we do. Concurrent work (Savani et al., 2025) also exploits related setting (token-level antidistillation) rather than per-example data selection.
symmetry, but in different
The recent work of Xia al. (2024) proposes LESS, data selection method for instruction tuning that adapts influence ideas to Adam and long sequences. In particular, these authors emphasize the challenge of computing and storing gradients to compute influences. They address this problem via random projections and low-rank approximation. Engstrom et al. (2024) apply the datamodel framework (Ilyas al., 2022; Park et al., 2023) to select pretraining data. Separately, replay algorithm that stores only logarithmic number of checkpoints is proposed in Engstrom et al. (2025). Methods that align training data distributions a small target set include TSDS (Liu et al., 2024) and DSIR (Xie et al., 2023); Nomain/task-adaptin pretraining also improves transfer (Gururangan et al., 2020). Broader LLM data-efficiency work proposes LLM LLMgguided quality scoring (Ask-LLM) and density sampling (Sachdeva et al., 2024), and clustering- guarantees (Axiotis et al., 2024). Finally, Data Filtering Networks (DFN) also leverage a held-out, high-quality set, but with different goal and setup (Fang al., 2023).
sensitivity sampling with provable
Our contribution differs by (i) inverting train/validation roles to approximate per-example influence using only forward losses and doesn't 't require per-example gradients, or Hessian vector products, and (fx) showing that this simple, symmetry-based score is computationally inexpensive and outper- forms recent data selection approaches for instruction tuning and NER.
2 DATA SELECTION FOR TOKEN-BASED LEARNING
In this section we describe our implementation of the general idea described in the introduction for token-based learning and present empirical results demonstrating its effectiveness. Since pre- diction takes place at the token level, while data selection takes place at the example level (e.g.. instruction/output pair), we compute token scores and aggregate them as described in Section 2.1. Section 2.2 gives brief overview of instruction-tunit introduced in Section 2.3. Empirical results are presented Sections 2.4 and 2.5.
and NER tasks. Experimental settings are
2.1 SCORE COMPUTATION FOR TOKEN-BASED LEARNING
Each example z consists of an input t and an output n both of which are strings and may differ in length. Let z0 denote the output vocabulary, and let T(z) denote the length of the output string ="" which we write as = = (1) 2 (2),
(T(z))).
Given a model parameterized by 0, its prediction on example a sequence of T(z) conditional distributions, [pr(- 2, 0) T(*) ))) where each pt 0) denotes the model's predictive distribution over the output token at position 1. Note that (- | 2, 0) depends on z solely through z and 20 (1)...., 2n 1). We train models using the log-loss
T(*)
((0.2)
Σ
log Pe (t)
[20]22000)) 2)
T(2)
4

---

Published as a conference paper at ICLR 2026
To compare two models, 0 and 0', on example define a per-token difference of log-loss
Pt(2" (3 (t) 2: 8¹)
(+ 0,0'
log
pi 2m (t) 2; 0)
Since our setting involves selecting entire examples rather than individual tokens, we aggregate the per-token differences into single score per example. Specifically, we apply transformation function F R to each A, before averaging across positions. The final score for example is:
T(x)
6(z;0,0')
1 T(z)
Σ F(Ar(z:0.0)).
(9)
1=1
We consider three instantiations of the function F, leading to three different scoring methods:
MAXIMUM-IMPROVEMENT F(y) y emphasizes raw improvement.
MAXIMUM-ABSOO.UT CHANGE: Fly) = |v| captures the magnitude of change.
MAXIMUM- IMPROVEMENT: Flv) = max (v. 0} ignores degradations.
The algorithm therefore the same in Algorithm with the adaptation o; (k) Φ( 0. 0%
Given budget of n examples, we choose S S [M] \ U, |8| = using one of these rules:
SCORE-ONLY: Choose the examples [N] U that have the largest score di.
SCORE+ Choose the n/2 examples i & [N] U that have the largest score oi and add n/2 more examples chosen uniformly random (without replacement) from U.
Our scoring schemes tend to favor shorter examples due to their higher variance, which arises from having fewer tokens. To mitigate this bias, we partition the set [N]] U into 10 bins based sequence length, ensuring each bin contains an equal number of examples. We then select an equal number of top-scoring examples from each bin.
After selecting S of size |S| scheme. We refer to this stage as final training.
we train (or fine tune) a model on S to evaluate the selection
We compare our schemes with three baselines:
RANDOM: The set S selected uniformly random subject its size.
MAXIMUM UNCERTAINTY: Instead of the scores we defined, we use the following hardness score:
T Σ
3/ = T. 1=1
log (p((2)
Pe (() 0))),
(10)
This score extends the method of Ting & Brochu (2018); Wang et al. (2018); Ai et al. (2021);
Kolossov et al. (2024) to token-based learning.
LESS:
We used the publicly available implementation from Xia et al. (2024); see Appendix A.I.
2.2 PREDICTION TASKS
We evaluate our data selection framework in two distinct token-based tasks: instruction tuning (IT) and named entity recognition (NER). The framework we introduced above captures both tasks:
Instruction Tuning (IT) involves training a language model to follow natural language instructions. Each training example consists of: Input "": user instruction prompt: Output n"!:; the desired model response.
The output is typically multi-token and highly variable in content and length, depending on the instruction. The model learns generate n conditioned on 2 This naturally fits our framework, which models predictions as token-level distributions (- 2, 0)-
Named Entity Recognition (NER) is sequence labeling task where the model assigns probabil- ity distribution over entity tags (c.g., PERSON, ORGANIZATION, ...) to each token. In this case: Input z": tokenized sentence; Output z0l, sequence of entity labels, aligned with the input.

---

Published as a conference paper at ICLR 2026
In NER. predictions computed as token-wise classification distributions and therefore output is of the same length as input sequence. In this case, as a base model we take a pretrained language model and replace its prediction head with classification head.
2.3 EXPERIMENTAL SETTING
In all of our experiments the training set consisted of N = 36 x 1024 samples. For the base model training, we used [U] size is ma = 10, 000. We vary the selected set size n E {1, 4, 8} x 1024.
1024 samples. The validation set size maat 1024 and the test set
Number of epochs. Both for surrogate model training and final model training we determine the number of epochs by L = (16 x 1024)/m... We use a batch size of 16 whence the above ensures that the number batches used in training remains constant, and equal to 1024. other words, all experiments in this section at constant compute. Since base model training uses |U] 4 x 1024 samples, the number of epochs is L =
Learning rate. The learning rate for both surrogate and final model training is selected using hyper- parameter optimization for each selected set size n. The learning-rate optimization was carried out for random data selection hence placing our approach
disadvantage.
We use linear learning rate scheduler and LoRA training Hu et al. (2022) with LoRA parameters 32 and dropout 0.2. For NER experiments, we used PEFTrank 1 and for instruction tuning experiments, we used PEFTrank 256. The learning rate for the validation examples is E 1/10 of the one for the base examples. We present here results with RANDEM and refer to the appendix for SCORE-ONLY.
2.4 EXPERIMENTS FOR INSTRUCTION TUNING
For these experiments we used 3 different datasets, which we will refer to as o == {Slmm Orca, Alpaca GPT-4, Alpaca GPT-3.5). As the foundation model, details of the model and datasets used are provided in the Appendix.
Meta-LJama---SB. Additional
We designed five experimental setups. In each experiment, dataset from y is selected as the targel distribution. We randomly sample validation and test sets, Z and zn, without replacement from the target dataset. These samples formed by randomly sampling an equal number of examples from one more datasets in y (ex- cluding the validation and test samples), such that the total number of selected training samples is fixed at N. We denote by P C. the datasets used to generate the training pool. The choices of the target dataset and of P. for each of the five experiments are summarized in Table All reported re- sults are averaged over 10 independent runs. In each run, we freshly sample the training, validation, and test sets. These experiments are designed to evaluate performance across a range of data con- figurations. In particular: in Experiments and 4, the training set includes samples from both target distribution and other distributions; in Experiments and 5, the training set includes samples only from non target distributions; in Experiment 3, it includes only samples from the target distribution.
excluded from further use. The training pool is then
Figure I summarizes our results for instruction tuning for a fixed select size n = 8 x 1024. We plot the improvement in test log-loss over random data selection for several data-selection strategies within the general framework described in Section 2.1, using Interleaved ToV scoring in algorithm for scoring the examples and SCORE+RANDOM for selecting. We observe that the proposed strate- gies yield significantly better instruction tuning than random data selection or selecting by max- uncertainty. We observe improvement (albeit small one) even when both train and validation data are from Slim Orca (Exp 3), which is case in which random selection should perform well. The proposed strategies also yield a significant improvement over LESS (Xia et al., 2024), with the exception of Experiment 2 in which LESS performs slightly better.
Figure displays the evolution of test log loss with selected sample size n. We observe that good choice of the data selection method results in model improvements that can be equivalent to or larger than doubling n. Plots show standard error (with scaling factor 1) for 10 runs.
In NER, typically token level probabilities are combined to assign labels a whole word.

---

Published as a conference paper at ICLR 2026
Table 2: Summary of named entity recognition experiments. Abbreviations: MN = Multinerd, A4p Ai4p, C4 C4, SB Syn-big.
Table Summary of instruction tuning exper- iments. Abbreviations: SO Slim Orca, A4
Alpaca GPT-4, A3.5 Alpaca GPT-3.5.
Exp
Target
Training pool
Exp
Target
Training pool
MN
MN, A4p, C4, and SB
SO
SO, A4, and A3.5
2
MN
A4p, C4. and SB
SO
A4 and A3.5
3
MN
MN
SO
SO
A4p
MN, A4p, C4, and SB
A4
SO, A4, and A3.5
5
A4p
MN, C4, and SB
A4
SO and A3.5
6
A4p
A4p
56 improvemeal Peeveeeaatt in log loss over random
Comparison
rxp 2
epp
7.5
oxp
5.0
2.5
-2.5
-5.0
-7.5
max abs cag
max pos Improv
max Improv selection method
max uncert
Jess
Figure 1: Test log-loss improvement (%) over random selection for instruction tuning with 8x 1024 samples. Each group of bars represents a data data-selection strategy (maximum couuuumm--nnetitttyy and LESS as baselines): colors show turget/training pool configuration (Table 1). Results use Interleaved ToV scoring (Algorithm 1) with the SCORE+RANDON selection strategy.
2.5 EXPERIMENTS FOR NAMED ENTITY RECOGNITION
The task is to classify whether a token is part person name or not. For these experiments we used 4 different labeled datasets, which will refer to / { Multinerd, Ai4p, C4, Syn-big). We use xlm-roberta-base as the foundation model. Further details on the experiment, model and datasets used are presented in the Appendix.
We conducted six sets of experiments. As for the case of instruction tuning, for each set of exper- iments, we select one of the datasets S defining the target distribution, and one or more other datasets to define the training pool (denoted by P..). The choices of target datasets and V are sum- marized in Table The construction train, test and validation sets same in instruction tuning.
Figure 3 summarizes our experiments with NER. We plot the improvement in test log-loss over random data selection for several score definitions. Throughout these experiments, we use SCORE+RANDS ANDOM. We observe that the strategies of Section 2.1 yield systematic improvements over random data-selection. Unlike instruction tuning, maximum uncertainty also improves per- formance in most settings; however, ToV achieves the largest gains. In this case, LESS (Xia et al., 2024) does not improve over random selection.
We also evaluated token-level Fl on the NER task and observed trends consistent with the log-loss improvements across selection methods (App. A.4).
2.6 RUNTIME AND MEMORY.
We compared the computational cost of ToV and LESS under identical hardware and software set- tings (Appendix K.3) using the official LESS implementation. We report wall-clock runtime and

---

Published as a conference paper at ICLR 2026
exp
exp 2
exp 4
exp
motled max ong
1.20
cos ingrew
0872
irprov
0862
1.100
1055
1.075
1og
0.002
0.34
0.755
1.002
0.723
2002
1050
5200
2000
2002
A000
$000
6002
2000
4602
5000
0002
2000
4002
G3CO
1000
Select Count
Select Count
Select Count
Select Count
Figure 2: Test log-loss vs. number of selected samples n for instruction tuning. (Due to space limits, Exp. 3 plot in the Appendix.) Lines show mean log-loss over 10 runs; error bars are =1 standard error Results use Interleaved ToV scoring with the SCORE+RANDOM strategy.
% improvemenn log lass over randon
New Methods
Compatison
exp
30-
oxp 2
20
etp
oxp
20-
-30
mas abs cag
max pos improv
max improv selection method
IDEK uncert
Less
Figure 3: Test log-loss improvement (%) relative random selection for NER at 8x 1024. Each group of bars represents data-sclection strategy; colors show target/training pool configuration (Table 2). Results use Interleaved ToV scoring (Algorithm 1) with the SCORE+RANN RANDOM strategy
disk storage required by each method (e.g., checkpoints and auxiliary tensors), Reported values are averages over five runs per task. Across both instruction tuning and NER, ToV substantially faster and more storage-efficient. Results are summarized in Table 3.
Table 3: Runtime and disk storage for ToV vs. LESS under identical hardware (mean of 5 runs).
Setting
Method
Runtime
Storage
Instruction tuning
LESS
4h 5m
4.9 GB
Instruction tuning
ToV
2h 3m
1.84 GB
NER
LESS
46m
4.1 GB
NER
ToV
8m
0.24 GB
ToV reduces runtime by 2-6x and disk storage by 2.5-16x relative LESS across the evaluated
tasks.
3 A FORMAL JUSTIFICATION
In this section, we present a mathematical analysis of our approach in the case of batch gradient descent (GD). We focus the implementation Parallel ToV (Method B), described in Algorithm 2.
Parallel ToV differs from Interleaved ToV because at each training cycle k. training on the base set X is initialized with the output of the previous aain-on-validation phase. Empirically Interleaved
8

---

Published as a conference paper at ICLR 2026
Algorithm 2 ToV Scoring Algorithm: Parallel ToV
1: Input: Pretrained model On. validation set Z training pool X (#;; E [N]),
2:
selected data count n S N, base model count m
3: Output: Set of examples
[N] of size
4: Sample base subset U - [M] of size m randomly; define Xu = (0; U)
5: Initialize models: O f- Oo. 0g Oo: set scores T, +- 0 for all E [N]
6: for k = to L do
7: Train for one epoch on Xy with learn. rate The and init. 0, Denote the output by 0
8: Train for one epoch on Zd with learn. rate Th and init. 0u. Denote the output by 0%
9: Train for one epoch on Xy with learn. rate The and init. Og - Denote the output by 0
10:
for each E [N]] U do
r - m(0m:: x2.) ((0
11:
12: + + r
end for
14:
end for
15: Select S C [N] U with size |S| = n using scores Y;
ToV performs somewhat better than Parallel ToV, see Appendix C. We use Parallel ToV for analysis just because the resulting mathematical expressions simpler.
We find empirically that the ToV works well beyond token-based learning, and hence our focus will be understand it in generic learning problem. Appendix D demonstrates this point by considering a simple logistic regression problem.
3.1 IDEAL SCORES, LINEARIZATION, INFLUENCE FUNCTIONS
In order estimate the model improvement produced by sample E [N]] U we could train model on two training sets Xy and XUus, using empirical risk functions (O), Ruvu(O). We thus would run GD for L steps, with initialization 0:z- = Ouu+. = 0g:
0: mm Ry (0 @ 0 17(m + R
(11)
At iteration we have thus two models @ and 0 that differ uniquely whether sample used or not. We define the ideal score to be the difference in validation error between these two models, averaged over epochs
1 Σ 1 S: :
1 ( 8=1
L maal
(((0".) (hastti ;2, " (12)
Evaluating this score is computationally expensive, hence several groups (Pruthi et al., 2020; Bae et al., 2024; Xia et al., 2024) proposed to use first order Taylor expansion to approximate the difference between the two models. Expanding S, with respect to the contribution of ((;;,.) yields
S
7
Σ (Q M, v((0.";;
(13)
OSACIEL
where Mct = Ia and M.r captures the propagation of perturbations along the GD trajectory:
M == H H
H := rmm 2 Ry (0th)
(14)
The next result shows that Sh approximates well S; in quantitative way, under local convexity.
Proposition 1. Assume there exist co. C1, M > 0 such that V Ru > co || ( xx)) <
I M| - 02ll2. [[;(0 vi(02: æ:) < M] 02112- Further assume that V (rr S C1 and V R. (0))
C₁ for all and, for all 0,, Oz (² - V
2 (2)llon < M| O2l12 for all 0,, O2 well. Finally, assume there exists a constant C
such that S C,/m Vk. Then there exists C' = C(co. C1. Cyp M) such that
|s.
S!mn < C/m².
(15)
9

---

Published as a conference paper at ICLR 2026
The assumption 7 Cz/m is justified by the fact that we expect the Hessian of ( to be of order one, and hence the stepsize for this objective (which is given by rm see Eq. (11)) should be of order one. As shown in the proof, the typical size of S! is of order 1/m, and hence Eq. (15) establishes that the difference |S Slin is negligible.
We emphasize that Proposition does not make any assumption about the data distribution and instead entirely relies on regularity properties of the risk function.
3.2 TRAIN-VALIDATIO DUALITY
We consider Interleaved ToV and Parallel ToV defined in Algorithms 2. We emphasize the de- pendence on by writing oi (c(c) and Y; = Y,(e). It is easy to derive the small asymptotics 6 6 + o(E), T,- T. olE), where, for Is.s == VE(O,";; 22).
8 7l
Ri g..a). 99..))
!! 17Ml
Σ
(Vima(0 M s, 19-s). (16)
L
L
0SICeSL
We show that these are good approximations of T,(:), (i(E) uniformly in dimension, sample size. Theorem 1. Consider Algorithms I, 2 with fixed stepsize Mi = 1 (and F(x) = - in Algorithm 1). Under the assumptions of Proposition I, further assume I Rual (0m)) S Ci for all k, and [[ x))lop S C2. Then there exist C. c.,coo M, C1), C = C(co, M, O2) such that, for em m I Ca.
(.(e) - r S (Emma/m)) ((x(e)
T. S ((emaa/
(17)
Note that T. differ from S. because of: (i) The different order of s and t: (ii) The fact that Mr is replaced by its transpose in Eq. (16). I measures the influence of training on validation data when making inference at æs, while S measures the influence of training on æ; data when making inference on validation. These two measures of 'influence' differ by the replacement of Mr by M s.t. However, in a number of cases we expect these two matrices to be not too different, and hence the two scores to yield similar results. We can prove that Y!i and SILm coincide (for large L) under local convexity conditions.
Theorem 2. Assume 0 + {(;; x) to be twice continuously differentiable and that || Rrnal )| <
k
C1 || V æz)ll I C1 for all k. Further assume that gradient descent iterates (@tm. k 2 0)
converge to Oee lim 0% which is local minimum of Rv)(0) with Qoo V2 Rey (ter)
>
(strictly). Then
1 1 _T = lim 5ThH (Vina (0Th) xc) Q V 0 ax:)
lim L-000 Mya
(18)
8,00
L
The last expression in Eq. (18) (denoted by S is the classical formula for influence functions of M-estimators (van der Vaart, 2000). Both our approach and the dynamical influence function Sfi(() (L)
can be regarded as approximations of Shin in this case.
In fine tuning, the model likely be overparametrized, and is unrealistic to assume convergence to strict minimum (with 2 2aRg (0 > 0). On the other hand, the weights will not change signifi- cantly during this phase and it is reasonable to approximate fine-tuning as fitting an overparametrized linear model with respect to the empirical neural tangent features learnt in the pre-training phase.
Theorem 3. Consider the loss function ((0:x) (y(x) - (xb(x), 0) /2 for some response vari- ables y(x)), and featurization map ub RRd + RP, p > m. Let V E R U be the matrix with rows (1b(2))) j E U), V E RR XP be the matrix with rows (+b(2)a)) j S m.a)). Ps the projector to the kernel of V. y (v(2r) : E U), θ := viy, yu = (v(2ze)) (0, vb(2)))) ; S mmaa), r(1) = y(x:) (0, ((x))) If GD is initialized with 0, O. and use constant stepsize
n < | /2, then
lim L Liiza
Theh
lim L
1 Shin (L) STTn
17 r1)(TT v Pvb(e))
(19)
L
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We are grateful to Neeraja Abhyankar, Alankrita Bhatt, Joseph Gardi, Mukur Gupta, Germain Kolossov, Mare Laugharn, Rahul Ponnala, Sahasrajit Sarmasarkar, Andreas Santucci and Pulkit Tandon, for several conversations about this work.
REFERENCES
Mingyao Ai, Jun Yu, Huiming Zhang, and Ying Wang. Optimal subsampling algorithms for big data regressions. Statistica Sinica, 3122)749-772, 2021.
Al@Meta. Llama model card. 2024. URL https://gittub..om wwwp:/ttttt.....tttootteooaoata. /lama/
1lama haama///llbba htw.o///aai//OELLL..... CARD md.
Kyriakos Axiotis, Vincent Cohen-Addad, Monika Henzinger, Sammy Jerome, Vahab Mirrokni, David Saulpie, David Woodruff, and Michael Wunder. Data-efficient learning via clustering- based sensitivity sampling: Foundation models and beyond. arXiv preprint urXiv:2402.17327 2024.
Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger Grosse. Training data attribution via approximate unrolled differentation. arXiv:2405.12186, 2024.
Peter L Bartlett, Andrea Montanari, and Alexander Rakhlin. Deep learning: statistical viewpoint.
Acta numerica, 30:87-201, 2021.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. Scaling instructionffinttun lan- guage models. Journal of Machine Learning Research, 25(70): 2024.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. CoRR, abs/1911.02216, 2019. URL
http://arxiv org/ abs /19////9911 19111. 02116.
Logan Engstrom, Axel Feldmann, and Aleksander Madry. Dsdm: Model-aware dataset selection with datamodels. arXiv preprint arXiv:2401.12926, 2024.
Logan Engstrom, Andrew Ilyas, Benjamin Chen, Axel Feldmann, William Moses, and Aleksander Madry. Optimizing ml training with metagradient descent. arXiv preprint 1335, 2025.
Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks. arXiv preprint arXiv:2309.14225 2023.
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. Don't stop pretraining: Adapt language models domains and tasks. arXiv preprint arXiv:2004. 10964, 2020.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, and Aleksander Madry. Data- models: Predicting predictions from training data. Proceedings of the 39th International Con- ference on Machine Learning, 2022.
Germain Kolossov, Andrea Montanari, and Pulkit Tandon. Towards statistical theory of data selection under weak supervision. In The Twelfth International Conference on Learning Repre- sentations, 2024.
Wing Lian, Guan Wang. Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". Slimorca: An open dataset gpt-4 augmented flan reasoning traces, with verifica- tion, 2023. URL https: htth/httt///hhttt/ ing co/ Opt00/ppnnn/aaeeeee imrrca.

---

Published as a conference paper at ICLR 2026
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv
preprint arXiv:1806.09000 09055, 2018.
Zifan Liu, Amin Karbasi, and Theodoros Rekatsinas. Tsds: Data selection for task -psc-iie model finetuning, Advances in Neural Information Processing Systems, 37:10117-10147, 2024.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In International Conference on Machine Learning, pp. 22631-22648. PMLR, 2023.
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orea: Progressive learning from complex explanation traces of gpt-4, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclere, and Aleksander Madry. Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303. 2023.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint (rXiv:2304.03277 2023.
Garima Pruthi, Frederick Liu, Satyen Kale. and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19920-19930, 2020.
Noveen Sachdeva, Benjamin Coleman, Wang-Chenp Kang, Jianmo Ni, Lichan Hong, Ed H Chi, James Caverlee, Julian McAuley, and Derek Zhiyuan Cheng. How to train data-efficient llms. arXiv preprint arXiv:2402.09666 2024.
Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Avi Schwarzschild, Alexander Robey, Mare Finzi, and J Zico Kolter. Antidistillation sampling, arXiv preprint arXiv:2504.13: 13146, 2025.
Andrea Schioppa, Katja Filippova, Ivan Titov, and Polina Zablotskaia. Theoretical and practical perspectives on what influence functions do. Advances in Neural Information Processing Systems, 11-27560-27588 2023.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instructionfollloinn llama model. https://githube com wwwootttttteeeeooooooooceooootet htt/stattooddaaaaaaaaaaaaaa 2023.
Simone Tedeschi and Roberto Navigli. MultiNERD: multilingual, multi-genre and fine-grained dataset for named entity recognition (and disambiguation). In Findings of the Association for Computational Linguistics: NAACL 2022, pp. 801-812, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/1/20222.fmm naac.... URL
https: http///cllaiinn http:///aaaaaatcgo....o.o orgg 2022. findings naacl. 60.
Daniel Ting and Eric Brochu. Optimal subsampling with influence functions. Advances in neural information processing systems, 31, 2018
Aaad W van der Vaart. Asymptotic Statistics. Cambridge University Press, 2000.
HaiYing Wang, Rong Zhu, and Ping Ma. Optimal subsampling for large sample logistic regression. Journal of the American Statistical Association, 113(522):829 844, 2018.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi. Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, al. How far camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information Processing Systems, 71477764-74788 2023.
12

---

Published as a conference paper at ICLR 2026
Zifeng Wang, Hong Zhu, Zhenhua Dong, Xiuqiang He, and Shao-Lun Huang. Less is better: Un- weighted data subsampling via influence function. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 6340-6347, 2020.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: selecting influential data for targeted instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, pp. 54104 54132, 2024.
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. Advances in Neural Information Processing Systems, 36: 34201-34227, 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2026
A ADDITIONAL EXPERIMENTAL DETAILS AND RESULTS
A.1 EXPERIMENTS FOR TOKEN- -EEEE LEARNING
In these experiments, we used pretrained models as base models and constructed training, validation, and test sets from real-world datasets. Details of the datasets and models are provided in Section K.
For each training example count both for surrogate model training (used for scoring) and final model training we selected the learning rate from the following grid:
[3e-6, 1e-5, 3e-5, 1e-4, 3e-4, 1e-3, 3e-3, 1e-2].
The optimal learning rate was determined by training models on randomly sampled subsets from the training pool and evaluating their test log-loss. For each learning rate, the loss was averaged over 10 runs, with a new random subset used in each run. The best-performing learning rate was selected separately for each experimental configuration listed in Table 1 and Table 2.
Implementation details for Less (Xia et al., 2024) We used the public implementation from the authors' GitHub repository. The projection dimension was set to 8192. Learning rate and other hyperparameters were tuned identically for all approaches. For both our method and LESS, the surrogate model used the same number of samples and was trained for four epochs, matching the settings in the LESS paper. Following the original LESS procedure, we selected the top-scoring examples.
A.2 EXPANDED RESULTS FOR INSTRUCTION TUNING
In the main paper, we compared our scoring methods for the SCORE+RANDOM strategy. Due to space constraints, Figure omitted results for Experiment 3. In Figure 4, we provide an expanded version that includes results for Experiment 3 as well.
exp 1
exp
exp 3
0.855
Impenv
1.325
0.55
1.300
0.825
1.050
0.80
0.775
0.250
0.725
5.875
6005
7000
3000
4500
seioo
raioo
2050
7000
Select Count
exp
1.07
3.06
3.05
log lo5s
2.04
0.94
100
0.22
1.00
0.99
2000
3000 4003
5000
6000
7000
8000
8000 2300
3000
4300
60co 7000
Select Count.n n
Select Count n
Figure 4: Expanded version of Figure 2 including the Experiment 3 plot.
14

---

Published as a conference paper at ICLR 2026
A.3 EXPANDED RESULTS FOR NAMED ENTITY RECOGNITION
In Figure 3 of the main paper, reported results for SCORE+RANDOM using fixed selected sample size of
1024, across all experiment configurations in Table 2.
In Figure 5, we show how the test log-loss varies with the selected sample size n for different scoring methods under the SCORE+ REIE++RANOOO strategy, and how these compare to random selection.
exp 1
exp
3
5.073
0.055
0.000
2.063
0.045
0.040
0.045
0.035
0.040
0.050
0.000
0.035
3.045
0.000
0.020
0.025
0.040
EXP
eXp
exp
0.011
0.010
log loss
2.042
0.007
3.005
0.006
0.005
8.080
0.92
2000 3000 4002 5000 6000 7000
sdo0
2000 3000
4000 s000
seco 7000
2000
2020
3000
4000
5000
6000
7000
5000
Select Count
Select Count
Select Count n
Figure 5: Test log-loss vs. number of selected samples n for NER. Lines show mean log-loss over 10 runs; error bars +1 standard error. Results use Interleaved ToV with the SCORE+RANDOM strategy.
A.4 ADDITIONAL NER RESULTS: TOKEN-LEVEL.
To complement the log-loss evaluation reported above and the main paper, we also measure token-level Fl the NER task. Figure 6 shows test 1 - F1 as function of the number of selected samples n, using the same experimental setup as Fig. 5. Lower values therefore, indicate better sequence labeling performance.
We observe trends consistent with the log-loss results: our selection methods outperform random selection and competing baselines in several settings, confirming that cross entsopyy gains translate into improved token-level F1.
B COMPARISON OF SCORE+RANDOM AND SCORE-ONLY SELECTION
In this section, we examine how the performance of our strategies changes when all training amples are selected from the top-scoring set (SCORE-ONLY) instead selecting only half of them from the top and the other half at random (SCORE+RANDOM),
Recall that our scores approximate how much benefit each example provides when added to a ran- domly chosen pool of training data. A higher score therefore indicates example expected to be
15

---

Published as a conference paper at ICLR 2026
exp
exp
exp 3
scoring method
0.20
0.13
max abs
5.375
0.300
0.16
0.34
0.200
0.055
0.30
0.045
eXp
6
0.249
0.24
0.085
9.255
3200
0.77
0.2
0.075
0.50
9125
0.2/0
6.16
0.085
6000
2020
2300
4000
6002
2000
6000
2000
a006
Select Count n
Select Count n
Select Count
Figure 6: Test 1- F1 vs. number of selected samples n for NER. Lines show mean F1 over 10 runs; error bars denote +11 standard error. Results use Interleaved ToV with the SCORE+RANDOM strategy.
Loss Ratio (SCORE ONLY SCORE+RANDOM
abs cng
improv
1.02
1.01
1.00
0.98
exp
exo
0.97
exp
exp
2000
3000
4000 5000
6000 7000 8000
2000 3000 4000 5000 6000
7000 8000
2000 3000 4000 5000 6000 7000 8000 Select Count
Select Count
Select Count
Figure 7: Ratio of log loss for SCORE-ONLY versus SCORE+RANDOM across our three scoring strategies and all instruction-tuning setups in Table 1. Scores are computed using Algorithm 1.
more helpful in that setting, SCORE+RANDON selects half of the final training set from the highest- scoring examples and fills the rest with random examples, whereas SCORE-ONLA takes only the top-scoring examples. This design creates a trade-off:
Pure exploitation:
Selecting only top-scoring examples can maximize immediate gain be- cause every chosen example has a high estimated contribution.
Score validity and diversity: The scores are defined relative to adding examples to a random pool. If select only top examples, the resulting set may differ substantially from the
16

---

Published as a conference paper at ICLR 2026
max abs cng
max pos improv
max improv
Loss Ratio (SCORE-ON) ONLY SCORE+RANDOR
2.0
exp
1.8
exp
1.4
1.2
0.8
2000
3000 4000 5000 Select Count
6000 7000 8000 2000 3000 4000 5000
6000 7000 8000 Select Count
2000 3000 4000 5000 6000 7000 8000
Select Count
Figure 8: Ratio of log-loss for SCORE-ONLY versus SCORE+RANDON across our three scoring strategies and all NER setups in Table Scores are computed using Algorithm
random reference, making the scores a less accurate guide. Randomly adding half the examples keeps the final set closer to the conditions under which the scores were computed and also protects against loss of diversity.
Which effect dominates varies by task.
Figures 7 and 8 show the ratio of log-loss for the selection strategies instruction tuning and NER respectively. In each figure the three subplots correspond to our three scoring strategies; different lines indicate the various experimental setups. Algorithm is used to obtain the scores.
For instruction tuning, SCORE+RANDOM performs better in three setups (3, 4, 5), while SCORE- ONLY better in the remaining two (1, across most selection sizes and scoring methods. For NER, SCORE+RANDON tends to outperform SCORE-ONILY more often, particularly for the Max- Improvement scores.
C PARALLEL ToV INTERLEAVED ToV
All previous plots used Interleaved ToV (Algorithm 1) for scoring. Here we compare the per- formance of the two scoring methods: Interleaved ToV (Algorithm 1) and Parallel ToV (Algo- rithm 2) across our experiments, using the SCORE+RANDOM selection strategy for both.
Figures 9 and 10 show the ratio of test log-loss obtained with Parallel ToV relative to Interleaved ToV for instruction tuning and NER, respectively. Each figure contains three subplots corresponding to our three scoring strategies, and different lines represent the various experimental setups.
The results indicate that for instruction tuning, Interleaved ToV is most often superior, while NER there is consistent winner. A possible explanation is that Parallel ToV uses two distinct training trajectories. Our analysis assumes that the resulting models differ only slightly, but practice the two training trajectories diverge substantially. This effect is likely be stronger with large and highly overparameterized models such as Meta- Llama-3-8838-8 which we used for instruction tuning, We expect the larger distance between the two models to result in less accurate estimation in Parallel ToV, as compared to Interleaved ToV.
D LOGISTIC REGRESSION EXPERIMENTS
In these experiments, we synthetically generated the training pool, validation set, and test set. We begin by defining parametric family of distributions used to construct the data.
For given > and parameter vector E IR,, we define distribution P, over pairs (z, y). where TRP and y (0, 1). The features sampled x ~ Ao0 T), and the label y is drawn according
17

---

Published as a conference paper at ICLR 2026
max abs cng
max pos improv
max improv
1.07
ex9
1.06
exp
Loss Ratte (Aig Aig
1.05
exp
1.04
1.03
1.02
1.01
2000
3000 4000 5000 6000 7000 8000 2000 3000
4000 5000
6000 7000 8000 Select Count
2000 3000 4000 5000 6000 7000 8000 Select
Select
Figure 9: Ratio of test log-loss for Parallel ToV (Method B; Algorithm 2) relative to Interleaved ToV (Method A; Algorithm 1) for instruction tuning. Results use SCORE+ R ANDOM selection strat- egy. Each subplot corresponds to one scoring strategy; lines denote different experimental setups in Table
abs cng
pos Improv
max Improv
1.8
exp
Loss Ratio (Alg B [Agg
exp
1.2
1.0
6000
Select Count
Select Count
Select Count
Figure 10: Ratio of test log-loss for Parallel ToV (Method Algorithm relative Interleaved ToV (Method A; Algorithm I) for NER. Results use the SCORE+RAC RANDOM selection strategy. Each subplot corresponds one scoring strategy; lines denote different experimental setups in Table 2.
to logistic model:
1
Pr(y 1 x 1 exp(-x-
Pr(y = = Pr(y = 1 z))
We randomly sample a unit vector H* from the unit sphere to serve as the target direction. second unit vector 0' is then drawn such that it lies at angle from 0* In our experiments, set p 10 and - -= 7/2.
The training pool consists of N 128 1024 samples, drawn independently from the mixture distribution:
1 Po. + PPrr 2
Danin
The validation and test sets contain Mya = 1024 and m. = 10,000 samples respectively, both drawn i..d. from the target distribution Po-.
18

---

Published as a conference paper at ICLR 2026
Strategy: Rand-frm-top
Strategy: Score Random
Strategy: Score-only
Improv
max
classificatonn error
random oracie
0.425
0.400
0.375
0.350
0.325
0.300
Final Size
Final Size
Final Size
Figure 11: Data selection experiments with Parallel ToV for logistic regression on synthetic data in 10 dimensions (4 epochs of training). Each color corresponds a distinct method score data in the training pool, and each frame to distinct method to use the score to form the selected set. Each symbol corresponds the average of 10 experiments.
For scoring, we used Parallel ToV as described in Algorithm 2: similar experiments with Interleaved ToV produced comparable results, so report only Parallel ToV here. Algorithm 2 does not specify the method to select data on the basis of scores. Figure we compare SCORE-ONLY and STORE-RANDOM, (already introduced above) with third one RANDOM-FROM-TON that selects at random from the top 50% subset of data with highest scores.
The RANDOM-FROM- ToP method is included only for these synthetic logistic- reggesior experi- ments, for theoretical interest as by construction, the training pool contains half of its examples from the target distribution.
The base set used for initial training contains [U] = x 1024 examples.
For both the scoring model and the final model training, used 4 epochs of batch gradient descent with a linear decay learning rate scheduler. The initial learning rate was set to 0.5. We used e TO for adjusting the learning rate on validation examples.
The selected subset size varied from 128 to 8192 in multiplicative steps of v2. All results averaged over 10 independent runs. The final performance curves are presented in Figure 11.
E PROOF OF PROPOSITION 1
Throughout this proof, we denote by generic constant that can depend on co, C1 M, C, and whose value is allowed to change from line to line.
Letting A₂ Ourti 0 Eq. (11) yields
A (+) (1) - vymn 2 Po. HOH
(1) A
; (1)
AH (i) 77 ((0 x. + (i),
(20)
where H3 is defined in Eq. (14) and
err (i) = =1 ( ( - ve(O æ.)] 1ym L [T - V2
- (i))dz
where (2) = (1 =) + 2 By assumption 0 ve(0: and 0 + 22 (0) are
M-Lipschitz, whence
| (i) < 1 0 1mMM
M A la (i) + mm A 1
(21)
Define A Iim by letting A A (i) and, for & 2
*+ (i) H A lin ( ;
(22)
19

---

Published as a conference paper at ICLR 2026
Comparing with Eq. (20), obtain
A x+ (i)) H 1 A in (+)(1) erra(n,
1-1
(1) Σ M Mrs+1err,,n, (n.
(23)
Since V R2 (0. = colo, we have HHllop & (1 - comn), and therefore
(i)(1) AT $
M + 1llopll (rr, m))l
(1 comm) t- lerr, (n. m)ll
(24)
Further, from Eq. (22), and using (v((0m x.)I S C1 we we get
A (1) n M1 VE(0... x.),
| (1)l S Cin Σ comm))) 1-0-1
C
(25)
m
Let D,(1)= == max,5: A.)l (1) = max [I)rl Using Eqs. (21), (24) and (25), we get
D.(i)= S + com 1i),
Fx(1) S (D(( + (i))
Using these inequalities together, we obtain, for all m 2 (and eventually adjusting the constant
C*)
C Ed(1)
Cn m
D2(1) S m
(26)
whence, using again Eq. (24), we get
C m
(1) (1)l
(27)
Notice that we can rewrite
L
S.D
I Σ
R.,. (0..)
A (1))
(28)
whence, using the fact that assumed bound || 2 R.n (0
| R ll) I C for all 0 E
10 @ratij (this follows from the
lpp C1 and the Lipschitz property of 2 Rr (0)), get
[S; S < C'
max L
A (1)
Rom
(i))
L
< C
max
(i)
2 +
C
max L
(2)
lin (i)l
m
and this completes the proof.
F PROOF OF THEOREM I
Throughout this proof, we denote by a generic constant that can depend on Co. C₁ M, C, and whose value is allowed change from line to line.
20

---

Published as a conference paper at ICLR 2026
F.1
BOUND ON Y,
The iteration for mh and 0% specified by Algorithm 2, reads
x++ 0 Tp V Ry (
(29)
k+1 0,k+1 En V Riyal (0 0,k+1).
0 0,++1
17m V (0
(30)
Letting A = aut me and D 0
we obtain
A k+1 u e
V Ra O0r k+1 + eerr k+r
(31)
00,+11 = A + err
(2)
(32)
where, letting 0,, (2) = 111=))0 2)0 k+1
S and 0x(=)= = -1 =2)00a + =0 we have
err (1)
V2 Pra (2)) dz,
k+1
mEmaa
(2) err
== mm
(2 V Ruu hmm)
A dz.
Using the assumption that V Re (Orus (=) C and 0 + V (O) is M-Lipschitz, we get:
||err 6+1 S Cermm Do |ll 0 [1
(33)
On the other hand, since + 7 Ry ( also M-Lipschitz, we have
| (2) S mmMM Ax
(34)
whence, using Eq. (32) and H llop 1
A o || S x + M
| & CaryA A + || + m72m3|14111))
err
(35)
where in the last line we used the assumption that rmm S Cy--
Substituting Eqs. (34) and (35) in Eq. (31), (32), we obtain (using again 1ym S Cty)
A R+1 H expm V Pean (0 + errk:
(36)
||erra|| S Crem. (1 A + || CrmmlAA:
(37)
We define A im and, for l: >
lin k+1
H lin Enmm V Resl (
(38)
whence
t-1
[-1
A lin
Mr.+1 (Qthm A
Mrst
(39)
==0
s=U
Define D, := max, ||a F, = max,s1 /|err Using the fact that | S (1
commy))--11 and the assumption || Rem (0.z))l S C₁, we get, from Eqs. (37), (39),
D < Cemmal
+ C
mi Ec:
(40)
m
E, < Chema (D, + D CrmD,
(41)
Using the assumption em /m this is easily seen to imply
D, < C Mmaal
E, < C. (em..a)
1j.
(42)
m
21

---

Published as a conference paper at ICLR 2026
Substituting in Eq. (39), we get
A 7"l S Σ (1
comm))==-1100
||err,11
S C
(43)
Yn
a=0
The linearized score of Eq. (16) can be rewritten as
1
Th =
Ve(O xx),, Am
(44)
L
Using the fact that
|| [ ax
Ve( ; m.pp C1, get
(.(e)
T.
S
L Σ
1
e(Oh:: xii ((0)mm:;a))
ax;) + : mn). )
(45)
C'
C
A 2|
Σ I. !
(46)
I
L
0
Emm
(47)
F.2 BOUND ON Oi
The iteration for 6 and 0 as specified by Algorithm 2, reads
pas 1 0
17m V Rs (0 0%
(48)
Elpm
V R. Obas 1++
(49)
Hence, we can rewrite
1
VE( ;b,) 0.m0 0mm)
T,
Using the assumptions ((0m0 x.)) I C1. (6 1 C1, we obtain
1
(d((E) - @1
E(@m xx)) - E(@rur; x;)) - (VE( 0rss xx)), 9tm 0
(50)
L
==1
C
|
0 12 < C(Emm)aa)
(51)
L
The claim thus follows by recalling that 7) S C./mm m..
G PROOF OF THEOREM
To lighten notation, define rk V Pum and vn (x) Vi(0R. m.)).
22

---

Published as a conference paper at ICLR 2026
For any L L, Z, we have
(L) -m. (L) + m. (L) (T.) + (L))
(L) == mm
Σ
(ret1, M M5,++13 (1))
os
(L) = L
E
(T+I,, M 8,1+1 v.t))
OStS Lo LoSCEL
(L) = mp
E
(r411, 10,++100100)
Lu <
bo<k<osLool-- Lo:le-
(L) = mp
E
(re+1, M 8, (i))
L
Ln<k<esoolo-122
(L): ==
my
Σ
(re+1, M 1,,441.217 (i))
La LockcosLolls
Since by continuity we have limg-+oe R2R (0 Q for any å E (0, 1/2), we can choose Lo large enough so that (1 k) 3 V2 Rr 0 3 (1 + 0) for all k > La- In particular there exists co (independent of E) such that H& op S (1 comn) for all Lo.
Clearly |T!.. (L)) S C(Lo)/ 0 as + oo. Further
| (L) < Cnmmal
Σ
|Mat+1llp
Cnmma
Σ
(1- (0-omm))----
Cmmmaal
Lo + 0.
L
Comn
Finally, by increasing Lo. we can ensure that, for Los H - Hollop I 8, 1|rk - rxoll S
(i) V.o (i)ll S 8 (where Hx I - qm x and Too Uoo (i)). Hence
(T++1, M. (.)) (Poos 1 Clt s 1|(1 - comm))--66-
Therefore, letting
T! (L) = apMaall
(Too, H -- 100 (i))
(52)
L
Fostrash
have |r
(L) (L)
-T.mm (L)
< maa
Σ
(T41, (THI, (Too, H 2 (1))
T.
Fostcash
qmnal
Σ
Clt- 11(1 - comm))
L
LostesLL
< Mmaal
(cumm)2
Finally, using again (rx,, H H 1
Uxx (i)) S (1- {11-cauy)++
we have
lim 1m (L) = lim
Σ
(rx, H U20 (1))
L-000
L
L
LocigL==tt1
mm
(Poor H oc vo0
k=0
(Too - Hox 1 voo
Myal
(Too: Q-1 Uox (i))
23

---

Published as a conference paper at ICLR 2026
This finishes the proof of the part of Eq. (18) which concerns the limit of T.p. The calculation of lim (L) is completely analogous and we omit it.
H PROOF OF THEOREM
To simplify notations, we write y/ = y(a,) ) for the response variables and v; vbxx) for the feature vectors. Similarly, for the yj y(z)")) (b(z))) 1/
With these notations, we have H = independent of k and
vi(0: xx;) = -(y { 0)) Rime (0) Tp y 40
(53)
(54)
m
= - L
(55)
Hence,
( (41 H Ha-
(56)
L
OSICSLL
nd
26 0ur
(1( y/ (vbi, )
(57)
H+1
Since Py is the projector onto the null-space of H, and by our choice of 1,, we have H = Pv + H 1.
where the row/column space of H_ is orthogonal to the of Ps and | llop (1 c()))
[0, 1). As consequence
H = Pv llop S -
Define
T!
Σ
( Pv
(58)
L
0Stco5L
Then we have
Tlin
1
Σ
Parbi)
L
2
OSTCOSL
Ir.(i)
P 1lop [abs]
1,2
USIC8LL
(a)
C
Σ
- (
1
OSACHSL
C MyMoa
1 1-20 0.
L
In (a), we used the fact that lim, 6p θ (Bartlett et al., 2021), and therefore (r)( || |/rr) remain bounded as s, t + Do.
In view of the above, lim Thth lim /L For the latter, we have
lim T lim 10 L
Σ
(
Lo
L
OSACAET
lim
My
Σ
(i
Lo
L
LoStegL
lim L
Σ
T(r)(Trr' Puub,) + err L),
I
LosteesL
where
lerr lerr(Lo, 1)] S C sup ||r(()) - r(i)] + C sup ||r -r
(59)
22
Lo
24

---

Published as a conference paper at ICLR 2026
Since lim te 0, we have Lo- lim sup-- err(Lo, L) 0. Therefore,
lim L Lo oc +00
lim - T
lim
My
Σ
r(i) Pevi)
LOSAEEEL
1
m (1) (pr Pot))
This proves the limit for (L) in Eq. (19).
The limit of (L) is computed essentially by the same argument and omit the derivation.
I GRADIENT ACCUMULATION IN ToV
A potential concern in Train-on- -alliiatioooii (ToV) scoring is that repeated updates on the validation set across scoring checkpoints could accumulate on the same model parameters and lead to overfit- ting of the selection policy to Z In Interleaved ToV, however, each scoring model is initialized from a base-model checkpoint that has not previously been updated on the validation set and is then updated Zd only once. Thus, validation gradients do not accumulate across epochs. In Parallel ToV, validation-updated models continue training and may, in principle, accumulate such effects; in practice, this mitigated by using a much smaller learning idation training. Nevertheless, Parallel ToV may remain more susceptible to validation overfitting, which could contribute to its slightly inferior performance relative Interleaved ToV.
the same parameters
for val-
LIMITATION
The core idea of ing function F.) and aggregation strategy might need adaptation for different problem settings.
impacting training examples is general, but the specific scor-
The SCORE+RANDOM selection strategy often outperformed SCORE-ONLY in our experiments, suggesting that diversity plays an important role beyond simply selecting the "most affected" ex- amples. While this is practical improvement, also indicates that our current scoring mechanism might not fully capture the optimal diversity coverage needed for effective generalization. It will be interesting to explore more sophisticated diversity-aware scoring or selection mechanisms that explicitly balance our scoring methods with representation across the data space.
Although we mitigated bias toward shorter examples through length- based binning, more refined length- nongtanoomaaiiitiion or task-specific weighting might further enhance the selection process. Fur- thermore, will be interesting to see if the performance of our strategies improves further compared to random selection when the learning rate is also tuned for these strategies, rather than just for random selection.
Finally, theoretical analysis relies stylized settings that are plausible for simple models but may not hold many large-scale applications.
MODELS AND DATASETS INFORMATION K
K.
DATASET INFORMATION
Slim Orca:
Link
Citations- Citations-Longpre et al. (2023); Mukherjee et al. (2023); Lian et al. (2023)
Licence: mit
Alpaca GPT-4:
Paper:Peng et al. (2023)
Repository
Link
25

---

Published as a conference paper at ICLR 2026
Licence: co-by-nc-4.0
Alpaca GPT-3.5:
Paper: Taori et al. (2023)
Link
Licence: cc-by-nc-4.0
Multinerd:
Paper: Tedeschi & Navigli (2022)
Link
Licence: cc-by-nc-sa-4.0
Ai4p:
-Link
-Licence link
C4 dataset:
Link
Labeled for NER task using llms.
Licence: terms of use
Syn-Big:
Synthetically generated by us using llms.
Proprietary dataset
K.2 PRETRAINED MODEL INFORMATION
3-8B AI@ Meta (2024)
Link
License: Ilama3
xlm-roberta-base Conneau et al. (2019)
Link
License: mit
K.3 COMPUTE RESOURCES INFORMATION
All experiments were conducted on a single machine equipped with 8 NVIDIA A100 (80 GB) GPUs and 128 AMD EPYC 9354 (32-core) CPUs.
Instruction- seatrcttoo-tuning experiments across all five setups, with 10 runs per setup and including hyper- parameter search, required approximately two weeks of wall-clock time using parallel execution across GPUs. NER experiments completed within 4-5 days under the same setup. Logistic regres- sion experiments were lightweight and required less than 30 minutes in total.
26


Reviews:
[{"review_id": "kcqDqucYpy", "reviewer": "Reviewer_JbT7", "summary": "The paper tackles data selection from a large training pool, with small datasets for validation and test. The proposed method efficiently scores each example from the training pool by how much one GD step on it contributes to lower validation loss, only with gradients on samples from the validation dataset, not from the training pool, by a clever trick based on first-order Taylor approximation. In section 2, experiments with tasks like instruction tuning and NER show that the proposed method (Method A) is more effective than baselines with respect to log-likelihood loss. Theoretical analysis is provided for a variant of the proposed method (Method B) in Section 3, justifying the proposed method from an asymptotic viewpoint.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The problem addressed in this paper, data selection from a large training pool with a small validation dataset, is important for real-world fine-tuning.\n- The derivation of the proposed method, which eliminates the need for computing gradients on the training pool, is straightforward and effective.\n- Experimental results indeed show the effectiveness of the proposed method (Method A) compared to baselines in log-likelihood loss.\n- Theoretical results provide further justification for their approach beyond just Taylor approximation, based on the asymptotic analysis.", "weaknesses": "- The main trick in this paper, i.e., swapping gradients between training and validation data in computing the difference in validation loss after one or multi-steps gradient descent, is not novel itself. Such a trick have already appeared in [Liu et al. 2019] and [Savani et al. 2025] for example. Nevertheless, as far as I know, its application to data selection may be novel.\n- I have some concerns on the experimental design in Section 2. While the experiments report only log-likelihood loss, especially in instruction tuning, it does not imply actual improvements for downstream tasks and the instruction-following capability. Also, the choice of the NER task is not reasonable since entity classification for each token is less useful in real world and there are more suitable tasks than NER.\n- There is a non-negligible mismatch between the experimental results (focusing on Method A) and the theoretical results (focusing on Method B). Even if Method A is empirically better than Method B, I think Method B should be mainly evaluated also in experiments (or Method A should be mainly analyzed theoretically) for consistency throughout the paper.\n- The paper title `Train on Validation` is somewhat misleading. The paper just proposes data selection using gradients on validation data, and actual training is mainly performed on the selected data from the training pool.\n\n[Liu et al. 2019] DARTS: Differentiable Architecture Search (ICLR'19)\n\n[Savani et al. 2025] Antidistillation Sampling, https://arxiv.org/abs/2504.13146", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Dsuw7HwFc", "reviewer": "Reviewer_E8Vn", "summary": "Train on Validation (ToV) is a simple data-selection method for fine-tuning. Leveraging from a principle they drove in the paper, train–validation symmetry, they built a fast scoring rule that does not depend on the per-example gradients, or Hessian-vector product, as opposed to prior works. They simply compute each pool example’s loss before and after a brief train-on-validation step and rank by the loss drop. Through experiments on instruction tuning and named entity recognition (NER), they show that ToV typically beats random/uncertainty and often outperforms a strong baseline (LESS), despite requiring less machinery.", "questions": "1 - Repeated training on the validation set during scoring raises concerns about overfitting the selection policy. Please clarify how ToV prevents overfitting to $Z^{val}$, and whether performance on the test set deteriorates as the number of ToV cycles increases.\n\n2 - It would be helpful to discuss regimes where the train-on-val = train-on-x symmetry assumption does not hold. Reporting these failure modes or limitations would add practical value and clarify the boundaries of ToV’s effectiveness.\n\n3 - The shift from token-level to sequence-level selection may introduce biases stemming from uneven sequence lengths, as discussed in LESS. Please clarify how ToV accounts for or corrects this bias.\n\n4 - What is the effect of base subset size $m$?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1 - The core idea is elegantly simple. By exploiting a symmetry between training and validation loss changes (Eq. 6), the method derives a tractable influence proxy that requires only two loss evaluations and a brief “train-on-val” phase, thereby avoiding N validation passes or per-example gradient computations.\n\n2 - They propose two algorithms (Method A/B) and theoretically connect their scores to linearized influence in a gradient descent setting, framing ToV as an influence-style selector that avoids gradients and HVPs.\n\n3 - Empirically, ToV consistently outperforms random and max-uncertainty baselines and, in most cases, exceeds LESS on instruction tuning.", "weaknesses": "1 - Because efficiency is central to the paper’s contribution, the claim of practicality should be backed by quantitative evidence, wall-clock time, memory usage, and forward/backward pass counts on identical hardware. LESS has warm-up and gradient-store overhead, whereas ToV involves repeated validation fine-tuning and two full loss sweeps per cycle. A detailed cost comparison would make the “fast” claim more credible.\n\n2 - Since LESS explicitly warns that loss and accuracy may not align in LLM evaluation, relying solely on test log-loss in ToV is limiting. Including complementary metrics, such as accuracy, MMLU-style subsets, or task-specific measures like exact-match or F1, would strengthen the evidence that lower log-loss corresponds to better task quality.\n\n3 - For completeness, ToV should be compared with at least one alignment-based selector (e.g., importance resampling) and a datamodel-style [1] to evaluate whether its advantage persists when competing methods also avoid per-example gradients.\n\n[1] Park et al, \"Trak: Attributing model behavior at scale\" ICML 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N433F5RdDZ", "reviewer": "Reviewer_5rLJ", "summary": "This paper studies how to efficiently select training samples so that the resulting model performs better on a small validation set drawn from a target distribution. The authors observe that standard training-sample scoring methods rely on per-sample gradient computations over both the training and validation sets. To reduce this cost, they propose an approximation based on the change in training samples' loss after performing a gradient update using the full validation set. Two algorithms are developed based on this insight—the second being theoretically guaranteed but empirically less effective. Experimental results demonstrate consistent performance improvements over two baseline methods.", "questions": "1. Instead of selecting important samples, why not consider applying target-aware domain generalization techniques here, especially in the context of LLM tuning?\n2. Since both training and validation sets are accessible, how about directly measuring sample similarity between them to guide selection?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation of this paper is strong, especially for scenarios where some existing packages do not support per-sample gradient computation. \n2. The observation of train–validation symmetry is interesting. It is also a good choice to put this point in the introduction section, as it helps readers quickly grasp the core insight of the paper.\n3. The proposed method could be applied to certain LLM-related tasks, such as instruction tuning and named entity recognition.\n4. The theoretical analyses and experiments look good and convincing.", "weaknesses": "1. Although the train–validation symmetry is an interesting observation, it is established under the assumption of independence of x. In Proposition 1, the Hessian matrix is expected to be of order 1, which supports the utility of the proposed estimator. However, it would be beneficial to discuss this point in more detail, particularly in relation to influence functions and data Shapley. I understand that Strategy 2, which increases diversity via random sampling, can heuristically mitigate this issue to some extent.\n2. The proposed method can also be interpreted as a task-dependent data pruning technique. From this perspective, it would be helpful to include a discussion comparing it with representative works in this area.\n3. The original problem can be formulated as a bi-level optimization problem, typically with continuous sample weights. I did not look into whether Xia et al. (2024) is following this idea, but the current baseline choices seem somewhat weak in comparison.\n4. It would be better to experimentally quantify the efficiency (how fast) of the proposed method, as this is claimed as the key contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OszIl9xqBm", "reviewer": "Reviewer_YYKn", "summary": "The paper proposes a method for selecting task-specific fine-tuning data using a validation set representative of the target distribution. The approach finetunes a model on the validation set and performs inference on candidate training samples, selecting those whose losses change the most. Experiments demonstrate that the method outperforms uncertainty-based selection and a recent task-specific selection method. Finally, the authors further provide a theoretical justification for its effectiveness.", "questions": "- What is the purpose of fine-tuning on a small random subset before fine-tuning on the validation set? Additionally, why is it necessary to repeat this process for multiple iterations?\n- In exp 3 and NER tasks, why does LESS perform so badly as a task-specific selection method?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Training on the validation set introduces a new perspective in task-specific data selection.\n- The method is conceptually simple and readily implementable.\n- A theoretical analysis is presented, providing provable error bounds.\n- The manuscript is well structured and written with clarity.", "weaknesses": "- The paper claims that the proposed method avoids the computational burden of computing influence functions, but does not provide any empirical or theoretical evidence. It is unclear how expensive the proposed method is compared to training on the full set or influence-based selection.\n- Only log loss is reported in the evaluation. For instruction-following and NER tasks, log loss does not adequately capture output quality. Metrics such as F1 (for NER) or semantic similarity and human preference scores (for instruction-following) would provide a more meaningful assessment.\n- A gap exists between the empirically evaluated method and the theoretical formulation, and the paper does not make the connection between them explicit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)