T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
Authors: Yanjun Fu
Faisal Hamman
Sanghamitra Dutta
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Instruction tuning
Data Selection
Token-selective Quality Score
Robust Hierarchical Selection
Primary area: deep_learning
TL;DR: We introduce T-SHIRT, a new data selection method for instruction tuning LLMs that scores data at the token level and emphasizes robustness.

Abstract: Instruction tuning is essential for Large Language Models (LLMs) to effectively follow user instructions. To improve training efficiency and reduce data redundancy, recent works use LLM-based scoring functions, e.g., Instruction-Following Difficulty (IFD), to select high–quality instruction-tuning data with scores above a threshold. While these data selection methods often lead to models that can match or even exceed the performance of models trained on the full datasets, we identify two key limitations: (i) they assess quality at the sample level, ignoring token-level informativeness; and (ii) they overlook the robustness of the scoring method, often selecting a sample due to superficial lexical features instead of its true quality. In this work, we propose Token-Selective HIeRarchical Data Selection for Instruction Tuning (T-SHIRT), a novel data selection framework that introduces a new scoring method to include only informative tokens in quality evaluation and also promote robust and reliable samples whose neighbors also show high quality with less local inconsistencies. We demonstrate that models instruction-tuned on a curated dataset (only 5% of the original size) using T-SHIRT can outperform those trained on the entire large-scale dataset by up to 5.48 points on average across eight benchmarks. Across various LLMs and training set scales, our method consistently surpasses existing state-of-the-art data selection techniques, while also remaining both cost-effective and highly efficient. For instance, by using GPT-2 for score computation, we are able to process a dataset of 52k samples in 40 minutes on a single GPU. Our code is available at https://github.com/Dynamite321/T-SHIRT.

T-SHIrt: Token-Selective Hierarchical Data Selection for Instruction Tuning
Yanjun Fu Faisal Hamman
Sanghamitra Dutta
University of Maryland, College Park (yanjunfu, fhamman, sanghamd) (oaraammd@@md..... edu
Abstract
Instruction tuning is essential for Large Language Models (LLMs) to effectively follow user instructions. To improve training efficiency and reduce data redun- dancy, recent works use LLM- based scoring functions, e.g., Instruction Instruction-Following Difficulty (IFD), select high-quality instruction-tur tunnnn data with scores above threshold. While these data selection methods often lead to models that can match even exceed the performance of models trained the full datasets, identify two key limitations: (i) they assess quality at the sample level, ignoring token-level informativeness; and (ii) they overlook the robustness of the scoring method, often selecting a sample due to superficial lexical features instead of its true quality. In this work, we propose Token-Selectee Tuning (T-SHIRT), a novel data selection framework that introduces a new scoring method include only informative tokens in quality evaluation and also promotes robust and reliable samples whose neighbors also show high quality with less local inconsistencies. We demonstrate that models instruction-tun (only 5% of the original size) using T-SHIRT can outperform those trained on the entire large-scale dataset by up 5.48 points on average across eight benchmarks. Across various L.MSs and training set scales, our method consistently surpasses existing vate-of-the--a data selection techniques, while also remaining both cost- effective and highly efficient. For instance, by using GPT-2 for score computation, we are able to process a dataset 52k samples in 40 minutes on a single GPU. Our code is available at https: // //gittte github..... cwm hemm/Dynaiitteen wtt.aammte21/SSSSTTTT
HIcRarchical Data Selection for Instruction
on curated dataset
1 Introduction
Large Language Models (LLMs) have shown remarkable success in various tasks [1-5]. One key component adapting these powerful LLMs for practical use instruction tuning (also known as supervised fine-tuning, SFT) [6] that fine-tunes pre-trained L.MSs on instruction instruction-response pairs. During this stage, researchers often use millions of samples [3, 4, 7]. However, LIMA [8] challenges this norm with the Superficial Alignment Hypothesis, which indicates that instruction tuning might not need to be data-intenssie, shifting the focus from data quantity to data quality. Fine-tuned on just 1,000 manually selected instruction-tunnin samples, LIMA achieves strong performance. Data selection for instruction tuning offers several key benefits. First, full-dataset tuning is extremely time-consee conee-oonumingggg and costly. Furthermore, synthesized data becomes more common [9-12], datasets often include redundancy and noise, making data selection for instruction tuning absolutely essential.
To select instruction tuning data, it is common to use scoring functions to assess sample quality and only keep samples with high scores. L.MA [8] relies on manual annotation, while others [13-15] prompt proprietary LLMs via APIs to assign quality scores. However, both methods costly. An alternative is Instruction-Follooiing Following Difficulty (IFD) score [16, 17] (elaborated in Equation (1)), which measures the ratio of response's perplexity when conditioned on the instruction to its unconditioned
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Perturbed Token Embeddings
Hierarchical Selection
Noise
S-IFD
[lstruction]
ieency sciennst. [eepoose]
Full Dataset
[nstructioon]
Summarae
The
WB
[Response]
The itte Prince belowed...
T-SHIRT
[nstructiue]
Congute the avenge of
[Roppase]
The ovenge of the namber
Figure 1: An overview of our approach, T-SHIRT. WE denotes the model's embedding layer (e.g.- GPT-2) used to compute S-IFD scores. For each enstruction-respomee pair, we generate its neighbors by perturbing token embeddings. Then we only use selected response tokens (green squares) for S-IFD computation, while excluded tokens are marked in red squares. Finally, we use hierarchical selection to choose samples whose neighbors exhibit high average S-IFD and low variance.
perplexity. Essentially, higher IFD scores indicate better quality. IFD is quite appealing due to its efficiency and low cost, and even small models like GPT-2 [18] can used to compute it.
However, current scoring functions have multiple limitations as we identify in this work. First, they evaluate the training data at the sample level, overlooking aner-grained token-level information. Recent studies show that not all tokens contribute equally during training [19, 20J. Instruction tuning
significantly affects models' output on only small subset of response tokens [21, [2], These findings motivate us to reconsider previous sample-level scoring paradigms. We revisit the computation of IFD scores to examine the informativeness of individual tokens. We find that the IFD score treats all response tokens equally, and sample where none of the individual tokens are informative for instruction tuning may still receive a high overall score (see our counterexample in Figure 2). Empirically, we find that over 20% of the response tokens in instruction tuning datasets are not informative. Excluding these tokens from IFD calculation can substantially impact the final score.
Second, existing works using LLM-based quality scores overlook the robustness of quality assessment, particularly with respect to the local neighborhood of training sample. These methods typically apply a fixed threshold and treat samples with scores above it as high quality. However, fluctuations may exist within a sample's neighborhood, where some of its neighboring samples fall below the threshold. We find that the IFD score is not a robust metric. Small, semantic- semantic-preserving changes, such as replacing a word with its synonym, can lead large shifts in the IFD score (see our counterexample in Figure 3 with more details in Section 3.2). This suggests that high scores may also result from reliance on spurious features rather than reflecting true sample quality, motivating the need to also examine the local neighborhood of a sample during quality assessment.
To address the two challenges above, we propose T-SHIRT, short for Token-Selective HIcRarchical Data Selection for Instruction Tuning, which is a new framework for data selection that promotes both token-level informativeness and robust quality assessment (also see Figure/ for an overview). Towards introducing our framework, we first propose Selective IFD (S-IFD) (Equation (3)) that evaluates each response token individually and includes only informative tokens in the final quality score. Next, to avoid selecting samples whose high scores result from superficial lexical cues rather than true semantic quality, we go beyond previous pipelines that naively select samples based on a fixed quality score threshold. Instead, we apply a hierarchical selection strategy that favors samples whose neighhors exhibit high average and low variance in S-IFD scores. Experiments across multiple instruction tuning datasets and pretrained LLMs show that our proposed method T-SHIRT outperforms existing baselines on a wide range of downstream tasks. At the same time, our method remains computationally and financially efficient. For example, T-SHIRT requires only about 40 minutes to select data from the sample Alpaca-GPT-4 [11] dataset using GPT-2 on a single GPU.
1.1 Related Work
Data Selection for Instruction Tuning LIMA [8] challenges the need for data-intensive instruction tuning [3, 4, 7] by proposing the Superficial Alignment Hypothesis (SAH) and showing that 65B

---

model instruction eestruction-tued on just 1,000 manually curated samples can outperform models trained with massive data. To reduce the cost of human annotation, subsequent research explored automated quality scoring functions for data selection. Early methods use simple metrics such as response length [23] and perplexity [24]] Later approaches rely on proprietary L.MSs for quality scoring. For example, ALPAGASUS [13] and DS² [15] use OpenAl APIs, while DEITA [14] trains custom L.M scorers using SeaaGGPT-generatee
labels. To avoid expensive API calls, researchers propose using the IFD score [16], which can be computed even with small models like GPT-2 [17]. Other methods assess data quality via gradient-basee influence functions [25] and Shapley values [26], but these approaches are computation- heavy and require additional validation sets. Besides sample quality, some works like LIMA [8] DEITA [14] and DS² [15]. also emphasize sample diversity. However, most existing methods evaluate data quality at the sample level, overlooking token-level quality. They also do not consider the robustness of quality scores to small input variations.
Token-level Informativeness Recent works [21, 22] further support SAH by showing that alignment primarily affects models' outputs on small subset of response tokens. This leads us to hypothesize that token-level analysis might be important for data selection. Other studies [19, 20] show that not all tokens contribute equally during pre-training and instruction tuning. Then they use selective language modeling (SLM) to alter the training process itself by computing losses only selected tokens. Selecting tokens requires training a reference model on high-quality data and comparing token losses between the untrained and reference models. This makes data selection prerequisite for building the reference model. In contrast, our metric, S-IFD, does not need a reference model and works well even if it is computed by GPT-2. Moreover, method operates at the data preparation stage and does not change the training process. Thus, our method is orthogonal to SIM,, and it has the potential be combined with SLM for further improvement instruction tuning.
Robustness for LLMS Theoretically, LLMs are not robust to minor input perturbations because the self-attention mechanism [27] is not smooth and Lipschitz continuous [28, 29]. Empirically, even small and semantic-pred preeeriiig input changes can lead LLMs to misclassify text [30-32] or generate incorrect code [33]. Currently, robustness in LLM-based scoring functions for data selection remains underexplored. To the best of our knowledge, we are the first to incorporate robustness assessment into data selection for instruction tuning.
2 Preliminaries
Problem Setting Data selection for instruction tuning aims to identify the most informative subset from large instruction-response dataset to efficiently fine-tune LLMs while maintaining strong performance on downstream tasks. Formally, given a dataset containing N instruction-response pairs and selection budget b < N, selection policy æ selects subset S = (D, b), where S C DD and [S] = 6. Let 0 denote the original LLM parameters and 0s the parameters after fine-tuning on S. Under the budget b, the optimal selection policy x² maximizes the fine-tuned LLMs performance: n = arg max_ d(0s), where & is the performance metric, such as accuracy on test sets. Ideally, the policy π should be both financially and computationally efficient scale with large dataset sizes N.
Scoring Function Selection policies typically use scoring functions to evaluate each instruction- response pair in the dataset D, favoring samples with higher scores. Some functions focus on a single aspect, such as quality or diversity of samples [13, 16, 17, 23], while others assess multiple attributes [8. 14, 15]. In this work, we focus solely on quality, using a well-established metric called Ciaructiio-Followee
Difficulty (IFD) [16]. is a desirable metric because it is purely statistical and casy to compute. For a given cestruction-respone pair (I, y). the IFD score is computed using an LLM with parameters 0'. It is defined as the ratio of the perplexity (PPLg)) of the response conditioned on the instruction to the unconditional perplexity of the response:
PPLor(y(x) exp 7 IFD(z, = PPL (2)
Ei= log Por (yi (e),,)))
(I)
T log Por (uelves))
exp
where T is the length of the response, yt is the t-th token in the response, V<t represents all response tokens before the 1-th token, and Po. is the probability assigned by the model 0' to the token y/. A higher value of Pør indicates that the token y, is easy to predict under the model. The IFD score measures how much the instruction x helps the model generate the response y. lower score indicates that the instruction made the model easier to generate the response y, while a higher score
3

---

[ustruction]
Instruction]
Identify famous scientist.
Summarize the story of The Littl Prince.
[Response]
[Respense]
One lamous soientist is Albott Einstein, widely regarded the greatest thcoretical physicisss in histery. for developing theory general telativity and the famous equation E-mes ns well his contributions to the development of the atomic during World
The Little Prince is beloved novella (Ba by Antoine de Saint-Exupéry, published 1943, The story begins with pilcr who has crash- land-l his plane the where he moets a young bay, the Little Prince, who has journoyed from his home small asteroid, callied B-612. The pilor home and his beloved rose, to rivel among the planets and moet their inhabitans....
Einstein
desert
Einstein's a discoveries have had a significant impuct on our understanding of the universe and have played key tule...
Prince
IFD 0.998, 00
IFD 0.998, S-IFDts to 0.968
12
Ropuas Tokas Iodect
Respase Takan Baios
(a) Example 1: high IFD but low S-IFDr5-
(b) Example 2: high IFD and high S-IFD75-
Figure 2: Two examples from the 41pa dataset with nearly identical IFD scores but markedly different S-1FD,s scores. Top: Instructions and partial responses from two examples, with tokens highlighted where | I 0.01. Bottom: Plots of A values corresponding to the same examples. Tokens highlighted above are marked as red dots in the plots.
suggests greater difficulty and thus a more informative sample for fine-tuning. However, samples with IFD y) 2 1 are discarded, they indicate that the instruction does not aid the response generation at all. The model used to compute IFD scores can be the same as the model being instruction-tun (0' = O) or different one (0' * 0). SUPERFILTERING [17] shows that even small models like GPT-2 can be used for computing IFDs.
3 Main Contributions
3.1 Selective IFD to Capture Token-Level Informativeness
LLMS acquire most of their knowledge during pretraining. So even without the instruction, the token following a response prefix can be easy for them to predict. As shown in Figure 2a, when tokens "famous scientist" appear early in the response, pretrained LLMs can easily predict that the next token after "Albert" is "Einstein", regardless of whether the instruction is "Identify a famous scientist." " or "Who proposed the theory of relativity?" This suggests that not all tokens are informative for instruction following. Therefore, we reconsider current data selection approaches, which typically evaluate instruction-respeee response pairs as whole units without examining each token individually, and propose the question: "Within an instruction respons pair, how informative is each response token?" ?
To answer this, we revisit the IFD score. Using A, to denote log Po (Veluce, - log Por (ye (uel)))) we first rewrite Equation (1) as follows:
IFD (x v) = exp {
T
T T
(log Por (VelDee,)) log Po (walwee)) = exp expp
A
(2)
T
Here, A measures how much the log- likeliidddood of token y changes after the instruction r is provided. A small A indicates that the instruction makes little difference in generating the token, implying lower informativeness of y/- Therefore, we propose A, as a measure of token-level informativeness.
The IFD score treats all tokens equally, which might be misleading. We use the following thought experiment to demonstrate the potential issue intuitively. If for all y/ in an instruction- respoons pair, A, ~ 0.01, the IFD score would be approximately 0.99, suggesting that this training sample has very high quality. However, at the token level, none of these tokens significantly depend on the instruction, indicating low informativeness. Empirically, we find that a large fraction of response tokens in instruction tuning datasets exhibit small |A,] values, regardless of the choice of for computing IFD scores. For instance, in the Alpaca-GPP GPT-4 dataset [1]], using GPT-2 as 0', approximately 22% of

---

Compuse 1,
[[sstrccion]
High Quality Quality
itss threchold for IFD
[Resposs]]
by legother scl,
Sample
22. IFD 0,988, a 0.867
mican
IFD 8.848, 2
Sample
Sample
Beat
Figure 3: Sensitivity of IFD and S-IFD scores to a semantics-preserving word substitution. Top: An instruction-respone pair with blank, filled with either "average" or "mean". Bottom: Plot of |r] values for each variant.
Figure 4: Illustration of the selection policy Each circle represents an instruction-respone pair in the embedding space. Existing methods distinguish between high- and low-quality train- ing data using fixed threshold.
response tokens have A < 0.01, shown in Figure 5 of Appendix C.I. This proportion is around 28% for both Llama-3.1-88 3.B-BB [3] and Qwen-2.5-7B [4]. We further illustrate this by selecting two samples from the Alpaca-GPT- dataset and computing token-level |Ael using GPT-2. As shown at the bottom Figure 2, we observe that | for most tokens is small.
Our Approach Building our observation that not all tokens are equally informative, we propose Selective IFD (S-IFD), refinement of the original IFD score that focuses only on the most informative tokens. Specifically, instead of treating all response tokens equally. S-IFD considers only those with highest A.l values. To determine which tokens are informative, we introduce fixed ratio k% and select the top k% of response tokens in the entire training dataset D based on their | rankings. Formally, S-IFD with a token selection ratio 4% defined as:
S-IFDa (I, exp Σi=
Σ
wr
We
if |A, ranks top k% in the dataset D,
where wt
0
otherwise.
We illustrate the distinction between IFD and S-IFD using two examples in Figure as they share similar IFD scores but exhibit notably different S-IFD values. We view that the second example, with a higher S-IFD score, has better quality.
3.2 Hierarchical Selection Based on Local Neighborhood
We note that LLMs can be highly sensitive to subtle input changes, even those that preserve semantics. As result, for an instruction responsse pair (z. y), the computation of A for token ye may vary significantly with minor changes in the instruction variations can accumulate, leading to substantial shifts in IFD and S-IFD scores. Even small perturbations to (x. y) can lead to large fluctuations in IFD and S-IFD scores, as we illustrate through an example from the Alpaca- GP--- dataset in Figure 3. Here, the instruction asks for the average of four integers. When the word "average" is replaced with the synonym "mean"" both IFD and S-IFD7s scores drop sharply. This highlights the model's sensitivity to superficial lexical variations and suggests that high scores may sometimes capture sensitivity to spurious surface features rather than genuine semantic quality.
the preceding response tokens yer- These
The sensitivity of IFD and S-IFD scores to small input shifts motivates us to rethink these scores. Commonly, they regard training samples as high- or low-quality only based on whether their quality scores (e.g.. IFD or S-IFD) exceed given threshold [16, 17], which has several drawbacks as we illustrate in Figure 4. For example, sample B may not be reliable high-quality sample because many nearby points have low scores (essentially casy). This suggests that high average quality
5

---

among neighbors is desirable for sample. However, average quality alone might be insufficient. For instance, sample C' lies in a region with high variance. indicating instability. In contrast, the neighbors of sample A exhibit both high average scores and low local variance, making it a more robust candidate. Thus, we propose selecting samples based not only on high local average quality but also on low variance within their neighborhood.
Our Approach Motivated by our observation that local properties, including the average and variance of sample's ' neighbors' S-IFD scores, are valuable for selection, we propose to use these measures as an additional dimension for evaluating data quality. To estimate these properties for training sample (z. y), we first generate its neighbors. Rather than using complex lexical perturbations such as paraphrasing or word substitution [31, 32]. we opt for a simpler approach: injecting random noise into the token embeddings. We add uniformly distributed noise to each token embedding. For each token, we sample a noise vector with entries uniformly drawn i.i.d. from [-1, 1], then scale by e = aL + T)d, where a is a hyperparameter, L and are the lengths of the instruction and response, respectively, and d is the embedding dimension. This scaling results in perturbed samples whose expected f--norm distance from the original sample is a/ /3 [34]. We denote the perturbed embeddings of (x; y) as (x + o6 y + 6("),, ) and compute the S-IFD score for each perturbed sample. We repeat this process M times to estimate the average and variance of the S-IFD scores:
A(z, v) M
Σ
S-IFDA + .y+ )
(4)
M
(x = M
(S-IFDA (x+ 800) . 5 51 (xx v))
where ) ~ (lexd(-e,)) and (/0) (uTxx(-e,)) c) with U denoting the uniform distribution.
After computing these local properties, apply a hierarchical selection strategy to identify high- quality training samples. Given a budget b, we first select xb samples whose neighbors have the highest average S-IFD scores, where 7 denotes select the final b samples with the lowest variance in S-IFD scores among their neighbors. We outline our data selection method in Algorithm I and visualize it in Figure 1. In practice, to improve efficiency, we batch the noisy embeddings instead of using a naive for-loop.
upsampling factor. From these, we further
Algorithm I: Token-Selective Hierarchical Data Selection for Instruction Tuning (T-SHIRT)
Input: Dataset D, selection budget b, token selection ratio k% oversampling factor 7, base noise scale a, and number of perturbations M
foreach (I, 3) E 'D do
Compute + a/ (LL TTd
for i + 1 to M do
Sample noise (( ~ (=e,( e), and o64) ~ -(Txx(-ee
Compute perturbed embeddings x' +- x + ((0)) y/ + y + 8%)
Compute S-IFD, (z',1/) via Equation (3)
Compute ((x,,) v) and 8 y) via Equation (4)
Select top b samples from D with highest faa,, y) to construct $
From S, select final samples with lowest (²(2x, y) to construct S
Output: Selected subset S D of size b
4 Experiments
4.1
Experimental Setup
Instruction Tuning Datasets We conduct data selection on two widely used Instruction-tuning
datasets of different initial qualities and scales: APpaca-GPT-4 [11] and Magpie [10]. Following
For simplicity, we use jáæz, y) denote #(S- (x. v)). and o² (x, y) to denote o² (S-1IDx v)).

---

Table Performance comparison between our method, T-SHIRT, and other baseline methods on A1paca-GPT-4, We use each method to select 5% of the Alpaca-GPT-4 training samples to instruction-tune Ilaam-3.1 1-88 and Qwen-2.5-7B. We use ARC-C for ARC-Challenge HS for HellaSwag, TQA for TruthfulQA, GSM for GSM8k, AH for Arena-Harde and AE-2 for AlpacaEval 2.0. The best results are shown bold, and the second-best are underlined. T-SHIRT achieves the best or comparable performance across all cight benchmarks and outperforms existing baselines in terms of PALL. Notably, it exceeds the full dataset performance by up to 5.48 points in HALL.
OpenLLM Leaderboard
LLM-as-a-Jude
PaL
ARC-C
HS
MMLU
TQA
BBH
GSM
AnrEN
AH
AE-2
Llama-3.1-88
FULL
55.97
77.89
59.77
53.32
43.86
32.75
53.93
5.20
9.10
7.15
42.23
RANDOM
57.94
81.29
61.37
53.39
45.62
30.71
55.05
5.50
8.13
6.82
42,99
LONGEST
58.45
83.07
61.83
54.86
46.40
51.25
59.31
6.80
11.91
9.36
46,82
DEITA
59.73
81.92
62.65
51.32
46.95
45.87
58.07
7.10
8.83
7.97
45.55
DS
61.26
82.62
63.68
54.28
47.67
37.60
57.85
7.70
9.49
8.60
45.54
IFD
61.35
83.00
62.88
54.23
46.99
51.63
60.01
7.20
8,20
7.70
46.94
T-SHIRT (k- 50)
60.15
83.14
64.06
57.09
46.90
51.02
60.39
6,70
9.64
8.17
47.34
T- T-SHIRT (k- 75)
61.95
83.07
64.11
56.11
46.47
53.75
60.91
6,20
10.03
8.12
47.71
FULL
61.69
79.54
72.19
58.37
50.82
77.10
66.62
11.50
8.81
10.16
52.50
RANDOM
64.68
81.97
74.27
59.06
53.34
33.13
61.08
14.70
16.45
15.58
49.70
LONGEST
65.61
82.47
73.92
61.48
53.52
84.61
70.27
14.30
19.15
16.73
56.88
DEITA
65.02
81.94
74.15
58.77
52.65
82.79
69.22
14.60
18.41
16.51
56.04
DS
65.10
82.07
74.35
60.58
54.11
82.11
69.72
13.80
15.25
14.53
55.92
IFD
64.76
82.66
74.33
60.86
53.76
86.50
70.48
15.60
16.01
15.81
56.81
T-SHIRT (k- 50)
66.21
82.39
74.23
61.58
54.21
86.81
70.91
16.40
18.94
17.67
57.60
T-SHIRT (k= 75)
65.78
82.45
74.06
61.12
54.14
87.19
70.79
16.40
18.98
17.69
57.52
various data selection methods to select 5% of the 52k samples in 4PP-4 and approximately 3.3% (10k samples) of the 300k-sample Magpie dataset. These sampling ratios have been shown to reach an effective balance between model performance and data efficiency [15-17]. We provide additional details on these two datasets in Appendix B.1.
prior works [15-17],
Baselines On Alpaca-GPT-4 we compare our method against the following baselines: (1) FULL, which uses the entire dataset for instruction tuning: (2) RANDOM, which randomly selects data from the full training set; (3) LONGEST [23], which selects samples with the longest response lengths; (4) DEITA [14], which first trains LLM scorers on ChatGPT-labeled data to assess sample quality and complexity, then selects data based quality, complexity, and diversity; (5) DS [15], which first uses GPT-4o-mini to score training samples, then learns a score transition matrix to correct potential scoring errors, and finally selects data based both quality and diversity; (6) IFD [17]. which uses GPT-22comppteed IFD scores [16] to select nigh-quality samples. On Magpie, due to limited resources, evaluate only the best-performing baselines identified on Alpaca-GPT- More details about the baselines are provided in Appendix B.2.
Benchmarks We evaluate the performance of the trained LLMs across eight diverse bench- marks. These include six standardized benchmarks from the OpenLLM Leaderboard [35, 36]: IRC-Challenge [37], HellaSwag [38], MMLU [39], TruthfulQA [40]. BBH [41], and GSM8k [42]. For these benchmarks, we use the AMM-Evluation-Hareee
[43] and report their default evaluation metrics. We denote their average score MOPEN. In addition, assess the instruction-tunee LLMs on two recommendations of the respective benchmark creators, we report the style-controlled win rate [44] for Arena-Hard and the length-controlle win rate [45] for AlpacaEval 2. O. The average score across these two is denoted MLLM- Finally, report the overall average across all eight benchmarks as MaL.- Additional details about the benchmarks are provided in Appendix B.3.
benchmarks: Wvena-Hard [44] and AlpacaEval 2. [45]. Following the
Implementation Details of T-SHIRT To implement our method, T-SHIRT, we use GPT-2 to compute S-IFD scores. For the hyperparameters in Algorithm 1, we set the token selection ratio 4% to either 50% or 75%, the oversampling factor to y 2, the base noise scale to 5 following
7

---

Table 2: Performance comparison between our method, T-SHIRT, and other baseline methods on Magpie. We use each method to select 10k samples from Magpie to instruction-tune tune Qwen-2.5- 7B. We use abbreviated benchmark names as shown in Table The best results are shown in bold, and the second-best are underlined. Our method achieves the best performance on seven out of eight benchmarks and has the highest HALLL-
OpenLLM Leaderboard
LLM LLM-as-a-Judee
FALL
ARC-C
HS
MMLU
TQA
BBH
GSM
MoPEN
AH
AE-2
MLLM
Qwen-2.5-7B
LONGEST
63.05
80.11
73.67
56.90
53.64
82.18
68.26
19.40
27.59
23.50
57.07
IFD
64.16
80.00
73.51
57.62
53.65
84.84
68.96
19.20
31.40
25.30
58.05
T-SHIRT (k 50)
64.16
79.63
73.69
57.79
54.17
85.82
69.21
20.80
31.51
26.16
58.45
NEFTTUNE [34], and the number of perturbations to M 30. We include more implementation details in Appendix B.4.
Instruction Tuning Details We use selected data fine-tune two open-source LLMs, Llama-3.1 8B [3] and Qwen-2.5-7B [4]. Following the same training settings as prior works [16, 17, 46], we use learning rate of 2e-5 and train for 3 epochs. For data selected from Magpie, e, we instead follow the setup in Magpie [10] and train for 2 epochs. Additional training details are provided in Appendix B.5.
4.2 Main Experimental Results
In this section, we present the experimental results demonstrate the effectiveness of T-SHIRT. Additional results are provided in Appendix C.
T-SHIRT outperforms all baselines. We evaluate the performance of our method, T-SHIRT, and six baseline methods on the Alpaca-GPT-4 dataset using eight benchmarks, with results presented in Table 1. Across two base models, T-SHIRT with different token-selection ratios (k%) consis- tently outperforms all baselines in terms of PALL- Our data selection method achieves the highest performance both the OpenLLM Leaderboard and LLM-as-a-Judee benchmarks. Specifically, T-SHIRT achieves the highest Moor and PLLM scores on Qwen-2.5-7B, as well as the highest PoPeN and competitive PLLM performance
Compared to the strongest baseline, IFD, T-SHIRT delivers superior results on six out of eight benchmarks and performs comparably on the remaining two. These results demonstrate that our method significantly enhances the overall quality of selected data. Notably, selecting only 5% of the data, T-SHIRT improves MAL.. by 5.10 to 5.48 points compared to instruction tuning using the full dataset.
Llama-3.1-88
T-SHIRT is effective at selecting data across datasets of varying initial qualities and scales. We further use T-SHIRT, and the two best pess-performin baselines from Table to select 10k samples from Magpie, a recent statc-of-the-an instruction tuning dataset. Magpie is approximately 5.8x larger than Alpaca-GPT and has undergone multi-stage filtering to select 300k samples from an initial pool of one million, leading to higher initial quality. We use the selected data to fine-tune Qwen-2.5-7B and report results in Table 2. Across eight benchmarks, T-SHIRT outperforms the baselines on seven of them and achieves the best overall scores in JоPeN, PLLMS and PALL. These results demonstrate that T-SHIRT scales effectively and remains beneficial even on large datasets with high initial quality. We also fine-tune Qwen-2.5-16 14B using T-SHIRT-Selece samples from Magpie and show detailed results in Table 5 and Appendix C.2.
T-SHIRT both cost-effective and computationally efficient. When having strong performance, T-SHIRT relies only lightweight GPT-2 model to compute S-IFD scores, avoiding the need for API credits required by DEITA and 2 This makes our method both accessible and efficient. We present runtime comparisons in Table 3. On single NVIDIA A6000 GPU, T-SHIRT runs 2.7 to 3.7x faster than DEITA and D Although it is slower than LONGEST and IFD, its total runtime remains about 40 minutes, which is acceptable trade-off given its superior overall performance. Moreover, we can further improve its efficiency by reducing the number of perturbations M without hurting the performance (see ablation study on M in Section 4.3).
Proprietary LLMS are not (all) you need for data selection. As shown in Table 1, the baselines LONGEST, IFD, and our method T- T-SHIRT outperform both DETTA and D LONGEST uses
8

---

Table 3: Runtime (in hours) for data selec- tion methods on A1paca-GPT-4, We report total time including CPU, GPU, and API query time. All methods are run using single GPU. For DEITA, we do not count the time for scor- ing data via API and training LLM-based scor- ers on scored data. T-SHIRT completes data selection in just 40 minutes, making it signifi- cantly faster than both DEITA and DS
Table Ablation study evaluating the effects of S-IFD and hierarchical selection. / indicates that the component is used, while indicates not. Specifically, X for S-IFD denotes the use of the original IFD as the quality score. Thus, the first row corresponds the baseline IFD, and the last row corresponds to our method. The results show that both S-IFD and hierarchical selection contribute positively to data selection performance, and their combination yields the best results.
Runtime (h)
LONGEST
0.0
Hicrarchical Selection
PoreN
S-IFD
DRITA
1.9
Llama-3.1
Qwen-2.5-7B
DS
2.6
60.01
70.48
IFD
0.2
V
X
60.66
70.78
T-SHIRT (M 20)
0.5
X
V
60.84
70.78
T-SHIRT (M 30)
0.7
60.91
70.91
single heuristic: response length. IFD and T-SHIRT assess data quality using only weak model, GPT-2. In contrast, DEITA and DS² rely on expensive proprietary LLMs and employ more complex data selection strategies. Moreover, data selected by DEITA and DS² might contain intrinsic biases, because both of them show consistently lower performance on GSMBk, and DEITA also underperforms on TruthfulQA, regardless of the base LLM used for instruction tuning.
4.3 Ablation Studies and Analysis
In this section, we conduct ablation studies using the Alpaca-GPTI GPT-4 dataset and OpenL Leader- board benchmarks. Additional details are provided in Appendix D.
Ablation on S-IFD and Hierarchical Selection In Table we evaluate the contributions of S-IFD and hierarchical selection. The results show that, compared to the baseline IFD, both components significantly improve MorEx of instruction-tun tuned LLMs, highlighting the importance of token-level informativeness and local quality consistency in data selection. Combining both factors, our method achieves the best overall performance.
Ablation on Neighbor Variance in Hierarchical Selection As shown in Figure 6 in Appendix D.I, Morex drops significantly, by 2.29 to 4.86 points, when hierarchical selection chooses samples whose neighbors have high variance in S-IFD. This further highlights the importance of choosing data whose neighbors display consistently high quality.
Ablation on Token Selection Ratio Figure 7a in Appendix D.2 shows how MОPЕE of LLMs changes with different token selection ratios 1%. For Llama-3.1-8B the optimal ratio is 75%, while for Qwen-2.5-7B. it is 50%. These results support our observation in Section 3.1 that over 20% of the tokens in the instruction tuning dataset are not informative for effective data selection.
Ablation on Token Weights Equation (3), we assign weight (we²) of to informative tokens and 0 to uninformative ones, effectively removing the latter from the quality score computation. In Table (Appendix D.3), we compare this binary weighting scheme with alternatives that upweight informative tokens instead of fully discarding uninformative ones. Specifically, informative-to soriii--ttouuinfoomatiii token weight ratios of 1.5:1 and 2:1. Our results show that the binary weighting scheme consistently outperforms these softer alternatives. This suggests that completely ignoring uninformative tokens yields a cleaner and more reliable signal for computing quality scores.
experiment with
Ablation on Number of Perturbations Figure 7b in Appendix D.4 illustrates the impact of varying the number of perturbations M used in hierarchical selection. We observe that PoPen for Llama- 3. 3BB reaches its highest value at M = 10, while paPEy for Qwen-2.5-7B peaks at M = 20. Both models perform well at M = 30. This suggests that reducing M to lower values can improve the efficiency of our method without compromising performance.

---

Discussion
5.1 Token Embedding Perturbations vs. Lexical Perturbations
In the hierarchical selection process of T-SHIRT, we choose to generate neighbors of each sample using continuous token embedding perturbations rather than lexical perturbations. This decision is motivated by several challenges associated with lexical perturbations.
Utility Rule-based lexical perturbations often struggle to preserve semantic equivalence and fluency. As a result, prior studies [30, 32] typically rely on costly human evaluations to assess the quality and utility of perturbed examples.
Efficiency Generating lexical variants by prompting large language models through APIs both computationally and financially expensive. For example, perturbing each of the 52k Alpaca-GPT4 samples 30 times would require over 1.5 million API calls.
5.2 Limitations and Future Work
While our method shows promising results in data selection for instruction tuning, it has several limitations, which we outline along with directions for future work.
Model Scale Due to computational constraints, our experiments are limited to models in the 7B-14B parameter range, is common in several published papers in instruction tuning [10, 15, 20]. We will apply our data selection method to larger models in future work.
Data Security Considerations Our approach focuses solely identifying high-quality subsets of instruction tuning datasets rather than security concerns. Future work could extend our method to incorporate data selection strategies that explicitly focus on security in LLMs.
Theorefical Foundations Our hierarchical selection method is inspired by research in adversarial machine learning [47, 48], robustness measures [49, 50]. and LLM robustness [29, 34]. Empirical results support its effectiveness. However, deeper theoretical understanding of our approach for data selection is still lacking. In future work, we aim to establish stronger theoretical foundation to better explain our method and potentially even improve its performance.
Conclusion
In this paper, we present Token-Sclective Hierarchical Data Selection for Instruction Tuning (T- SHIRT), a framework for selecting high-quality subsets from instruction tuning datasets. Our method introduces S-IFD, quality metric that selectively incorporates only informative response tokens into quality assessment, and a hierarchical selection strategy that favors samples surrounded by consistently high-quality neighbors. Extensive experiments across range of base LLMs and instruction tuning datasets demonstrate that T-SHIRT is both effective and scalable, consistently outperforming existing baselines. At the same time, remains cost-effective and computationally efficient. These results underscore the importance of fine-grained and robust quality evaluation for data selection.
Acknowledgments and Disclosure of Funding
We thank Peng Cao, Xianhang Cheng, and Yuehui Qian for their valuable comments on this paper. This work was partially supported by NSF CAREER 2340006. F. Hamman was also supported by Ann G. Wylie Dissertation Fellowship. Y. Fu gratefully acknowledges support from the NeurIPS 2025 Travel Award.
References
[1] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Hosss,t-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler. Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models sho-shot learners. In Advances in Neural Information Processing Systems, volume 33, 2020.
10

---

[2] OpenAL. Gpt-4 technical report. arXiv:2303.08774, 2024.
[3] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and The llama herd of models. arXiv:2407.21783 2024.
[4] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng. Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, and et al. Qwen2.5 technical report. arXiv:2412.11115, 2025.
DeepSeck-AI. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv:2501. 12948, 2025.
[6] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. Finetuned language models are zero-shot learners, In International Conference on Learning Representations, 2022.
[7] Nathan I Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Cu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang. Pradeep Dasigi, and Hannaneh Hajishirzi. Tulu 3: Pushing frontiers in open language model post-training, 2025.
[8] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, LILI YU, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less is more for alignment. In seirtty-eventh Conference on Neural Information Processing Systems, 2023.
[9] Yizhong Wang. Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), 2023.
[10] Zhangehen Xu, Fengging Jiang, Luyao Niu, Yuntian Deng. Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned LLMs with nothing. In The Thirteenth International Conference on Learning Representations, 2025.
[11] Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv:2304.0322 03477777.. 2023.
[12] Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. WizardLM: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations, 2024.
[13] Lichang Chen, Shiyang Li, Jun Yan, Hai Wang. Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training a better alpaca with fewer data, The Twelfth International Conference on Learning Representations, 2024.
[14] Wei Liu, Weihao Zeng, Keqing He, Yong Jiang. and Junxian He. What makes good data for aligament? comprehensive study of automatic data selection in instruction tuning. In The Twelfth International Conference on Learning Representations, 2024.
[15] Jinlong Pang, Jiaheng Wei, Ankit Shah, Zhaowei Zhu, Yaxuan Wang, Chen Qian, Yang Liu, Yujia Bao, and Wei Wei. Improving data efficiency via curating LLM-driven rating systems. In The Thirteenth International Conference on Learning Representations, 2025.
[16] Ming Li, Yong Zhang, 7hitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang. Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting L.IM performance with self-guided data selection for instruction tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
[17] Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang. Ning Cheng, and Tianyi Zhou. Superfiltering:
data filtering for fast ceatuuction-tuning In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024.
[18] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models unsupervised multitask learners. OpenAl blog. 2019.

---

[19] Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu. yelong shen, Ruochen Xu. Chen Lin, Yujuu Yang, Jian Jiao, Nan Duan, and Weizhu Chen. Not all tokens what you need for pretraining. In The Thirty-eighth Annual Conference on Neural Information Prucessing Systemx, 2024.
[20] Jinlong Pang, Na Di, Zhaowei Zhu, Jiaheng Wei, Hao Cheng, Chen Qian, and Yang Liu. Token cleaning: Fine- time-grained data selection for Ilm supervised fine-tuning, arXiv: 01968, 2025.
[21] Xiangyu Qi. Ashwinee Panda, Kaifeng Lyu, Xiao Ma. Subhrajit Roy. Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should he made more than just few tokens deep. In The Thirteenth International Conference on Learning Representations, 2025.
[22] Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Mclanic Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base LLMs: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations, 2024.
[23] Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Long is more for aligument: simple but tough-to-beat baseline for instruction fine-tuning. In Proceedings of the 41st International Conference on Machine Learning. 2024.
[24] Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. Instruction mining: Instruction data selection for tuning large language models. In First Conference Language Modeling, 2024.
[25] Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjecv Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, 2024.
[26] Yexiao He, Ziyao Wang, Zheyu Shen, Guoheng Sun, Yucong Dai, Yongkai Wu, Hongyi Wang, and Ang SHED: Shapley-based automated dataset refinement for instruction fine-tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[27] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
[28] Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention. In Proceedings of the 38th International Conference on Machine Learning, 2021.
[29] Faisal Hamman, Pasan Dissanayake, Saumitra Mishra, Freddy Lecue, and Sanghamitra Dutta. Quantifying prediction consistency under fine-tuning multiplicity in tabular LLMs. In International Conference on Machine Learning, 2025.
[30] Di Jin, Zhijing Jin, Jocy Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. arXiv: 1907. 127./1993 1932, 2020.
[31] Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word- level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
[32] Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial GLUE: multi-task benchmark for robustness evaluation of language models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
[33] Shiqi Wang, Zheng Li. Haifeng Qian. Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, and Bing Xiang. ReCode: Robustness evaluation of code generation models. In Proceedings the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), 2023.
[34] Neel Jain, Ping yeh Chiang, Yuxin Wen, John Kinchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. NEFTune: Noisy embeddings improve instruction finetuning. In The Twelfth International Conference on Learning Representations, 2024.
[35] Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. Open I1m leaderboard (2023-2024) https: //huggingface co/spaces/opent 11m wewdereoardddd-eeeeeeeeerrrreede wateeeerbbrrdood///ppoeeeeeee open 11m _ill__eedeeeeeeeeeeeeeeeeeeeeeee leaderboard, 2023.
12

---

[36] Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. Open Ilm leader- board v2. https://huggingface http://wwwggggaaacco///////ppppa http:ppnn----nadderrrrrrdddddddd /oee/ 11m_ leaderboard. 2024.
[37] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sahharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. 103i::8833.055571,.2 2018.
[38] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
[39] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazcika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. In International Conference on Learning Representations, 2021.
[40] Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehonds. In Proceedings the 60th Annual Meeting of the Association for Computtaioonal Linguistics (Volume I: Long Papers), 2022.
[41] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Lc, Ed H Chi, Denny Zhou, and Jason Wei. Challenging big-benel tasks and whether chain-of-thought can solve them. arXiv:2210. 2el:2226000 2022.
[42] Karl Cobbe. Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv:2110. 10. 11110.44.0 /4168, 2021.
[43] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black. Anthony DiPofi. Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac h. Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang. Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 2024. URI. https://zenon
org/records/ 12608602....0...
[44] Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks; Arema- and benchbuilder pipeline. arXiv:2406.1193 11939, 2024.
[45] Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. AXiv:2404.044755 2024.
[46] Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloingg Ilama model. https://gittub. com/tatsu- wtwpaaa/ssaaooddaaaaaaaaaee
2023.
[47] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning, 2019.
[48] Aditi Raghunathan, Jacob Steinhardt, and Percy S Liang. Semidefinite relaxations for centifying robustness to adversarial examples. Advances in neural information processing systems, 2018.
[49] Faisal Hamman, Erfaun Noorani, Saumitra Mishra, Daniele Magazzeni, and Sanghamitra Dutta. Robust counterfactual explanations for neural networks with probabilistic guarantees. In Proceedings of the 40th International Conference on Machine Learning, 2023.
[50] Tessa Han, Suraj Srinivas, and Himabindu Lakkaraju. Characterizing data point vulnerability as average- case robustness. In Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, 2024.
[51] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch. Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso. and al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023.
[52] Wei- Chiang. Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang. Michael I. Jordan, Joseph E. Gonzalez, and lon Stoica. Chatbot arena: an open platform for evaluating Ilms by human preference. In Proceedings of the 41st International Conference on Machine Learning, 2024.
13

---

[53] Wenting Zhao, Xiang Ren. Jack Hessel. Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference Learning Representations, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction in Section I clearly reflect our paper's contribu- tions and scope. We also provide a summary of our contributions in Appendix A.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Section 5.2.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
This is not theoretical paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We present our method in Section 3, and detail the experimental setup in Sec- tion 4.1 and Appendix B.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: All data used this work publicly available, and our code available in a public GitHub repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /CodeSubmiiiie //n ccppb.c//pullic//uu hut.b/iiiguudddeeddduuubbiiiiii woabbbisssiini ssisnsolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide details of the experimental settings in Section 4.1 and Appendix B.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviation of evaluation results across three random seeds in Appendix C.3. However, due to limited computational resources and API budget, we do not report standard deviations for all experiments and benchmarks.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details of the computational resources used for instruction tuning are provided in Appendix B.5, and the runtime for executing both the baselines and our methods is reported in Table 3.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https /etn://ipppppp cc cccppblic//////cceeccccecccllclt wtt.ttiiiiiiiieeeeeeeeeeee
Answer: [Yes]
Justification:
We adhere strictly to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification; This work primarily focuses data selection using publicly accessible datasets and therefore does not pose any direct societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: As this work relies solely public datasets and open-source LLMs, not aware of any related risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
In this paper, we credit them properly.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com /atasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have open- open-sourced our code in publicly available GitHub repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this paper does involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https:// http://eeeeuneeeeeeeeee httpp/////errps..cooofeeeeeeeeee /ttpa////ooooeeeeeee//////////// for what should or should not be described.
21

---

Appendix
Contents
Section Summary of Contributions
Section B: Experimental Setup Details
Section B.1: Instruction Tuning Datasets
Section B.2: Baselines
Section B.3: Benchmarks
Section B.4: Implementation Details of T-SSIRT
Section B.5: Instruction Tuning Details
Section Additional Experiments
Section C.I: CDF of A with GPT-2 as 6'
Section C.2: Performance of T-SHIRT Qwen-2.5-14B
Section C.3: Standard Deviation of Results
Section D; Ablation Stuyy Details
Section D.1: Ablation on Neighbor Variance in Hierarchical Selection
Section D.2: Ablation Token Selection Ratio
Section D.3: Ablation on Token Weights
Section D.4: Ablation Number of Perturbations
Summary of Contributions
Our contributions in this paper are summarized as follows:
We analyze existing instruction tuning datasets and find that 20% of the response tokens are uninformative. Excluding these uninformative tokens from IFD score computation significantly alters the results (see Section 3.
We investigate the robustness of IFD and observe that it is sensitive to small, semanties- preserving perturbations. Our findings suggest that high IFD sometimes reflects superficial features the training sample rather than its intrinsic quality (see Section 3.2).
To address these limitations, we present T-SHIRT, novel data selection framework. We first introduce S-IFD, new quality metric that incorporates token-level informativeness into assessment. In addition, we propose a robust, hierarchical selection pipeline that prioritizes samples with high average neighborhood quality and low variance (see Section 3).
We conduct extensive experiments showing that our method achieves superior performance using more than 5% of the full instruction tuning datasets. Our approach consistently outperforms prior data selection methods across varying dataset sizes and base LLMs. Moreover, it remains cost-effective and efficient: single GPU, it processes 52k-sample dataset in approximately 40 minutes, with no additional API costs (see Section 4).
B Experimental Setup Details
B.1 Instruction Tuning Datasets
In this paper, we perform data selection on two instruction tuning datasets, Alpaca-GPT-4 and Magpie, and they have varying scales and quality:
Alpaca- A1pacc-GPT-4 GPT-4 [11] contains 52k instructiin-ressonse pairs. It keeps the original instruc- tions from the Alpaca dataset [46], but regenerates the responses using GPT-4 [2] instead of GPT-3.5, resulting in improved response quality.
22

---

Magpie [10], specifically the Mert version," is a fully synthetic dataset comprising 300k hugh-quality instruction-response pairs. is constructed by prompt- ing allama3--70B-Istrree to generate one million samples, followed by multi-stage filtering. Compared Alpaca-GPP Magpie not only larger but also of much higher quality.
B.2 Baselines
We detail the three baselines: LONGEST, DEITA, and DS?, which are used in our experiments:
LONGEST [23] applies a simple heuristic that selects the longest samples from the dataset, based on the assumption that longer responses tend to more informative and useful for instruction tuning. We tokenize responses using the Llama-3.1-88 tokenizer and select those with the highest token counts.
DEITA [14] follows four-step selection process. It begins with small seed dataset and evolves its samples to span different levels of quality and complexity, similar to WIz- ARDLM [12]. Then, it prompts ChatGPT to label these samples by quality and complexity. Using the labeled data, it trains its own LLM-based scorers. Finally, it selects high-scoring. diverse samples from the full dataset, skipping those similar to ones already chosen. In our experiments, we use their released scorers and codebase to select data.
DS² [15] also follows four-step selection process. first scores the full dataset by prompt- ing GPT-4o-mini. Then, it learns ecore-transition matrix to correct possible scoring errors. Next, estimates each sample's diversity using feature embeddings by measur- ing the average cosine similarity to its K-nearest neighbors. Finally, it selects samples with both high corrected quality scores and high diversity. In our experiments, we use CPT-40-mini-20 202---22222--222 and DS codebase to score and select the data.
B.3 Benchmarks
We extensively evaluate instruction-tuned LLMs using eight benchmarks, including six standardized ones from the OpenI..M Leaderboard [35, 36]:
ARC-Challenge [37]: challenging subset of the multi-choice question answering bench- mark ARC, consisting 2,590 natural science questions.
HellaSwag [38]: commonsense inference benchmark with 10k test samples, designed to assess models' ability to complete sentences plausibly.
MMLU [39]: diverse question-answering benchmark covering 57 tasks (e.g., STEM, US history), used to measure models' world knowledge and problem-solving skills.
TruthfulqA [40]: benehmark of 817 questions focused on assessing models' truthfulness in the face of common misconceptions and false beliefs.
вBH [41]] a collection of 23 challenging tasks from BIG-Bench [51], often requiring multi-step reasoning.
GSM8k [42]: a test set of roughly 1 grade-school-level math word problems targeting multi-step mathematical reasoning.
Following the OpenLLM Leaderboard, we use the codebase of MMM-Evauatioo-Harreee [43] for evaluation. We report normalized accuracy for ARC-Challenge, HellaSwag, and BBH; accuracy for MMLU and TruthfulQA: and exact match for GSM8k.
Then we evaluate the instruction LLMs act as proxies for human annotators by comparing responses from the evaluated model and baseline model, then indicating preference. Specifically, we use:
LLMs using two LLM-as-a-Judge benchmarks, where strong
Arena-Hard [44]: comprises 500 challenging queries sourced from Chatbot Arena [52] and WildChat- 1M [53]. It introduces the style-controlled win rate (SC) to mitigate judges' bias related to stylistic features, including answer length and density of markdown headers. We use CPT-4-0314 as the baseline model and Gemini-2 --w.aah--revioo-00--111111 as the judge, reporting SC scores.
Atctp::/hhuggingaacccooo/// httpp///wwgggaaeooo////aaaataaaa hwtpgiiee--lgggggge-eeeeeeee Pro-3000-------------- 600k-Piltered
23

---

90%
70%
60%
50%
30%
20%
10%
0%
0.1
0.2
0.3
0.4
0.5
1,,1
Figure 5: CDF of A (Equation (2)) for response tokens in Alpaca-GPT-4 computed using GPT-2.
The red dash line indicates A = 0.01.
AlpacaEval 2.0 [45]: contains 805 queries from AlpacaEval [46] and uses the length-controlled win (LC) mitigate judges' bias on response length. We 197-4-1106-Previees as the baseline model and GPT-4o-2024-08-00 as the judge, re- porting LC scores.
Compared to the official settings of Arena-Hard and AlpacaEval 2.0, we use the same baseline models but different judges due to API budget constraints.
B.4 Implementation Details of T-SHIRT
To implement hierarchical selection in our method, straightforward approach would be to iterate M times and recompute the S-IFD score for each perturbed sample individually, as shown in Algorithm 1. However, this is computationally inefficient, especially since GPT-2 is relatively small and leaves much of the GPU unused. Instead, we generate a noise tensor of shape M x (L + T)) x d, where M is the number of perturbations, L is the instruction length, T is the response length, and d is the embedding dimension. This noise is added to the sample's 's token embeddings using broadcasting. This allows to compute perplexities for all M perturbed samples in single forward pass through GPT-2, which significantly reduces the time cost compared to repeated evaluations with a for-loop.
B.5
Instruction Tuning Details
For data selected from 4Ppaca-GPT-4, we follow the training settings used in prior works [16, 17, 46]: learning rate of 2e-5, training epochs, batch size of 128, warmup ratio of 0.03, and cosine learning rate schedule. For data selected from Magpie, we keep the same settings but reduce the number of training epochs to 2, in line with the official configuration of Magpie [10]. All instruction tuning experiments are conducted on server equipped with NVIDIA A6000 GPUs.
C
Additional Experiments
C.1 CDF of |A,] with GPT-2 as 0'
Figure shows the cumulative distribution function (CDF) |A,] (as defined in Equation (2)) for response tokens in the Alpaca-GPP CPT-4 dataset, computed using GPT-2 as 0', Notably, 20% of tokens have |A, < 0.009, and 50% have | S 0.050, indicating that substantial portion of tokens are relatively uninformative even for GPT-2.
24

---

Table Performance comparison between our method, T-SHIRT, and other baseline methods on Qwen-2.5-14B and Magpie. We use each method to select 10k samples from Magpie to
instruction-tune Qwen-2.5-14B, We use abbreviated benchmark names as shown Table The best results are shown in bold, and the second-best are underlined.
OpenLLM Leaderboard
LLM ALM--s-a-Judae
PALL
ARC-C
HS
MMLU
TQA
BBH
GSM
MoPEN
AH
AE-2
Qwen-2.5-14B
LONGEST
68.17
84.01
79.23
58.69
61.00
85.22
72.72
34.40
35.01
34.71
63.22
IFD
68.94
84.05
79.07
57.87
61.78
83.02
72.46
33.10
37.15
35.13
63.12
T-SHIRT (k 50)
68.60
83,9
79.26
58.60
62.04
84.15
72.76
35.30
38.45
36.88
63.79
Table Average performance and standard deviation RANDOM and T-SHIRT across three random seeds on Qwen-2.5-7B and Alpaca-GPT-H We use abbreviated benchmark names as shown in Table 1.
Op Leaderboard
ARC-C
HS
MMLU
TQA
BBH
GSM
Qwcn-2.5-7B
RANDOM
64.42
81.89.0.07
74.15
59.30
53,461109
50.52
63.9612.32 87.34 10.91
T-SHIRT (A: 50)
06.04.0.24
82.36.10.03
74.09
61.42...17 54.21
C.2 Performance of T-SHIRT on Qwen-2.5-14B
In the main experiments (Section 4), primarily evaluate T-SHIRT models with 7B 8B-8B parame- ters. Here, we extend the evaluation to Qwen-2.5-14B and compare T-SHIRT against LONGEST and IFD using experimental settings similar to those in Section 4.2. The results shown in Table 5. On Magpie, T-SHIRT consistently outperforms both baselines across MOPEN. PLaM and PALL Notably, achieves average improvement of least 1.75 points the LLM-as-a-Judge benchmarks.
C.3 Standard Deviation of Results
Both the baseline RANDOM and our proposed method T-SHIRT involve stochasticity. Within our limited budget, we run each method on Qwen-2.5-7B and Alpaca-GPT- using three different random seeds, and report the mean and standard deviation of their performance on the OpenLLM leaderboard benchmarks. The results, shown in Table 6, are consistent with the observations reported in the main paper.
D Ablation Study Details
This section provides details of our ablation study, where we use the average performance on the OpenLLM Leaderboard benchmarks, popes, the primary metric. For ease of reading, make this section self-contained by including the discussion also provided in the main paper with additional details.
D.1 Ablation on Neighbor Variance in Hierarchical Selection
We examine the role of selecting samples whose neighbors exhibit low quality variance in the hierarchical selection process (see Algorithm and Section 3.2 for details). As shown in Fig- ure 6, performance for both Llama-3, 1-BBBB and Qwen-2.5-7B drops significantly when high-variance neighborhoods are preferred over low-variance ones. This highlights the importance of selecting samples from consistently high-quality neighborhoods and further validates the effectiveness of our hierarchical selection approach.
'While conducting this experiment, Gemini-2, S--Flaah-Peevii 04-11 became unavailable, so we used Gemini-2.55 5-Fassh instead.
25

---

High Variance
Low Variance (Ours)
75.0
68.8
66.05
NoPN 62.5
60.91
58.62
56.3
L.1ma-3.1-BB
Qwen-2.5-7B
Figure 6: Ablation study on the impact of preferring samples with low-variance (z, y) in Equation (4)) versus high-variance neighbors during hierarchical selection. The results indi- cate that selecting samples whose neighbors exhibit high variance significantly degrades performance.
T.lmms-3.1-
Qwen-2.5-7R
Lluma 3.1 11:80
Qwen 2,5-70
61.0
71.0
61.0
71.0
60.7
70.9
60.9
70.5
Porzy 60.4
70.8
Jorex 608
70.7
60.1
70.7
60.6
70.5
99.3
70.6
60.5
70.3
25%
50%
75%5
100%
10
30
Selcction Ratio
Number Penurbations
(a) Ablation study token selection ratio k%.
(b) Ablation study number of perturbations M.
Figure 7: Ablation study on two hyperparameters: the token selection ratio 1% in S-IFD and the number of perturbations M in hierarchical selection. both subfigures, the left y-axis represents Hopex for 1.ama-3.11-88 optimal token selection ratios are 75% for Llama-3.1-88 and 50% for Qwen-2.5-7B, (b) indicates that the optimal number of perturbations is 10 for Llama-3. 1.taa-..1--3.. 1-8B and 20 for Qwen-2.5-7B, Both models maintain strong performance when M = 30.
and the right y-axis represents HoPEN for Owwen-2.5-7B. (a) shows that the
Table 7: Ablation study on token weights (w, in Equation (3)) for informative uninformative tokens. We use abbreviated benchmark names as defined in Table 1. The best results are shown in bold, and the second-best are underlined. The results indicate that completely removing uninformative tokens from the S-IFD computation is more effective than merely upweighting informative tokens.
OpenLLM Leaderboard
We Ratio
ARC-C
HS
MMLU
TQA
BBH
GSM
PoPEN
Qwen-2.5-7B
1.5:1
65.44
82.37
74.09
61.44
53.83
86.05
70.54
2:1
65.53
82.17
74.19
60.73
53.95
86.81
70.56
1:0 (Ours)
66.21
82.39
74.23
61.58
54.21
86.81
70.91
D.2 Ablation on Token Selection Ratio
We ablate the token selection ratio 4% to assess impact on data selection performance. Here, ratio of 100% corresponds to using IFD (i.e., all tokens are used for computing quality scores), rather than S-IFD. As shown Figure 7a, Llama-3.1 achieves optimal performance k = 75, while Qwen-2.5-7B's performance peaks at k 50. These results support observation in Section 3.1
26

---

that over 20% of response tokens in instruction tuning datasets are uninformative. Using S-IFD to focus only on informative tokens enhances data quality assessment and improves data selection for instruction tuning.
D.3 Ablation on Token Weights
We ablate the choice of token weights wa used in computing the S-IFD score in Equation (3). In addition to removing uninformative tokens by setting their weights to O, we also experiment with upweighting informative tokens by setting the weight ratios between informative and uninformative tokens to 1.5:1 and 2:1. As shown in Table 7, removing uninformative tokens entirely yields better performance than merely upweighting informative ones.
D.4 Ablation Number of Perturbations
We ablate the effect of the number of perturbations M in hierarchical selection. Llama-3. 1-8B performs best M = 10, while Qwen-2.5-7B reaches optimal performance at M 20. Both models maintain strong performance at M = 30, suggesting that our method, T-SHIRT, can be made more efficient by reducing M without sacrificing performance.
27


Reviews:
[{"review_id": "lDqtVLUnF4", "reviewer": "Reviewer_WDFx", "summary": "This paper proposes a new instruction tuning method, called T-SHIRT, i.e., Token-Selective Hierarchical Data Selection for Instruction Tuning.  The general idea is to select high–quality instruction-tuning data to improve training efficiency and reduce data redundancy.  Some recent studies use LLM-based scoring functions, e.g., Instruction-Following Difficulty (IFD). While these data selection methods often lead to models that can match or even exceed the performance of models trained on the full datasets, this work identifies two key limitations: (i) they assess quality at the sample level, ignoring token-level informativeness; and (ii) they overlook the robustness of the scoring method, often selecting a sample due to superficial lexical features instead of its true quality. \n\nThis work proposes Token-Selective HIeRarchical Data Selection for Instruction Tuning (T-SHIRT), a novel data selection framework that introduces a new scoring method to include only informative tokens in quality evaluation and also promote robust and reliable samples whose neighbors also show high quality with less local inconsistencies. The paper demonstrates that models instruction-tuned on a curated dataset (only 5% of the original size) using T-SHIRT can outperform those trained on the entire large-scale dataset by up to 5.48 points on average across eight benchmarks.", "questions": "I raised a few questions in the weakness section.  Please check those questions and have some clear explanation.", "limitations": "The limitation section discussed a few issues related to the current work.  I think the limitation discussion section is clear and informative.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1.  It identifies two limitations of recent instruction tuning works which use LLM-based scoring functions: (i) they assess quality at the sample level, ignoring token-level informativeness; and (ii) they overlook the robustness of the scoring method, often selecting a sample due to superficial lexical features instead of its true quality.  \n\n2. It proposes a new instruction tuning method, T-SHIRT, which selects high–quality instruction-tuning data to improve training efficiency and reduce data redundancy.  The new methodology on instruction tuning: It uses hierarchical selection to choose samples whose neighbors exhibit high average S-IFD and low variance. T heir token-selective hierarchical data selection for instruction tuning method introduces a new scoring method to include only informative tokens in quality evaluation and also promote robust and reliable samples whose neighbors also show high quality with less local inconsistencies\n\n3.  High performance: The new method T-SHIRT can generate new instruction-tuned models on a much small, curated dataset (5% of the whole set) but outperforms those trained on the entire large-scale dataset.\n\n4.  The overall analysis seems to be clear and in high quality.\n\n Weakness:\n\n1. For Table 1 (the major performance comparison between our method, T-SHIRT, and other baseline methods on Alpaca-GPT-4), the performance gain seems not that great.   Especially using LLM as a judge (which is closer to human evaluation), the other method seems to show higher performance on Llama 3.1-8B.   I am not sure how you can explain this result.\n\n2. The ablation study does not show quite impressive differences.  I am not sure how you can explain this performance result (Table 4).\n\n3.  The experiments are all done on relatively smaller models (e.g., Llama 3.1-8B, Qwen 2.5-7B).  You mentioned this in the limitations but it should have some discussions on how the method can be extended to large models and what can be difficulties based on your experiments.", "was_revised": true, "final_justification": "Based on my reading of authors' rebuttals and other reviewers' comments, I would like to maintain my current evaluation comments and rating.", "rebuttal": "We thank the reviewer for appreciating our technical contributions and high-quality analysis. We also thank the reviewer for the constructive comments. We hope the following responses address the weaknesses (W) raised by the reviewer.\n\n### **Notations and clarifications.**\nIn our responses, we follow the draft’s notations for several benchmark names: ARC-C (ARC-Challenge), HS (HellaSwag), TQA (TruthfulQA), GSM (GSM8k), AH (Arena-Hard), and AE-2 (AlpacaEval 2.0). We use $\\mu_{\\text{open}}$ to denote the average performance across six OpenLLM Leaderboard benchmarks, $\\mu_{\\text{LLM}}$ for the average performance across two LLM-as-a-Judge benchmarks, and $\\mu_{\\text{all}}$ for the average performance over all eight benchmarks. Additionally, we clarify that *Gemini-2.5-Flash-Preview-04-17* was used in our draft as the judge for evaluating AH. Since this checkpoint is no longer available via API, we now use the stable version of *Gemini-2.5-Flash*, which affects the absolute scale of AH scores.\n\n### **W1: Performance gains of our method.**\n\nWe believe this concern primarily stems from the hyperparameter settings used in our main experiments, particularly the token selection ratio $k$% defined in Equation (3). In the draft, we limited our evaluation to $k = 50$ and $k = 75$ to demonstrate that our method performs well even without extensive hyperparameter tuning.\n\nTo address this concern, we conducted a finer-grained sweep over $k \\in \\lbrace 50, 60, 70, 80, 90 \\rbrace$, using the same instruction tuning dataset and benchmark setup as in Table 1. On *Llama-3.1-8B*, performance peaks at $k = 70$. The full results are provided below.\n\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:------: |:------: |:------: |:------: |:------: |:------: |:------: |:------: |:------: |:------: |:------: |\n| **IFD**                     | $61.35$ | $83.00$ | $62.88$ | $54.23$ | $46.99$ | $51.63$ | $60.01$ | $\\mathbf{5.50}$ | $8.20$ | $6.85$ | $46.72$ |\n| **T‑SHIRT ($k=70$)**        | $\\mathbf{63.48}$ | $\\mathbf{83.26}$  | $\\mathbf{63.92}$  | $\\mathbf{55.19}$  | $\\mathbf{47.13}$  | $\\mathbf{53.15}$  | $\\mathbf{61.02}$  | $5.40$ | $\\mathbf{11.22}$  | $\\mathbf{8.31}$  | $\\mathbf{47.84}$  |\n\nWith $k = 70$, our method, T-SHIRT, yields a clear improvement over the strongest baseline, IFD:\n\n* On $\\mu_{\\text{open}}$, T-SHIRT outperforms IFD by 1.01 points.\n* On $\\mu_{\\text{LLM}}$, which better approximates human preferences, T-SHIRT shows a 1.46-point gain.\n* Overall, T-SHIRT achieves a 1.12-point improvement in $\\mu_{\\text{all}}$.\n* T-SHIRT outperforms IFD on 7 out of 8 benchmarks and performs comparably on the remaining benchmark (AH).\n\nThese results indicate that after tuning a single hyperparameter $k$, T-SHIRT consistently delivers stronger performance. We will include these results in the revised version.\n\n### **W2: Interpretation of ablation study results.**\n\n**Both components bring improvements.** Our ablation study in Table 4 demonstrates that both of our proposed components, (1) S-IFD, which estimates token-level informativeness, and (2) hierarchical selection, which favors samples surrounded by consistently high-quality neighbors, contribute positively when applied independently. As shown in our response to W1, careful tuning of the hyperparameter $k$ further increases the performance gains of our method.\n\n**There is synergy between the two components.** While combining both components results in improved performance compared to using either one alone, the gains are not strictly additive. We believe this is due to the synergy between the components, as they may target partially overlapping aspects of data quality. We will clarify this interpretation in the revised version and plan to investigate their interaction more thoroughly in future work.\n\n### **W3: Applicability to larger models.**\n\nWe thank the reviewer for this valuable suggestion. To demonstrate the scalability of our method, we follow Table 2 and apply our approach (with $k = 50$) to the Magpie dataset and fine-tune a larger model, *Qwen-2.5-14B*. Due to time limitations, we do not conduct further hyperparameter tuning. The evaluation results are shown below:\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| **Longest**                 | $68.17$ | $\\underline{84.01}$ | $\\underline{79.23}$ | $\\mathbf{58.69}$ | $61.00$ | $\\mathbf{85.22}$ | $\\underline{72.72}$ | $\\underline{34.40}$ | $35.01$ | $34.71$ | $\\underline{63.22}$ |\n| **IFD**                     | $\\mathbf{68.94}$ |\t$\\mathbf{84.05}$ | $79.07$ | $57.87$ | $\\underline{61.78}$ | $83.02$ | $72.46$ | $33.10$ | $\\underline{37.15}$ | $\\underline{35.13}$ | $63.12$ |\n| **T‑SHIRT ($k=50$)**        | $\\underline{68.60}$ | $83.90$ | $\\mathbf{79.26}$ | $\\underline{58.60}$ | $\\mathbf{62.04}$ | $\\underline{84.15}$ | $\\mathbf{72.76}$ | $\\mathbf{35.30}$ | $\\mathbf{38.45}$ | $\\mathbf{36.88}$ | $\\mathbf{63.79}$ |\n\nOur method consistently outperforms the strongest baselines across $\\mu_{\\text{open}}$, $\\mu_{\\text{LLM}}$, and $\\mu_{\\text{all}}$. Notably, it achieves an average improvement of at least 1.75 points on LLM-as-a-Judge benchmarks. These results suggest that our data selection method remains effective and beneficial for instruction tuning of models larger than 7-8B.\n\nWe will include these results in the revised version."}, {"review_id": "eWCZ3JPQpJ", "reviewer": "Reviewer_6LJR", "summary": "Many recent papers have studied how to select samples to use for instruction tuning by scoring with an LLM. In particular, an approach called IFD scores samples based on the change in response perplexity when conditioned on the instruction. However, this approach has two issues: 1) the change in response perplexity is computed over all tokens, even though some tokens in the response are very predictable regardless of the instruction (i.e., low change in PPL), and 2) the change in response perplexity may be sensitive to the choice of instruction --- small perturbations may impact this score. To address this, the paper presents T-SHIRT, which scores samples based on the top k% of token-level perplexity changes, rather than the perplexity change over the entire response. This scoring mechanism reduces the noise from uninformative tokens, like \"Einstein\" often coming after \"Albert\". Second, T-shirt more robustly selects samples based on a smoothed score within a neighborhood, where the neighborhood is constructed by adding Gaussian noise to the token embeddings. Experiments are conducted on two instruction tuning datasets and Llama/Qwen as base models.", "questions": "1. In Table 1/Table 2, there are some tasks (HS, MMLU, BBH, AH) where IFD does better than both versions of T-SHIRT. Does there exist a k besides 50%/75% that does allow for T-SHIRT to outperform IFD? \n2. Can we better understand the inconsistency in the relationship between k% and the performance across Qwen and Llama? For instance, perhaps a more fine-grained sweep of k or an assessment on another LM would be helpful. \n3. What does token embedding perturbation look like in terms of the actual text? Does it look vastly different from the word substitution example provided that swaps \"mean\"/\"average\"?\n \nOverall, my score is a borderline accept. The proposed modifications are well-motivated and likely to be useful in practical, budget-constrained settings. However, I am not giving a 5 because I believe the experiments could be expanded to more thoroughly assess the limits of T-SHIRT and its behavior across different base models. I also find the novelty somewhat incremental, though I defer to reviewers with deeper familiarity in this area, and I am open to changing my opinion on this.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nQuality: \n- T-SHIRT's performance is demonstrated on both Alpaca-GPT-4 and Magpie datasets. This is promising, given that these datasets differ in how old they are and in terms of how heavily curated they are. This suggests that T-SHIRT is robust and may continue to offer improvements on future instruction tuning datasets.\n\nClarity:\n- Paper is well-motivated, with illustrative examples of 1) cases where IFD and S-IFD have different vs similar scores and 2) cases where IFD/S-IFD are sensitive to the wording in the instruction. \n\nSignificance:\n- The ability to score samples for data selection in a cheap and effective way is important. In comparison, many existing data selection methods involve using relatively large, costly LLMs.\n\n**Weaknesses**\n\nQuality:\n- In Table 1/Table 2, there are some tasks (HS, MMLU, BBH, AH) where IFD does better than both versions of T-SHIRT. Is this due to the choice of T-SHIRT's hyperparameter k? That is, is there a setting of k where T-SHIRT consistently outperforms IFD, but this was not identified due to coarse sweeping? The method would be more compelling if you could confirm that there always is some k/other hyperparams such that using top-k% token scoring always outperforms scoring based on all tokens equally.\n- In Figure 6a, it seems like the trends of k% vs performance are not consistent between Llama and Qwen. For Llama, higher k% translates to better performance, with approximately a 1 point difference between k=25 vs 75 - this is quite significant when you look at the range of accuracies in Table 1. For Qwen, performance appears to slightly decrease as k% increases. This inconsistency is surprising given that 28% of tokens have low $\\triangle_t$ for both Llama and Qwen (L152). This raises concerns about whether we fully understand how T-SHIRT interacts with different base models, and about how practitioners using other base models should set k if they want to use this method. \n- A minor suggestion to strengthen the case for T-SHIRT in comparison to methods that use API-based models to score samples: it would be interesting if one could implement DEITA/DS^2 using GPT2-based scores. Perhaps, you could show that GPT-2 fails to produce meaningful scores, and thus T-SHIRT/IFD are the only methods that would work at all when using a smaller LM. That is, can you argue that if a practitioner is budget constrained and must use a weaker LM, then no construction of DEITA or DS² will be viable? This would help clarify a practical niche that T-SHIRT/IFD has.\n\nClarity:\n- What does token embedding perturbation look like in terms of the actual text? Does it look vastly different from the word substitution example provided that swaps \"mean\"/\"average\"?\n\nOriginality: \n- While the proposed modifications to IFD are sensible, they have somewhat limited novelty, given that token-level scoring and neighborhood smoothing for robustness are well-known in many contexts. However, I am not completely familiar with all the works in the instruction tuning data selection space, so this concern is minor.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive assessment of our method’s scalability, well-motivated design, and efficiency, and we appreciate the constructive feedback. We hope the following clarifications help address the weaknesses (W) and questions (Q) raised by the reviewer.\n\n### **Notations and clarifications.**\nIn our responses, we follow the draft’s notations for several benchmark names: ARC-C (ARC-Challenge), HS (HellaSwag), TQA (TruthfulQA), GSM (GSM8k), AH (Arena-Hard), and AE-2 (AlpacaEval 2.0). We use $\\mu_{\\text{open}}$ to denote the average performance across six OpenLLM Leaderboard benchmarks, $\\mu_{\\text{LLM}}$ for the average performance across two LLM-as-a-Judge benchmarks, and $\\mu_{\\text{all}}$ for the average performance over all eight benchmarks. Additionally, we clarify that *Gemini-2.5-Flash-Preview-04-17* was used in our draft as the judge for evaluating AH. Since this checkpoint is no longer available via API, we now use the stable version of *Gemini-2.5-Flash*, which affects the absolute scale of AH scores.\n\n### **W1 & Q1: Performance comparison between IFD and T-SHIRT across benchmarks.**\nWe thank the reviewer for the thoughtful suggestion regarding finer hyperparameter tuning. In our draft, we used $k = 50$ and $k = 75$ to show that T-SHIRT can perform well even without heavy hyperparameter tuning. To explore hyperparameter choice further, we conduct a finer sweep over $k \\in \\lbrace 50, 60, 70, 80, 90 \\rbrace$ using the same dataset and evaluation benchmarks from Table 1. We found that for *Llama-3.1-8B*, performance peaks at $k = 70$. We show the results in the following table.\n\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| **IFD**                     | $61.35$ | $83.00$ | $62.88$ | $54.23$ | $46.99$ | $51.63$ | $60.01$ | $\\mathbf{5.50}$ | $8.20$ | $6.85$ | $46.72$ |\n| **T‑SHIRT ($k=70$)**        | $\\mathbf{63.48}$ | $\\mathbf{83.26}$  | $\\mathbf{63.92}$  | $\\mathbf{55.19}$  | $\\mathbf{47.13}$  | $\\mathbf{53.15}$  | $\\mathbf{61.02}$  | $5.40$ | $\\mathbf{11.22}$  | $\\mathbf{8.31}$  | $\\mathbf{47.84}$  |\n\nWith $k = 70$, T-SHIRT outperforms IFD on 7 out of 8 benchmarks and performs competitively on the remaining one (AH). Specifically, T-SHIRT improves over IFD by:\n* +1.01 points on $\\mu_{\\text{open}}$,\n* +1.46 points on $\\mu_{\\text{LLM}}$,\n* +1.12 points on $\\mu_{\\text{all}}$.\n\nThese results confirm that with appropriate tuning of $k$, T-SHIRT can consistently outperform IFD. We will include these results in the revised version.\n\n\n### **W2 & Q2: Different choices of hyperparameter $k$ for *Llama* and *Qwen*.**\n**Clarification on the token selection ratio $k$%.** We clarify that, in Equation (3), we include a token $y_t$ in the computation of S-IFD only if its token-level informativeness $|\\Delta_t|$ ranks in the top $k$% of the dataset. This means that we filter out the least informative $(100-k)$% of tokens, which are often easy to predict and potentially noisy. \n\n**Interpretation for Figure 6a.** The trends in Figure 6a show that:\n* On *Llama-3.1-8B*, using $k=75$, i.e., removing 25% of the least informative tokens from S-IFD computation, yields strong performance. In contrast, setting $k=25$, which removes 75% of the tokens, is too aggressive and degrades performance.\n* On *Qwen-2.5-7B*, we observe the best performance at $k=50$, indicating that this stronger pre-trained model can benefit from filtering out a larger fraction (50%) of the least informative tokens.\n\nThese trends align with our pilot analysis (L151–L152 in our draft), which indicates that at least 22–28% of tokens are uninformative and can be excluded from S-IFD computation. We will include our clarification and additional results (e.g., for $k = 70$ that removes 30% of the least informative tokens) in the revision.\n\n### **W4 & Q3: Token embedding perturbations and word substitutions.**\n**The random noise is continuous.** The noise injected into token embeddings is continuous and cannot be directly mapped back to discrete tokens, unlike word substitutions (e.g., replacing \"mean\" with \"average\"). By choosing an appropriate noise level, the perturbed embedding remains close to the original token embedding in the embedding space. \n\n**Intention of choosing token embedding perturbations.** We clarify that we deliberately designed embedding perturbation as a computationally efficient alternative to simulating lexical perturbations such as word substitutions. Below, we provide further discussion of our rationale for choosing continuous embedding perturbation:\n* **Utility.** Rule-based substitutions may struggle to preserve semantic equivalence and fluency. Prior works \\[1, 2\\] require costly human evaluations to verify the utility.\n* **Efficiency.** Prompting APIs to generate perturbed variants is prohibitively expensive. For example, perturbing each of the 52k Alpaca-GPT4 samples 30 times would require over 1.5 million API queries.\n\nTherefore, embedding perturbation is a lightweight, efficient way to consider robustness in scoring.\n\nWe will clarify this distinction and rationale more clearly in the revision.\n\n**References**\n\n\\[1\\] D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment, 2020.\n\n\\[2\\] B. Wang, C. Xu, S. Wang, Z. Gan, Y. Cheng, J. Gao, A. H. Awadallah, and B. Li. Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models, 2021.\n\n### **W3: Feasibility of DEITA/DS$^2$ with smaller models (e.g., *GPT-2*).**\nWe thank the reviewer for this thoughtful suggestion. It inspires us to reflect on how T-SHIRT compares with methods that rely on API-based scoring.\n\n**Discussion on DEITA.** For DEITA, two critical components depend on powerful LLMs like *ChatGPT*:\n* **Dataset evolution.** DEITA uses the API to generate increasingly complex and higher-quality instruction data following WizardLM \\[1\\]. *GPT-2* is not capable of performing this evolution.\n* **Scoring and supervision.** DEITA prompts *ChatGPT* to score each evolved sample, then uses these scores as supervision to train scorer LLMs. *GPT-2* is not suitable for this direct scoring via prompting. Using *GPT-2* would likely require fine-tuning it to serve as a weak classifier.\n\n**Discussion on DS$^2$.** DS$^2$ relies directly on prompting large LLMs for sample scoring, so it would likewise be infeasible to use *GPT-2* as a substitution.\n\nWe believe this discussion illustrates a practical advantage of our method and will include this comparison in the revision.\n\n**References**\n\n\\[1\\] C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, Q. Lin, and D. Jiang. WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions, 2024.\n\n### **W5: Novelty of our method.**\nWe acknowledge that token-level scoring and neighborhood smoothing for robustness have been explored in other contexts. We have reviewed related work in other contexts in Section 1.1 of our draft. However, to the best of our knowledge, we are the first to question the well-accepted paradigm in instruction tuning data selection, where each sample is scored as a whole unit and selected via a hard threshold. One of the key contributions of our work is a novel perspective: combining fine-grained token-level informativeness with neighborhood-level robustness for data selection. Our approach offers new insights into what makes a data point valuable for instruction tuning."}, {"review_id": "K6stZmsos5", "reviewer": "Reviewer_wj2T", "summary": "This paper introduces a novel framework for selecting high-quality instruction tuning data by addressing two important gaps in the existing literature: (1) overlooking token-level information for selection and (2) failing to incorporate robustness as a selection metric. The authors propose to firstly be selective when computing IFD and only consider the influence of tokens that exhibit large and informative signals, and secondly consider the neighborhood of samples through random perturbation in the embedding space and select data with high score and low variance. Through empirical experiments, the authors show impressive performance and efficiency of the proposed method.", "questions": "I don’t have much to complain about the paper. I am happy to have read a high-quality write-up with insightful methodology design and comprehensive experiments. \n\nSome minor points: \n\n1. It would be good to add repeated experiments and hence error bars for the experiments. Especially, baselines like RANDOM or DS2 have randomness in the selection, and the proposed T-SHIRT also uses random noises, it is critical to show the stability of the method through repeated, reproducible experiments. I could not be certain about the statistical significance right now.\n2. It seems that the selection ratio $k$ is an essential hyperparameter that affects the performance much according to Table 1 and Figure 6. As your method does not rely on a validation dataset, what would be a recommended method to choose, or potentially optimize for this hyperparameter.\n3. While the authors have included “Model Scale” as a limitation of the paper, the paper should at least show some larger-scale model experiments to show the scalability and generalizability of the proposed method. I suggest the authors to run these experiments.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper builds on an interesting observation that not all tokens are contributing equal to the quality of the instruction tuning sample, hence proposing a novel method that carefully eliminates the effects of those less useful (or noisy) tokens when performing data selection.\n2. The robustness perspective in scoring the utility of the data is new and useful.\n3. Throughout the paper, there are good illustrative examples to demonstrate the rationale behind the specific designs of the methodology.\n4. The proposed method exhibits impressive empirical performance, computational cost. At the same time, the paper includes extensive ablation studies.\n\nFor weaknesses, refer to the Questions section below.", "was_revised": true, "final_justification": "I have nothing much to complain about this paper. For the suggestions raised during the rebuttal, the authors have provided sufficient clarification with additional results.", "rebuttal": "We thank the reviewer for the encouraging reviews on the technical novelty, empirical strength, and efficiency of our approach, as well as the quality of our writing. We also greatly appreciate the insightful feedback. Below, we provide detailed responses to each question (Q) raised by the reviewer.\n\n### **Notations and clarifications.**\nIn our response, we follow the draft’s notations for several benchmark names: ARC-C (ARC-Challenge), HS (HellaSwag), TQA (TruthfulQA), GSM (GSM8k), AH (Arena-Hard), and AE-2 (AlpacaEval 2.0). We use $\\mu_{\\text{open}}$ to denote the average performance across six OpenLLM Leaderboard benchmarks, $\\mu_{\\text{LLM}}$ for the average performance across two LLM-as-a-Judge benchmarks, and $\\mu_{\\text{all}}$ for the average performance over all eight benchmarks. Additionally, we clarify that *Gemini-2.5-Flash-Preview-04-17* was used in our draft as the judge for evaluating AH. Since this checkpoint is no longer available via API, we now use the stable version of *Gemini-2.5-Flash*, which affects the absolute scale of AH scores.\n\n### **Q3: Applicability to larger models.**\nWe thank the reviewer for this valuable suggestion. To address this concern, we use the settings presented in Table 2 and fine-tune a *Qwen-2.5-14B* model on data selected from the Magpie dataset. Here, we still use the token-selection ratio $k$% of 50% without further hyperparameter tuning because of time limitations. The evaluation results are shown below.\n\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| **Longest**                 | $68.17$ | $\\underline{84.01}$ | $\\underline{79.23}$ | $\\mathbf{58.69}$ | $61.00$ | $\\mathbf{85.22}$ | $\\underline{72.72}$ | $\\underline{34.40}$ | $35.01$ | $34.71$ | $\\underline{63.22}$ |\n| **IFD**                     | $\\mathbf{68.94}$ |\t$\\mathbf{84.05}$ | $79.07$ | $57.87$ | $\\underline{61.78}$ | $83.02$ | $72.46$ | $33.10$ | $\\underline{37.15}$ | $\\underline{35.13}$ | $63.12$ |\n| **T‑SHIRT ($k=50$)**        | $\\underline{68.60}$ | $83.90$ | $\\mathbf{79.26}$ | $\\underline{58.60}$ | $\\mathbf{62.04}$ | $\\underline{84.15}$ | $\\mathbf{72.76}$ | $\\mathbf{35.30}$ | $\\mathbf{38.45}$ | $\\mathbf{36.88}$ | $\\mathbf{63.79}$ |\n\nOur method consistently surpasses the strongest baselines in terms of $\\mu_{\\text{open}}$, $\\mu_{\\text{LLM}}$, and $\\mu_{\\text{all}}$. In particular, it yields an average gain of at least 1.75 points on LLM-as-a-Judge benchmarks. These findings indicate that our data selection approach remains robust and advantageous for instruction tuning of models beyond the 7–8B parameter scale.\n\nWe will include the new experiment in our revision.\n\n### **Q1: Statistical significance of results for randomized methods.**\nWe thank the reviewer for raising the importance of demonstrating the stability of our method, given the stochasticity in both T-SHIRT and certain baselines. We conduct repeated experiments on *Qwen-2.5-7B*, running both the Random baseline and our method (T-SHIRT with $k = 50$) three times with different random seeds. We report the mean and standard deviation of their performances across the OpenLLM Leaderboard benchmarks. Due to the budget constraints, we were unable to re-run DS$^2$ \\[1\\], since its randomness primarily stems from API queries, and re-scoring the dataset would require an expensive number of API queries. For context, we reiterate the performance for DS$^2$ and IFD from Table 1. Standard deviations for the Random baseline and T-SHIRT are shown in brackets.\n\n| Method                      | ARC‑C |   HS   |  MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |\n| :-------------------------- |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----:|\n| **Random**                  | $64.42$ $(\\pm 0.31)$ | $81.89$ $(\\pm 0.07)$ | $74.15$ $(\\pm 0.11)$ | $59.30$ $(\\pm 0.18)$ | $53.46$ $(\\pm 0.09)$ | $50.52$ $(\\pm 14.17)$ | $63.96$ $(\\pm 2.32)$ |\n| **DS**$^2$ | $65.10$ | $82.07$ | $\\mathbf{74.35}$ | $60.58$ | $\\underline{54.11}$ | $82.11$ | $69.72$ |\n| **IFD**                     | $64.76$ | $\\mathbf{82.66}$ | $\\underline{74.33}$ | $\\underline{60.86}$ | $53.76$ | $\\underline{86.50}$ | $\\underline{70.48}$ |\n| **T‑SHIRT ($k=50$)**        | $\\mathbf{66.04}$ $(\\pm 0.24)$ | $\\underline{82.36}$ $(\\pm 0.03)$ | $74.09$ $(\\pm 0.10)$ | $\\mathbf{61.42}$ $(\\pm 0.17)$ | $\\mathbf{54.21}$ $(\\pm 0.11)$ | $\\mathbf{87.34}$ $(\\pm 0.59)$ | $\\mathbf{70.91}$ $(\\pm 0.07)$ |\n\nThese results show that our method consistently outperforms other baselines. Our method provides a considerable margin of 1.19 points over DS$^2$ on $\\mu_{\\text{open}}$, which makes it quite likely that this margin would remain robust, even under repeated trials for DS$^2$.\n\nWe will include these statistical significance results in the revised version.\n\n### **Q2: Selection and tuning of hyperparameter $k$.**\nWe agree that the token-selection ratio $k$ used in S-IFD is a critical hyperparameter. We offer the following recommendations based on our findings:\n\n* **Default recommendation.** When resources are limited, we recommend starting with $k = 50$, which performs well across multiple models and datasets, as shown in both Table 1 and Table 2.\n\n* **Fine-grained tuning.** On *Llama-3.1-8B* with the Alpaca-GPT4 dataset, we conduct a sweep over $k \\in \\lbrace 50, 60, 70, 80, 90 \\rbrace$. We find that our method works well across different values of $k$. The performance peaks at $k = 70$, with T-SHIRT outperforming IFD on 7 out of 8 benchmarks and remaining competitive on AH. It also achieves a 1.12-point improvement in $\\mu_{\\text{all}}$.\n\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| **IFD**                     | $61.35$ | $83.00$ | $62.88$ | $54.23$ | $46.99$ | $51.63$ | $60.01$ | $\\mathbf{5.50}$ | $8.20$ | $6.85$ | $46.72$ |\n| **T‑SHIRT ($k=70$)**        | $\\mathbf{63.48}$ | $\\mathbf{83.26}$  | $\\mathbf{63.92}$  | $\\mathbf{55.19}$  | $\\mathbf{47.13}$  | $\\mathbf{53.15}$  | $\\mathbf{61.02}$  | $5.40$ | $\\mathbf{11.22}$  | $\\mathbf{8.31}$  | $\\mathbf{47.84}$  |\n\nThese findings suggest that $k$ can be flexibly chosen within the range of 50–90. When tuning is feasible, it can provide additional gains.\n\n* **Transferability across model scales.** For the *Qwen-2.5* family, we find that $k = 50$ performs robustly across both 7B and 14B models, and across different instruction-tuning datasets (Alpaca-GPT4 and Magpie). We hypothesize that models within the same family share similar pretraining distributions, making it feasible to tune $k$ on a smaller model and transfer that setting to a larger one.\n\nWe will include this discussion and guidance in the revision."}, {"review_id": "8Mf9OmFdZt", "reviewer": "Reviewer_MFfG", "summary": "This paper identifies two key limitations of prior data selection approaches (e.g., IFD) for instruction tuning: (i) they evaluate sample-level data quality, ignoring token-level information; and (ii) they overlook the robustness of data quality assessment. To address these issues, the authors propose T-SHIRT, a new data selection framework that incorporates fine-grained token-level information and favors samples with consistently high-quality neighbors. They conduct comprehensive experiments demonstrating that datasets curated using T-SHIRT outperform full datasets and those filtered by other data selection methods.", "questions": "- Have you considered other scoring models used in score computation? Do you have any insights into how the choice of scoring model influences the quality of data selection?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- The authors use a semantics-preserving word substitution example to illustrate that IFD-based scores are highly sensitive to subtle textual changes, suggesting that selecting samples with high-quality neighbors is more effective than relying solely on fixed score thresholds.\n- This study provides fine-grained signals for high-quality data selection by incorporating token-level information, emphasizing that not all tokens contribute equally when assessing example quality.\n- They conduct comprehensive experiments using two instruction-tuning datasets of different scales, evaluate performance on eight benchmarks, and compare results against six baseline methods.\n- The authors perform comprehensive ablation studies to evaluate the contribution of each major component in their proposed method.\n### Weaknesses\n- Since this study emphasizes token-level information when selecting training examples, it would be interesting to investigate whether examples selected by T-SHIRT offer advantages at the token level compared to those selected by other baselines during instruction tuning.\n- I’m not fully convinced that removing all tokens with small delta_t values is justified. Since these values are already quite small, their cumulative contribution to the final score may be negligible. Could you please provide a plot showing the distribution of delta_t values to better understand their overall impact? Additionally, have you considered weighting tokens by their delta_t values instead of removing the less significant ones entirely?", "was_revised": true, "final_justification": "The author's reply about the token-level advantage of their method makes sense, though they do not provide direct evidence. The new ablation experiments on removing tokens solve my question well.", "rebuttal": "We thank the reviewer for the positive feedback on the technical novelty, comprehensive experiments, and ablation studies in our work. We also appreciate the reviewer’s insightful comments and questions on token-level informativeness and the choice of scoring models. Below, we provide itemized responses and hope they can address weaknesses (W) and questions (Q) raised by the reviewer.\n\n### **Notations.**\nWe use the following notations throughout our response, and they are consistent with our draft.\n* $\\mathcal{D}$: instruction-tuning dataset;\n* $\\theta$: LLM parameters to be fine-tuned;\n* $(x, y)$: instruction–response pair;\n* $y_t$: the $t$-th token in the response;\n* $|\\Delta_t|$: token-level informativeness for the token $y_t$;\n* Benchmark abbreviations: ARC-C (ARC-Challenge), HS (HellaSwag), TQA (TruthfulQA), GSM (GSM8k);\n* $\\mu_{\\text{open}}$: average score over six OpenLLM Leaderboard benchmarks.\n\n### **W1: Token-level advantages of T-SHIRT.**\nWe appreciate the reviewer’s suggestion to investigate whether T-SHIRT offers token-level advantages during instruction tuning.\n\n**Tokenizer difference limits direct comparison.** The models used in our pipeline, *GPT-2* for scoring, and *Llama-3.1-8B* and *Qwen-2.5-7B* for fine-tuning, employ different tokenizers, and this makes direct token-level comparisons nontrivial. Therefore, we analyze the training dynamics as a proxy.\n\n**Higher training loss indicates more informative tokens.** We observe that LLMs fine-tuned on data selected by T-SHIRT consistently exhibit higher training loss compared to those trained on baseline-selected data. This pattern holds across both *Llama-3.1-8B* and *Qwen-2.5-7B*, and across datasets (Alpaca-GPT4 and Magpie). Since the instruction-tuning loss is defined as: $\\mathcal{L}(\\theta) = -\\frac{1}{\\sum_{(x,y)\\in\\mathcal{D}} |y|} \\sum_{(x,y)\\in\\mathcal{D}} \\sum_{t=1}^{|y|} \\log P_\\theta\\left(y_t | x, y_{<t}\\right)$, this suggests that data selected by T-SHIRT provides more informative and challenging tokens for instruction tuning. We believe this observation provides meaningful evidence of token-level advantages.\n\n### **W2: Distribution and impact of low $|\\Delta_t|$ values.**\nWe thank the reviewer for raising thoughtful questions about our decision to remove tokens with low $|\\Delta_t|$ when computing S-IFD.\n\n**Cumulative effect of small $|\\Delta_t|$ values.** First, we clarify that tokens with small $|\\Delta_t|$ values can still have a significant cumulative effect. For instance, when using a token selection ratio of $k$% $= 75$%, we remove the bottom 25% of tokens (those with $|\\Delta_t| \\le 0.013$) from the S-IFD calculation. For a relatively long input, a lot of tokens are less informative and have low $|\\Delta_t|$, but cumulatively, they could still add up. As illustrated in Figure 2 in the draft, we can differentiate between two samples with nearly identical IFD scores (0.998 vs. 0.998) effectively by their S-IFD scores (0.751 vs. 0.968).\n\n**Distribution of $|\\Delta_t|$.** Second, since the rebuttal guideline does not allow presenting figures, we include a table to show the distribution of $|\\Delta_t|$ values computed by *GPT-2* across the Alpaca-GPT4 dataset.\n| Ratio            | $10$% | $20$% | $30$% | $40$% | $50$% | $60$% | $70$% | $80$% | $90$% |\n| :--------------: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| $\\|\\Delta_t\\|$     | $\\le 0.002$ | $\\le 0.009$ | $\\le 0.018$ | $\\le 0.032$ | $\\le 0.050$ | $\\le 0.078$ | $\\le 0.121$ | $\\le 0.210$ | $\\le 0.509$ |\n\nFor example, 20% of the tokens have $|\\Delta_t|$ below 0.009, and 50% of the tokens have $|\\Delta_t|$ below 0.050. suggesting a considerable portion of tokens are uninformative.\n\n**Alternative weighting schemes.** Third, following the reviewer’s suggestion, based on Table 1, we conduct new experiments using *Qwen-2.5-7B* and Alpaca-GPT4 to explore alternative weighting schemes. In Equation (3), our default approach assigns a weight $w_t = 1$ to informative tokens and $w_t = 0$ to the rest. We compared this to two alternative schemes:\n* **Weights $1.5:1$.** $w_t = 1.5$ for informative tokens, $w_t = 1$ for uninformative tokens.\n* **Weights $2:1$.** $w_t = 2$ for informative tokens, $w_t = 1$ for uninformative tokens.\n\nThe results are shown below.\n| Method | ARC‑C | HS | MMLU | TQA | BBH | GSM | $\\mu_{\\text{open}}$ |\n| :--------------------------- | :-----: | :-----: | :-----: | :-----: | :-----: | :-----: | :-----: |\n| **IFD** | $64.76$ | $\\mathbf{82.66}$ | $\\mathbf{74.33}$ | $60.86$ | $53.76$ | $\\underline{86.50}$ | $70.48$ |\n| **T‑SHIRT ($k=50$, Weights $1.5:1$)** | $65.44$ | $82.37$ | $74.09$ | $\\underline{61.44}$ | $53.83$ | $86.05$ | $70.54$ |\n| **T‑SHIRT ($k=50$, Weights $2:1$)** | $\\underline{65.53}$ | $82.17$ | $74.19$ | $60.73$ | $\\underline{53.95}$ | $\\mathbf{86.81}$ | $\\underline{70.56}$ |\n| **T‑SHIRT ($k=50$, Ours)** | $\\mathbf{66.21}$ | $\\underline{82.39}$ | $\\underline{74.23}$ | $\\mathbf{61.58}$ | $\\mathbf{54.21}$ | $\\mathbf{86.81}$ | $\\mathbf{70.91}$ |\n\nExperimental results show that both weighting schemes outperform the IFD baseline on $\\mu_{\\text{open}}$, confirming the benefit of emphasizing more informative tokens during data selection. However, our default setting that removes uninformative tokens (i.e., setting $w_t = 0$ like Equation (3)) achieves even stronger performance, both on average and across all six individual benchmarks.\n\nThese results validate our design choice. We will include the analysis and new experiment results in our revision.\n\n### **Q1: Other scoring models.**\n***GPT-2* vs. larger models.** In our early explorations, we considered several alternative scoring models and ultimately chose *GPT-2* for its strong efficiency and accessibility, as it can run quickly on a single consumer-grade GPU. While different scoring models can influence the quality of selected data, we refer the reviewer to the related work Superfiltering \\[1\\] for further information. Its Table 3 shows that larger models may offer only marginal improvements in data selection quality.\n\n**Open-source models vs. proprietary models (APIs).** Additionally, we chose open-source models over closed-source APIs because they provide access to token-level information and token embeddings, which are essential for our method. In contrast, querying more powerful proprietary LLMs via APIs typically returns only coarse-grained, categorical quality scores, as used in methods like DEITA \\[2\\] and DS$^2$ \\[3\\]. As shown in our Table 1, this lack of fine-grained information negatively affects the quality of selected data.\n\n**References**\n\n\\[1\\] M. Li, Y. Zhang, S. He, Z. Li, H. Zhao, J. Wang, N. Cheng, and T. Zhou. Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning, 2024.\n\n\\[2\\] W. Liu, W. Zeng, K. He, Y. Jiang, and J. He. What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning, 2024.\n\n\\[3\\] J. Pang, J. Wei, A. Shah, Z. Zhu, Y. Wang, C. Qian, Y. Liu, Y. Bao, and W. Wei. Improving Data Efficiency via Curating LLM-Driven Rating Systems, 2025."}, {"review_id": "5piIYN1R73", "reviewer": "Reviewer_VJnr", "summary": "Selection of high quality data is critical for efficient and effective instruction-tuning. Existing methods turn toward sample-level quality scores assigned by humans or proprietary LLMs, which are expensive to compute and overlook token-level features of instruction samples. Further, semantically similar samples can often have varying quality score assignments, which is not accounted for by threshold-based data selection. This paper addresses these challenges through T-Shirt, a reference-free data selection framework that considers only informative tokens when assigning quality scores and prioritizes samples whose neighbors achieve high quality scores to ensure robustness and consistency. Empirical evaluations show T-Shirt is highly scalable and efficient while incurring low monetary and time costs.", "questions": "* How does random noise injection compares to systematic lexical perturbations when it comes to IFD and S-IFD scores, and selection efficacy? \n* How effective is the pipeline toward fine-tuning LLMs across model scales (e.g., efficacy of the selected data for larger models trained with LoRA, will the observed performance gains still hold)?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n* The paper introduces a novel proposal to consider token-level information and local quality scores of related neighbor data points to evaluate training data quality.\n* The proposed methodology is well-motivated and clearly described; figures and data presentation are effective.\n* Experiments have good coverage of relevant benchmarks, baselines, and ablations, and results are strong even when using only GPT2 for score computation.\n* The paper provides evidence that using even a weak selector model can get effective samples in short amount of time, consistent with some recent works.\n\nWeaknesses\n* No major weaknesses. While results are convincing, the T-Shirt-selected data is only used to fine-tune two moderately sized base LLMs (<=8B parameters), with efficacy at larger scales unclear.", "was_revised": true, "final_justification": "The authors have addressed my concerns regarding scalability and hyperparameter selection as it relates to the method. It appears other reviewers' concerns are mostly resolved as well. I therefore maintain my positive rating of the paper.", "rebuttal": "We thank the reviewer for the encouraging assessment on the technical novelty, efficiency, experimental thoroughness, and presentation quality of our work. We also appreciate the thoughtful questions and constructive suggestions. Below, we address the weakness (W) and the questions (Q) raised by the reviewer.\n\n### **Notations and clarifications.**\nIn our response, we follow the draft’s notations for several benchmark names: ARC-C (ARC-Challenge), HS (HellaSwag), TQA (TruthfulQA), GSM (GSM8k), AH (Arena-Hard), and AE-2 (AlpacaEval 2.0). We use $\\mu_{\\text{open}}$ to denote the average performance across six OpenLLM Leaderboard benchmarks, $\\mu_{\\text{LLM}}$ for the average performance across two LLM-as-a-Judge benchmarks, and $\\mu_{\\text{all}}$ for the average performance over all eight benchmarks. Additionally, we clarify that *Gemini-2.5-Flash-Preview-04-17* was used in our draft as the judge for evaluating AH. Since this checkpoint is no longer available via API, we now use the stable version of *Gemini-2.5-Flash*, which affects the absolute scale of AH scores.\n\n### **W1 & Q2: Applicability to larger models.**\nWe thank the reviewer for this constructive suggestion. To evaluate the scalability of our method, we fine-tuned a *Qwen-2.5-14B* model using data selected from the Magpie dataset, following the setup described in Table 2. Due to time constraints, we used a token-selection ratio of $k$% $= 50$% without further hyperparameter tuning. The results are presented below.\n\n| Method                      | ARC‑C |   HS   | MMLU |  TQA |  BBH |  GSM | $\\mu_{\\text{open}}$ |   AH  | AE‑2 | $\\mu_{\\text{LLM}}$ | $\\mu_{\\text{all}}$ |\n| :---------------------------|:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |\n| **Longest**                 | $68.17$ | $\\underline{84.01}$ | $\\underline{79.23}$ | $\\mathbf{58.69}$ | $61.00$ | $\\mathbf{85.22}$ | $\\underline{72.72}$ | $\\underline{34.40}$ | $35.01$ | $34.71$ | $\\underline{63.22}$ |\n| **IFD**                     | $\\mathbf{68.94}$ |\t$\\mathbf{84.05}$ | $79.07$ | $57.87$ | $\\underline{61.78}$ | $83.02$ | $72.46$ | $33.10$ | $\\underline{37.15}$ | $\\underline{35.13}$ | $63.12$ |\n| **T‑SHIRT ($k=50$)**        | $\\underline{68.60}$ | $83.90$ | $\\mathbf{79.26}$ | $\\underline{58.60}$ | $\\mathbf{62.04}$ | $\\underline{84.15}$ | $\\mathbf{72.76}$ | $\\mathbf{35.30}$ | $\\mathbf{38.45}$ | $\\mathbf{36.88}$ | $\\mathbf{63.79}$ |\n\nT-SHIRT consistently outperforms the strongest baselines across $\\mu_{\\text{open}}$, $\\mu_{\\text{LLM}}$, and $\\mu_{\\text{all}}$. Notably, it achieves an average improvement of at least 1.75 points on the LLM-as-a-Judge benchmarks. These results suggest that our method remains effective and robust when applied to instruction tuning of larger models beyond the 7–8B scale.\n\nWe will include these results in the revised version of the draft.\n\n### **Q1: Comparison between random noise injection and lexical perturbations.**\nCompared to random noise injection at the embedding level, lexical perturbations present several challenges:\n* **Utility.** Rule-based lexical perturbations may fail to maintain semantic equivalence and fluency. Prior studies \\[1, 2\\] typically rely on expensive human evaluations to verify the utility of perturbed examples.\n* **Efficiency.** Generating lexical variants via prompting powerful LLMs through APIs is computationally and financially expensive. For instance, perturbing each of the 52k Alpaca-GPT4 samples 30 times would require over 1.5 million API calls.\n\nGiven these limitations, we adopt random noise injection as a lightweight and scalable alternative to lexical perturbations. Investigating how systematic lexical perturbations can affect selection efficacy remains an exciting direction for future work.\n\n**References**\n\n\\[1\\] D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment, 2020.\n\n\\[2\\] B. Wang, C. Xu, S. Wang, Z. Gan, Y. Cheng, J. Gao, A. H. Awadallah, and B. Li. Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models, 2021."}]

Author rebuttal:

Meta-review: This paper presents a new framework, T-SHIRT, for selecting high-quality instruction tuning data by addressing two key gaps in the existing literature: (1) the lack of token-level information in data selection, and (2) the absence of robustness as a selection criterion. T-SHIRT incorporates fine-grained token-level signals and prioritizes samples with consistently high-quality neighbors. The authors provide comprehensive experiments showing that datasets curated with T-SHIRT outperform both full datasets and those filtered by alternative selection methods.

The reviewers found the results to be novel, interesting, and relevant to the NeurIPS community. I therefore recommend acceptance.
Decision: Accept (poster)