Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
Authors: Yulei Qin
Yuncheng Yang
Pengcheng Guo
Gang Li
Hang Shao
Yuchen Shi
Zihan Xu
Yun Gu
Ke Li
Xing Sun
Venue: Accepted by TMLR ()

Abstract: Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Despite the vast amount of open instruction datasets, naively training a LLM on all existing instructions may not be optimal and practical. To pinpoint the most beneficial datapoints, data assessment and selection methods have been proposed in the fields of natural language processing (NLP) and deep learning. However, under the context of instruction tuning, there still exists a gap in knowledge on what kind of data evaluation metrics can be employed and how they can be integrated into the selection mechanism. To bridge this gap, we present a comprehensive review on existing literature of data assessment and selection especially for instruction tuning of LLMs. We systematically categorize all applicable methods into quality-based, diversity-based, and importance-based ones where a unified, fine-grained taxonomy is structured. For each category, representative methods are elaborated to describe the landscape of relevant research. In addition, comparison between the latest methods is conducted on their officially reported results to provide in-depth discussions on their limitations. Finally, we summarize the open challenges and propose the promosing avenues for future studies. All related contents are available at https://github.com/yuleiqin/fantastic-data-engineering.

Published in Transactions on Machine Learning Research (12/2024)
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
Yulei Qin Tencent To Lab
yuleigin@lencent.com
Yuncheng Yang
yaphacbates@isjuuucccb.c
Tencent You Lab, Shanghai Jiao Tong University
Pengcheng Guo
guupengchheug122200gmaa.ooo
Tencent YouTu Lab
Gang Li Tencent YouTu Lab
wang..i@@njuut..deee edu.cc
Hang Shao Tencent YouTu Lab
tulkkingghh@gmaiilccm
Yuchen Shi
oldeittystal@@mmailccom
Tencent YouTu Lab
Zihan Xu Tencent YouTu Lab
ianczzu@@tencent.ccom
Yun Gu
gurigu@icceeco org
Shanghai Jiao Tong University
Ke Li Tencent YouTu Lab
tristanli. @etiisshssmmmmm.m com
Xing Sun Tencent YouTu Lab
winnfred.ssn@@grmaa
Reviewed on OpenReview: http::///oorrrriii.eee/////////a
Abstract
Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Despite the vast amount of open instruction datasets, naively training LLM on all existing instructions may not be optimal and practical. To pinpoint the most beneficial datapoints, data assessment and selection methods have been proposed in the fields of natural language processing (NLP) and deep learning. However, under the context of instruction tuning, there still exists gap in knowledge what kind of data evaluation metrics can be employed and how they can be integrated into the selection mechanism. To bridge this gap, we present comprehensive review on existing literature of data assessment and selection especially for instruction tuning of LLMs. We systematically categorize all applicable methods into quality- diversity-based, and importance based ones where a unified, fino-grained taxonomy is structured. For each category, representative methods are elaborated to describe the landscape of relevant research. In addition, comparison between the latest methods is conducted their officially reported results to provide

---

Published in Transactions on Machine Learning Research (12/2024)
in-depth discussions on their limitations. Finally, summarize the open challenges and propose the promosing avenues for future studies. All related contents are available at http::/wwww....eeeeee//aaaatttta
1 Introduction
One of the ultimate goal of developing large Inguage models (LLMs) is to unlock their potentials of generalization to unseen natural language processing (NLP) tasks. Towards this goal, a series of LLMs such as GPTs Brown et al. (2020); Achiam et al. (2023) LLaMAs Touvron et al. (2023a b): AT@Meta (2024)) and
Mistrals Jiang et al.
22023a 2024a]] have delivered high-level text understanding and generation capabilities
via utilizing vast amount of high-quality web and human-annotated datasets for pre-training and preference
aligmment [Liu et al. (2023a 2024ch; |Sun al. (2024b); [dunov et al. (2019); Dong et al. (2019) During
preference alignment, instruction tuning plays an important role in refining the pre-trained LLMs to provide
accurate, pertinent, and harmless responses on collection of downstream tasks [Wii et al. (2021 ) Sanh
ct al. (2021]; Zhang et al. (2023d): Peng et al. (2023 Longpre et al. (2023) ); Shu et al.] (2023 Jang et al.
(2023): J; Ghosh et al. 2024]: : Kung & Peng [2023) For efficient and effective instruction tuning, existing
studies Ouyang et al. (2022):
: Taori et al. (2023); Zhou et al. (2024a); Xia et al. (2024ah] have noticed that
improving the quality of instruction tuning data (e.g., formulation well-defined and complete contexts),
rather than simply piling up instructions without analysis (e.g,, exhaustive collection of open datasets), is of prioritized concerns.
In this work, we aim to unify wide array of data assessment and selection methods under the context of instruction tuning of LLMs. As revealed from the probabilistic view John & Draper (1975 : Murphy (2012); Albalak et al. (2024)), the statistical patterns inherent in datasets determine the modeling performance. The overall evaluation of datapoints not only deciphers the distribution in various aspects (e.g., composition, task, and domain) but also helps cherry-pick the most beneficial subsets for higher performance with less training cost. Through this survey, we demonstrate that: 1) existing resourceful data assessment. methods can be categorized into three main perspectives: quality, diversity, and importance (see Fig. [ 2) systematic view of selection methods can be unified even they more or less exhibit coupling with the assessment techniques (see Fig. 2 It is noted that quality, diversity, and importance might be used interchangeably without strict discrimination in previous studies. But here we provide a rationalized organization taxonomy for structured elaboration. Despite the goal of being comprehensive, the present survey only provides details of certain typical, representative mods to avoid being lediously long. We hope the in-depth explanations ions and discussions on the selected methods provide insights into developing robust data assessment and selection pipelines for future studies.
1.1 Related Surveys
Liu et al. (2024d)) studied the mainstream datasets for building LLMs, including the pre-training corpora, instruction tuning datasets, preference datasets, evaluation benchmarks, and traditional NLP datasets. Their work focuses on the descriptions of dataset statistics (e.g., categorization, sources, and domains) without providing guidelines on utilization. In contrast, we emphasize the selection of intrrcction-tuiigg data for the improved downstream performance. Albalak et al. [2024]] presented a systematic overview of constructing the data pipeline for language models. Any selection method, either via distribution matching or diversification, can be composed of: 1) utility function; 2) selection mechanism. During different stages of the pipeline, the selection method should be adjusted according to different selection objectives (e.g., language filtering, data quality control, domain knowledge division, deduplication, toxic and explicit content removal, and data mixing). Their work pays extra attention to the processing of the pre-training corpora while neglecting the fine-grained analysis of cxisting selection methods specifically designed for instruction tuning. In this case, our survey serves as an indispensable exlension on the selection of instruction datasets. Wang et al. [2244)] focused on the data preparation for instruction tuning. Existing methods on building instruction tuning datascts include: 1) reformulating the discriminative NL. datascts into generative ones; 2) self self-instruct with seed prompts; 3) prompt mapping and evol-instruct Popular methods on dataset selection can be simply classified as: 1) system of indicators; 2) trainable LLMs; 3) powerful I.I.Ms; and 4) small models.
2

---

Published in Transactions on Machine Learning Research (12/2024)
Hand-crafted Indicators $4.1
Model-based Indicators $4.2
[0, H0,00.,,
Nile
1 x
0
Nitai
Nylm)
Geometry-base based Coreset Sampling $4.3
Bilevel Optimization- based Coreset Sampling $4.4
min d'O 83)
Σ 4
arg S
83
#:E S
Ar?
Hand-crafted Indicators $3.1
IND; F(IND' (n)), (&))
Diversity
(..).....N (z,)),
Model-bused Indicators $3.2
Quality
Hand-crafted Indicators $5.1
IND;= ((NN)(()) Dija)) [N ...e.....IIII INDE (r)).
Data
# Syllables # Words # Words # Sentences
Importance
GPT Score $3.3
GeTre = G(a),PC):
Model-based Indicators $5.2
Prompt Pg for scoring 2;
would like request your leedback the
0
Human Evaluation $3.4
Loss and Error-based Coreset Sampling $5.3
Label Score; [Label (m;)).
# Samples
Li
Li
Guidelines for annotations
10}
Below list of guidelines that should be adhered to for each possible task...
Forgelling
Gradient Cradient-based Coreset Sampling $5.4
C
Figure 1: Categorization of data assessment and selection methods for effective instruction tuning of LLMs.
Comparatively, our survey stems from the characteristies of data themselves, namely quality, diversity, and importance, for categorization of selection methods. In each category, we further provide subdivided groups by the selection philosophy, which deepens the understanding of selection for practical take-home messages.
Existing surveys on general data selection also shed light on the principles of developing selection methods. Guo al. (2022) started from the general coreset selection methods in the field of deep learning and categorize all selection manners into: 1) geometry-based methods (e.g., herding, k-center greedy); 2) uncertainty- based methods (forgetting: SraNDD/EL21 importance resampling); 4) decision boundary-baser (adversarial dcepfool; contrastive active Icarning); 5) gradient matching-basee (gradient approximation lowards full set); 6) bi-level optimization--asee (inner loop of model optimization and outer loop of datapoint selection); 7) sub- hoao--oddutttytaaaaaaaeaa
methods (e.g., least aeffiidennceentroop... /margin); 3) error/loss-bar
(e.g., graph cut; facility location); 8) proxy-basod (preference of a small model on data selection). [Zhou al. (2024bb) investigatod the potential metrics and aspects for data quality measurement. They provide list of available tools for data evaluation. Apart from data assessment. and selection methods that are specifically designed for NL.P
3

---

Published in Transactions on Machine Learning Research (12/2024)
Hand craffed Indicators $3.1
Vacabulary; Linguistics; Lexical Analysis; Semanties
Perplesily; Learning Complexity; Reward Score; Error Norm;
Model-based Indications $3.2 ?
Memorization, Instruction Folluwing Difficulty; Predictability; Uncertainty
bused Selection $3
Direct Scoring, Ranking, Justification
Instruction Writing: Respanse Labeling:
Human Evaluation $3.4
Direct Scoting, Pairwise Ranking
Hand-crafted Indicators
Lexical Diversity; Semantic Diversity; Statisties
Comprchensive Data Assessment and Selection
Entropy; Simpson's i's Index; Vendi Soor:;
Model-based Indicators 64.2
Fisher Information Matrix; Tagging
Diversity-baxed Selection $4
Geometry-haeet Coreset Sompling $4.3
K-center Greedy, Herding Greedy; Clustering
Retrieve; Glister; Consistency Regularization;
Bilevel Optimization-ausi Coreset Sampling $4.4
Entropy Regularization; Minimal
Hand crafted Indication $5.1
Readahility Indices;
Education level Difficulty
Model-based Indicators $5.2
Uneertainty: Reward Score; Datamodels
Importance-bused Selection $5
Emror-basad Coreset Sampling $5.3
Forptting, Memorizzation; Influence by Loss
Gradient Sampling $5.4
Gradiens Matching:: Influence by Gradient
A high-level overview of comprchensive data assessment and selection. The analysis aspects that apply to either individuals or the overall dataset can be categorized into three groups marked in blue italic.
Figure 2:
LLM applications Moore & Lewis (2010); Chen et al. (0224a]; Dodge et al. 2020 Kandpal et al.
2022
et al.
(2022); Feng et al. (2021b; Lee et al. (2021) Malhotra & Bakal 2015 Liu et al. (2024e) there exist
many
survey studics that tackle general quality mcasurement in machine learning Gupta ct al. 2021 Zha
el al.
2023 Ehrlinger W88 (2022 Mohammed el al. (2024); Li el al. 2024c : Lu al. (023b); Dix el al.
(2023
Priestley et al. (2023): Byabazaire al. (2020): : Roh et al. (2019); Sidi et al. (2012); Batini al,
(2009
for constructing safe, unbiased, and accurate datasets. In this paper, we mainly focus reviewing
data selection methods for utilizing instruction tuning data in the context of large language models.
1.2
Survey Scope
Although "data evaluation' has been so frequently mentioned that it appears as cliché problem in developing machine learning algorithms, the optimal solution establishing an overall data assessment and selection pipeline still remains open question. Especially under the context of instruction tuning of LLMs, existing studies proposed various measurements and strategies to select the "high-quality" instructions. However, very few studies noticed that there exists no unified dimensions or aspects in measuring data 'quality" where previous works tend to put emphasis on the domain-specific and task-dependent characteristics. In addition, the inherent, systematic coupling between data assessment and subset selection is not well demonstrated.
Under such circumstance, the present study strives to provide a comprchensive review on evaluating and decomposing massive instruction tuning datasets. We categorize the main aspects of data assessment in terms of quality, diversity, and importance. To reduce ambiguity, their definitions are first provided below.
Quality refers to the intrinsic value of the data. High-quality data typically satisfy two conditions: 1) The instructions are clear, accurate, and explicit in explaining the task at hand and the expected behavior of LLMs. 2) The responses are correct, coherent, and pertinent to the instructions. All the requirements and constraints specified in instructions should be met. Accordingly, most existing methods not only underline the clarity and fluency of instructions but also filter out mismatched, incorrect, and harmful responses via various judging and measuring techniques.
4

---

Published in Transactions on Machine Learning Research (12/2024)
Diversity refers to the variety and richness of the dataset. During training, models that are exposed to datapoints under wide range of scenarios enjoy high level of generalization to unseen tasks. High-diversity data imply that each constituting datapoint is unique. The diverse dataset should cover different knowledge domains and task definitions with varying input contexts and output constraints. In light of this statement, existing methods that prioritize diversity all emphasize the removal of homogeneous, near-deduplicated datapoints in terms of lexical and semantic representations.
Importance refers to the impact of specific data points on the LLM's performance. It implics the necessity of adding one datapoint into the selected subset for instruction tuning. The importance of datapoint is up to factors such as the difficulty of following the instruction, the capability of the LLM under investigation, and the scale of the dataset to be selected. Under such circumstance, various methods of importance- selection have been developed by measuring the difficulty of each datapoint, matching the gradients or performance of models that are trained on the selected and the full set, and investigating the dynamics of models in learning rich and scarce datascts.
In each aspect, provide detailed survey both traditional (e.g., hand-crafted indicators) and machine learning (e.g,, model-based indicators) methods. Besides, the coreset sampling methods that bridge evalua- tion and selection are introduced separately in diversity and importance oriented subset construction. In consideration of the properties of instruction tuning, we focus on the text modality and start from classical text analysis metrics. Metrics that are either specific to instruction tuning or compatible with pre- training and preference alignment are included since they all share general rules in data assessment.
The survey organized follows. First, we present the preliminaries for assessment and selection of rustron tuning datasets (Sec. $2). Next, we present the surveying methods of data asscssment and selection methods in terms of quality (Sec. $3), diversity (Sec. $ and importance (Sec. $5). Then, discussions on the existing methods are provided in (Sec. $6) followed by the promising directions for future research (Sec. 77). The final conclusion is given in (Sec.
1.3 What's Beyond the Survey Scope
The instruction tuning methods for de-bias and fairness of LLMs not covered in the present study. We acknowledge that the bias and fairness aspects of instruction data are valued in developing responsible LLMs Gallegos et al. (2024); Chu et al. (0044) et al. 2023dl). However, they are beyond the scope of the present study for the following reasons:
Most data selection methods on instruction tuning do not even notice the bias or fairness of data. They are in lack of explicitly designing steps reduce negative impacts of biased data.
Existing fairness tuning methods follow their own definitions of quality and diversity in terms of gender, race, religion, profession, age, and political ideology. It difficult to bring in all their corresponding evaluation techniques under our selection taxonomy.
The bias and fairness not only intersect with diversity but also with quality control, which is not an inclusive concept. For example, to control the harmful contents like hatred and racism responses, reward models or GPT scoring from quality-based selection can be used because the publicly released reward models and LLMs are already aligned with human preference.
The evaluation of bias and fairness is not covered in existing studies on data selection, which makes difficult to validate the offectiveness of selection techniques in improving the fairness of LLMs.
However, we emphasize that bias and fairness should be spocifically handled because instruction-tuned LLMs tend to exhibit more bias than the pre-trained LLMs Itzhak al. (2024)). Therefore, we believe the selection for less-biased dataset would be promising future direction in developing comprehensive selection methods.
5

---

Published in Transactions on Machine Learning Research (12/2024)
Synstem
Assistant
User
thhot_tepploe hoppllte" "ox message in messages 8y48 loep, first and sesxages(8)
["rote]] "systes' 1E xgystorintoo a hetaful assistent cltc.nnd "0
"essoeerrolee" rale"]
contant')
1- of
"eltnunddi"" "Mn' endfor Nje8 ods ww..geesttooromme S36( (<iingstort) sassistantvee 13 ondif at"
Template Wrapping
Tokenization
I;
Pi
23
clie sttarllssystenn you are helpful
("a())))
"instructiun" "Explain fifereence between USL
ass issistntt. clin andlaland in stuturi seresserrreeeaaa Explain the aifference between a UAI, and a donatn nawe irgend i-nnel sustart loasst snd unI whtch soonds fnifore Resource Locosor, is entire address used visit westlse It ireluse HITPS), proteusl other elenents donati
34096
"irput"; ti
"'"tput":"A "A unt, which stards for Uni form Resource Locotor, is the eneire address used to visit o website or access a speci onl line ressurce, Ie includes the protocal (with os HTTP or HTTPS), the donatn nows and other sleeents like the poth, poraneters, or onchor. exomle, UR. of this page hetpss// play.ol aidunggnn... Larsa in/hoossaagee An the hand,
the path,
#2((21)
perowters, anchar. For eumple, the URL of this pigt is hetps://play. honellage vrlatnn the other sand, a doecie nore is a cosporent of a (RIl that specifically idontifics the wessite or wes page. It the port of the URL shat cores after the protscat and before the path. above, the desain nawe sumple, in the Vplay, dinggenn.toV.. you spectfic location donatin the spect location you' rw visiting. is
http:///aaaallaggeeeee dtdungeon. to/nain/
32 5548 11 B9G 32552 380 47096
31765 5ICSS 31 314
279 4453
351
1400
2047 24- 2013 476 3055 964
3155 2560
S165
3364
J66 316 11507
328
component
Joacs cs 10:.8
ei #1806
N11 2V8 N44 536
1090
595
150
spect after protocol, and before URL that For sxumple, in the unl shove, the damain nome Voltey LoV", Vrivii simplar bares, you can think of o URI as address that leads you to specifiie fire location on on the wet, while donatn none is the spectfte nane of the location you're re vistiingg"
5005 9639
L1105
3752 3316
279
5548
413
3150
3703
3130 130 3779
4545 15154 18225
2808
1525
279 1308
1404
3452 315
4.5 31406
XUSLE
325
3445
siepler teres,
2150 1045 UX
Figure 3: The pro-processing of an instruction dataset includes: 1) template wrapping, and 2) tokenization. In the first step, we wrap the raw texts 4 with pre-defined chat template into the textual prompts ps- In the second step, we perform tokenization on p; with the I.--aaasociated tokenizer for the datapoint zi- Given the index t for indicating where the loss mask of language modeling starts taking effect, we split z; into ((t) and Ti(2n), respectively denoting the instruction part (input) and the response part (output).
2 Preliminaries
In this section, we briefly introduce the instruction tuning of LLMs and the problem statement for dataset assessment and selection.
Instruction Dataset Preparation In instruction tuning, each text sample 1; is usually composed of three parts: 1) instruction (either with or without system prompt), 2) input (can be empty), and 3) response.
For an off-the-shelf pre-trained LLM parameterized as 0, pre-determined instruction template is used to wrap the T; into a textual prompt ps with special tokens like '<tin_start\>' and "<lim_end|>> for separation of roles (e.g,, system, user, assistant, function, and observation) and their contents. It is noted that the prompt template for organizing a text sample I differs across model families. Especially, the special tokens are unique to the tokenizers and play an important role in differentiating multi-turn conversations and stopping model generations. Then, a LLM-associatod tokenizer performs tokenization on the textual prompt. p; for sequence of token-id integers: x; = [Mv()), T2(2); (o, Fi()), where Ti()) denotes the j-th token of I; and is the total number of tokens. Out of simplicity, the token sequence z; can be simply split into two parts by the index t: 1) the instruction part (zi(<e) that is fed into the LLM without being involved in loss computation, and 2) the ground-truth response part (Z((2) that is expected as the LLM output via minimizing its language modeling loss. Without losing generality, we use the terminology of "datapoints' throughout the paper to refer to the tokenized samples Zt.
The process of processing an instruction sample x; is illustrated in Fig. 3 A Apaca dataset Taori et al. (2023) and perform Qwen tokenization Yang et al. (2024) for demonstration.
We choose one example from the
6

---

Published in Transactions on Machine Learning Research (12/2024)
Instruction Supervision Given the tokenized instruction tuning clttaset S { the supervised tuning is performed via cross-entropy loss:
)
C= Σ Li:
2,68
(1)
[asl
C,
Σ jmn
For each 24s given all previous tokens T((<s) the model iteratively predicts the next token Fuy) at the j-th index. Note that the previous tokens here include both the instruction context part T((()) and the response completions up the current token 21(2),<5)
Data Assessment and Selection We aim at finding the most informative subset Sp S from the entire set S under the given budget |S s b. Mathematically, the selection of S, requires: 1) quantitative evaluation function q(-) that assesses each datapoint and 2) an elaborated sampling mechanism æ that determines the rules of selection:
S. = ((,bb,g)
(2)
With respect to the detailed implementation of म, either an iterative, greedy algorithm or a batch-wise heuristic rule can be adopted for compatibility with q()). For example, the greedy sampling is represented
S = greeddy (S,b, g) = arg max S'SS,181E6 ES
Σ (x.)).
33)
The greedy algorithm Tgrondy consistently selects datapoint with the highest score ((z;) until the budget target b is met. The heuristic sampling algorithm, on the other hand, chooses each datapoint according to a pro-defined rule. For example, the probability derived by the score ((x;) can be employed as reference:
S = (S,ob(S,b, b, 4) = Sample(S,b.)) b,p,),
q(1,)
(4)
p(zx) = Ex,es q(zz)'
where Sample(S,b,) performs sampling times upon the normalized probability distribution over q(-). More advanced sampling techniques can be developed in accordance with the domains and tasks at hand.
Such data assessment and selection paradigm is expected to bring about the following benefits: 1) the reduction of noise by ignoring those mislabeled, mismatched instructionn-essposee pairs, 2) the re-balance of data distributions by down-sampling those easy, common, and similar examples while up-sampling hard, rare, and unique ones, and 3) the expedition of training in return for efficient. optimization of L.LMs.
3 Qualdty-based Selection
In this section, we present methods on quality assossment and selection. Without lose generality, we present the unified formulation of quality measurement. The evaluation function g(x;) in quality-based methods can be decomposed into Lwo fine-grained parts: 1) instruction quality (r())) and 2) response quality gn(xx).
q(x,) (
5
where fg is aggregation function that combines the instruction and response quality scores either explicitly or implicitly. Specifically, the instruction quality qı can be further broken down into: 1) clarity of that measures the ease of understanding the lask, 2) accuracy of that measures how well the instruction aligns with the expected task, and 3) explicitness of that measures how explicitly the instruction defines the output constraints (c.g., formats and styles). Consequently, we have dr (*tct) = gr (#vct); of (Fict), (#vee)) with the aggregation function gr. Similarly for the measurement of response, its quality qR can be assessed via: 1) correctness of that measures whether the response correctly answers the instruction, 2) coherence off that
7

---

Published in Transactions on Machine Learning Research (12/2024)
measures the logical consistency of the response, and 3) pertinence q that measures the relevance of the response to the instruction. The final response can be judged as gn(x121) = (xiet)) o (Fiet)) (f (xvert) with an aggregation function gR- It is noted that all the mentioned quality measurement components above are only demonstrative and are not enforced explicitly in the development of existing quality- methods. Therefore, certain fine-grained aspects might be integrated into one formulation without extra considerations.
3.1 Hand-crafted Indicators
Overview Traditional methods develop hand-crafted indicators to evaluate the data quality in terms of linguistic analysis such as vocabulary, syntax, and inter-sample somantic similarity. Each indicator is manually, empirically designed with prior knowledge on the language, domain, and lask of the corpus under investigation. The calculation of each indicator is explicitly defined and does not require training and inference of proxy models or language models. Although the indicators are hand-crafted, deep learning models such sentence encoders might be leveraged to extract embedding representations for each datapoint T₁. An indicator IN Dj can be lypically defined as:
IND, = I(IND) IND? (T." (z:)),
(6)
where M denotes the total number of indicators and f is the aggregation function which depends on both the instruction task and dataset. One can simply use linear combinations with pre-defined dynamically adjusted weights. However, meticulous tuning might be needed for the ultimate f. Given the indicators IND; for cach 2;, two intuitive selection methods can be adopted: 1) to filter out datapoints whose indicator scores are below a threshold; 2) to keep only the samples whose indicator scores rank within a certain range of percentiles, Mathematically, these two selection mechanisms can be respectively represented as:
S = [xi//Tmin < f(xs) < Tmax: 1 SiiSN,
(7)
S = [Lj]Pmin S ((((((a)) S Pmax, 1 UiiSN),
(8)
where Tmin and Tmax respectively denote the left and right threshold boundaries. The estimated Fy is the empirical cumulaatve distribution function of all indicators f. Puin and Pmax respectively refer to the minimum and maximum percentile for enclosing the selection range. In practice, both the threshold and percentiles are syper-paramcters that require task-specific fine-tuning.
Technical Details Mishra et al. [020aa] and Mishra et al. [2020b) introduce a data quality metric, namely the DQI, to quantify the differences between successive benchmarks by giving high scores to generalizable samples and low scores to biased samples. Such a metric implies whether a well-trained model truly learns the underlying task rather than overfitting the spurious bias of specific benchmarks. Specifically, DQT has seven components including vocabulary, inter-sample N-gram frequency and relation, inter-sample semantic textual similarities (STS), intra-sample word similarity, intra-sample STS, N-Gram frequency per label, and inter-split STS. Based on the DQT, Mishra & Sachdeva (0200) propose to prune existing huge NLP datasets and demonstrates that the model trained on only 2% of the SNLI dataset achieves near-equal performance with that on the entire set. It first performs AFLite Le Bras et al. 2020)]] which is detailed in Sakaguchi
et al. (202 ], to keep samples with predictability scores over a threshold and then delete bottom k samples with the lowest. DQT scores. Dang & Verma (2024)) further split DQT components into linguistic indicators and semantic indicators, and validate their respective roles in detecting outliers, noises, and duplications. Apart from data selection for training LLMs, quality indicators can also be employed to identify the most discriminative samples in the evaluation set to expedite evaluation of LLMs. Saranathan et al. investigate key indicators such as spelling errors Yannakoudakis & Fawthrop [1883)] ; average word length, excessive word repetition, and the compound probability distribution. These indicators stem from the traditional studies on text readability (i.e., readability formulas and sophisticated features) Klare et al. 1963 1984]; ; Dubay (2044];
Kintsch & Vipond (2014); Kemper (1983)] ]. Recent studies on readability leverage NLP systems rxcr
more advanced and informative features for readability measures
Si & Callan (2001) ): Collins-Thin Tolimss-hoomson &
Callan 2005);: Schwarm & Ostendorf (2005); Feng al. (2010)] François (010 201 François & Fairon
8

---

Published in Transactions on Machine Learning Research (12/2024)
2012 systematically analyze the lexical features, syntactic features, semantic features, and language-specific features with up to 46 indicators. François & Miltsakaki (2012) validate these manually-designe (classical) and NLP-enabled (non-classical)
readability formulas, implying that high-quality texts can be pinpointed by such carefully designed metrics. Felice & Specia (0122) find that the hand-crafted linguistic features should
be combined with other shallow leatures for better quality estimation.
The hand-crafted indicators often stem from studies on linguistic analysis and readability mea- surement. Although these indicators help filter out instruction samples that are unreadable, nonsensical, and incoherent, they cannot detect mismatched instruction-response pairs and therefore fail to guarantee the instructionfollowiii capability of LLMs trained on highly-scored datasets.
Remark
3.2 Model-based Indicators
Overview
The model-based indicators, on the other hand, leverage trainable models to predict the indicators for each datapoint. The trainable models used for data quality measurement can either share the same similar architecture with the language model under dovelopment, possess completely different. implementations. Accordingly, these indicators can be simply defined as:
IND, f(IND), (a:), IND (a,), DN (N.)....INN (r))),
(9)
where the learnable parameters 0,, 02, ... Om highlight the difference between model-based and hand-crafted indicators. Based on such indicators, similar selection mechanisms (Eqs. and can be adopted.
Technical Details One of the most intuitive model-basee indicators is perplexity [Shannon (2001): Jelinek
et al. (1777) Jelinek 1980 J. It. is frequently mentioned the evaluation metric for pre-trained language
models Penedo et al.
2023 Radford et al. (0118 2019;; Brown et al. (2020); Achiam et al. (2023) but
also be employed as data quality indicator. Ankner et al. [2024]] propose to use a small GPT-style reference
model such as MPT 125M Team (2023) to prune datasets via perplexity-based sampling for training a 3B
model. For any datapoint T;, the perplexity is defined as the exponential negative likelihood with base of 2:
NLL: 1 laa
JogP(arg)
25
(10)
==1
PPLX; 2PLX:=2NEAL NLL
Based on the perplexity inferred from a small model, samples at the high and medium percentiles are chosen by Eq. for downstream fine-tuning. Deng et al. (2021]] develop unified evaluator framework to score the generated outputs for natural language generation tasks. A RoBERTa-based [Liu et al. (2019)) discriminator learns to score responses in terms of consistency, relevance, preservation, engagingness, and groundedness. One could simply adopt such a discriminator for evaluation of instruction-response pairs. [Zhong al. [2022)] further propose multi-dimensional scoring evaluator. For cach evaluation dimension, the original instruction-response pairs are converted into positive samples in the form of boolean question-answer problems. The negative samples are respectively constructed via rule-based transformation. The evaluator itself is implemented as T5 model Raffel et al. (2020) and trained on these positive and negative samples for scoring in the range from 0 to 1. Jiian et al. [2024ch prune the UltraChat Ding et al. (2023) dataset. by scoring each datapoint by the learning complexity of a pre-trained Qwen-1.8P model Bai et al. (2223
Specifically, the learning complexity calculated as the averaged prediction confidence of different subnets:
Š(x;) = 1 Σ PPLX
(11)
3=1
where I is the number of subnets. Each subnet θ, is obtained by adjusting the dropout rate from 10% 90% the original θ of any pre-trained language model. Such dropout technique imitates the learning
9

---

Published in Transactions on Machine Learning Research (12/2024)
progress in triniin-free manner. Datapoints with small S(x;) are considered easy ones. They are often learned at an earlier training stage with less model capacity compared with hard ones. In a data-poor regime, easy datapoints are more informative and should be kept first. On the contrary, in a data-rich regime, hard datapoints should be treasured in that more samples can be preserved for fino-tuning. Both [ukhhrin & Zhao (2023) and [Du et al. (2023)) employ reward models to assess the quality of each instruction pairs. They respectively utilize the raft model [Dong et al. (2023) and the deberta-v3-largevv2 [ for reward scoring:
R² re ((2))
(12)
where To denotes the reward model. t is the index where T((<() and T2(2)) respectively denote the instruction
Q and response A.
Marion et al. (2023) investigale three classic metrics in clean set selection Guo el. al.
2022);
Song et al. 2022): Natarajan et al. (2013); ; Qin et al. (2024)]: perplexity (Eq. [0], error 12-Norm
(EL2N)
Paul et al.
2021 and memorization ranking Biderman et al. 2024 Specifically, EL2N is defined
88:
1
EL2N,
|00 Σ
[ - -Y-))l2-
(13)
where Yτg) denotes the one-hot ground-truth vector as the target of the probability vector (( 0)- Specifically, both P(z((<)):) 0) R Moccab and Yı() E RNocab are of the same dimension, where Nyocab denotes the vocabulary size associated with the tokenizer. For the vector of Yig): all elements are zero except that the element indexed at 2i(j) is one. The memorization ranking is represented as:
Nun
MEM
1 (B +5) (i(Moomn+)))
(14)
Nwin
where Winn denotes the length of a consecutive sequence and Motfset is an offset of the starting index. The
Mi +i refers to the generated token given imput Ti(<< Monaa and [i(Manna+t) is its ground-truth. Cao
et al. 2023 combine both hand-crafted indicators (e.g., input length, output length, MTLD McCarthy &
Jarvis (2010 and kNN-i [Dong et al. (2011)) and model-based indicators (e.g., reward score, perplexity, and Uni- metrics [Zhong et al. (2022) for fitting the loss of a LLM on the evaluation set. The linear
regression model is optimized via least squares method Bjork [988)] and the optimal selection of instruction data achieved via BlendSearch [Wang et al. (2021a/b) for minimizing the estimated evaluation loss. Li ct al. [2023a)] propose one of the most pioncering works that leverages the target language model itsclf to perform self-guided data selection. The language model is first. 'warmed-up' with very few samples randomly chosen from the pool to learn from brief experience. Then, such an experienced model evaluates each instruction-response pair via the instruction-followng difficulty (IFD) score. The IFD score measures how much guidance or assistance the instruction provides to the generation of ground-truth response, by comparing the loss of causal language modeling on the response with and without instruction:
NLI, AIQ IFD; NLIA
NTT. AIQ
log (ran))))))
(15)
(0(22))
NLLA (0(20)
(ogg((a)))
where the index t splits apart the instruction Q and the response A. Samples whose IFD scores over Taax
are invalid datapoints with misaligned, mismatched instruction-response pairs. The empirical setting of
Tiin affects the trade-off between quality and diversity of the selected datapoints. Zhao & Fang (2024) comprelensively employ hand-crafted indicators for low-level quality filtering, and uses perplexity and IFD score for high-level filtering. voting mechanism is additionally introduced with IFD scores from one
10

---

Published in Transactions on Machine Learning Research (12/2024)
pre-trained base model and one fine-tuned experience model. Li et al. [2024b) corroborate that both the perplexity and IFD scores inferred from rather small GPT2-125M Radford et al. (2019) are indicative in
selecting high-quality datapoints for training LLaMA2-7B and LLaMA2-13B Touvron et al. [2023b)], which greatly improves selection efficiency.
Another popular model-based quality fiteringg method AF-Lite Te Bras al. (2020) which has been applied and validated in recent NLP studies Mishra & Sachdeva (2020); Sakaguchi et al. (2021) J. It randomly partition all available datapoints into the training set and the validation set. Then, model (e.g., linear classifier language model) is trained the training set and inferred on the validation set. Such process iterates m times for calculation of the predictability score, which is defined as the ratio of the mumber of correctly predicted response over the number of total predictions:
PRED, H&g Es, s.t. I; x4
(16)
E, = (a 201
20m
where a. denotes the generated response from the model parameterized as 0z. It is noted that 2; is not involved for optimizing fg, and therefore a higher PRED; suggests better quality.
Bhatt et al. [2024)] present uncertainty-basee indicators such as mean entropy Settles 2011 Kremer el al.
[2014)], least confidence [Settles] (1995) 2011]], mean margin [Tong & Koller (2001); Balcan et al. (2006);
Settles (201 ], and min margin Nguyen et al. [2022)] Mathematically, such uncortainty indicators
defined
laral P(aan)aaE)))))))
U Ventropy
(17)
log (
lm:
Uronfidence
(18)
|xd
Umarginn
0 Σ
(3r (P((((
(19)
B2 ((((()()))))))
U min-marrin
min je
(81 (P(
(20)
B2 (1))
where 31 and 32 denote the largest and sccond largest clements of the probability P(a 0) E R Nvoosh for the newly generated j-th token. However, Wwu et al. (0233 find that such uncertainty- bbaeed data sampling methods perform worse than random sampling Patabricks-Dolle Conover et al. (2023)), SelfInstruct-
Davinci Taori et al. (2023)), ), and SelfInstruct-GPA Peng et al. (2023)
Remark Hybrid techniques that simultaneously combine perplexity, uncertainty, reward scores, and other training-aware metrics are promising in selecting unbiased high quality samples. In consideration of the training and inference cost, it is feasible to employ small proxy models as alternatives for computing model-based indicators.
3.3 GPT Score
Overview The invoking of OpenAl APls Tingiris & Kinsella 2021];
Lappalainen & Narayanan 2023 ;
Sun ct al. (2023): Kublik & Saboo (2023) for ChatGPT services (c.g., GPT3.5, GPT4) allows automatic
scoring of instruction tuning datasets. Recent studies on bringing LLMs as judges Zheng (0024); Wang
et al. 2023a); Zhu al. (2023): Huang et al. (2024) Zeng et. al. (0233 Chan et al. (2023) Pang al.
11

---

Published in Transactions on Machine Learning Research (12/2024)
2024)] reveal that powerful language models like ChatGPT highly align with human preference on judging the quality of instructions and responses. Given well-designed prompt with clear definition on grading
criteria, ChatGPT produces justified quality scorings with explanations for the raw textual instruction data I
GPTScore: = PG),,
(11)
where PG denotes the prompt template that defines the task and grading scheme with format constraints on outputs (see Fig. The G(-,-) represents the process of quality scoring and response parsing, which produces the GPT score Gcore, Samples with high G can be selected using Eqs.
andd
Prompt PG for scoring I, with instruction (input) and response in the <dimension>>
We would like to request your feedback on the performance of Al assistant in response to the instruction and the given imput displayed following.
Instruction: <instruction>
Input: <input>
Response: <response>>
Please rate according to the <dimension>> of the response to the instruction and the imput. Each assistant receives score on scale of 0 5, where higher score indicates higher level of the <dimension>> Please first output a single line containing the value indicating the scores. In the subsequent line, please provide comprehensive explanation of your evaluation, avoiding any potential bias.
Figure 4: The prompt PG for scoring the raw text 15 with ChatGPT.
Technical Details Chen et al. 2023b propose surprisingly casy-vet-effective method that directly uses GPT3.5 to score datapoints in Lerms of helpfulness and accuracy. Both instructions and responses are scored on a scale from 0 to 5 and experimental results show that general instruction datasets, except coding-related samples, can be distilled into smaller subsets for better downstream performance. [Bukharin & Zhao [2023) follow [Chen al. (2023b) for filtering Alpaca [aori al. (2023)) Chen & Mueller (2024)) employ the BSDetector Chen & Mueller (2023) to estimate the confidence of GPT3.5/ CPT3.5/GPT4 GPT4 on the give instruction-response pair. It takes both the self-consistency and direct scoring into consideration. Only highly confident samples are kept for fine-tuning domain-specific LLMs and those less confident ones are corrected automatically by these LLMs, Xu et al. [2023b)] directly evaluate instruction datasets in terins of accuracy, explanation, clarity, and difficulty for weighted scorings from GPT4. Then, both hand-crafted indicators (i.c., lengthwise semantic evaluation) and GPT4 scorings are employed for final ranking. Liu et al. (2023b) argue that the direct scoring of GPT4 on one single instruction sample is not well-calibrated and instead gives relative ranking of multiple instruction variants once. The complexity of instructions Xu al. (203aa) and the quality of cnstruction-resposee pairs sequentially obtained from GPT3.5. Zhang et al. (002cc
use GPT scorings to judge: 1) whether the given text. contains mathematical contents; 2) and if yes, whet these maths contents are of high quality for education purpose. Such scores are proved more effective than traditional "mathematical" classifiers Paster et al. (2023) Lu et al. (2023a) propose to use ChatGPT for annotating open-ended, Ene-grained intention tags on open datasets. Then, the quality of the tag dataset is evaluated by humans and GPT4 in terms of tagging precision and consistency. Instead of fully relying on the GPT4, [i al. [2023c)] exploit the model under investigation itself (e.g., LLaMA 65B) to iteratively derive quality scores on each augmented example on 5-point scale. Then curated clean set is chosen via the Eq. QuRator WWettig et al. (2024b) manually define quality criterion such writing style, facts and trivia, educational value, and required expertisc. Then, quality comparison is conducted on two instructiio-rossooss samples via GPT3.5 scoring. Such pairwise scorings are used to fine-tune a sheared-LLaMA 1.3B model Xia et al. [023) in a manner similar to DPO Ouyang et al. (2022); Rafailov et al. (2024) It noted that the pairwise scoring [Ouyang et al. (2022); Dubois et al. (2024); Zeng et al. (2023) Liu et al. (2023b) has been found more reliable, consistent, and unbiased than the individual scoring Gunasekar et al.
(023): Chen et al.
[2023b)] during GPT-based quality analysis.
12

---

Published in Transactions on Machine Learning Research (12/2024)
Remark Closed-source LLMs such as ChatGPT enjoy high level of alignment with human preference and therefore can be utilized for quality scoring. It would be cost-efficient to collect few (e.g., <100K) GPT-scored samples first and then fine-tune an open-source LLM for quality measurement on massive corpus.
3.4 Human Evaluation
Overview
Evaluation with human-in-the-loop is indispensable in constructing preference alignment datasets Wang et al. (202bb); Ouyang al. (2022 for helpfulness, honesty, and harmlessness. Specifically,
human annotators deliver gradings following specific criteria (see Fig. 5 in multiple dimensions:
LabelScore, = (LabelScoree (z:), LabelScore (zz). LabelScore" (x:)))
(22)
where LabelScore""(ae (z;) can be both bool or integer (e.g., range from 0 to 5) for the m-th fine-grained aspect. The aggregation function f is commonly chosen as summation or averaging.
Guidelines (excerpts) for human annotations
# Guidelines
Below is a list guidelines that should be adhered to for each possible task available when building the dataset. To see some examples of how the guidelines can be applied, visit the examples document.
## 1. General rules
Always make sure to read and understand the guidelines to each task before fulfilling it. Try to follow the guidelines closely possible. If you unsure whether message violates guidelines, contact our Discord.
Use the thumbs-u up/ tttumbs-soooe system further mark messages that are of high or low quality.
# ## 2. Providing an assistant reply #ussistant-reply
### # Do:
Remain polite and treat the user with respect, even when not given the same courlesy.
Figure 5: The guidelines (thumbnails) for human experts to create and annotate instruction datasets.
Technical Details The OpenAssistami Köpf et al. (2024)) dataset featured by its high-quality human- generated, human-annotated multi-lingual conversations for both instruction tuning and reinforcement learning from human feodback. For each instructionrrsponne pair along the conversation tree, the human annotators are asked to categorize them according to three dinensions: spain delection, guideline adherence, and quality. The quality score is rated on five-point Likert scale across aspects including quality, creativity, humorousness, politeness, and harmlessness. These scores are used to sort instructions for analysis and preference optimization of LLMs. Lu et al. (2023a)] enroll human annotators to provide judgements on the tagging each instruction. To verify the quality scores provided by humans, counterfactual cases are prepared respectively for precision and consistency tasks. Results show that human annotators have low false positive rales at lagging precision, but lack proof of confidence et al. (0244b) propose to use human annotators for creation of yfffeeive instruction datasets. To collect questions and answers from various sources, simple hand-crafted indicators such as text length are used to filter low-quality datapoints. Then, high quality instructionn-responee pairs are manually selected (750) and written (250) via subjective quality control. The databricks dataset [onover et al. (2023) contains 15K human-genan gennrrted instruction-respone pairs. Although quality emphasized during large-scale annotation, imporfeet samples still exist. For example, low-quality and inaccurate responses, incomplete and vague instructions, problematic lexts with toxic language and grammar errors are found He et. al. (2024))
their original quality judgements. Zhou
Remark Human evaluation plays an irreplaceable role in quality control of preference alignment. To reduce the inter-annotator inconsistency, detailed guidelines should be prepared for quality measurement. In addition,
13

---

Published in Transactions on Machine Learning Research (12/2024)
supplementary quality measures such as GPT scores can be provided for reference during evaluation and selection of high-quality datapoints.
4 Diversity-based Selection
In this section, we introduce methods that emphasize the diversity of instruction datasets. When it comes to diversity, existing researches either measure the individual diversity of each sample (e.g., lexical and semantic richness) or the overall diversity of the entire dataset (e.g., the volume of the enclosed embedding space). Datapoints whose tasks and domains are of minority classes in long-tailed distribution subset selection. Such sampling philosophy strikes to maintain or approximate the spread of the original embedding clusters but with much less sparsity. In the case of diversity, the evaluation function ((x;) can be represented in the unified formulation as:
preferred during
<(x;)
(23)
where qr measures the lexical diversity of I₁ and qs assesses the semantic diversity. The fa denotes the aggregation function in diversity measurement. Typically, 9r. often investigates the diversity of n-grams, tokens, words, and sequences. Complementarily, 9s emphasizes semantic diversity that the variety of representations of the selected datapoints should be maximized in the embedding space. Both the two aspects of diversity can be sequentially or jointly considered to remove any duplicates in the instruction datasets.
4.1
Indicators
Overview The diversity of datasets is the key to develop less biased, more generalizable machine learning models. However, recent studies (Zhao et al. (2024c bj] show that existing datasets do not share unified and concrete definition of diversity in terms of dataset composition, source, domain, subject, annotator, and promote (lairness). With respect to the diversity measures specific in instruction tuning datasets, hand-crafted indicators, similar to Eq. 6 in traditional NLP studies, can be used as a good starting point.
Technical Details One of the most popular diversity measure is lexical diversily, which refers to the range of different words occurring in one text. The greater range implies greater diversity and quality. Type-token (1987) is originally proposed as:
ratio (TTR) Templin 1957);
Richards
TTR;- Unnique[::]
(24)
where Unique(x;) denotes the set of unique tokens present in x.. To reduce the sensitivity of TTR to the variation of (ext. length, several studies Covington & Mclall 2008 2010 Kettunen (2014): Mallach el al. (2021) standardized the length by introducing logarithms or n n-grams into the formula.
Later, computational approaches to measure lexical diversity have been developed such as vocabulary diversity (vood-D) Malvern Richards (197);; Malvern et al.] (2004); [ilverman & Rat (2002); deBoer (2014) the measure of textual lexical diversity (MTLD) [MCaartyy & Jarvis (2010): Jarvis Daller (2013) and
hypergeometric distribution diversity (HD-D) [Jarvis (2013); McCarrhhy (2005)). All these metrics require multi-step computation for approximation. Specifically for vocd-D, random sampling is first performed on 2; for series of sub-sequences with varying lengths l (e.g., 10, 20, 30 tokens). Then, TT TTR* is:
Unniqu(((ggssstt))
TT TTR--
1 15
< b k.
(25)
where T((jS,j++b) denotes the sub-sequence of x4 starting from the randomly chosen index j and ending at the index + k. Then, the curve of TTR versus the lengths k is plotted and a mathematical model is built. for fitting the curve:
TTR
D [(1 [ k
)
11.
(26)
k
14

---

Published in Transactions on Machine Learning Research (12/2024)
where D is the only parameter required to be estimated. By approximating TÊR towards TTR with the least squares, we have Dbest fit. = D:
vocd- D, D.
(27)
A larger reffects the higher diversity of 24. The computation of MTLD, on the other hand, first determines the TTR, as a pre-defined threshold, and then partitions H; into M different. contiguous subsequences {at, "i:,,,, ,
a.M), ,..,,,;?",, }. Each subsequence x T105,<Jt)) Vk 0, V1 S j S | k maintains TT R. .....
above the threshold T. The MTLD is defined as:
M
MTLD; 1 M
(28)
The HD-D shares the same idea behind vocd-D but stems from the hypergeometric distribution McCarthy & Jarvis (2010). ]. With M-times sampling, the HD-D represents the probability of drawing certain mumber of tokens of the given type from the subsequence of xy with a particular size h:
[nnique(a))
M
HD-D, Σ
Σ
(<7()) = ut), 1 Snn [
(29)
Ue E (nniuue(ss,.aa oT" = IIUSK Vk > O, 0, S S [z k.
Variants of TTR indicators such as MTTRSS Malvern et al. 2004 J. MSTTR Malvern et al. (2004)) ) MATTR Covington & McFall (2010), and MTLD-W W Vidal & Jarvis (2020) Kyle el al. (2021) all larget al
the solutions to two fundamental problems Bestgen (2023)] 1) the sensitivity of indicators to text length, and 2) the impact of the indicator parameters. Li et al. (2015) propose two rather simplified TTR scores
distinct-1 and distinct-2, where the number of distinct unigrams and bigrams of are respectively divided
by the total number of tokens. Other studies [Ca & Clark 2017 Zhu et al. (018)); 8); Shu et al. (0019) Tevel
& Berant
| extend the application of n-gram-based diversity for model generated responses.
Apart from lexical diversity, there exists many efficient diversity indicators that are built upon the semantics of each example. [ong et al. (201T)) propose to approximate k-nearest neighbor (k-NN) graph Peterson [2009]] with arbitrary similarity measures on semantic embeddings of large-scale datasets. Such efficient. construction of k-NN graph allows the distance of x; to its j-th nearest neighbors to be feasible diversity measure:
KNN: d(g(az), ((N,
(30)
where (2s) denotes the j-th closest neighbor of x; in the embedding space projected by 9(-). The common choices of the distance function d(-,-) include the Euclidean distance, cosine distance, and Jaccard coelficient
distance Huang et al. (2008)] The projection from text (e.g., instruction-responsee pairs) into the embedding space can be achieved with pre-trained sentence BERT Reimers & Gurevych (2019); Feng et al. (2020)) ], where an additional pooling operation is performed on the final output of BERT Devlin et al. (2018) for sentence embeddings. Note that a higher INN;; N; implies that the sample x; is more unique and should kept:
in subset selection for higher diversity. Due to the fne-grained representation capability of BERT, existing
hand-crafted indicators often rely on BERT embeddings for similarity or diversity measurement
Tevel &
Berant 2020); Zhang et al. (2019): Larson et al. (2019) Yauney et al. 2023
To improve the generalization of diversity mcasure, Xu ct al. (2023b) argue that the statistics of feature embedding of each sample itself should be considered. It does not require additional prior knowledge on the structure of embeddings. Given all datapoints 2; E S, their semantic embeddings from any sentence encoder R15)x xH, The row variance Var, of each embodding g(x;)) in the reduced dimensional space Risix by principal components analysis (PCA) Wold et al. (1987)
can be represented as X (g(z1). ((2)))))()))))....)))))))))))))) ((.2),.... gxx)),)).....)
is used as the diversity indicator:
Var:
EG = 1) -4:)
k-
(31)
1
14
k
Yij
j=l
15

---

Published in Transactions on Machine Learning Research (12/2024)
where the PCA chooses the top-k eigenvectors (V [U1, U2 ... Uk] with A 2 2 A₂) of the covariance
matrix Coc = QAQ = - ux)² (xx), μx = [S] - X; to project the original embeddings into more compact and reduced ones via Y = (X /x)V. Samples with the highest 20% Var; (via Eq. are selected as the variety-curatee dataset.
When it comes to the overall diversity of a dataset S, the average distance of any datapoint 2, to its closest neighbor the dataset, namely N;,, can be leveraged intuitively:
1 19 Σ
DANN(S)
[S] KNN, x; E S.
(32)
Such diversity measure has been widely usod dataset construction and retrieval Stasaski al. (020 :
Stasaski & Hearst )2022]; Mithun et al. (2019) Spyromitros- Xioulis al. (2015); Sun al. (2024a]; Tonescu
et al. 2018 Du & Black (2019) simply perform clustering on all samples with k-means Ikotun et al.
(2023 into K clusters CC,,C2....CK in the embedding space, and then uses the cluster inertia as diversity
indicators:
Dinertio (S) = Σ l1 - -m,11
j=1 E,C
(33)
My = [C 2,EC,
g(x)).
Lai et al. [2020)] develop diversity metric on the dispersion of a cluster induced by embeddings of all samples, where the cluster is approximated by multi-variate Gaussian distríbution:
H II
Dradiue (S)
Jj
(34)
where H is the dimension of the projected embeddings g(I;) E RH and denotes the radius of the ellipsoid along the j-th axis of the dataset S. The inter-cluster (class) distance can also be used for diversity
measure Dang & Verma (0024]]:
K Σ div.s(P)l|PP))
(SD
(35)
K
where P3 denotes the inverse-document frequency (IDF) distribution Sparck Jones (1972 of the cluster C₁ and div is the Jensen-Shannon divergence.
Remark Both lexical and semantic diversity should considered with hand-crafted indicators. The optimization of individual diversity would contribute the overall diversity of the entire dataset.
4.2 Model-based Indicators
Overview Similar to Eq. 9 model-based indicators on diversity also rely on the target or proxy language model for computing the indices.
The diversity of a dataset S can be intuitively defined as the sum of rarity measures of each constituting element z;- Accordingly, entropy-related methods are proposed to estimate such rarity. The more uncommon, various samples exist, the higher diversity the dataset becomes. Mathematically, the vanilla entropy (Shannon (1948)] is proposed for diversity measures:
Technical Details
Dentropu
(S)= - Σ P(z.00) (Pgx((()))))
(36)
N.ES
16

---

Published in Transactions on Machine Learning Research (12/2024)
where P(zz) denotes the probability of z4 occurring in the dataset. Later, Rényi entropy Rényi 1961 introduce an additional parameter > 0, * 1 for generalized entropy definition:
Da RE (5) 1
log2( P(a,10))).
(37)
2,68
The parameter a adjusts the element-wise emphasis on rare or frequent events.
Studies on biology and ecology Mouillot & Lepretre (1999 : Peet
1974);
He & Hu (2005); Gregorius &
Gillet (2008) investigate Simpson's Index (SI) (Simpson (1949); Wu et al. (2024 2022) for measuring the
biodiversity of species and genetics. Zhou (2020) propose a variant of the original SI with a more
flexible statistic metric:
DS1(S) E (z; = xz(0)
(38)
(|S( +1)
where the equivalence of H, and zj is judged by indicator function parameterized as O.
Vendi Score (VS) Dan Friedman & Dieng 2023
Pasarkar & Dieng 2023 Nguyen & Dieng 2024 is
rencently proposed for diversity measurement in machine learning researches. Inspired by the Rényi entropy,
a generalized VS metric Pasarkar & Dieng (022 is defined as below:
SI Σ
+ exp( 1
loga( 1-a (=1,EEeapp())
(39)
where λyo denotes the normalized eigenvalues of the similarity kernel matrix Ksjo, and supp(A) is the set of indices of all non-zero eigenvalues. The smaller a 1 makes the scoring more sensitive to rare classes and therefore allows accurate diversity measurement even under severe class imbalance. One simple implementation of the similarity kernel Ksjø is to use the Gaussian Radial Basis function k with feature embeddings as ([g(x; [0). [.gzz110)) (0)) = exp(-}lles (0) (()r)10)1??) Nguyen & Dieng (2024) further introduce quality scoring into Eq. 39 where for each subset S, C S, its average quality score ((a))
1 Ex,ss, IND;
15.
is multiplied with SVS
(Sp) for comprehensive evaluation in terms of quality and diversity.
Miranda et al. (022)) propose an intrinsic diversity coefficient to measure the diversity of a dataset with Task2Vec embeddings Achille et al. (2019); Nguyen et al. (2019) for distance computation between different. tasks. The Task2 encodes data from different tasks by the diagonal entries of the Fisher Information
Matrix (FIM). The FIM results from fine-tuning only the final (e.g., token classification) layer of a pre-trained
model, namely a probe model (e.g., GPT2 Radford et al. 2019 [), to solve the task. Given a batch of samples B, the mathematical representation of FIM is defined as:
FB EmiiiRe Ve log P(am)la
(40)
V Volog
where (gg) denotes the j-th token predicted from the model parameterized as 0 given the real sequence input Z(((5). The expectation E,, takes an average over the sequence length |a,] for each z, sampled randomly
from the batch x; E B. The Task2Vec embedding fB diog(FB), where diag(-) denotes the diagonal entries of Fn. Based on the Task2Vec embeddings, Lee et al. 2023 propose to compute the diversity coelficients div specifically for NLP datasets:
Dddi((S) = En B. Sma). (ii S2) = AA Sy dl Sm2):
(41)
where d denotes distance measurement (e-g., cosine distance). Both Bj and B, are two batches sampled respectively from the same or different datasets for diversity measures within across datasets. Experiments confirm that hand-crafted indicators such as the number of latent concepts Xie et al. (2021)) and the richness of vocabulary are positively associated with the proposed div coefficients.
17

---

Published in Transactions on Machine Learning Research (12/2024)
Lu et al. (2023a) develop diversity mcasure by open-ended tagging. Specifically, tagging model parameterized by is trained with GPT4-labeled tagging pairs. To label such tags of each datapoint, the GPT4 first performs open-set fine-grained tagging and then all the collected tags are normalized to filter out low-frequency ones and aggregate near-duplicate oncs. The specifically trained tagging model describes each instruction turing datapoint. 24 by its fine-grained, alomic intent ions and semantics (e.g., lasks and domains). Correspondingly, the number of tags can be viewed as a diversity indicator for sampling instruction subset. S. from the whole set S (see Alg.
Algorithm TagLM- Diverse Sampling Lu (2023a) ]
Require: data x; S,, tagging LLM To, visited tag set D and budget b
1: Initialize S. = 0
2: for each E S do
3: Obtain tags Dz, = T);
4: end for repeat
Initialize D 0
for each H; = arg m8 Dr. do
if D U D.,
D
then
S, = S U (z;}
10:
D = D UDz
11:
S S {
12:
end if end for
13:
14:
until [S,] b
15:
return Sb
Remark The model-based indicators are highlighted by their flexibility in handling various aspects of diversity either implicilly or explicilly.
4.3 Geometry-based Coreset Sampling
Overview Instead of explicitly calculating the diversity-aware indicators, recent studies selecting instruction datasets tend to introduce coreset sampling methods for systematic consideration Guo et al (2022) Specifically, coresct sampling aims to find the most informative-and-dierr subset that reprosents the entire dataset the most, so that close or even surpassing performance can be achieved on the language model trained on the subset with respect to that on the entire set.
Technical Details Among different categories of coreset sampling methods, geometry-based methods are the most intuitive and widely-used ones Chen et al. (2012); Agarwal et al. (2020); [Sener Savarese
(2017); Sinha al. (2020); Kamalov (2020); Rezazadegan Tavakoli et al. (2011) ; Kirchenbauer al. (2024); Zhou et al. [2023]]. The intuition behind is that close samples in the embedding space often share similar properties with low diversity. Therefore, redundant information can be effectively suppressed by controlling the minimurn distance between any two samples for subset selection. Specifically, k-center greedy is a typical diversity-orienter sampling method for massive pretraining and instruction-tuning corpus Chen et al. (2023a) ;
Bhatt et al. (2024): ; Wu et al. (2023): ) Zhao & Fang (2024); Du et al. (2023) ]. It solves the minimax facility location (FL) problem [Coruujjoss
al. (183); Farahani & Hekmatfar (2009)), i.c., sclecting the subset Sb under the given size budget b from the full set. S that the largest distance between an example in SA,S,
and its closest example in S₁ is minimized:
max min d(g(xz). S.CS, b ES) 2,55.
min
The direct solution Eq. 42 Na-hard Cook et al. 1994] and greedy approximation is proposed Sener
& Savarese (2017) (see A
2 Instead of simultaneously retrieving all the datapoints that can maximize
18

---

Published in Transactions on Machine Learning Research (12/2024)
the diversity of the selected subset, k-center greedy iteratively finds the most heterogeneous datapoint until the budget b runs out. For initialization of Sg. one can either choose randomly sampled datapoints from S, or use the cluster center points from K clusters C2... "0........, Ck) of S via k-means clustering. Similarly, the farthest point sampling method Eldar et al. (1997) sharcs the same principle that each iteration time only the farthest datapoint relative to the already selected coreset is chosen from the candidates.
Algorithm K-Center Greedy Sener & Savarese [2017)
Require: data H; S, existing pool SR and budget
1: Initialize S. = 5g
2: repeat
3: u = arg max m
mingzes, d(g(z)),
S. = (u)
until [S, = b b-b+1.. (Sg
return S S.
In addition to the k-center greedy, the herding methods Chen et al. (0012) Welling 2009); ; Huszár &
Duvenaud (2012); Adhikary & Boots [2022]] select datapoints H; so that the distance between the coresct center and the full set center is minimized in the embedding space. For efficiency, it is also approximated via sample each time into the S, to minimize the distance between two centers (sce Alg. 3 Similar to k-center greody, the herding greedy also performs iterative sample selection. However, it differs in that in each step, the distance between the centers of the selected set and the full set is minimized.
greedy implementation [Chen et al. (2016): Harvey & Samadi (2014) by adding
Algorithm Herding Greedy Harvey & Samadi (2014))
Require: data T2 S, budget b
1: Initialize μ 1 Σ o(x))
2: Initialize So = 0
3: for t = 1 to do
4: u = arg min I|m-
s. En {(2,)|1
S. S. (u)
end for return So
Furthermore, recent studies tend to develop complex heuristic sampling methods that takes geometry-based diversity into consideration Jiang et al. (2023e); Chan et al.) (2021): Xia et al. (2022)). Specifically, the inter-sample similarity of the selected coreset is minimized in return for an overall high diversity. [Jiang et al.
(2024c) propose to preserve informative subset with the learning complexity (sce Eq. 11 and implicitly puts constraints on its diversity via sampling the k-means clusters:
Ddist (S) = S
Σ 2,E8
min 2(1, C,
(43)
where C denotes the constant that controls the degree of diversity. A larger represents the larger diversity of the dataset. The detailed procedure be found in Alg. [ It can be seen that datapoints are first sampled bluster-by-chuster ensure more balanced data distribution. Then, bersentile-based selection is performed on each cluster to select datapoints with easy learning complexily.
Bukharin & Zhao [2023] propose the quality-diversity instruction tuning (QDIT). It also uses FL functions for diversity measure of the subset Sp:
DFL( Σ max sim(g(x)).g(z)))
(44)
E.E9, 25 es
19

---

Published in Transactions on Machine Learning Research (12/2024)
Algorithm Easy and Diverse First Sampling [Jiang et al. (2024c)
Require: data E S, existing pool Sr, budget b, and the number of clusters K 1: Initialize So = Sg
k arg mine 2j=1 E Ex,Ec,Cs ||
a(a;) |ل(a;))| myll?
Mj = Cj Ex,ec, [|a(2;]||
3: for
to K do
S = {zi (zile((5(z;)) S E.EEC
4:
So So US,
end for
7: return Sb.
where .i denotes the similarity function (e.g,, cosine similarity). Tf the selected S. can be well- representative the entire set S, then S, is assumed high diversity. Given quality scores defined by Eq. 21 the detailed mechanism of QDIT is described in Alg. 5 with greedy approximation. Different from the previous sequential setup that respectively prioritizes quality and diversity in two successive steps, QDIT adopts dynamic weighting the GPT mooring-based quality and the FI-based diversity for selection.
Algorithm QDIT sampling Bukharin & Zhao 2023
Require: data v; E S, a budget b, and the trade-off hyper-parameter 1: Initialize S = 0 for 1 u arg es a)) QS (xx)) a GPTScore;
do
S = S, u {u}
end for 6: return S.
Liu et al. [2023b]] adopt the quality score-first and diversity-aware data selection method (DEITA), where all datapoints are first scored and sorted by quality measurement, and then selected by geometry-based heuristic criterion (i.e., Repr Filter). Specifically, it considers that for each chosen datapoint in S, its NNN! (Eq. 30) should be above a certain threshold т so that the overall diversity (N,N((S)) (Eq. 32) can be improved. As shown in Alg. G the quality and complexity of each sample x; are respectively measured by the trained complexity scoring model AC and the quality scoring model 0g with prompts PC and pq. Then, samples with high Geq are prioritized but only those dissimilar ones be kept for the diversity of Sy.
Algorithm DEITA Sampling Liu et al. 2023b
Require: data x, S and budget b
1: Compute the combined complexity and quality score Goq(#:) = G(N,pclOc) ((1:.))(00)
2: u arg max,,es Gzo(z,(0)
3: Initialize So = {u}
SV(u) S=S)(()
while [S,] < b do
u arg maxe,es Goo(z;(0)
if d(g(u), ((N(u)) No(u) So then
S, = S, (u) 9: end if
8:
10: S=S\
11: end while
12: return S.
Another series of goometry-based methods focus on the organization of data structures via developing clustering-based sampling techniques Citovsky et. al. (0021) Tirumala al. (2024) Axiotis et al. (2024)
20

---

Published in Transactions on Machine Learning Research (12/2024)
Shao et al. 2024): Alcoforado et al.
(2024): Saranathan al. With respect to the clustering criterion,
traditional methods employ topic modeling with LDA Blei et al.
2003
Raghuveer et al. 2002 : Bui et al.
[2017]], NMF [ee & Seung (2000); Waan & Zhang 2012]; Shen & Si (2010); Lazar & Doncescu (2009)],
TF-IDF Sparck Jones (1972): Bafna ct al. (2016) Patil & Atique (2013): Roul et al. (2014)) and latent
concepts Xie al. (2021) to assign text. corpus into thematic clusters. Most recent studies exploit sentence
encoding methods Reimers & Gurevych (2019); Feng et al. (2020] to perform clustering in the embodding
space, where the vanilla k-means clustering and its variants Sinaga & Yang (2020); Kanungo al. (2000);
Bandyapadhyay & Varadarajan 2015 DBSCAN Deng 2020 Khan et al. (2014): ): Cretulescu et al. [2019]]
and spectral clustering BBach & Jordan 2003 Von Luxburg (2007): Jia et al. (2014h) are widely used.
Specifically, (Tirumala et al. (2024]] propose to use SemDeDup Abbas et al. (2023)
to remove semantically
similar examples for deduplication, which provides a basis of diversily sampling. Then, k-means clustering is performed in the embedding space. In each cluster, the prototype-based sampling technique [Sorscher et al. [2022]] is used. The "prototypical' samples, whose distance to their assigned cluster centers are small, should be discarded first to allow more "outliers" to be kept in S, during iterative sampling (see Alg. 7 It is noted that two times of clustering operations conducted, where the semantic deduplication and prototype-based sampling are respectively performed on Ki and K2 clusters.
Algorithm D4 Sampling Liu et. al. (2023b))
Require: data I; S, budget b, the number of clusters for SemDeDup K₁ and the number of clusters for prototypicality K2
1: Initialize Sd K1 Σj=
S, 0
E Ex,cc,ss ll a(a.) My 1yl1?
arg mine
[[([))]]|
My C2
Ex,ec, [|s(,;]|
s(z))
3: for j j= 1o K1 do
Cy= 0
while |C < Cyl do
arg min ,ic, vo sim(g(E:) (b))
7: if max sim(g(u), (om(g(u),gaag ((x))) < then
U end if 10: Cÿ = C CRUCUUG (u}
8:
11: end while
12: end for
13: arg mine
Σ j=1
E E8,E0,cs, |ل(لتل|| 1
My Cl Ex,ec, [(x))
[[(
14: for j to K2 do
15:
S/ (adlFa(d( 14) > 24 E C2)
16: S. So US!
17: end for
18: return S₁
Axiotis et al. (024 propose a k-means cluster- based sensitivity sampling technique. For each datapoint in a cluster, both distance the cluster center and a proxy evaluation loss Feldman & Langberg (201T) mcasured on that cluster center contribute proportional
the probability of being chosen. [Shao al. [2024)] propose the balanced ClusterClip sampling. It first performs k-means clustering and then sample datapoints uniformly from each cluster. Different from the uniform sampling, the proposed ClusterClip puts constraints on the maximum number of each cluster being sampled, and therefore avoids overfitting of small clusters.
Alcoforado et al. ) comprehensively compare different geometry- based diversity sampling techniques such as similarity or distance-based greedy sampling and clustering- based sampling. It proposes three approaches select subsets S, for human annotation: 1) reverse semantic search, 2) ordered clustering, and 3) limited lexical similarity. For the reverse semantic search, two datapoints (x1,2y) that share the least.
21

---

Published in Transactions on Machine Learning Research (12/2024)
semantic similarity are first sampled as Sg. and then itcrative selection of the next most dissimilar element from S is added into Sg. Its implementation is quite similar to the k-center greedy algorithm (see Alg, 2 except for the initialization of Sg. For the limited lexical similarity approach, the first sample zo is chosen randomly for initialization of sg. For the remaining b b-1 1 quota, cach sample 2; is also randomly chosen from 5%,S, as long as sém( 1 S т, where sira(;.)) here denotes the lexical similarity such as BLEU Papineni ct al. [2002)] and ROUGE scores Lin 2004)]. The ordered clustering applics a hicrarchical and density-based clustering algorithm like HDBSCAN [Campello et al. (2013)] on all samples and sequentially (i.e., from large to small clusters) choose the samples of the lowest membership in each cluster into the subset Sp. Experimental results show that the reverse semantic search performs most consistently and competitively, while the limited lexical similarity is sensitive to the hyper-parameter threshold т. The ordered clustering is not robust across datasets and fails to select high-quality samples.
Remark Geometry-based sampling is intuitive and effective in diversity control. Most solutions to optimizing the overall diversity can be reformulated as variants of an iterative, similarity or distance- based, greedy sampling technique. Clustering does play explanatory role in deciphering the embedding structures, making it easier and preciser to control the proportion of selection.
4.4 Bilevel Optimization-base Coreset Sampling
Overview
The selection of coreset can also be viewode as a bilevel optimization problem Colson et al.
2007 0: Zhang (2024); Sinha et al. (2017): Borsos et al. (2020); Killamsetty al. (2021b Zhang et al. (0022);
Borsos et al. (0224]; Pan et al. (2024) that consists of two loops: 1) the outer loop of optimizing the hard masks or soft weights for selecting the subset Sy from S: 2) the inner loop of optimizing the model parameters
0 on Sg. Without lose of generalizability, the bilevel optimization with the self-supervised language modeling loss can be written as follows:
S. = arg min
Σ
NLL 1Q
$.Cs
(45)
s.t. 0 = arg min Σ
NLLA AQQ
2,
Technical Details The retrieve method proposed by Killamsetty et al. (2021c) takes both labeled and unlabeled datasets into consideration, where the self self-supervised loss from the unlabeled set (e.g., consistency regularization Xie et al. (2020); Wang et al (2021c) and entropy regularization [Zhao et al.
(2020b); (Grnnvvaeet & Bengio (2004); Erkan & Altun (2010)) contributes to the inter-level and outer-level optimization as well. To improve the robustness, Glister Killamsetty et al. (2021b)) optimizes the outer-level coresel selection on the additionally prepared validation set for the minimized validation loss. Li el al. [2023e]] further emphasize the role of the validation set in bilevel optimization. It not only computes the loss the validation set for adversarial training, but also introduces gradient matching Killamsetty et al. 2021a where the gradient of the model on the selected subset S, should be close to that on the entire S.
Borsos et al. [0044]] reformulate the coreset sampling as a cardinallty-connstraieed bilevel optimization problem. It proposes greedy forward selection and first-order methods that apply to any twice differentiable models. Variants of the solution for acceleration are extended: 1) binary weights, inverse-hessian-
hessian hessian-vectorr vector product approximations, and batch-wise selection; 2) small proxy models for fast estimation; 3) enforced sparsity-inducing penalty in the outer loop.
The ScaleBiO Pan et al. [202g)] specifically address the data reweighting problem for large-scale LLM instruction tuning. It also prepares an extra validation set gual for the minimization of the outer loop. ScaleBio transforms the bilevel optimization into the single loop framework with an outer-level problem plus a constraint of the inner-level problem. A multiplier a > and a proxy u. for optimizing the original inner loop (i.e., model weights 0) are introduced into the minimax formulation Kwon et al. (0023 Lu & Mei (024)
In contrast to a fixed budget b, Xia al. (2024c)) propose a lexicographic cilevcl-optimizaaion method Borsos
et al. (202)); Killamsetty et al. 22021b where the inner loop optimizes model parameters and the outer
22

---

Published in Transactions on Machine Learning Research (12/2024)
loop optimizes data selection. During optimization of the data selection mask, the loss terms are relaxed to allow the size of the final coreset smaller than b.
The bilevel optimization methods often involve regularization tricks as a relaxation to the original problem with nested outer-inner loops. Compared with the boolean selection, soft weights- based objective guarantces a higher level of diversity as each sample contributes more or less the overall optimization.
Remark
5 Importance-based Selection
This section provides the review of methods on importance mcasurement and selection. By importance we mean the necessity of adding one instruction-resposee sample into the training set. Due to the pre-training nature of L.Mss, a wide range of materials have been "parameterized" several common tasks can be correctly solved without additional fine-tuning, In this case, alignment is not required for easy samples but becomes indispensable for difficult ones. The selected datapoints provide supplementary knowledge to activate the pre-trained LLMs on following complex instructions. Generally, the evaluation function ((x;) in the context of importance can be alternatively implemented as: 1) the complexity of the datapoint itself (c that can be perceived by the task definition and the model's uncertainty, 2) the contribution of each datapoint to the overall performance qP that can be reflected in the losses and errors during learning, and 3) the degree of gradient matching ga that indicates influential datapoints for effective selection. Consequently, we can unify the importance-based measurements as:
internal knowledge and therefore
g(z;)) = fs (z))) (p(x;) ga(x:)),
(46)
where fi denotes the aggregation function. It is noted that most existing methods simply choose one of the above evaluation implementations and therefore Si can be designed as an exclusive choice function.
5.1 Hand-crafted Indicators
Overview Existing researches on importance measurement of datapoints often stem from two aspects: 1) from the perspective of a datapoint itself, i.c., the difficulty or complexity of cach datapoint and the amount of information it provides; 2) from the perspective of the model under development, i.c., the necessity of learning from such datapoint. based on the current. performance and confidence (uncertainty), Most. hand-crafted indicators are proposed to analyze the text difficulty.
Technical Details The readability indices Young & Shishido (2023) can be used to assess both quality (see Sec. $3.1) and difficulty of text samples. Specifically, samples with intricate grammar, advanced vocabulary, and inference dependency are deemed as difficult ones and can be used to evaluate robustness of models
across benchmarks of various difficulty levels (Smith & Johnson (2020); Kiela el al. (2021): : Ethayarajh el. al.
(2022); [eliinkvv& & Glass (2019); Nie et al. (2019); Ribeiro et al. [2020] ]. For specialized domains such as
solving maths problems, the education level (c.g., clementary-lovel high school-level, and university-level)
determines the difficulty of samples [Patel et al. (2021) Huang al. (0016) Koncel-Kedziorski et al.) (0016)
One of the pioneering studies on readability scores for difficulty assessment is to compute the percentage of difficult or easy words in one sentence Klare 1974]] )3 Begeny & Greene (2014)) The words on a pre-defined list are counted as familiar words, and those not listed are unfamiliar, advanced words. Besides, the average number of syllables per word, the number of single-syllable words, and the number of multi-syllable words are also indicative in assessing the text materials [Connatser (1999); Carrell (1987); Zakaluk & Samuels (1888): Dale & Chall [1949)] Notably, there exist three representative readability metrics: 1) the Dale Chall formula Chall & Dale [,95]]] J, 2) the flesch reading ease Flesch (1948)], and 3) the gunning fog index Gunning 1952p) Given these metrics, Saranathan et al.) conduct a thorough analysis on existing NLP datasets S to select the most challenging subsets for efficient evaluation of LLMs. The easiest and hardest. samples from the TruthfulQA Lin et al. (202I)] via these indicators are confirmed positively correlated with the actual complexity. The selection of difficult instruction-responee pairs via E alloows a wider distribution of performance for the models under investigation. The introduction of difficult datapoints into the subset S, helps keep the relative rank of different models unchanged compared with that measured the entire set S.
23

---

Published in Transactions on Machine Learning Research (12/2024)
Remark
The difficulty indices help comprehensively analyze the robustness of models across samples and datasets. In addition, it also presents guidelines of curating and constructing discriminating NT. benchmarks.
5.2 Model-based Indicators
Overview
To avoid potential confusion, the model-based importance indicators discussed in this section are mainly categorized as three kinds: 1) uncertainty-based, 2) reward score- score-based; and 3) data model-based based. Methods that employ training/infern /inforence losses, errors (motrics), and gradients are not included despite their involvement of the language model for importance sampling.
Technical Details Inspired from uncertainty indicators, Siddhant & Lipton (2018); Kung al. (2023
Nieth et al. (2024)) propose the prompt uncertainty which measures the disagreement between model responses on different perturbed versions of the same instruction:
Uprompt
P(rn)))())))))))
(47)
( 0)
where K denotes the number of perturbations and i is the k-th perturbed prompt. Note that only the instruction part Zi(<e) is perturbed and sent to the model for the following likelihood measurement on the original response T(g)- = t, t + 1,... |as]- Samples with high prompt uncertainty should be chosen for fine-tuning since the model does not perform consistently on such instructions.
Jiang al. 2023b target the over-confidence problem of LLMs after instruction tuning [Kadavath et al. 2022 and propose the CAPE to calibrate the uncertainty with augmented prompt ensembles. They first transform all discriminative and generative tasks into multiple-choice problems, and use the LLM's predicted probabilities over answer choices (e.g., A, B, C) for uncertainty estimation. Then, prompt augmentations are performed via paraphrasing of the template, permutation of the in-context examples and the answer choices. Multiple predictions over the answer choices are collected with the augmented prompts as inputs, which helps calibrate the uncertainty an ensemble manner. Such calibrated uncertainty tells if an instruction-tunct LLM simply memorizes the response to given prompt rather than truly understanding the instruction. Therefore, it can be used to precisely choose important datapoints with high uncertainty.
Apart from the incertaint, the reward model can also be used beyond quality scorer. Since most of the knowledge and capabilities are acquired during pre-training Zhou et al. (2024a]), the instruction tuning datasets are aimed at aligning the behavior of models with human preference and expectations. Therefore, for any given instruction I4, if the generated response is of high quality, then the necessity of fine-tuning on this instruction is low. Accordingly, 2; is deemed as 'unimportant" and will not be chosen into the subset. In that case, the language model parameterized as is first. prompted with T(({)) to generate the response s0 ((2))) Then, a reward model parameterized as & acts as necessity evaluation model:
R = Te (Zi(<e)) (22))
(48)
Samples whose necessity score R below a pre-determined threshold are selected via Eq. 7 implying that the model 0 does not own the capabilities to handle x; and requires fine-tuning.
; Another series of model-based importance estimation methods are based on datamodels Tlyas et al. (002 Park al. (2023); Jain et al. (2023); Kang et al. (2024); Chhabra al. (2024): Saunshi ct al. (2022): Ye et al. (2024)) ) where the contribution of each datapoint to the model's behavior is estimated. The datamodels can be implemented in any machine learning model which targets at predicting the influence of each datapoint
the performance of the trained model Koh & Liang 2017): Jain et al. 2022 Liu et al. (024bb Picard et al.
2024 Bae et al. (022); Covert et al. (0024).
Engstrom et al. 2024 propose to use datamodels to select subsets that maximize the overall performance. Specifically, it chooses the subset S, C S, (21, 2is } by estimating the loss of the model trained on
it. Out of simplicity, the datamodel Fg can be implemented as linear model and it learns to approximate the actual loss via the TARK estimator Park et al. 2023)
24

---

Published in Transactions on Machine Learning Research (12/2024)
0z, arg min E (m) S cseesgg Ls, (Si)):
11 00 otherwise.
1, if Sb.
(49)
1s, (0,1)18), (1s,)s =
To, (1s.) = 0% 15.
where Lz,(S) denotes the loss of the model (trained on Si) on the sample zz.- The (m() is a m-sample empirical expectation and (:rg((:)) .) is a regression loss function (e.g,, mean squared error). Intuitively, what the datamodel тæ, does is approximate the real loss Lx, (Si) under various compositions of subsets St Sb. Given any subset Sy, the averaged loss approximated by the datamodel on all 2j E Sevat is calculated on the evaluation set Seval and minimized to find the optimal Sb, |Sa] = b:
S = arg min (S)),
S.Cs
L eval (5h) = By Sevut [THLg
(1s;)]
(50)
Σ
0 1s; = 13x
02,)
SSeval
|eeval zyESevut
ZyESeval
The importance of v4 E S is therefore mcasured by [Seaal] Exjes Ozy and its smallest elements are chosen for the minimum loss Ls..ar
Liu el al. [2024b]] also propose simulation- Guu al. (0233 linear datamodel that correlates the training samples with the validation test set loss. A featured simulator, namely GPTTueence, models the training dynamics (e.g., loss, BLEU and ROUGE scores) across time via an n-th order Markov process. It extracts representations g(z;), x4 E S from BERT or GPT, and generates both multiplicative and additive factors to reflect the influence of any training example on the testing set. The testing performance Φe at any time affected by: 1) its performance at preceding n times and 2) the current training batch G;:
(x(xx) = Σ
a3(c))(1)(1x)) + B(cx), Van E Seval,
j=1
lodl
(51)
ag(c)= Σ
Aiji ((c)) Σ
Bi, E C,
1=1
==1
Ag (WT ())() Ung(E))), B, (W" g(x))s U'g(sx))
where WT ()) ()) Yt being the ground-truth metric score monitored during training at step L. The (5-) denoles the Frobenius inner product. Samples that reduce the evaluation loss the most are selected as influential data.
UT W U are learnable weights which are optimized by minimizing Et (xxx)) with
Instead of performing off- line selection, YYu et (2024) propose MATES where small datamodel continuously selects the most effective subset. for the current. training of the LLM. The datamodel, like a partner, is updated alternatively to adapt to the constantly changing preferences of the model under development.
Unlike previous datamodels that predict the influence of datapoints via the testing performance of the model, Xie et al. (2023) propose the DSTR with importance scores estimeted by the distributional resemblance. Tt. simply assumes that training samples that resemble the evaluation set are important, and these datapoints should be selected with higher probability. Given the hashed n-grams features h((x)) E N of F,, its importance score Wi is calculated as:
Dreat (h((a))))))) Areat (h(xz)))
W/
0
Σ -1=1
W
frat (h(x:)) = II
M(z.)s
Dfeat (M(x:)) (a0z))== = II aorrsss
(52)
1=1
j=1
3. Σn
Σ
hxx)),
Σ Mx,)),
En.ES E8cool
17h(x:)
Enes 17h(z,)
ZyES
25

---

Published in Transactions on Machine Learning Research (12/2024)
where S and Seval respectively denote the training set and the evaluation set. Given the budget b, the subset S is obtained by importance-wwighteed sampling without replacement b times. Zhang et al. (0023c) also proposes to use a independent-cascadee diffusion model [i et al. (2018); Du et al. [2014)] to mimic the information diffusion process upon a directed graph on emboddings of datapoints. The most influential datapoint are selected for annotation and serve as in-context learning examples for L.Mss.
Remark Compared with uncertainty and reward score, datamodel-based importance indicators are more correlated with the downstream performance since the task-specific evaluation set introduced to provide foedback on the selection scheme.
5.3 Loss and Error-based Coreset Sampling
Overview During training, samples that contribute more to the total loss or cause worse performance are considered more important. In the light of this statement, the influence of each datapoint can also be measured via the losses and errors for coresct sampling. Compared with the datamodel-based mcasurement that estimates individual importance with specifically designed datamodel, loss and error-based measurement is performed with the same LLM under development.
Technical Details One kind of methods that record the errons of each sample during training to estimate importance forgetting score or forgetting event [Tonvaa al. (2018)). It counts how many times the forgetting happens with the itcration of training step t. For any given sample x; in batch B (a, B C S), if the previous accuracy acc;-1 surpasses the current accuracy acc; (ace) > acc;*1), then the example 2; undergoes forgetting event. Conversely, learning event occurs if acef aect+1 ++1 The number of forget events implies whether the sample is difficult and indispensable for training, An example x; is defined as unforgettable satisfics:
1, Et < 0o, s.t. acdt < acet+1
Unforgel: =
and Vk t", acck acc
(53)
0,
otherwise.
The easy samples with Unforget; can be simply discarded and the important subset S, {a;lUnforget, 2, E S] is selected for training. Recent studies on both pre-training and instruction tuning have investigated
the effectiveness of using the forgetting score for efficient data pruning [Sorscher et al. (2022 Paul et al.
(2021) Zhang et al. (2023a): Jin & Ren 2024a); Maini al. (2022
In contrast to the term 'forgetting', researchers introduce the concept 'memorization' Feldman 2020
Tirumala et al. (2022) Antoniades et al. (2024) for analysis on the generalization of deep models Zhang
et al. [2021]]. The memorization of training samples is necessary for reducing close-to-optimal generalin lintralization
error especially when a long-tailed disttribution observed for the training set [Feldman (2020) Specifically, the amount of label memorization on the Enteruction-rsspons pair TT(()) defined as follows:
lms
Memo; 1
(54)
P(zg)) [())):: 0S%,.)),
where 8 and 8% respectively refer to the language model parameters optimized with the entire set with and without 24. Accordingly, the influence of the sample z; other samples Ib. 26 * be defined as Feldman & Zhang 20200]]
lx
Infla
I
(55)
J=
P(T (ACC): ((r)))
26

---

Published in Transactions on Machine Learning Research (12/2024)
where Tk(c() and ZR(24)) respectively denote the instruction and response part of 2g. In practice, the memorization and influence scores are approximated via batch-wise sampling where N batches B1, B2,... Bx are sampled from S with [B n. For each batch B;, a language model parameterized as 03: is trained to compute the memorization and influence scores of each sample F₁. It is noted that some batches contain 2; and the others do not. Therefore, the two probability terms Eqs. 54 and 35 are respectively averaged over multiple probability outputs of the models trained on batches with and without 24-- [Sorscher et al. (2022)
confirm that memorization scores (Eq. 54] demonstrate stronger performance on pruning the dataset into a significantly smaller subset Sb than random sampling, EL2N (Eq. [3], and influence scores (Eq. 55 Suzuki
et al. (2023) and [Schoch et al. (2023)) also follow Feldman & Zhang 22200) to select the high-quality influential subset for LLM training.
Furthermore, Chen et al. (024b)] use the evaluation loss to check whether the current task requires certain skills or capabilities that can be obtained by learning from the prerequisite tasks. For each task, it selects the skill-dependent datapoints that reduce evaluation loss. [Mishra & Sachdeva (2020) propose a rather simple method that adopts proxy model (e.g., logistic regression and SVM) to train the randomly selected subset Sp and evaluate on the remaining set S",S, Such process iterates over multiple times to ensure that cach sample is at least validated once. The probability of cach sample being correctly predicted is used as importance measurement. Likewise, Lin et al. (2022) also quantify the average marginal effect (AMF) as influence of 24. It can be viewed as a variant of shapley value [Jia et al. (2019) Ghorbani & Zou
20199 Schoch et al. (2023) Kwon Zou (2021)] Different subsets are randomly sampled to train multiple
submodels and each submodel is evaluated for jointly estimating the AME via LASSO regression Lecué &
Mendelson ).18)
The loss and error-based selection methods are intuitive and effective to select the datapoints with high difficulty and influence. To accelerate the computation of marginal effect (gain) of each datapoint, iterative approximations can be adopted with small proxy models.
Remark
5.4 Gradient-based Coreset Sampling
Overview Since gradients directly affect the optimization of language models, two kinds of intuitive
methods for data selection are presented: 1) gradient matching Zhao et al. (2020a) Killamsetty et al. (021a);
Jiang et al. (2023d); : Zhao & Bilen (2023); Du et al./ (2024): Balles et al. 2022 Zhang et al. (2024a) ie..
the gradients of the entire set being approximated by the weighted gradients of the subset Sp. and 2) gradient-based influence Pruthi et al. (2020); Brophy al. (2023); Koh & Liang (2017); [asu et al. (2020):
Picard et al. (2024); Alaa & Van Der Schaar (2020)), i.e., the influence of each sample 2y on testiing datapoint Ie being measured by upweighted gradient multiplication. Specifically, the gradient matching aims to minimize the difference below:
8", 8 arg min d( VONLL
S
FEES
(56)
Σ Wc VONLL AQ 19), S. S,w, 0,
Σ. W S4S
where d(.,.) denotes the distance measurement and w, is the weight for the gradient of zz
The gradient-based influence methods, on the other hand, aim at selecting the most influential datapoints in terms of the variation of model parameters O. Given the optimal parameters 0*, the updated parameters
8 ((r)) by up-weighting the lass of x, with e can be derived as the first-order Taylor series expansion as follows:
8r, = arg min
NLIAA. + +.NII
S
ES
(57)
0- 0 - cH VLLL
where Ho- represents the Hessian with respect the model parameters 0*, Accordingly, the influence function of a sample z; the model parameters and its effect the performance of a particular sample xy can be
27

---

Published in Transactions on Machine Learning Research (12/2024)
respectively denoted as:
InflF, = de de mi
c= = -H VoNLL
Vo AQQ
(58)
InflFi, VONLLA Hg. VONLL
The importance indicator In,f approximately measures the change of the loss on Ij when 22 is removed from the training set. To expedite the computation of Hessian matrix for large models, a combination of
Hessian-vector product and optimization techniques are developed Pearlmutter [199]] Nilsen et al. (019
Mathieu & LeCun (2014); Agarwal et al. (2016); Shewchuk et al. (1994).
Another kind of influence score is defined as the expected gradient norm (GraNd score) Paul et al.
202
Kirsch 2023): : Böther et al. (2023) where the GraNd score controls the contribution of a training sample to
the change of the loss.
GraNd; = Fy Vo.LL
A1Q
(59)
Experiments Paul ct al. (001 suggest that the GraNd score (Eq. 59 can be well approximated by EL2N score (Eq. 13 for elficient data pruning.
Technical Details Xia et al. [2244a] propose to find the most influential training data that resemble the testing set the most via low-rank gradient similarity search. [Tan et al. (2024a)] introduce the moving-one sample-out (MoSo) by pinpointing the least informative samples via gradient-
influence assessment. To avoid the costly retraining procedure by iteratively moving one sample out, a gradient-based approximator is proposed to select samples whose gradients are consistently aligned with the average gradients of the entire training set.
For the detailed definition of distance measure of Eq. 56 Everaert & Potts 2023 exploit the KL-divergence to measure the difference between the selected subset and the lesting set. Note that here the objective is to approach the distribution of the testing set rather than the entire training set. Killamsetty et al. (2021a) speod up the gradient matching between the selected datasot and the validation set via an orthogonal matching pursuit algorithm. [in et al. (2024) apply gradient- based influence scores on recommendation datasets for effective LLM instruction tuning. [Schioppa et al. [2021] choose different way [Arnoldi (1951) to accelerate the computation of the inverse Hessian matrix in Eq. 57 and successfully scales up the influence scoring for LLMs with several hundreds of millions of parameters. Grosse et al. (2023) use the influence functions to study the generalization properties of LLMs To scale up influence functions for I.I.Ms up to 52 billions, an approximation technique via Eigenvalue-corrotte (EK-FAC) George et al.] [2021)] to elficiently find the most influential samples to the pre-trained LLMs over maths and programming abilities, cross-lingual generalization, and role-playing behavior. Zhao al. (2021)) condense the datasets into small informative synthetic samples where the gradients of the model the synthetic data are matching those the real data of the entire training set.
Kroneckor-Factoree
Approximate Curvature
Remark
The gradient-based coreset sampling techniques are highly dependent on the LLMs under develop- ment, where the gradients describe the model's inherent knowledge and uncertainty about each datapoint. Despite the precision of gradient- based selection methods, it is noted that approximation is unavoidable for application on LLMs. The efficiency and accuracy of various approximation techniques should be considered.
6 Results and Discussions
In this section, we classify different methods according to their respective emphases, and then demonstrate their effectiveness in the selection of "high-standard" datapoints for instruction tuning. First, we provide explanations on the classification and selection of recent instruction tuning methods. Then, we summarize the representative methods with their data statistics in Tab. [ Finally, we provide the detailed experimental results of each method respectively under our structures of quality, diversity, and importance.
Criteria on Classification and Selection of Existing Methods For the classification of existing methods into quaei,y-based, diversity-based, and importance-based categories, we adhere to the following
28

---

Published in Transactions on Machine Learning Research (12/2024)
Table 1: Statistics of datasets in existing representative data assessment and selection methods.
Methode
Quality
Diverdit
Importance
Training Set
Training Set. Size
IPD
Li al
2028a
Alpaca WisardL.M
62K
70K
LPPT
Xu
20236
/
X
Open. Platypus CodeAlpaca
20K
25K
DQ Zhou et
D023
x
Alpaca
52K
Aukner
[024g]
The
NA
InttroationMining Cao 65
2023
/
x
x
OpenOreu
Dally
50K
15K
Pl. Blast et
1024
FLAN
99X
Alpaguoua
Clen
2023b
/
Alpuca
8QuAD-N
NA
BSDetcctor
Cleu Mueller
2024
/
x
x
Emails-N
NA
DROP-N
NA
DEITA
Liu et all
3023b]]
Mixed[Share(iPTy Uhira +
200K
AusoD8
Zhang et
/
X
OpanWebMatk
NA
X
QultanadPsjaams
ClasterC1ppp
Shan at al
2034
x
X
Proof-Pile-2
OpanOecs
2.7M
4:2M
UltraChat
1.3M
LMSYS
Bukkaria Zhno
X
Alsuca
52K
Mixed (Alpec)+01G++Dlly))
270K
Dally
15K
DaDms
Eagetrom et ml.
2024
C4
NA
MATES
Yu
2024
C4
NA
DSIR
Xic
x
The
1.6B
Slall-it
x x / (FLAN Ve--Dally--ponnnmnsnntt---CO
x
LE38
Xia al.
2024n
270K
evidence: 1) the data characteristies emphasized in the motivation of their method development, and 2) the optimization objectives organized in this survey that are closest to those in their pipelines. For instance, the concept of diversity is stressed in DEITA [Liu et al. (2023b)) where a similarity-based filtering step is set to remove highly duplicated datapoints. Therefore, it falls under the category of diversity- selection methods. DSIR [ic et al. [2023]], on the other hand, directly estimates the importance of datapoints by matching the distribution between the selected subset and the target evaluation set. Tt. noted that many existing methods design compound, multi-facct selection criteria that strike balance between quality, diversity, and importance. In this case, we choose the most representative methods under each category to highlight their corresponding facets without misinterpreting their mechanisms. Furthermore, we select the most recent methods that exploit open-sourced LLMs for verifying the effectiveness of their proposed data selection techniques. Such accessibility to public LLMs avoids the privacy issues and inference costs brought by requesting API services from close-sourced proprietary models.
Quality The quality data directly impacts the effectiveness of model training. Quality control measures include data scoring, quality assessment, and more. In Tab. L we have summarized the results of different. methods focusing on data quality. In the table, we list the data used by different methods and the proportion/sizz /size of the data selected. It can be seen that the method of selecting data based quality match the results of training with full data even under the dala-poor regime. They are also superior to the results of randomly selecting a subset from the original datasct. In the table, WK stands for World Knowledge, CR stands for Commonsense Reasoning, LU stands for Language Understanding, SPS stands for Symbolic Problem Solving, and RC stands for Reading Comprehension.
Diversity Data engineering enhances the generalization ability of models by improving the diversity of datasets. Such improved diversity may be implemented via encompassing data from different sources, with varying features, and of distinct distributions. Rescarches indicate that datasets that are similar to the target data from the downstream tasks. Tab. B demonstrates the importance of diversity in data selection. Compared with random selection and uniform selection, the scheme of selecting
insufficient to merely select
29

---

Published in Transactions on Machine Learning Research (12/2024)
Table 2: Experimental results of quality- based selection methods are directly cited from their papers. BT denotes billions of tokens. WK, CR, LU, SPS, and RC respectively stand for compound datasets of World Knowledge, Commonsense Reasoning, Language Understanding, Symbolic Problem Solving, and Reading Comprehension.
Meshods
Training Set
Model
Ratio/Siae
Reported Recults on Teating Sets
HellaSwan
Trathful(Q
AspacaKval
Alpacs
LLMMA
0%
0.383
0.347
W.
7B
10%
0.631
0.378
0.429
0.020
cb al.
IFP
2023c
Alpucn
Full
107
6%
0.480
0.344
0.558
0.767
0.801
0.879
0.470
0.100
0.804
0.410
0.002
0.442
0:278
0,388
NA
MaMA2
1576
0.300
0.574
0.807
0.004
NA
711
Full
0.378
0.820
0.311
0.415
0.350
WasrrLM
10%
075
0.024
0.030
0.840
0.839
0.553
0.307
0.425
0.415
0,408
NA
18%
0.024
0.835
0.556
0.434
ARC
HellaSwag
MMLA
Trathnatt)
Open-
Mietral
Handam TMK
0.00Y
0.830
0.025
Platypua
0.043
0.844
0.645
0,400
RumannEv
Code- Alpsen
SldrCeder
Raudou T0K
Full 50%
0.085
0.112
The Pile
50%
0.101
0.187
0.085
0.034
0.109
PPL Ankour ni.
MPT
High 80% Full
0.182
0.128
0.332
0.004
0.106
2024
IB
0.165
0.123
0.289
0.036
0.080
Dolam
Low 10%
KIA 50%
0.101
0-180
0.100
0.101
0.213
0.319
0 0.1
0-033t
0.070
0 TU4
High 8019
0.311
0:012
0.086
A1G
HellaSwag
MELLU
Trorkduiig
IM 10K
0.367
0.798
0.499
0:483
Instriationn Minins Clso at al (0233
OpenOrea
LLaMA2 7B
IM 40K
0.648
0.801
O. 0.326
0.40
& Dolly
Random TOK
0.100
0.110
Handam 40K
0.709
0.11
MMLU
BBR
Random 20K
6.349
0.300
0:451
0.383
FLAN v2
LLaMA2
Randour
TOR
40K
MML.U
Aloagaaus Clen al. 2023
Alpacs
LLaMA2
LLaMA2
13B
7B1
Alpagaziu 9K
Random SK
Ramdoon UK
Full 12x
52R
IT
030
0.38Y
0,880
6.360
0.255
00000
0.884
0.338
0.344
0.167
0.10
0.162
0.3
0.450
0.401
0.479
0.405
0.305
SQUQA-N
DROP.N
Full Auto-fiter
0.450
NA
NA
SQuAD-N
6.899
NA
NA
Auto-cocreet
6.714
NA
NA
LLaMA2
Full
NA
0.507
NA
Chen & Maeller
2024
Rmaile-N
Chat
Anto-marrect
Anto-Riiter
Full
DROP-N
Antowllter
GSM0K
ARCS-
A
TETT
Boolg
NQ
MELGU
A Zhang et al. 2024c
Open- WehMaat
Mietral 7B
Random T
U.835
0.622
AstoD8 2.6BT
6.310
0.881
0.291
0.323
0.627
PIQA
W Wanogroadd
Randow 2.FT
0.222
0.802
0.372
Autob8 2.5TT
0.822
0.800
0.988
Sheated GLaMA 1.5B
RC
CR
WK
QuRator Westig es al.
QuRated= Pajama
Random 2011
0
0 10
2024
Qurater 30BT
0.621
0.565
0.152
30

---

Published in Transactions on Machine Learning Research (12/2024)
Table 3: Experimental results of diversity-based selection methods are directly cited from their papers.
Methods
Training Set
Model
Ratia/Size
Reported Rocnlte on Teating Sete
ARC
HelloSwug
MMLU
TruthfuIQA
MaMMA-13
Random TOK
DEITX 10K
0:595
0.358
0.800
0.820
0'475
0:608
0:350
0:374
Mixed
LLaMA2-13B
Randam 10K
DENTA
0.61b
0.837
0.852
9.488
Musrall7B
DETTA OK
0.578
0.803
0.019
0.595
SaperGLUR
OBQA
Beach
Random
OpenOrra
Mistral-7B
Unitorm 5B
0.030
0.085
0.783
0.750
Cluster Shao ml.
2024
Rosdom 205 tkn
ClasterCippp Pp
MAPII
0.096
0.G
GSMAR
0.250
0.08Y
MML0
0.814
0.488
0.418
0,500
Preaf-Pie-2
LLaMA2-7B
Unilora 201 Oka
0.076
0.200
0.000
0.425
ClasterCi:pp
0.079
0.204
0:511
0:428
MMLU
BBH
ARC
UlrraChat
Hardom TOK
QDVF 10K
0.321
0.391
0.321
0.007
0.583
LMSYS
Ratelo T0K
QDDT 10K
0.351
0.373
0.325
0.320
0.014
0.002
Alpaca
Nandom
QDIT
0:356
0:383
0 320
0:817
Mixod
Raralom
Nukchario Ahao
QDIT
2023
Chara
Delly
LLaMA-7B
Randam 10K
QDIT 10K
QDIT IK
DROP
0.282
0.335
0.287
LAMBADA
0.898
0.898
0,309
0.028
Setg
0.834
8.868
LMBYS
Rasdam 10K
QU1l 10K
0.251
0-201
0.085
0.053
0 R500
Alpaca
Handom MK
QD1T SK
0.270
0-20
0.00Y
0.Y10
0 0:870
8.841
Mixed
Ratalou TOK
0,208
0.661
0.841
QDDT T0K
0.200
0.007
0:3558
Dolly
Randam iK
QDIT
0.173
0:226
0.723
0.717
0.808
0.807
DROP
MML.1
Haman
0.320
0.410
0.100
Alpaco
LLaMA-7B
0.353
0.002
data with diversity criteria is superior. In addition, compared to only selecting high-quality data, the criteria that combine quality and diversity can achieve better performance than simply selecting high-quality data.
Importance It is non-trivial to accurately identify and utilize datapoints that significantly affect model performance As shown in Tab. A the importance-based data selection approaches often maximize the final performance by integrating the data selection and model optimization processes. To address the challenges of potential huge computation overhead brought by the implementation framework, they propose to use instance-wise data impact and propose efficient parameterization. Moreover, by performing importance resampling in the feature space that depicts geometric structures, they select examples of both high importance and similarity to the larget distribution, thereby enhancing the performance on the target, tasks. Existing work has confirmed that the importance sampling- sampling-based based approach can effectively improve the performance on the target tasks with enhanced capabilities of LLMs.
Hybrid Selection It is noteworthy that many data selection methods such as LIFT Xu et al. (2023b) ]. FL Bhatt et al. [,024)] DEITA Liu et al. (2023b)] ], and QDIT Bukharin & Zhao (2023) attempted to combine multiple aspects of data assessment into their selection pipelines. Most of them emphasize the overall assessment of instruction data in that the definitions of 'good data' are varied under different scenarios in the era of LLMs. A unitary measurement would inevitably cause a biased selection of datapoints and thereafter leads to the degraded performance of LLMs. Technically, existing bybri selection methods can be categorized into: 1) parallel setups, and 2) sequential setups. For the former, an operation of maximization or weighted sum is performed on scores or proxy indicators (i.e., losses and gradients) from multiple aspects (e.g., quality and diversity) Xu et al. (2023b): Bhatt et al. (2024); Bukharin & Zhao (2023)). For the latter, a sequential setup of quality-, diversity- or inportance-oriented selection techniques is performed step-by-step for hicrarchical, stratified filtering Liu et al. (2023b)] Comparatively, the parallel sctups allow dynamic trade-offs bet ween multiple data aspects by adjusting the aggregation operations. The sequent ial setups, on the other hand, fail to retrieve the candidates that are filtered out in the preceding quality control steps even if those candidates are of high importance or varioty. Therefore, it would be preferred to develop hybrid assessment techniques that simultancously weigh quality, diversity, and importance. Furthermore, we notice that the importance-based assessment is often overlooked in existing hybrid approaches, implying
31

---

Published in Transactions on Machine Learning Research (12/2024)
that the investigation of integrating importance with quality and diversity is of high potentials for future studies Zhuang et. al. 22024) Lv et. al. (2024))
Distinctions and Connections the similar philosophy of data assessment and selection but differ in the detailed implementations.
The methods that are respectively reported in Tabs. 2
3 and 4 share
For the quality-basef selection, most of existing methods like IFD Li et al. (2023 PPL Ankner et al. (2024)], AutoDS Zhhang et al. (2024c)), and QuRator [Wettig et al. (2024). mainly use the model-based indicators to measure the quality of cach datapoint. However, their indicators are derived from different perspectives such as the perplexity of LLMs and the predicted quality score of a regression model. Both the Alpagasus Chen et al. (2023b) and BSDetector [Chen & Mueller (2024) mainly employ ChatGPT scoring for quality evaluation. Alpagasus directly prompts GPT to score and filter out datapoints, but BSDetector considers both the consistency of the generated responses from GPT and the quality of the ground-truth response. On the other hand, LIFT [uu et al. (2023b)], Instruct ionMining Cao et al. (2023) [, and FL. [Bhatt] et al. [2024)] develop composite methods that involve hand-crafted or model-based indicators, together with GPT scoring, for quality verification of the selected datapoints.
For the diversity-based selection, both DEITA Liu et al. (2023b)) and ClusterClip Shao et al. (024g) adopt the geometry-based measurement where the geometric structure of datapoints is maintained via clustering. DEITA measures the individual-level relationship between one datapoint and its closest. neighbor for sampling while ClusterClip simply performs the cluster-level uniform sampling with the quantity limit. QDIT [ukharii & Zhao [2023) performs the iterative selection where each step only one most promising datapoint from the remaining dataset is selected to maximize the overall diversity of the selected subset. Despite the fact that DQ Zhou et al. (2023) also exploits iterative coreset sampling, it first divides the entire dataset into multiple subset bins and performs sampling across bins for aggregation and deduplication.
For the importance-based selection, DsDm Engstrom et al. (2024) and MATES Yu et al. (2024) employ model-based indicators to estimate the influence with datamodels. However, these two methods choose different implementations of datamodels. DsDm develops linear model and approximates the loss of each datapoint via the TARK estimator while MATES directly leverages a small language model to Iearn to predict the individual influence of each datapoint. DSIR Xie et. al. (2023) on the contrary, bypasses the need to maintain a specific datamodel. Instead, estimates the importance of each datapoint by comparing the distributions of the entire selected subset with those of the validation set. Both Skill-it Chen et al. (2024b) and LESS Xia al. (2024a) pinpoint the samples that resemble the most to the target set. Skill-it performs sampling via minimizing the evaluation loss while LESS utilizes the gradient similarity a proxy.
7 Future Directions: Challenges and Opportunities
In this section, we present the existing challenges and potential solutions to developing advanced data assessment and selection methods.
7.1 Benchmarking Instruction-Tunar LLMs
There exists a gap between the effectiveness of data selection and the reported performance on benchmarks. In existing researches, the ablation studies on the effectiveness of assessment and selection methods are often carried out by comparing the performance of LLMs fine-tuned with the selected and the full dataset. However, for coreset sampling methods that use losses and gradients as proxies for data quality, the downstream performance may not be positively correlated with the selection effectiveness. The reason behind is that the evaluation loss itself Yang et al. (2022); ; Hoffmann et al. (2022); Kaplan et al. [2020)] is not informative enough for universal estimation of benchmark performance. A (2024) demonstrates that the correlation between the negalive log-likelihood loss on downsttream lasks and the accuracy metrics should be modeled task- by-tsk and impractical to simply count on losses or gradients to pinpoint the most beneficial data for improving the downstream performance, let alone methods that try to predict the loss based on various indicators CCo et al. 2023) Furthermore, even if the metrics exhaustively computed for the selection of each sample,
In the light of this statement, is
32

---

Published in Transactions on Machine Learning Research (12/2024)
Table 4: Experimental results of importance based selection methods are directly cited from their papers.
Mathode
Training Set
Model
Ratla/Size
Reported Recalts on Teeting Set
COPA
OBQA
PIQA
CBT
Hellnwwog
Handam
0.827
0.334
0.889
B.964
Eugotrom of
DaDa 2624
Chinckilla- opsimel-1.95
Randam
DaDm
Winggrande
0.830
BeeIQ
0.312
COQA
0.890
ARC-B
0.882
Trivia
0:423
BeiQ
ARC-E
ARC-C
LociQ
Py1bis-410M
MATES 80% MATRS
20%
860
MATES Ya 2024
Pythis-1B
ONIA
075
035
Hellawwang
0 0.200
0,209
Winogrand
Pythis-410M
Randon 20%
MATES 20%
0.2
303
0.000
0.580
0. 0.307
0.410
0.087
0.071
0.327
0.30/
Pythia-11
Handom 20%
MATES 20%
0.322
0:318
0.609
0.802
0:455
0:338
0.889
0.895
0.324
0.307
MNL
QNL.1
QQP
RTE
Random 61,2M BRIR $1 2M
0.824
0.589
0.896
0.874
18518 Xie et al
2023
The Pile
Base (125M)
RoBERTs
Random $1:2M
8212
0.901
MRPC
0.591
0.874
COLA
0,508
0.384
0.761
0.888
D8IR 2M
905
0,877
0.540
0.892
ARC-C
ARC-E
Booly
Bkill-is
0.012
0.882
Uniforan
0.002
0.080
Uniffrmm
SRIlCG 1B
0.020
0.087
Sletll-te Chen al 20228
Red Pajamaa
GPT-Nee-331
Uniform
Hall Tawnn
0.350
0.005
0.072
Skill-is 1B
0377
0.870
0. 0.780
0.039
Unilora 1B
0.033
0.044
0. 1.5
0.022
Bklll- 1B
0.33
0.007
0. 782
0.0:3
Uniform 1B
0.858
0.659
0. 0.785
0.859
Skill-it 1B
6.9
0.680
0.757
0.851
Uniformn TN
040
0.008
0.700
0.033
MXLU
TYDIQA
BBB
Full
0.0
0.540
0. 0.432
LLaMA2-7B
Random 8%
03365
0:527
0.389
LESS
302
0.502
0.315
LEsS
1aMA2-138
Random
0.577
0.580
Mistral-BB
5%
0.645
LESS 5%
0.888
0.803
0.580
the gains brought by one sample might be limited in few tasks. Therefore, comprehensively reflect the effectiveness of sample selection, the evaluation of instruction-tuned models should be accompanied by the specialised evaluation of the selected datapoints. For the former, all sorts of evaluation strategies have been proposed to prociscly evaluate the LLMs Melis et al.] (2017]: Chang et al. [2024]; Xu et al. (2022) Liang et al. model truly understands the problem rather than simply memorizing the answer choices given the instruction context. For the later, a benchmark for documenting and comparing the statistics of the selected instruction-respouse pairs in terms of quality, diversity, and importance needs to be constructed in the future. It would benefit the task-wise customized data selection according to the statistical indicators on such benchmark.
[2022]] ]. The multiple-choice QA lasks are not enlightening in judging if the instruction
Test set contamination should be considered during instruction data selection. For instruction tuning on publicly released pre-trained L.MMs, it cannot, be too careful to check the potential data leakage
where the testing instructions are already modeled during pre-training Rae et al. 2021 0: Li et al. 2023b);
Magar& & Schwartz (2022); Carlini et al. (2019) Marone & Van Durme (0024) ; Deng et al. 2023 Cao et al.
(2024): Jiang et al. 2024b); Magar & Schwartz [2022)] ]. To improve the performance of pre- are-trained models
on downstream tasks, datapoints (i.e., instruction-ikke conversations) are already added into the anncaling
phase of pre-training [AI@Meta| (2024); Bilibili (2024); Yang et al. (2024); Bai al. (2023)) Therefore, potential risks of data contamination are raised for benchmarking the fine-tuned LLMs. To avoid the negative effect of data leakage on evaluation of the data assessment and selection, it is encouraged to follow Li et al. [2023a)] to adopt the pre-trained model for experiencing the datapoints before fine-tuning. If the model exhibits overfilting behaviors (i.e., accuralely generating the instruction part or producing the same answer choice even with permutation on the choice letters), data contamination is likely to exist and thereafter the testing set should be replaced. For future studics, would be more reliable to decouple the evaluation of data selection and that of fine-tuned LLMs, where the performance consistency between these two evaluation results can be analyzed to rule out the possibility of contamination.
33

---

Published in Transactions on Machine Learning Research (12/2024)
7.2 Unveiling the Definitions of Good Data
What signifies the most a good datapoint remains an open question. Unfortunately, there exists unified criteria discriminating 'good' instructions from "bad' ones. Essentially, the definitions on the general data 'quality' differ from task to task and domain to domain Evans & Murshudov (2013); Flach (2012); Albalak et al. (2224)] Although existing quality measurement methods can be categorized in terms of quality, diversity, and importance under the present study, they all exhibit more or less ad-hoc properties in methodology. First, studics on instruction tuning are often targeted at improving the performance of LLMs on downstream lask. No maller whether these lasks are of general-purpose (e.g., common NLP lasks on leaderboard [Myraakhan et al. (2024); Wolf et al. (2019])) or domain-specifie applications, such task-orientated data selection itself is only "proxy' for exploring the underlying "quality" mcasurement. Especially for coreset sampling methods that directly employ the evaluation set or testing set for distribution matching importance estimation, instructions that resemble the most to the testing set or bring about performance gains are judged as 'good' data. However, such 'good" data cannot be easily transferred to another LLM of completely different architecture and parameters. Each time the entire pipeline has to be enforced for a novel task, making it difficult to accumulate baweerralllyacckoowleeeeee
high-quality data for archiving. Second, each method has an individual quality ovaluation system and very few of them ever tried to justify their design and interpret the philosophy behind. It is difficult to validate whether certain component of the selection pipeline can be replaced removed for better serving a new task-of-interrst
Accordingly, further academic explorations include: 1) to present a more unified, generally applicable definitions on "good" datapoints in terms of fine-grained aspects, and 2) to improve interpretability and explanability of the selection pipeline beyond empirical design.
The expected model behavior retrospectively determines the trade-off between quality, diversity, and importance for data selection. The three aspects we used to calegorize data assessment methods are actually overlapping with each other, where the 'boundary' between two measuring dimension is often hard to explicitly defined. Under such circumstance, the definition of good data can be perceived as the weighted, biased mixture of quality, diversity, and importance. Existing methods are not flexible in dynamically adjusting the mixing weights to adapt to different downstream tasks. Instead, their priority order of the three dimensions is implicitly encoded into the selection of instructions. For instance, [iu et al. (2023b)) emphasizes quality and importance equally by first establishing the relative ranking of all samples in both quality and complexity. The subset is formed by consecutively selecting the top-ranked samples in sequence, with diversity intervened via ruling out heavily homogenized examples. Such hard-coded, greedy treatment to quality, diversity, and importance is not applicable to scenarios where the behavior of LLMs is expected to cater to varied preference.
In general, the data assessment and selection methods that can adapt to the model requirement under different application scenarios are yet to be systematically developed. For generation tasks like role playing and creative writing, the preferred instruction tuning datapoints should be distinct from those for discriminative tasks like named entity recognition and sentiment analysis.
7.3 Scaling Up Datasets
The optimal scale of the selected subset becomes less explicit with the expansion of datasets. In the analysis of the disadvantages exploiting the entire instruction dataset for alignment, putting aside the issue of long training time, we notice that the performance of fine-tuning the entire dataset might not be the optimal. There often exists critical point of the best selection proportion, and such proportion varies from dataset to dataset. When more instruction datasets from diverse domains and tasks are incorporated, it becomes more difficult to nail down the best solection proportion for three main reasons. First, a large proportion of noise exists in the open datasets and few noisy samples can already cause tremendous negalive impact on performance Song et al. [2022]] During the pre-processing of instruction dataset, noise can be unintentionally introduced instruction preparation (c.g., missing context or system prompt), response generation (e.g., unverified or mismatched answers), format wrapping (e.g., invalid JSON and unresolved code), and text augmentation (e.g., synonym replacement and reorder of words). Second, for specialized tasks in
34

---

Published in Transactions on Machine Learning Research (12/2024)
'vertical' domains, the overfitting of specific prompts occurs Ma et al. (2023) when the diversity of input instructions is rather limited. Despite the accuracy and rationality of the instruction--esponse pairs, L.Mss tend to overfit certain patterns of the input instruction rather than truly comprehend the task. Therefore, the increase of dataset size instead reduccs the generalization of trained LLMs with lower instruction following capabilities. Third, the lorgetting /Zhang & Wu (2024); Jin & Ren 2024b): Waang el al. [0044b] becomnes a severe problem when more instruction datasets are introduced without setting a proper re-playing schedule
of pre-training or previously visited instruction tuning datapoints [Parmar et al. (2024) ; Jin et al. (2021);
Ibrahim et al. (2024)). The skill cultivation of a LLM on any new instruction task heavily relies on its preceding skills acquired during pre-training or previous fine-tuning. Consequently, the expansion of samples for high-level skills would 'dilute' those for low-level skills and degrade performance.
To sum up, with the dataset scaling up, fine-tuning with the selected subset instead of the entire set becomes a must-have strategy. To help determine the optimal selection ratio, we suggest the following three guidelines: 1) One may first develop a complex quality mcasure scheme that uses both indicators and human verification estimate the noise percenlage of each constituting dataset. Without, lose of generality, random sampling [ia] et al. [024bb] can be performed to accelerate quality measurement. To combat noise, lower ratio (i.e., smaller Sb) should be considered for data selection from the noisy S. 2) To combat overfitting, both the diversity of datapoints within and across datasets should be emphasized. A higher keeping proportion should be established for datasets with diverse instruction styles, prompt formations, and response patterns, which helps improve the model's instruction following capabilities. 3) For continual fine-tuning, datasets that share similar distributions with pre-training and previous fine-tuning datapoints should be kept to fight against forgetting. The optimal selection ratio and proportion for each dataset is built upon the meticulous and thorough analysis each instruction datasct, and therefore case-by-casc adjustment is needed. For future studies, one may investigate the automation of assessment and selection recipe to minimize the human intervention.
The optimization of a scalable pipeline for data assessment and selection is of urgent need. In consideration of the cost of building human-labeled and human-verified instruction tuning datasets, methods that employ powerful LLMs like GPT4 for instruction synthesis Bradley et al. (2023); Li et al. (2023(); Xu
ct al. (2023a): Li et al. (024a)); Zhao et al. (022a)): Dong et al. (2024) have gained increasing attention. The hetic instructions proliferate cost-effectivvel with fine-grained control of characteristies such as difficulty and style. Therefore, it is expected to witness surge of datapoints (e.g., tens or even hundreds millions) in the short future. In that case, datasets of such quantity pose a significant challenge to the robustness, efficiency, and precision of the selection hods Previous studies like DSIM Xie el al. (2023) demonstrated that cheap approximation of features by bag-of-n-grams achieves similar performance but requires much less computing resources. For future research, one may draw inspiration from the data deduplication and filtering techniques in handling billions of pre-training tokens. Especially for the measurement of diversity, the computing of embeddingg pairwise similarity and clustering can be greatly reduced with simplified representations. In addition, the hierarchical philosophy [mida et al. (2016); Talavera, (1999); Cabezas et al. (2023): Ran et al. [2023)] might be a promising approach to select data from coarse-grained to fine-grained structures. One may apply the devide-and-conquee strategy to recursively handle each subset of the instruction dataset, limiting the peak resource consumption under budget.
7.4 Scaling Up LLMs
The cost-efficiency of data assessment and selection diminishes with larger LLMs involved in the pipeline. The model-based indicators and coreset sampling methods often require the language model itself to be involved for computation of metrics Li et al. (2023a]], losses Chen et all (2024b)), and gradients [Xia] et al. [2024a]]. With the increase of model sizes, it becomes more and more cumbersome to implement the entire pipeline for quality measurement and selection. To expedite the process, one important direction for future study is to develop proper efficient proxy models. Small proxy models have been successfully applied in accelerated fine-tuning of language models [Hoffmann et al. (2022): Liu et al.) (2024a)]. filtering datasets by perplexity [Ankner et al. (2024)), intervention on aangevval- generation Tan et al. (2024b)), and performance prediction Amugraha et al. (2024); Ngu et al. (2024]] Such proxy models often share the
same architecture design with the LLMs under development but own much less parameters. The scaling
35

---

Published in Transactions on Machine Learning Research (12/2024)
law Kaplan et al. (2200) confirms the expected consistent behavior between data quantity and model scale, providing practical guidelines on the development of such proxy L.MSs
On the other hand, under the context of data evaluation, it calls upon on rethinking of traditional machine learning techniques such as efficient optimization tricks and dimensionality reduction approaches. For example, in the assessment of loss-based datapoint influence Feldman & Zhang (2020)), the exhaustive measurement on the marginal performance by by iterative batch-wise sampling tricks with greedy principle behind. For efficient assessment, PCA Xu
and model re-training can be simply approximated
et al. (023b) and random projection Xia et al. (2024a): Park et al. [2023] are popular choices for obtaining low-rank representations of embeddings and gradients, which facilitates not only metric computation but also
storing of datapoints.
The marginal benefits of instruction tuning diminishes with increasing size of LLMs for knowledge supplement. Recent studics on the effectiveness of instruction tuning in injecting tpsk-specific or Nomain-specctfe knowledge into LLMs [Shi & Lipani 2023); Goyal et al. (2023); Zhang et al. (2024b);
Yildzz et al. [2024) show that the stand-alone instruction tuning might not be the most appropriate method.
Compared with strategies like continual pre-training Cosss et al. 2024); Ke (2024); Cossu et al. (2024)) and instruction modeling [ou et al. (2024): Cheng et al. (2024): Wang al. (2022): Xu et al. (2024): Shi et al. (2024)), , instruction tuning counts the response sequences for loss computation without sufficient, perception of instruction context. For specialized domains like medicine, finance, and laws, if the pre-trained LLMs are in lack of the prerequisite kuowledge, the instruction luning cannot properly activales the parameterized 'memory' for aligument but only causes overfitting of the given prompt. In that case, the benefits of data selection are limited with poor generalizability.
Another noteworthy phenomenon in data assessment and selection studies is that due to limited budgets of computing resources, most of the experiments are performed on LLMs of small and moderate size (e.g., less than 7B) to validate the effectiveness of the quality mcasurement and the selection strategy. Small pre-trained LLMs, by their nature of small parameter size, are more sensitive to the instruction datasets during fine-tuning or continual learning [chiick & Schütze (2020); Yildız et al. (2024)). They exhibit the most significant rates of both forggeting (old knowledge) and learning (new knowledge). In the light of such statement, small LLMs tend to sacrifice the task-irrelovant knowledge in return for rapid adaptation towards novel domains and tasks. The selected datasets by various quality measures can impose immediate effect on the parameters of small LLMs,, but may weaken on those of huge ones. It remains unknown whether the same quality measurement and data selection pipeline can achieve similar performance gains on both small and large LLMs. For future research of data assessment and selection, extensive experiments are required to validate their efficiency on huge LLMs (e.g., 70B and 405B) AT@Meta (2024) and LLMs of
mixture-of-cxperts (MoE) architectures (c.g., Mixtral 8x22B) Jiang et al. 2024a)]
In consideration of the pre-training corpus, extremely large LLMs already experienced a vast amount of multi- lintuiilinua,, multi-domain web texts during pre-training, and therefore the priority of the dimensions in data assessment (i.c., quality, diversity, and importance) differs from small LLMs. The association between the model scale and the data selection criteria is yet to be studied.
7.5 Validating the Bias and Fairness of Instruction-tuned LLMs
As mentioned in the Sec. 1.3 most existing data selection methods fail to specifically study the dataset bias for consideration of fairness. It is noted that for domain-specifie tasks such as solving programming and maths problems, the data bias might not be that severe. However, for general grestion-answeriin
recommendation, and creative writing tasks, even ChatGPT could produce biased answers that are intolerable in domains like education Doan et al. (2024): Zhang et al. (2023b); Chisca et al. (2024): Gao et al. (2024)). To improve the fairness of LLMs, it is necessary to take the measurement. of data bias into serious consideration. Future work includes: 1) the integration of embedding based (e.g., WEAT Caliskan et al. (2017)), SEAT May et al. (2019)), probability- basod (c.g., DisCo Webster et al. (2020)) CrowS-Pairs Score Nangia et al. (2020))), and generated texts- based (e.g., Co-Occurrnnce Bias Score Bordia & Bowman [2019)] Full Gen Bias Smith et al. (202 techniques for fairness evaluation of datapoints Gaalegos et al. (0024) and 2) the construction of
36

---

Published in Transactions on Machine Learning Research (12/2024)
bias and fairness benchmarks Jiao et al. (2024) under various domains and tasks supplementary kits to the current evaluation pipeline that only focuses on testing the capability of LLMSs
8 Conclusion
In this study, we have thoroughly examined the state-of-the-art data assessment and selection methods for instruction tuning of LL The present review presents a unified organization and categorizes these methods in terms of measuring dimensionality: quality, diversity, and importance. In each dimensionality, we outline the representative strategies in details and describe the factors consider when selecting data for instruction tuning. Furthermore, we report the performance of typical data selection methods and provide discussions the comparison between these methods. Last but not least, the existing challenges and potential solutions for future studies are summarized in hope for benefitting the rescarch community.
References
Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXim:2303.09540 2023.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint Stl:2333.08774,, 2023.
Alossandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning, In Proceedings of the IEFE/CVF international conference on computer vision, pp. 6430 1430-6439, 2019.
Sandesh Adhikary and Byron Boots. Sampling over riemannian manifolds using kernel herding. 2022 International Conference on Robotics and Aulomation (ICRA), pp. 3646 3653. IEEE, 2022.
Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization in linear time. stat, 1050:15, 2016.
Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. Contextual diversity for active learning, In Computer Aision-ECCV 2020: 16th European Conference, Glasgow, UK, August 28-28, 2020, Proceedings, Part XVI 16, pp. 137 153. Springer, 2020.
AetMeta. Llama 3 model card. 2024.
CARD. md)
Ahmed Alaa and Mihaela Van Der Schaar. Diseriminative jackknife: Quantifying uncertainty in deep learning via higher-order influcnce functions. In International Conference on Machine Learning, pp. 165-174. PMLR, 2020.
Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Mucnnighoff, Bairu Hou, Liangming Pan, Haewon Jcong, et al. A survey on data selection for language models. Xi preprint 2r 16827, 2024.
Alexandre Alcoforado, Thomas Palmeira Ferraz, Lucas Hideki Okamura, Israel Campos Fama, Arnold Moya Lavado, Bárbara Dias Bueno, Bruno Veloso, and Anna Helena Reali Costa. From random to informed data selection: A diversity- based approach to optimize human annotation and few-shot learning. arXiv preprint 1rX.v:2401.13929, 2024.
Zachary Ankner, Cody Blakeney, Kartik Sreenivasan, Max Marion, Matthew L Leavitt, and Manshooj Paul. Perplexed by perplexity: Perplexity- data pruning with small reference models. arxiv preprint 1r5iv:2405.20541, 2024.
37

---

Published in Transactions on Machine Learning Research (12/2024)
Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs memorization: Tracing language models' capabilities back to pretraining data. arXiv preprint arXim:2407.14985 2024.
David Anugraha, Genta Indra Winata, Chenyue Li, Patrick Amadeus Irawan, and En-Shiun Annie Lee. Proxylm: Predicting language model performance on multilingual tasks via proxy models. arXiv preprint arXiv:2406.0993 09334, 2024.
Walter E. Arnoldi. The principle of minimized iterations in the solution of the matrix cigenvalue problem. Quarterly of Applied Malhemalics, 9:17-29 1951. URL https://api.seennticscchooooo o
115852469
Kyriakos Axiotis, Vincent Cohen-Addad, Monika Henzinger, Sammy Jerome, Vahab Mirrokni, David Saulpic, David Woodruff, and Michael Wunder. Data-efficient learning via clustering- based sensitivity sampling: Foundation models and beyond. arXiv preprint arXiv:2402. 1 17327, 2024.
Francis Bach and Michael Jordan. Learning spectral clustering. Advances in neural information processing systemss 16, 2003.
Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger Grosse. Training data attribution via approximate unrolled differentation. arXio preprint 2xXi:2205.121866 2024.
Prafulla Bafua, Dhanya Pramod, and Anagha Vaidya. Document clustering: Tf-idf approach. 2016 International Conference on Electrical, Electronécs, and Optimization Techniques (ICEEOT), pp. 61-66. TERE, 2016.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 11129999.66000 16609, 2023.
Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In Proceedings of the 23rd international conference on Machine learning, pp. 65-72, 2006.
Lukas Balles, Giovanni Zappella, and Cédric Archambeau. Gradient Gradient-matching coresets for rehearsal baased continual learning. arXiv preprint arXi::2203. 2022.
Sayan Bandyapadhyay and Kasturi Varadarajan. On variants of k-means clustering. arXiv preprint 1ffi:1512.0005 2015.
Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. ai preprint arXiv:2006.14651, 2020.
Carlo Batini, Cinzia Cappiello, Chiara Francalanci, and Andrea Maurino. Methodologies for data quality assessment and improvement. ACM computing surveys (CSUR), 41(3):1 52, 2009.
John C Begeny and Diana J Greene. Can readability formulas be used to successfully gauge difficulty of reading materials? Psychology in the Schools, 51()):198-211 115, 2014.
Yonatan Belinkov and James Glass. Analysis mothods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49 72, 2019.
Yves Bestgen. Mcasuring lexical diversity in texts: The twofold length problem. Language Learning, 2023.
Gantavya Bhatt, Yifang Chen, Arnav M Das, Jifan Zhang, Sang T Truong, Stephen Mussmann, Yinglun Zhu, Jeffrey Bilmes, Simon S Du, Kevin Jamieson, et al. An experimental design framework for label-efficient supervised finetuning of large language models. arXiv preprint arXic:2401.0668) 2024.
Stella Biderman, Usven Prashanth, Lintang Sutawika, Hailey Schoclkopf, Quentin Anthony, Shivanshu Purohit, and Edward Raff. Emergent and predictable memorization in large language models. Advances in Neural Information Processing Systems, 36, 2024.
Bilibili. Index1.9h 1.9b technical report. 2024. URL httpsss//gahhbbooo/////b///////
38

---

Published in Transactions on Machine Learning Research (12/2024)
A Bjork. Least squares methods: Handbook of numerical analysis. Finâte Difference Methods Solutions of Equalions in Rn, 1, 1988.
David M Blei, Andrew Y Ng. and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, ((Jan):993-102 2003.
Shikha Bordia and Samuel R Bowman. Identifying and reducing gender bias in word-level language models. Xiv preprinl arXi::1904.0300 2019.
Zalán Borsos, Mojmir Mutny, and Andreas Krause. Corescts via bilevel optimization for continual learning and streaming, Advances in newral information processing systems, 33:14879 -47990, 2020.
Zalán Borsos, Mojmír Mutný; Marco Tagliasacchi, and Andreas Krause. Data summarization via bilevel optimization. Journal of Machine Learning Research, 25(73):1 13, 2024.
Maximilian Böther, Viktor Gsteiger, Ties Robrock, and Ana Klimovic. Modyn: A platform for model training on dynamic datascts with sample-level data selection. arXiv preprint wrXi::231.006254 2023.
Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Grégory Schott, and Joel Lehman. Quality-diversity through ai feedback. arXiv preprint arXiv:2310. 0200310.110000 13032, 2023.
Jonathan Brophy, Zayd Hammoudeh, and Daniel Lowd. Adapting and evaluating inffnence-estimation methods for gradient- gradient-boosted decision trees. Journal of Machine Learning Research, 24(154):1 48, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. A Advances in neural information processing systems, 33:1877 2020.
Quang Vu Bui, Karim Sayadi, Soufian Ben Amor, and Marc Bui. Combining latent dirichlet allocation and k-means for documents clustering: effect of probabilistic based distance measures. In Intelligent Information and Database Systems: 9th Asian Conference, ACIDS 2017, Kanazawa, Japan, April 9-5, 2017, Proceedings, Parl 1 9, pp. 248 257. Springer, 2017.
Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning. arXiv preprint WxXi:2311.147980 2023.
John Byabazaire, Gregory O'Hare, and Declan Delaney. Data quality and trust: A perception from shared data in iot. In 2020 IEEE International Conference on Commonications Workshops (ICC Workshops), pp- 1-6. IEEE, 2020.
Luben MC Cabezas, Rafael Izbicki, and Rafael B Stern. Hierarchical clustering: Visualization, feature importance and model selection. Applied Soft Computing, 141:110303, 2023.
Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334))1833-866 2017.
Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander. Density-based clustering based on hicrarchical density estimates. In Pacific-Asia conference on knowledge discovery and data mining, pp. 160 172. Springer, 2013.
Jialun Cao, Wuqi Zhang, and Shing- Chi Cheung, Concerned with data contamination? assessing countermea- sures in code language model. arXiv preprint arxiv:2403. 16898, 2024.
Kris Cao and Stephen Clark. Latent variable dialogue models and their diversity. ar preprint arv: 1rXiv:1702.05000 702.05962, 2017.
Yihan Cao, Yanbin Kang, and Lichao Sun. Instruction mining: High-quality instruction data selection for large language models. Xio preprint arXin:3307.02290 2023.
39

---

Published in Transactions on Machine Learning Research (12/2024)
Nicholas Carlini, Chang Liu, Ülfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX securily symposium (USENIX security 19), pp. 267 284, 2019.
Patricia L Carrell. Readability in esl. 1987.
Jeanne Sternlicht Chall and Edgar Dale. Readability revisited: The new dale-chall readability formula. Edgar Brookline Books, 1995.
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate. arxiv preprint ar Xix:2230800080000 2023.
KHR Chan, Y Yu, C You, H Qi, J Wright, and YR Ma. white-box deep network from the principle of maximizing rate reduction. arxiv. 2021. arXiv preprint 2rXi::2105.10444 2021.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1 45, 2024.
Daoyuan Chen, Yilun Huang, Zhijian Ma, Hesen Chen, Xuchen Pan, Ce Ge, Dawei Gao, Yuexiang Xie, Zhaoyang Liu, Jinyang Gao, et al. Data-juicer: A one-stop data processing system for large language models. In Companion of the 2024 International Conference on Monagement of Data, pp. 120 134, 2024a.
Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuctao Ma, Yifan Yanggong, and Junbo Zhao. Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning. arXiv preprint arxi2:2305. 09246, 2023a.
Jiuhai Chen and Jonas Mueller. Quantifying uncertainty in answers from any language model and enhancing their trustworthiness. 2023.
Jiuhai Chen and Jonas Mueller. Automated data curation for robust language model fine-tuning. arXiv preprinl Xiv:22403 12776, 2024.
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.0877 2023b.
Mayee Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. Skill-it! data-driven skills framework for understanding and training language models. Advances in Neural Information Processing Systems, 36, 2024b.
Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. 0rXiv:1208.3477 2012.
arXiv preprint
Yutian Chen, Luke Bornn, Nando De Freitas, Mareija Eskelin, Jing Fang, and Max Welling. Herded gibbs sampling. The Journal of Machine Learning Research, 17(1):263 291, 2016.
Daixuan Cheng, Yuxian Gu, Shaohan Huang, Junyn Bi, Minlie Huang, and Furu Wei. Instruction pre-training: Language models are supervised multitask learners. arXiv preprint 2xXiu:2/06.14499 2024.
Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. what data benefits my classifier?' enhancing model performance and interpretabilityy through influence-based data selection. The Twelfth International Conference on Learning Representations, 2024.
Andrei-Victor Chisca, Andrei-Cristian Rad, and Camelia Lemnaru. Prompting fairness: Learning prompts for debiasing large language models. In Proceedings of the Fourth Workshop on Language Technology for Equalily, Diversity, Inclusion, pp. 52 62, 2024.
40

---

Published in Transactions on Machine Learning Research (12/2024)
Zhibo Chu, Zichong Wang, and Wenbin Zhang. Fairness in large language models: A taxonomic survey. ACM SIGKDD exploralions newsletter, 26(1):34-48, 2024.
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, and Sanjiv Kumar. Batch active learning scale. Advances in Neural Information Processing Systems, 34: 11933 11944, 2021.
Kevyn Collins-Thomper and Jamie Callan. Predicting reading difficulty with statistical language models. Journal of the omerican society for information science and technology, 56(13):1448 1462, 2005.
Benoît Colson, Patrice Marcotte, and Gilles Savard. An overview of bilevel optimization. Annals of operations research, 153:235 256, 2007.
Bradford R Connatser. Last rites for readability formulas in technical communication. Journal of technical wriling and communication 29(3):271 187, 1999.
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tuned Ilm. Company Blog of Databricks, 2023.
William J Cook, William H Cunningham, William R Pulleyblank, and Alexander Schrijver. Combinatorial optimization. Unpublished manuscript, 10:75 93, 1994.
Gérard Cormuéjols, George Nemhauser, and Laurence Wolsey. The uncapicitated facility location problem. Technical report, Cornell University Operations Research and Industrial Engineering, 1983.
Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuytelaars, and Davide Bacciu. Continual pre-training mitigates forgetting in language and vision. Neural Networks, pp. 106492, 2024.
Ian Covert, Wenlong Ji, Tatsunori Hashimoto, and James Zou. Scaling laws for the value of individual data points in machine learning. arXiv preprint 2tii:2405204550 2024.
M Covington and Joe D McFall. The moving-averagge type-token ratio. Linguistics Society of America, Chicago, IL, 2008.
Michael A Covington and Joe D McFall. Cutting the gordian knot: The noving-average type-token ratio
(mattr). Journal of quantitative linguisties, 17(2):94-100) 100, 2010.
Radu Cretulescu, Daniel I Morariu, Macarie Breazu, and Daniel Volovici. Dbscan algorithm for document clustering. International Journal of Advanced Statistics and ITEC for Economics and Life Sciences, 9(1): 58-66, 2019.
Edgar Dale and Jeanne S Chall. The concept of readability. Elementary English, 26(1):19 26, 1949.
Dan Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. Thunsactions machine learning research, 2023.
Vu Minh Hoang Dang and Rakesh M Verma. Data quality nlp: Metrics and comprehensive taxonomy. In International Symposium on Intelligent Data Analysis, pp. 217 229. Springer, 2024.
Fredrik deBoer. Evaluating the comparability of two measures of lexical diversity. System, 47:139-145, 2014.
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. Investigating data contami- nation in modern benchmarks for large language models. arXiv preprint arXiv:2311.09788, 2023.
Dingsheng Deng. Dbscan clustering algorithm based on density. In 2020 7th international forum electrical engineering and automation (IFEEA), pp. 949-953. IEEE, 2020.
Mingkai Deng, Bowen Tan, Zhengzhong Liu, Eric Xing, and Zhiting Hu. Compression, transduction, and creation: unified franework for evaluating natural language generation. arXiv preprint XrX.v.2109.06379, 2021.
41

---

Published in Transactions on Machine Learning Research (12/2024)
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. orXiv preprinl or arXiv: X8o::1800004805, 2018.
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprinl ar Xiv:2305. 14233, 2023.
Marcel Dix, Gianluca Manca, Kenneth Chigozic Okafor, Reuben Borrison, Konstantin Kirchheim, Divyasheel Sharma, Kr Chandríka, Deepti Maduskar, and Frank Ortmeier. Measuring the robustness of ml models against data quality issues in industrial time series data. In 2023 IEEE 21st International Conference Industrial Informalies (INDIN), pp. 1 8. IEEE, 2023.
Thang Viet Doan, Zichong Wang, Nhat Nguyen Minh Hoang, and Wenbin Zhang. Fairness in large language models three hours. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 5514 5517, 2024.
Jesse Dodge, Gabriel Ioharco, Roy Schwartz, Ali Farhadi, Hannanch Hajishirzi, and Noah Smith. Finc-tuning pretrained language models: Weight initializations, data orders, and early stopping. aiv preprinl (:Xi::2002.06365, 2020.
Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capabilities of large language models. arXiv preprint 2rx: 13542, 2024.
Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finctuning for generative foundation model aligument. Xiv preprint arXiv:2344.06661, 2023.
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao Wuen Hon. Unified language model pre-training for natural language understanding and generation. Advances in newral information processing systems, 32, 2019.
Wei Dong, Charikar Moses, and Kai Li. Efficient k-nearest neighbor graph construction for generic similarity measures. Proceedings of the 20th international conference on World wide web, pp. 577 586, 2011.
Jiawei Du, Qin Shi, and Joey Tianyi Zhou, Sequential subset matching for dataset distillation. Advances in Neural Information Processing Systems, 36, 2024.
Nan Du, Yingyu Liang, Maria Balcan, and Le Song. Influence function learning in information diffusion networks. In International Conference on Machine Learning, pp. 2016-2024. PMLR, 2014.
Qianlong Du, Chengqing Zong, and Jiajun Zhang, Mods: Model-oriented data selection for instruction tuning. arXiv preprint drXtv:2811.15655 2023.
Wenchao Du and Alan W Black. Boosting dialog response generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
William H Dubay. The principles of readability. impact information. Costa Mesa, CA, 2004.
Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. Alpacafarm: A sinulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36, 2024.
Sergey Edunov, Alexci Bacvski, and Michael Auli. Pro-trained language model representations for language generation. arXiv preprint Xix: 09722, 2019.
Lisa Ehrlinger and Wolfram WöB. A survey of data quality measurement and monitoring tools. Frontiers in big data, 5:850611, 2022.
42

---

Published in Transactions on Machine Learning Research (12/2024)
Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y Zeevi. The farthest point strategy for progressive image sampling. IEEF transactions on image processing, 6(9):1305-1315, 1997.
Logan Engstrom, Axel Feldmann, and Aleksander Madry. Dsdm: Model-aware dataset selection with
datamodels. arXiv preprint orXin:2401.12 12926, 2024.
Ayse Erkan and Yasemin Altun. Semi-supervised learning via generalized maximum entropy. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 209-216. JMLR Workshop and Conference Proceedings, 2010.
Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding datasct difficulty with V-usable information. In International Conference on Muchine Learning, pp. 5988 6008. PMLR, 2022.
Philip R Evans and Garib N Murshudov. How good are my data and what is the resolution? Acta Cryslallogruphica Section D: Biological Crystallogruphy, 69(7):1204- -2111 2013.
Dante Everacrt and Christopher Potts. Gio: Gradient information optimization for training datasct selection.
arviv preprial ar Xin:23006 1 11670, 2023.
Reza Zanjirani Farahani and Masoud Hekmatfar. Facility location: concepts, models, algorithms and case studies. Springer Science & Business Media, 2009.
Dan Feldman and Michael Langberg. A unified framework for approximating and clustering data. In Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 569-578, 2011.
Vitaly Feldman. Does learning require memorization? a short tale about long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 954-959, 2020.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881-2891, 2020.
Mariano Felice and Lucia Specia. Linguistic features for quality estimation. In Proceedings of the Seventh Workshop Statistical Machine Translation, pp. 96-103, 2012.
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. Language-agnostic bert sentence embedding. arXio preprint vxXiw:2007.0182, 2020.
Lijun Feng, Martin Jansche, Matt Huenerfauth, and Noémie Elhadad. A comparison of features for automatic readability assessment. In Coling 2010: Posters, pp. 276-284, 2010.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. A survey of data augmentation approaches for nlp. arXiv preprint arXiv:2105. 0.0005 2021.
Peter Flach. Machine learning: the art and science of algorithmss that make sense of data. Cambridge university press, 2012.
Rudolph Flesch. A new readability yardstick. Journal of applied psychology, 32(3):221, 1948.
Thomas François. La lisibilité computationnelle: un renouveau pour la lisibilité du français langue première et seconde? ITI--Interaational Journal of Applied Linguistics, 160(1):75-95)) 99, 2010.
Thomas François. Les apports du traitement automatique du langage la lisibilité du français langue étrangère. PhD thesis, Ph. D. thesis, Université Catholique de Louvain. Thesis Supervisors: Cédrick .... 2011.
Thomas François and Cédrick Fairon. An "ai readability" formula for french as a foreign language. In Proceed- ings of the 2012 joint conference empirical methods in Natural Language Processing and computational natural language learning, pp. 466-477, 2012.
Thomas François and Eleni Miltsakaki. Do nlp and machine learning improve traditional readability formulas? In Proceedings of the First Workshop on Predicting and Improving Tert Readability for target reader populations, pp. 49 57, 2012.
43

---

Published in Transactions on Machine Learning Research (12/2024)
Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, pp. 1 79, 2024.
Rong Gao, Qin Ni, and Bingying Hu. Fairness of large language models in education. In Proceedings of the 2024 International Conference Intelligent Education and Computer Technology, pp. 1, 2024.
Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent in kronecker-factored eigenbasis, 2021. URL attps::/arxxi..or//bb//// 03884
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. International conference on machine learning, pp. 2242 2251. PMLR, 2019.
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Di- nesh Manocha, et al. closer look at the limitations of instruction tuning. arXie preprint arX1v:2402.05119, 2024.
Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. Finetune like you pretrain: Improved finetuning of zcro-shot vision modcls. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 19338 19347, 2023.
Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in neural informalion processing systems, 17, 2004.
Hans-Row Gregorius and Elizabeth M Gillet. Generalized simpson-diversity. Ecological Modelling, 211(1-2): 90-96, 2008.
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Fssi Durmus, Ethan Perez, Evan Hubinger, Kamilé Lukošiūté, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying large language model generalization with influence functions, 2023. URL https://arxiv ob3.s///.....////....... 2ttp://32000000.3663
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. Xiviv preprint ar Xr:::23306.11144 2023.
Robert Gunning. The technique of clear writing, (No Title), 1952.
Chengcheng Guo, Bo Zhao, and Yanbing Bai. Deepcore: A comprehensive library for coreset selection in deep learning. In International Conference on Database and Expert Systems Applications, pp. 181-195. Springer, 2022.
Nitin Gupta, Shashank Mujumdar, Hima Patel, Satoshi Masuda, Naveen Panwar, Sambaran Bandyopadhyay, Sameep Mehta, Shanmukha Guttula, Shazia Afzal, Ruhi Sharma Mittal, et al. Data quality for machine learning tasks. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 1040-4041 2021.
Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, lan Tenney, and Tolga Bolukbasi. Simfluence: Modeling the influence of individual training examples by simulating training runs. arXio preprint arXiv:2303.0811, 2023.
Nick Harvey and Samira Samadi. Ncar-optimal herding. In Conference on Learning Theory, pp. 1165-1182. PMLR, 2014.
Fangliang He and Xin-Sheng Hu. Hubbell's fundamental biodiversity parameter and the simpson diversity index. Ecology Lellers, 8(4):386 330, 2005.
Jimming He, Sanjana Garg, and Jonas Mueller. How detect bad data in your instruction tuning dataset (for better 1mm fine-tuning), 2024. URL httt:::///ceemllaaaa////l//////p wtw.:iittte-11mmmm----- htnp://lg..fflleeee---nnnunaaaad
44

---

Published in Transactions on Machine Learning Research (12/2024)
Hmida Hmida, Sana Ben Hamida, Amel Borgi, and Marta Rukoz. Hierarchical data topology based selection for large scale learning. In 2016 Iull IEFE Conferences on Ubiquilous Intelligence & Compuling, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress CLTC/ATC/Saalle /aa////aaaa//////C /CBDCCC/IDooo/oo /Smart (orld), pp. 1221-1226. TEEE, 2016.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. Xiv preprint arXiv:2203.1555 2022.
Anna Huang et al. Similarity measures for text document clustering, In Proceedings of the sizth new zealand computer science research student conference (NZCSRSC2008), Christchurch, New Zealand, volume 4, pp- 9 56, 2008.
Danqing Huang, Shuming Shi, Chin-Yew Lin, Jian Yin, and Wei-Ying Ma. How well do computers solve math word problems? large-scale dataset construction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 887-896, 2016.
Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, and Tiejun Zhao. An empirical study of lim-as judges--judde for Ilm evaluation: Finc-tuned judge modols are task-spocific classifiers. arXiv preprint arXiv:2403 03.0030,, 2024.
Ferenc Huszár and David Duvenaud. optimally-weighteed herding is bayesian quadrature. arXiv preprint orXiv:1204.1660 2012.
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, and Irina Rish. Simple and scalable strategies to contimually pre-train large language models. arxiv preprint arxiv22003 08763, 2024.
Abiodun M Ikotun, Absalom E Ezugwu, Laith Abualigah, Belal Abuhaija, and Jia Heming. K-means clustering algorithuns: A comprehensive review, variants analysis, and advances in the era of big data. Information Sciences, 622:178 210, 2023.
Andrew Tyyss, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Datamodels: Predicting predictions from training data. arXiv preprint X20:2200000022, 2022.
Bogdan Ionescu, Mihai Lupu, Maia Rohm, Alexandru Lucian Gínsca, and Henning Müllrr. Datascts column: diversity and credibility for social images and image retrieval. ACM SIGMullimedia Records, 9(3):777 2018.
Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, and Yonatan Belinkov. Instructed to bias: Instruction-tuned language models exhibit emergent cognitive bias. Transactions of the Association for Computational Linguistics, 12:771-785, 2024.
Saachi Jain, Hadi Salman, Alaa Khaddaj, Eric Wong, Sung Min Park, and Aleksander Mądry. A data-based perspective on transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. (613-3622, 2023.
Sarthak Jain, Varun Manjunatha, Byron Wallace, and Ani Nenkova. Influence functions for soquence lagging models. arXiv preprint arXiv:2210.14117 2022.
Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Leo, and Minjoon Sco. Exploring the benefits of training expert language models over instruction tuning. In International Conference on Machine Learning, pp. 14702 14729. PMLR, 2023.
Scott Jarvis. Capturing the diversity in lexical diversity. Language learning, 63:87 106, 2013.
Scott Jarvis and M Daller. Defining and measuring lexical diversity. Vocabulary knowledge: Human ratings and automated mensures. Amsterdam, The Netherlands, 2013.
45

---

Published in Transactions on Machine Learning Research (12/2024)
Fred Jelinek, Robert L Mercer, Lalit R. Bahl, and James K Baker. Perplexity mcasure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S66 S63, 1977.
Frederick Jelinek. Interpolated estimation of markov source parameters from sparse data. In Proc. Workshop on Pattern Recognition in Practice, 1980, 1980.
Hongjie Jia, Shifei Ding, Xinzheng Xu, and Ru Nie. The latest research progress spectral clustering. Neural Computing and Applications, 24:1477-1486, 2014.
Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pp- 1167-1176. PMLR, 2019.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arviv preprint Xrx:222300. 06825, 2023a.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401. 0000020, 2024a.
Mingjian Jiang, Yangjun Ruan, Sicong Huang, Saifei Liao, Silviu Pitis, Roger Baker Grosse, and Jimmy Ba. Calibrating language models via augmented prompt ensembles. Workshop on Challenges in Deployable Generalive AI al International Conference on Machine Learning, 2023b.
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training langnage models. arXiv preprint wxXv:2441.06055 2024b.
Wenyu Jiang, Hao Cheng, Mingcai Chen, Chongjun Wang, and Hongxin Wei. Dos: Diverse outlier sampling for out-of-distribution detection. Xiviv preprint arxiv:2306. 02031, 2023c.
Wenyu Jiang, Zhenlong Liu, Zejian Xie, Songxin Zhang, Bingyi Jing, and Hongxin Wei. Exploring learning complexity for downstream data pruning. arXie preprinl or Xix:222002 05356, 2024c.
Zixuan Jiang, Jiaqi Gu, Mingjie Liu, and David Z Pan. Delving into effective gradient matching for dataset condensation. In 2023 IEFE International Conference on Omni-layer Intelligent Systems (COINS), pp- 1 6. IEEE, 2023d.
Tong Jiao, Jian Zhang, Kui Xu, Rui Li, Xi Du, Shangqi Wang, and Zhenbo Song. Enhancing fairness in Ilm evaluations: Unveiling and mitigating biases in standard-ansser--asee evaluations. In Proceedings of the AAAI Symposium Series, volume 4, pp. 56 56-59, 2024.
Xisen Jin and Xiang Ren. Demystifying forgetting in language model fine-tuning with statistical analysis of example associations. arXiv preprint arxi::2406. 1400020666,00 2024a.
Xisen Jin and Xiang Ren. What will my model forget? forecasting forgotten examples in language model refinement. ar Xiu preprint ar Xrxiv:2402. 01861, 2024b.
Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang- Wen Li, Xiaokai Wei, Andrew Arnold, and Xiang Ren. Lifelong pretraining: Continually adapting language models to emerging corpora. aiv preprinl arXiv:21 10.08594, 2021.
RC St John and Norman R Draper. D-optimality for regression designs: review. Technometrics, 17(1): 15-23, 1975.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds Nova DasSarma, Eli Tran-Johnson, al. Language models (mostly) know what they know. Xin preprint 2207:2207002221 2022.
46

---

Published in Transactions on Machine Learning Research (12/2024)
Firuz Kamalov. Kernel density estimation based sampling for imbalanced class distribution. Information Sciences, 112:1112-1201, 2020.
Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks language models. In International Conference on Machine Learning, pp. 10697- 10707. PMLR, 2022.
Feiyang Kang, Hoang Anh Just, Anit Kumar Sahu, and Ruoxi Jia. Performance scaling via optimal transport: Enabling data selection from partially revealed sources. Advances in Neural Information Processing Systems, 36, 2024.
Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine Piatko, Ruth Silverman, and Angela Y Wu. The analysis of a simple k-means clustering algorithm. In Proceedings of the sicleenth annaual symposium on Computational geometry, pp. 100-109, 2000.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for ncural language models. arXiv preprint NNi:20011083660 2020.
Zixuan Ke. Contirual Learning with Language Models. PhD thesis, University of Illinois at Chicago, 2024.
Susan Kemper. Measuring the inference load of a text. Journal of educational psychologu, 75(3):391, 1983.
Kimmo Kettunen. Can type-token ratio be used to show morphological complexity of languages? Journal of Quantitative Linguistics, 21 (3):223 245, 2014.
Kamran Khan, Saif Ur Rehman, Kamran Aziz, Simon Fong, and Sababady Sarasvady. Dbscan: Past, present and future. In The fifth international conference on the applications of digital information and web technologies (ICADIWT 2014), pp. 232-2222222222222222 232-238. IEEE, 2014.
Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, et al. Dynabench: Rethinking benchmarking in nlp. arXiv preprial ar Xrxi::1144 14337, 2021.
Krishnateja Killamsetly, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad- match: Gradient matching based data subset selection for efficient deep model training. In International Conference on Machine Learning, pp. 5464-5474. PMLR, 2021a.
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Tyer. Glister: General- ization based data subset selection for efficient and robust learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 8110-8118, 2021b.
Krishnateja Killamsetty, Xujiang Zhao, Feng Chen, and Rishabh Iyer. Retrieve: Coreset selection for efficient and robust semi--upeee supervised learning. Advances in neural information processing systems, 34:14488 14501,
2021c
Walter Kintsch and Douglas Vipond. Reading comprehension and readability in educational practice and psychological theory. In Perspectives memory research, pp. 329 365. Psychology Press, 2014.
John Kirchenbauer, Garrett Honke, Gowthami Somepalli, Jonas Geiping, Daphne Ippolito, Katherine Lee, Tom Goldstein, and David Andre. Lmd3: Language model data density dependence. arXiv preprint arXiv:2405.06331, 2024.
Andreas Kirsch. Does* deep learning on data diet" reproduce? overall yes, but grand at initialization does not. arXiv preprint arXiv:2303. 1575,, 2023.
George R Klare. Assossing readability. Reading research quarterly, pp- 62-102, 1974.
George R Klare et al. The measurement of readability. Towa State University Press Ames, 1963.
George R Klare et al. Readability. Handbook of reading research, (6811744,, 1984.
47

---

Published in Transactions on Machine Learning Research (12/2024)
Pang Wei Koh and Percy Liang. Understanding black- box predictions via inffuence functions. In International conference on machine learning, pp. 1894. PMLR, 2017.
Rik Kedziorsk,,,,,,kii, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. Mawps: A math word problem repository. In Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies, pp. 1152-1157, 2016.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Auagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Due Nguyen, Oliver Stanley, Richárd Nagyfi, et al. Openassistant nnversations-democraaa large language model alignment. Advances in Newral Information. Processing Systems, 36, 2024.
Jan Kremer, Kim Steenstrup Pederson, and Christian Igel. Active learning with support vector machines. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 4(4):313 326, 2014.
Sandra Kublik and Shubham Saboo. GPT-3: The Ultimate Guide to Building NLP Products with OpenAI
API. Packt Publishing Ltd, 2023.
Po-Nien Kung and Nanyun Peng. Do models really learn to follow instructions? an empirical study of instruction tuning. arXiv preprint drXiv:235511388, 2023.
Po-Nien Kung, Fan Yin, Di Wu, Kai-Wei Chang, and Nanyun Peng. Active instruction tuning: Improving cross-task generalization by training on prompt sensitive tasks. arXiv preprint 0rXii:2331.000 2023.
Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. A fully first-order method for stochastic bilevel optimization. In International Conference on Machine Leorning, pp. 18083-18113. PMLR, 2023.
Yongchan Kwon and James Zou. Beta shapley: a unified and noise-reduced data valuation framework for machine learning. arXiv preprint arXio:2110.14449, 2021.
Kristopher Kyle, Scott A Crossloy, and Scott Jarvis. Assessing the validity of lexical diversity indices using direct judgements. Longuage Assessment Quarterly, 18(2):154-170, 2021.
Yi-An Lai, Xuan Zhu, Yi Zhang, and Mona Diab. Diversity, density, and homogeneity: Quantitative characteristic metrics for text collections. arXiv preprint arXiv:2003. 08529, 2020.
Yrjo Lappalainen and Nikesh Narayanan. Aisha: A custom ai library chatbot using the chatgpt api. Journal of Web Läbrorianship, 17(3):37-58 2023.
Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K Kummerfeld, Parker Hill, Michael A Laurenzano, Johann Hauswald, Lingjia Tang, and Jason Mars. Outlier detection for improved data quality and diversity in dialog systems. arXiv preprint arXiv:1904.01120 2019.
Cosmin Lazar and Audrei Doncescu. Non negative matrix factorization clustering capabilities; application on multivariate image segmentation. In 2009 International Conference on Complez, Intelligent and Software Intensive Systems, pp. 924-929. IEEE, 2009.
Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. Adversarial filters of dataset biases. In International conference on machine learning, pp.
Pmlr, 2020.
Guillaume Lecué and Shahar Mendelson. Regularization and the small-ball method i: sparse recovery. 2018.
Alycia Lee, Brando Miranda, Sudharsan Sundar, and Sanmi Koyejo. Beyond scale: the diversity coefficient. as a data quality metric demonstrates llms are pre-trained on formally diverse data. arXiv preprint 2rX6::2306.138 18840, 2023.
Daniel Lee and H Sebastian Seung. Algorithms for non-negative matrix factorization. Advonces in neural information processing systems, 13, 2000.
48

---

Published in Transactions on Machine Learning Research (12/2024)
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch and Nicholas Carlini. Deduplicating training data makes language models better. ar preprinl arXiv:2107.064900 2021.
Bohan Li, Yutai Hou, and Wanxiang Che. Data augmentation approaches in natural language processing: A survey. Ai Open, 3:71-90, 2022.
Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan
Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, al. Synthetic data (almost) from scratch: Generalized instruction luning for language models. arXi preprinl ar (tto:2402.300633 2024a.
Jiwei Li, Michel Galley, Chris Brocketl, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv: 03055, 2015.
Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting 1lm performance with self-guided data selection for instruction tuning. arXiv preprint or Xiv: ft::338.12032. 12032, 2023a.
Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfittrrng: tosk-to-strong data filtering for fast Instruction-tuiingg arXiv preprint arXiv:2402.00530 2024b.
Na Li, Yiyang Qi, Chaoran Li, and Zhiming Zhao. Active learning for data quality control: A survey, ACM Journal of Data and Information Quality, 2024c.
Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint ar (tle:2305.0661.. 2023b.
Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. Self-alignment with instruction backt ranslation. arXiv preprint arXiv:2308. 06259,, 2023c.
Yingji Li, Mengnan Du, Rui Song, Xin Wang, and Ying Wang. A survey on fairness in large language models. Xixiv preprinl ar Xrx:2030 10149, 2023d.
Yize Li, Pu Zhao, Xue Lin, Bhavya Kailkhura, and Ryan Goldhahn. Less more: Data pruning for faster adversarial training. arXiv preprint ar Xix::2302.123000 2023e.
Yuchen Li, Ju Fan, Yanhao Wang, and Kian-Lee Tan. Influecce maximization on social graphs: A survey. IEEE Transactions on Knowledge and Data Engineering, 30(10):1852-1822 2018.
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large language models for text classification: Potential and limitations. arxiv preprint arXiu:2310.07840. 2023f.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint ar arXim:2211.09110 2022.
Chin- Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Jinkun Lin, Anqi Zhang, Mathias Lécuyer, Jinyang Li, Aurojit Panda, and Siddhartha Sen. Measuring the effect of training data on deep learning predictions via randomized experiments. In International Conference on Machine Learning, pp. 13468-13504. PMLR, 2022.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXi::2109.07000 2021.
Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat- Seng Chua. Data-cfficient fine-tuning for llmebased recommendation, 2024. URL http:/pps//aa.v.oo////////////// 17197
49

---

Published in Transactions on Machine Learning Research (12/2024)
Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A Smith. Tuning language models by proxy. arxiv preprint or (xXx:2401.08666 2024a.
Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, and
Dong Yu. Mmc: Advancing multimodal chart understanding with large-scale instruction tuning. arXiv preprint ar arXie:231110071 2023a.
Qingyi Liu, Yekun Chai, Shuohuan Wang, Yu Sun, Keze Wang, and Hua Wu. On training data influence of gpt models. arXiv preprint or orx:22404..085 2024b.
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction luning, Xiv preprinl Xrx: 15685, 2023b.
Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Hao- liang Wang, Tong Yu, et al. Large language models and causal inference in collaboration: A comprchensive survey. arxiv preprint orXiv:2403. 09606, 2024c.
Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: A comprehensive survey. arXiv preprint arXiv:2402.180/ 2024d.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint ar Xiv: 11692, 2019.
Ziche Liu, Rui Ke, Feng Jiang, and Haizhou Li. Take the essence and discard the dross: A rethinking on data selection for fine-tuning large language models. arXie preprint 0rXiv:2406.1441 2024e.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoe V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In International Conference on Machine Learning, pp.
PMIR, 2023.
Renze Lou, Kai Zhang, and Wenpeng Yin. Large language model instruction following: A survey of progresses and challenges. Compulational Linguistics, pp. 1-10, 2024.
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations, 2023a.
Yingzhou Lu, Minjic Shen, Huazheng Wang, Xiao Wang, Capucine van Rechem, and Wenqi Wei. Machine learning for synthetic data generation: a review. arXiv preprint arXiw:2302.04000 2023b.
Zhaosong Lu and Sanyou Mci. First-order penalty methods for bilevel optimization. SIAM Journal on Optimization, 34(2):1937-1969, 2024.
Weijie Lv, Xuan Xia, and Sheng-Jun Huang, Codeact: Code adaptive compute-efleient tuning framework for code llms. orXiv preprint arXiv:2408. 02099,, 2024.
Chengeheng Ma, Yang Liu, Jiankang Deng, Lingxi Xie, Weiming Dong, and Changsheng Xu. Understanding and mitigating overfitting in prompt tuning for vision-language models. IEFE Transactions on Circuits and Systems for Video Technology, (3(9):4616 4629, 2023.
Inbal Magar and Roy Schwartz. Data contamination: From memorization to exploitation. arXiv preprint
0tl.2203.08222 2022.
Pratyush Maini, Saurabh Garg, Zachary Lipton, and J Zico Kolter. Characterizing datapoints via second-split forgetting. Advances in Neural Information Processing Systems, 35:30044-30057 2022.
Jyoti Malhotra and Jagdish Bakal. survey and comparative study of data deduplication techniques. In 2015 International Conference on Pervasive Computing (ICPC), pp. 1-5. IEEE, 2015.
50

---

Published in Transactions on Machine Learning Research (12/2024)
David Malvern, Brian Richards, Ngoni Chipere, and Pilar Durán. Lerical diversity and language development. Springer, 2004.
David D Malvern and Brian J Richards. A new measure of lexical diversity. British Studies in Applied Linguistics, 12:58 71, 1997.
Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining Ilms at scale. arXiv preprint (tfi::2309.05566 2023.
Marc Marone and Benjamin Van Durme. Data portraits: Recording foundation model training data. Advances in Neural Information Processing Systems, 36, 2024.
Michael Mathieu and Yann LeCun. Fast approximation of rotations and hessians matrices. arxie preprint arXiv:1404. 7195, 2014.
Vladimir Matlach, Diego Krivochen, and Jiri Milička. A method for comparison of general sequences via type-token ratio. Language and Tert: Data, models, information and applications. Amsterdam: John Benjamins, pp. 37-54, 2021.
Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. On measuring social biases in sentence encoders. arXiv preprint orXiv:1903, 10561, 2019.
Philip M McCarthy, An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of tertual, levical diversily (MTLD). PhD thesis, The University of Memphis, 2005.
Philip M McCarthy and Scott Jarvis. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381 392, 2010.
Gábor Melis, Chris Dyer, and Phil Blunsom. On the state of the art of evaluation in neural language models. arXiv preprint ar 1tt::707.000 775899559, 2017.
Brando Miranda, Patrick Yu, Yu-Xiong Wang, and Sanmi Koyejo. The curse of low task diversity: On the failure of transfer learning to outperform maml and their empirical equivalence. arXio preprinl arXiv:220S.015.. 2022.
Swaroop Mishra and Bhavdeep Singh Sachdeva. Do we need create big datasets to learn a task? In Proceedings of SustaiNLP: Workshop on Simple and Efficiintt Natural Language Processing, pp. 169-173, 2020.
Swaroop Mishra, Anjana Arunkunar, Bhavdeep Sachdeva, Chris Bryan, and Chitta Baral. Dqi: Measuring data quality in nlp. arXiv preprint arXiv:2005.00816 2020a.
Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva, Chris Bryan, and Chitta Baral. Dqi: A guide benchmark evaluation. arXiu preprint arXio:2008. 0000:496000000 2020b.
Niluthpol Chowdhury Mithun, Rameswar Panda, and Amit K Roy-Chowdhury. Construction of diverse image datascts from web collections with limited labeling. IEEE Transactions on Circuits and Systems for Video Technology, 30(4):1147-111 2019.
Sedir Mohammed, Hazar Harmouch, Felix Naumann, and Divesh Srivastava. Data quality assessment: Challenges and opportunities. arXiv preprint arXiv:2403.00598 2024.
Robert C Moore and William Lewis. Intelligent selection of language model training data. In Proceedings of the ACL 2010 conference short papers, pp. 220-224, 2010.
David Mouillot and Alain Lepretre. A comparison of species diversity estimators. Researches on Population Ecology, 41:203 225, 1999.
Kevin P Murphy. Machine learning: probabilistic perspective. MIT press, 2012.
51

---

Published in Transactions on Machine Learning Research (12/2024)
Aidar Myrzakhan, Sondos Mahmoud Bsharat, and Zhiqiang Shen. Open-Iln-laaderbaard From multi-choice to open-style questions for llss evaluation, benchmark, and arena. arXiv preprinl ar arXiv2406.07555 2024.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. Crows-pairs: A challenge dataset for measuring social biases in masked language models. arXiv preprint arXiv:2010.000 00133,, 2020.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances in neural information processing systems, 26, 2013.
Noel Ngu, Nathaniel Lee, and Paulo Shakarian. Diversity measures: Domain-independent proxies for failure in language model queries. In 2024 IEEE 18th International Conference on Semantic Computing (ICSC), pp. 176 182. IEEE, 2024.
Cuong Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto. Toward understanding catastrophic forgetting in continual learning. arXiv preprint trXiv:1908.011 0109, 2019.
Quan Nguyen and Adji Bousso Dieng, Quality- vendi scores and their application to diverse experimental design. arXiv preprint arXiv:2405.024. 2024.
Vu-Linh Nguyen, Mohammad Hossein Shaker, and Eyke Hüllermeier. How to mcasure uncertainty in uncertainly sampling for active learning. Machine Learning, 111(1):89 122, 2022.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial nli: A new benchmark for natural language underslanding. ai preprinl arxiv: 14599, 2019.
Björn Nieth, Thomas Altstidl, Leo Schwinn, and Björn Eskofier. Large-scale dataset pruning in adversarial training through data importance extrapolation. arXiv preprint arXiv:2406. 13283, 2024.
Geir K Nilsen, Antonella Z Munthe-Kaas, Hans J Skaug, and Morten Brun. Efficient computation of hessian matrices in tensorflow. arXiv preprint arXic: 005599 2019.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neurol information processing systems, 35:27730 27744, 2022.
Rui Pan, Jipeng Zhang, Xingyuan Pan, Renjie Pi, Xiaoyu Wang, and Tong Zhang. Scalebio: Scalable bilevel optimization for Ilm data reweighting. arXiv preprint 2rXi::2406. 19976, 2024.
Jinlong Pang, Jiaheng Wei, Ankit Parag Shah, Zhaowei Zhu, Yaxuan Wang, Chen Qian, Yang Liu, Yujia Bao, and Wei Wei. Improving data efficioncy via curating Ilm-driven rating systems. arXiv preprint ar.ivv:2 10. 10877, 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311 311-318, 2002.
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclere, and Aleksander Madry. Trak: Attributing model behavior at scale. arxiv preprint arXiw:2303.14:8 2023.
Jupinder Parmar, Sanjev Satheesh, Mostofa Patwary, Mohammad Shocybi, and Bryan Catanzaro. Reuse, don't 't retrain: A recipe for continued pretraining of language models. arXiv preprint arXiv:2407.07263, 2024.
Amey Pasarkar and Adji Bousso Dieng, Cousins of the vendi score: A family of similarity-based diversity metrics for science and machine learning. arXiv preprint arXiv:2310. 12952, 2023.
Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openwebmath: An open dataset of high-quality mathematical web text. arXiv preprint arXiv:2310.0071 2023.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word problems? Xiv preprint ar xrX::21033007191 2021.
52

---

Published in Transactions on Machine Learning Research (12/2024)
Leena H Patil and Mohammed Atique. A novel approach for feature selection method tf-idf in document clustering. In 2013 3rd IEEE international advance compuling conference (TACC), pp. 758-862. IEFE, 2013.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. Advances in neurul information processing systems, 34:20596 20607, 2021.
Barak A. Pearlmutter. Fast exact multiplication by the hessian. Newral Compulation, 6(1):147-160, 1994. doi: 10.1160 1t..2//ncco.199..........
Robert K Peet. The mcasurement of species diversity, Annual review of ecology and systematics, pp. 285-307, 1974.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon Ilm: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.011 2023.
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXix:20440022) 2023.
Leif E Peterson. K-nearest neighbor. Scholorpedia, 4(2):1883, 2009.
Agustin Picard, Lucas Hervier, Thomas Fel, and David Vigouroux. Influencia:: A library for tracing the influence back to the data-points. In World Conference on Esplainable Artificial Intelligence, pp. 193 204. Springer, 2024.
Maria Priestley, Fionntán O'donnell, and Elena Simperl. A survey of data quality requirements that matter in ml development pipelines. ACM Journal of Data and Information Quality, 15(2):1 39, 2023.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advunces in Neural Information Processing Systems, 33:19920- 19930, 2020.
Yulei Qin, Xingyu Chen, Yunhang Shen, Chaoyou Fu, Yun Gu, Ke Li, Xing Sun, and Rongrong Ji. Capro: webly supervised learning with cross-modality aligned prototypes. Advances in Neural Information Processing Systems, 36, 2024.
Alec Radford, Karthik Narasimhan, Tim Salimans, Nya Sutskever, et al. Improving language understanding by generative pre-training. 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Tlya Sutskever, et al. Language models are unsupervised multitask learners. OpenA/ / blog, 1(8):9, 2019.
Jack W Rac, Sebastian Borgcaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis insights from training gopher. arXio preprint 1rXiw:2112.114// 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Newral Information Processing Systems, 36, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning reseurch, 21(140):1 67, 2020.
K Raghuveer al. Legal documents clustering using latent dirichlet allocation. IAES Int. J. Artif. Intell, 2 (1):34 37, 2012.
53

---

Published in Transactions on Machine Learning Research (12/2024)
Xingcheng Ran, Yue Xi, Yonggang Lu, Xiangwen Wang, and Zhenyu Lu. Comprehensive survey hierarchical clustering algorithms and the recent developments. Artificial Intelligence Review, 56(8):8219 8264, 2023.
Nils Reimers and Iryna Gurevych. Sentence-bers Sentence embeddings using siamese bert-networks. arXiv preprint 1rXi::1908 10084, 2019.
Alfród Rényi. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium mathematical statisties and probabilily, volume 1: contributions to the theory of statistics, volume 4, pp. 547-562. University of California Press, 1961.
Hamed Rezazadegan Tavakoli, Esa Rahtu, and Janne Heikkilä. Fast and efficient saliency detection using sparse sampling and kernel density estimation. In Image Analysis: 17th Scandinavian Conference, SCTA 2011, Yslad, Sweden, May 2011. Proceedings 17, pp. 666 675. Springer, 2011.
Marco Tulio Ribciro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of nlp models with checklist. arXiv preprint AXXi:2005.04411 2020.
Brian Richards. Type/token ratios: What do they really tell us? Journal of child language, 14(2):201 -09,, 1987.
Yuji Roh, Geon Heo, and Steven Fuijong Whang. A survey data collection for machine learning: big data-ai integration perspective. IEEE Transactions on Knowledge and Data Engineering, 33(4)11328-1347, 2019.
Rajendra Kumar Roul, Omanwar Rohit Devanand, and Sanjay Kumar Sahay. Web document clustering and ranking using tf-idf based apriori approach. arXic preprint 4rXiv:1406.5611 2014.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd scherna challenge at scale. Communications of the ACM, 64(9):99 106, 2021.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arv NNix:2110.08200 10.08207, 2021.
Gayathri Saranathan, Mahammad Parwez Alam, James Lim, Suparna Bhattacharya, Soon Yee Wong, Martin Foltin, and Cong Xu. Dele: Data efficient I1m evalnation. In ICLR 2024 Workshop on Nanigating and Addressing Data Problems for Foundation Models.
Nikunj Saunshi, Arushi Gupta, Mark Braverman, and Sanjeev Arora. Understanding influence functions and datamodels via harmonic analysis. In The Eleventh International Conference on Learning Representations, 2022.
Timo Schick and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot. learners. Xixiv preprint arXic:2009.0711 2020.
Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions, 2021. URL httpss//arxiv.orr/b////112...8
Stephanie Schoch, Ritwick Mishra, and Yangleng Ji. Data selection for fine-tuning large language models using transferred shapley values. arXiv preprint arXiv:2306. 10165, 2023.
Sarah E Schwarm and Mari Ostendorf. Reading level assessment using support vector machines and statistical language models. In Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL '05), pp. 523 530, 2005.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv: nee:1708.000 00489, 2017.
Burr Settles. Active learning literature survey. Science, 10(3):237-38) -34,, 1995.
54

---

Published in Transactions on Machine Learning Research (12/2024)
Burr Settles. From theories to queries: Active learning in practice. In Active learning and experimental design workshop in conjunction with ATSTATS 2010, pp. 1-18. JMLR Workshop and Conference Proceedings, 2011.
Claude Elwood Shannon, A mathematical theory of communication. The Bell system technical journal, 27 (3):379-423) 1948.
Claude Elwood Shannon. A mathematical theory of communication. ACM SIGMOBILE mobile computing and communications review, 5(1):3-55, 2001.
Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahma Lin, and Xipeng Qiu. Balanced data sampling for language model training with clustering. arXiv preprint arXiv:2402.1 14526, 2024.
Bin Shen and Luo Si. Non-negative matrix factorization clustering on multiple manifolds. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, pp. 575 580, 2010.
Jonathan Richard Shewchuk et al. An introduction to the conjugate gradient method without the agonizing pain. 1994.
Zhengxiang Shi and Aldo Lipani. Don't stop pretraining? make prompt-based fine-tuning powerful learner. Advances in Neural Information Processing Systems, 36:5827-588 2023.
Zhengyan Shi, Adam X Yang, Bin Wu, Laurence Aitchison, Emine Yilmaz, and Aldo Lipani. Instruction tuning with loss over instructinns. Xiv preprinl or (xXi:2240511339) 2024.
Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning. Advances in Newral Information Processing Systems, 1161886-66156, 2023.
Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. Generating diverse translations with sentence codes. In Proceedings of the 57th onnual meeting of the association for computational linguistics, pp. 1823 1827, 2019.
Luo and Jamie Callan. statistical model for scientific readability. Proceedings of the tenth international conference on Information and knowledge management, pp. 574 576, 2001.
Aditya Siddhant and Zachary Lipton. Deep bayesian active learning for natural language processing:
Results of a large-scale empirical study. arx preprint arXiv: 1808.0567, 2018
Fatimah Sidi, Payam Hassany Shariat Panahy, Lilly Suriani Affendey, Marzanah A Jabar, Hamidah Ibrahim, and Aida Mustapha. Data quality: A survey of data quality dimensions. In 2012 International Conference on Information Retrieval Knowledge Management, pp. 300 304. TEEE, 2012.
Stacy Silverman and Nan Bernstein Ratner. Measuring lexical diversity in children who stutter: Application of voed. Journal of fluency disorders, 27(4):289 000444444444 2002.
Edward H Simpson. Measurement of diversity. nalure, 163(4148):688 688, 1949.
Kristina P Sinaga and Miin-Shen Yang. Unsupervised k-means clustering algorithm. IEEE access, 8: 80716-80722 2020.
Ankur Sinha, Pekka Malo, and Kalyanmoy Deb. A review on bilevel optimization: From classical evolutionary approaches and applications. IEEE transactions on evolutionary compulalion, 22(2):276-25) 295, 2017.
Samarth Sinha, Han Zhang, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, and Augustus Odena. Small-gan: Speeding up gan training using core-sets. In International Conference on Machine Learning, pp. 9005 9015. PMLR, 2020.
Eric Michael Smith, Mclissa Hall, Melanie Kambadur, Elconora Presani, and Adina Williams. i'm sorry hear that': Finding new biases in language models with a holistic descriptor dataset. arXiv preprint orXin:2205500920, 2022.
55

---

Published in Transactions on Machine Learning Research (12/2024)
John Smith and Lisa Johnson. Strategies for difficulty sampling providing diversity in datasets. Journal of Machine Learning Research, 10:100-120, 2020.
Hwanjum Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with doep neural networks: A survey. IEEE transactions on neural networks and learning systems, 34(11): 8135 8153, 2022.
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35: 19523 19536, 2022.
Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of documenlation, 28(1):11 21, 1972.
Eleftherios Spyromitros- Xioufis, Symeon Papadopoulos, Alexandru Lucian Ginsca, Adrian Popescu, Yiannis Kompatsiaris, and loannis Vlahavas. Improving diversity in image search via supervised relevance scoring. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 323-330, 2015.
Katherine Stasaski and Marti A Hearst. Semantic diversity in dialogue with natural language inference. arXiv preprint Xrx:2:2205. 01497, 2022.
Katherine Stasaski, Grace Hui Yang, and Marti A Hearst. More diverse dialogue datasets via diversity- informed data collection. In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 1958-4968, 2020.
Albert Yu Sun, Eliott Zemour, Arushi Saxena, Udith Vaidyanathan, Eric Lin, Christian Lau, and Vaikkunth Mugunthan. Does fune-tuning gpt-3 with the openai api leak personally-identifinn information? ar preprint ar xrXi::237716338 16382, 2023.
Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pottern Recognition, pp. 1390-9399, 2024a.
Wangtao Sun, Haotian Xu, Xuanqing Yu, Pei Chen, Shizhu He, Jun Zhao, and Kang Liu. Itd: Large language models can teach themselves induction through deduction. arXiv preprint axXiv:2403.0578, 2024b.
Jun Suzuki, Hciga Zen, and Hideto Kazawa. Extracting representative subset from extensive text data for training pre-trained language models. Information Processing & Management, 60(3):103249 2023.
Luis Talavera. Feature selection as preprocessing step for hierarchical clustering. In ICML, volume 99, pp. 389-397, 1999.
Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moring-one-smmpllont. Advances in Neural Information Processing Systems, 36, 2024a.
Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, and Ji-Rong Wen. Small models, big insights: Leveraging slim proxy models to decide when and what to retrieve for llms, arXio preprint arXiv:2402.1 12052, 2024b.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction-followiing model. Slanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/033 13/ /llpcco. html, 366):7, 2023.
MosaicML NLP Team. Introducing mpt-7b: A new standard for open-source, commercially usable 1ms,, 2023. URL
Accessed: 2023-05-05.
Mildred C Templin. Certain language skills in children; their development and interrelationships. University of Minnesota Press, 1957.
56

---

Published in Transactions on Machine Learning Research (12/2024)
Guy Tevet and Jonathan Berant. Evaluating the evaluation of diversity in natural language generation. arXiv preprinl Krx::2000 029900 2020.
Steve Tingiris and Bret Kinsella. Exploring GPT-3:
An unofficial first look the general-purpose language processing API from OpenAT. Packt Publishing Ltd, 2021.
Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 85:38274-382005 2022.
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari Morcos. D4: Improving Ilm pretraining via document de-duplication and diversification. Advances in Neural Information Processing Systems, 36, 2024.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural net work learning. orXiv preprinl arXiv: (ff::181100519 2018.
Simon Tong and Daphne Koller. Support vector machine active learning with applications to text classification. Journal of machine learning research, 2(Nov):45-66, 2001.
Hugo Touvron, Thibaut Lavril, Gantier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. orXiv preprint arxi:22002. 13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXie preprint arXiv:2307.09288, 2023b.
Karina Vidal and Scott Jarvis. Effects of english-medium instruction on spanish students' proficiency and lexical diversity in english. Language Teaching Research, 24(5):568 587, 2020.
Ulrike Von Luxburg. tutorial on spectral clustering. Statistics and computing, 17:395 416, 2007.
Chi Wang, Qingyun Wu, Silu Huang, and Amin Saied. Economic hyperparameter optimization with blended search strategy. In International Conference on Learning Representations, 2021a.
Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. Flaml: A fast and lightweight automl library. Proceedings of Machine Learning and Systems, 3:434-447, 2021b.
Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, and Dianhui Chu. survey on data selection for Ilm instruction tuning. arXiv preprinl ar Xiv:2400. 05123, 2024a.
Lingzhi Wang, Xingshan Zeng, Jinsong Guo, Kam-Fai Wong, and Georg Gottlob. Selective forgetting: Advanc- ing machine unlearning techniques and evaluation in language models. arXiv preprint 3rXiv:2402.05813, 2024b.
Yidong Wang, Zhuohao Yu, Zhengran Zeng Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, et al. Pandalm: An automatic evaluation benchmark for IIm instruction tuning optimization. arXiv preprint arXiv:2306. 05087, 2023a.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Aligning language models with self-generated instructions. arXiv preprint
Hajishirzi. 2022.
arxiv:2212. 20160000 10560,
Yu-Xiong Wang and Yu-Jin Zhang. Nonnegative matrix factorization: A comprehensive review. IEEF Transactions knowledge and data engineering, 25(6):1336 1353, 2012.
Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. Aligning large language models with human: A survey. arXiv preprint orXiv:2307.19666 2023b.
57

---

Published in Transactions on Machine Learning Research (12/2024)
Yulin Wang, Gao Huang, Shiji Song, Xuran Pan, Yitong Xia, and Cheng Wu. Regularizing deep networks with semantic data augmentation. IEEF Transactions on Potlern Analysis and Machine Intelligence, 44 (7):3733 3748, 2021c
Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint 2020.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe V Le. Finetuned language models are zero-shot learners. arXiv preprinl Xrv:22109.0010 2021.
Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th onnual international conference on machine learning, pp. 1121-1128, 2009.
Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. Qurating: Selecting high-quality data for training language models. arXiv preprint 2Xi1:2240..99739, 2024.
Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37-52, 1987.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Authony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiu preprint arXiv:1900.003 037711, 2019.
Haolun WI, Yansen Zhang, Chen Ma, Fuynan Lyu, Fernando Diaz, and Xue Liu. A survey of diversification techniques in search and recommendation. CoRR arXiv, 2212, 2022.
Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, and Xue Liu. Result diversification in scarch and recommendation: A survey. IEEE Transactions on Knowledge and Data Engineering, 2024.
Shengguang Wu, Keming Lu, Benfeng Xu, Junyang Lin, Qi Su, and Chang Zhou. Self-evolved diverse data sampling for efficient instruction tuning. Xi preprint ar 2xix:23111.00 1088122 2023.
Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared llama: Accelerating language model pre-training via structured pruning. arXiv preprint orXi:23310.060 066604 2023.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. arXiv preprint 2rxiv:2402.00 04333, 2024a.
Tingyu Xia, Bowen Yu, Kai Dang, An Yang, Yuan Wu, Yuan Tian, Yi Chang, and Junyang Lin. Rethinking data selection at scale: Random selection is almost all you need. arXiv preprint arXiv:24 10. 1..00335, 2024b.
Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coresct: A universal method of data selection for real-world data-efficient deep learning. In The Eleventh International Conference on Learning Representations, 2022.
Xiaobo Xia, Jiale Liu, Shaokun Zhang, Qingyun Wu, Hongxin Wei, and Tongliang Liu. Refined coreset selection: Towards minimal coreset size under model performance constraints. In Forty-first International Conference on Machine Learning, 2024c.
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in newral information processing systems, 33:6256-622 6268, 2020.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXim:2111.020 2021.
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy S Liang. Data selection for language models via importance resampling. Advances in Neural Information Processing Systems, 11:34201-34211 2023.
58

---

Published in Transactions on Machine Learning Research (12/2024)
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. or or preprinl W.Xiv:330.112444 2023a.
Frank F Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium Machine Programming, pp. 1 10, 2022.
Yang Xu, Yongqiang Yao, Yufan Huang, Mengnan Qi, Maoquan Wang. Bin Gu, and Neel Sundaresan. Rethinking the instruction quality: Lift is what you need. arXiv preprinl Xiv:22312 115008 2023b.
Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned Ilms with nothing. arXiv preprint xrXi:22406008 08464, 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXie:2407.1000 10671, 2024.
Yaoqing Yang, Ryan Theisen, Liam Hodgkinson, Joseph E Gonzalez, Kannan Ramchandran, Charles H Martin, and Michael W Mahoney. Evaluating natural language processing models with generalization metries that do not need access to any training or testing data. arXiv preprint ar arXi:2200.0028 2022.
Emmanuel J Yannakoudakis and David Fawthrop. The rules of spelling errors. Information Processing & Management, 19(2):87-99, 1983.
Gregory Yauney, Emily Reif, and David Mimno. Data similarity is not enough to explain language model performance. arXiv preprint 1xXi:2331.000000 2023.
Jingwen Ye, Ruonan Yu, Songhua Liu, and Xinchao Wang. Distilled datamodel with reverse gradient matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11954 11961, 2024.
Çağatay Yıldiz, Nishaanth Kanna Ravichandran, Prishruit Punia, Matthias Bethge, and Beyza Ermis. Investigating continual pretraining in large language models: Insights and implications. ar Xiv preprint arXiv:240.117400, 2024.
Julio Christian Young and Makoto Shishido. Investigating openai's chatgpt potentials in generating chatbot's is dialogue for english as forcign language learning. International journal of advanced computer science and applications, 14(6), 2023.
Zichun Yu, Spandan Das, and Chenyan Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models. arXiv preprint Xrxiv:2406.00 06046, 2024.
Beverly L Zakaluk and S Jay Samuels. Readability; Its Past, Present, and Future. ERIC, 1988.
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. arXie preprint orXiv:2310.076. 2023.
Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey. arXiv preprint arxi:22903. 10158, 2023.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107-115, 2021.
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual memorization in neural language models. Advances in Newral Information Processing Systems, 36:39321-3999 39362, 2023a.
Jipeng Zhang, Yaxuan Qin, Renjie Pi, Weizhong Zhang, Rui Pan, and Tong Zhang. Tagcos: Task-agnostic gradient clustered coreset selection for instruction tuning data. arXiv preprint orx::22477 15235, 2024a.
59

---

Published in Transactions on Machine Learning Research (12/2024)
Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Is chatgpt fair for recommendation? evaluating fairness in large language model recommendation. In Proceedings of the 1 17th ACM Conference on Recommender Systems, pp. 993 999, 2023b.
Lei Zhang. Bilevel optimization in the deep learning cra: Methods and applications. 2024.
Shaokun Zhang, Xiaobo Xia, Zhaoqing Wang, Ling-Hao Chen, Jiale Liu, Qingyun Wu, and Tongliang Liu. Ideal: Influence-driven selective annotations empower in-context learners in large language models. arv preprint arXiv:2310. 10873, 2023c.
Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey, ar preprint arXiv:2308.1000 10792, 2023d.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint PrXfu:194400677 2019.
Xiao Zhang and Wu. Dissecting learning and lorgetting language model finetuning. In The Twelfth International Conference on Learning Representations, 2024.
Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Yipeng Zhang, Haitao Mi, and Helen Meng. Self-tuming: Instructing Ilms to effectively acquire new knowledge through self-teaching, arXiv preprint arXiv:2406.000 06326, 2024b.
Yifan Zhang, Yifan Luo, Yang Yuan, and Andrew C Yao. Autonomous data selection with language models for mathematical texts. In JCLR 2024 Workshop Navigaling and Addressing Data Problems for Foundation Models, 2024c.
Yihua Zhang, Yuguang Yao, Parikshit Ram, Pu Zhao, Tianlong Chen, Mingyi Hong, Yanzhi Wang, and Sijia Liu. Advancing model pruning via bi-level optimization. Advances in Neural Information Processing Systems, 35:18309 8309-1 2022.
Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IRFE/CVF Winter Conference on Applications of Computer Vision, pp. 6514 6523, 2023.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.040 05929,, 2020a.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching, 2021. URL https://aarxw.org//bbs/ 05929
Chenyang Zhao, Xueying Jia, Vijay Viswanathan, Tongshuang Wu, and Graham Neubig. Self-guide: Better task-specific instruction following via self-synthetic finetuning. arXiv preprint arXiv:2407.18874 2024a.
Dora Zhao, Jerone TA Andrews, Orestis Papakyriakoppullss, and Alice Xiang. Position: Measure dataset diversity, don't just claim it. arXiv preprint 007.022407.88188 2024b.
Dorothy Zhao, Jerone TA Andrews, AI Sony, Tokyo Orestis Papakyriakopoulls, and Alice Xiang. Mcasuring diversity in datasets. International Conference on Learning Represenlations, 2024c.
Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain generalization via entropy regularization. Advances in neural information processing systems, 33:16096-16104 2020b.
Shuaijiang Zhao and Xiaoquan Fang. Technical report: Competition solution for bettermixture. arXiv preprint ar XrXv:24403.1323 13233, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. A Aduances in Newral Information Processing Systems, 36, 2024.
60

---

Published in Transactions on Machine Learning Research (12/2024)
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards unified multi-dimensionll evaluator for text generation. arXiv preprint or 2rx::2210.07197, 2022.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Tyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024a.
Daquan Zhou, Kai Wang, Jianyang Gu, Xiangyu Peng, Dongze Lian, Yifan Zhang, Yang You, and Jiashi Feng. Dataset quantization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17205-17216, 2023.
Jianghong Zhou, Eugene Agichtein, and Surya Kallumadi. Diversifying multi-aspect search results using simpson's 's diversity index. In Proceedings of the 29th ACM International conference on information & knowledge management, pp. 2345 2348, 2020.
Yuhan Zhou, Fengjiao Tu, Kewei Sha, Junhua Ding, and Haihua Chen. A survey on data quality dimensions and tools for machine learning. arXiv preprint wxXi:2406.19614 2024b.
Lianghui Zhu, Xinggang Wang, and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiu:3300116311, 2023.
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 4 1ss international ACM SIGIR conference research & development in information retrieval, pp. 1097 1100, 2018.
Xinlin Zhuang, Xin Mao, Yuan-Hao Jiang, Hongyi Wu, Shangqing Zhao, Cai, Shu Liu, Yang Chen, Yuxiang Song, Chenghao Jia, et al. Bread: A hybrid approach for instruction data mining through balanced retrieval and dynamic data sampling. In CCF International Conference Natural Language Processing and Chinese Computing, pp. 229 240. Springer, 2024.
61


Reviews:
[{"review_id": "ZdiT1j73u6", "reviewer": "Reviewer_r62g", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper does a good and detailed survey of an important topic. Each approach has been explained with clarity.\n2. The empirical results and discussions in section 6 covering different approaches, LLMs, and test datasets can be really helpful to the community. \n3. Summarized and proposed a list of open research questions for future research.\n\nWeaknesses:\n\nI did not find any major concern with this survey. \n\n1. There are some formatting issues with quotation marks throughout the paper.\n2. Some algorithms could benefit from bit more detailed descriptions for easy understanding by a reader without any background in this field. \n\nIn general, I appreciate the author's work. Overall, this is a very comprehensive survey analyzing an important field.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors present a survey on data assessment and selection methods for instruction tuning of LLMs. This is an important topic with a possibility of significant impact on future research. The authors systematically categorize the literature into three main perspectives: quality, diversity, and importance. Next, for each category, representative methods from both model-based and hand-based indicators are elaborated. The survey also includes an empirical comparison between different approaches. After introducing related works, the paper proposed a number of open questions for the research community to consider.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Fix the formatting inconsistencies with quotation marks throughout the paper.\n2. I request the authors to provide more detailed descriptions of the algorithms for easier understanding. Although it is optional.", "broader_impact_concerns": "There are no concerns about the broader impact of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fND9NGqHCn", "reviewer": "Reviewer_BZqh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The survey provide good collection of LLM data selection method that focus on three aspects: quality, diversity and importance. The survey nicely summarized representative methods of hand-crafted and model-based methods that is easy to understand and refer to for LLM community.\n\n2. The provided suggestions on future directions are insightful.\n\nWeakness:\n1. This draft contains a lot of typos and unfinished sentences.\n2. see requested changes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The survey examined data assessment and selection method for instruction tuning of LLMs. It presents a taxonomy based on three aspects: quality, diversity, importance. Within each aspect, further categorization of method is given and representative method are summarized and discussed. The performance of some representative method are summarized and discussed per aspect. Finally, the survey points out issues of existing methods and provide future directions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Related surveys are collected in the survey, but a compare and contrast is lacking.\n2. Figure 1 is not vector graph, small texts are hard to read.\n3. The mathematical formulation in Preliminaries Instruction Dataset Preparation is not clear to me.\n    1. instruction and input are used interchangeably while considered as two parts in first sentence. Also, what is x? is x the same as I? and what is the relationship between p and x?\n    2. The equation 2 is also confusing, the input of sampling mechanism \\pi takes input b and a single data point that is a result of argmax. Author should correct the formulation. Also \\S_{b}^{\\*} ‘s meaning is unclear.\nThe terminology is not unified across the survey. datapoint and instruction datapoint are also used interchangeably.\n4. Eq (6):  the indicators should not be sharing parameters in general?\n5. Eq (8): I don’t understand why small \\tilde{S}(x_i) here corresponds to “easy”, it should be the opposite since the perplexity would need to be small, no? (correct me if I am wrong)\n6. The sentence below equation 10 “where yi(j) ∈ R^{N_{vocab}} denotes the one-hot vector of the vocabulary size N_vocab,” . The wording here is confusing.\n7. Quality-based, diversity-based and importance-based method are discussed, but author did not elaborate the definition of those quantities. Also I think some high level rationale should be provided regarding what these three aspects are and why use them to build the taxonomy. I understand these concepts overlap practically, but some discussion of the usage of these concepts within the scope of the survey is appreciated. \n    A general formulation of methods in each of these aspects would be beneficial if possible. This might help in resolve the possible confusion of the concept for example in Ch 4, diversity seems to refer to lexical and semantic richness, but why bias, fairness aspects of diversity are not discussed in the survey is not clear.\n\n8. Data selection and assessment targeting bias, fairness finetuning is completely missing in this survey, I wonder the authors opinion on these.\n\n9. I suggest author to recheck their literature summary across the draft, some summaries are wordy but fail to convey main idea of surveyed work. Take an example of Jiang et al. (2023b) referred in 5.2 the paragraph below equation 42, I have a hard time understanding the calibration process despite it taking a whole paragraph.\n\n10.  How are table 1, 2, 3 constructed? How do authors pick methods to be summarized to the tables?\n\n11. Again, the draft should be checked thoroughly by the authors, as the current version contains a lot of typos, unfinished sentences, or empty references.", "broader_impact_concerns": "LLMs are known to have bias/fairness issues, author may consider adding discussion on how these are handled by data assessment and selection work currently.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AfZAkCklKi", "reviewer": "Reviewer_KHBS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1. The paper offers profound insights and a comprehensive analysis of data assessment and selection methods aimed at enhancing LLMs performance. The innovative approach of categorizing existing methods into three dimensions is noteworthy.\n2. The inclusion of a detailed link to related content, which categorizes the corresponding papers and provides summaries, significantly enhances the paper's impact within the LLMs community.\n3. The survey is both thorough and well-organized, with figures and structure that facilitate easy comprehension for readers.\n\n**Weaknesses:**\n\n1. Beyond the three main categories, there are algorithms that are hybrids of two categories, such as LIST, FL, DEITA, and QBIT. It would be beneficial for the authors to discuss these hybrid algorithms and their corresponding categories in a dedicated section.\n2. The paper should clarify the distinctions and relationships between the methods in these three categories.\n3. In the \"Survey Scope\" section, the authors claim that previous studies overlook the measurement of data quality. It is recommended that the authors more explicitly differentiate this survey from previous ones and emphasize the novelty of their work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a thorough survey on data assessment and selection methods in the instruction tuning of large language models (LLMs). It systematically categorizes the existing approaches into three key dimensions: quality, diversity, and importance. The paper provides a detailed comparison and analysis of these methods, contributing valuable new insights and perspectives on optimizing LLMs performance through effective data selection. Furthermore, it addresses the challenges faced in practical applications and proposes potential directions for future research.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the Weaknesses.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers unanimously agreed to accept, and agreed that the paper was well-written and easy to understand, with nice summaries of previous works. No reviewer found any 

There were some notational issues pointed out by Reviewer BZqh's careful reading, but these seem to have been resolved. Reviewer KHBS asked for the paper to clarify its novelty over previous works, and also cover hybrid methods - this was also resolved. Otherwise, no major issues.

There was a 2/3 vote on the survey certification as well - this is a high quality survey, and I agree with the decision for the certification.
Decision: Accept (poster)