CPQS-Tuning: A Model Self-Perception-Based Data Filtering Algorithm for Efficient Instruction Fine-Tuning
Authors: Yi Ren
Yanhui Li
Tianyi Zhang
Diandong Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Instruction Fine-tuning
LLMs
Data Filtering
CPQS
Hidden States
Primary area: foundation or frontier models, including LLMs

Abstract: Instruction fine-tuning is a key technique for enhancing the performance of large language models (LLMs), but low-quality and redundant data often hinder its effectiveness. Recent studies suggest that filtering a small amount of high-quality data for instruction fine-tuning can achieve faster and more efficient training performance. However, existing data filtering approaches predominantly depend on predefined evaluation models or manually designed metrics, without leveraging information from the target LLM itself. This limitation may result in a mismatch between the filtering criteria and the actual requirements of the LLM being fine-tuned, thereby reducing the effectiveness of the fine-tuning process.  To address these issues, we propose a novel perspective: the hidden states of LLMs implicitly reflect the quality of the training data. Based on this insight, we propose a novel data filtering method that extracts the hidden states that reflect the target LLM’s perception of the data as representative features, and builds a data classification model upon them, which outputs the Contrastive Perception Quality Score (CPQS) for dataset filtering. Our experiments are conducted in both general and downstream domains.
(1) In the general domain, our experiments show that training on under 10\% of the data from both the Alpaca\_GPT4 and DeepSeek-R1 synthesized reasoning datasets enables our method to outperform models trained on the complete datasets. Moreover, it surpasses the performance of current state-of-the-art data-selection techniques.
(2) In downstream tasks, our approach delivers an average performance gain exceeding 3.6\% over leading data-selection algorithms across multiple benchmarks, including GSM8K, HumanEval, and HumanEval-Plus.

Published as a conference paper at ICLR 2026
CPQS-TUNING: A MODEL SELF-PERCEPTION-BASER DATA FILTERING ALGORITHM FOR EFFICIENT IN- STRUCTION FINE-TUNING
Yi Ren
Yanhui Li
Key Laboratory for Novel Software Technology Nanjing University
Key Laboratory for Novel Software Technology Nanjing University Nanjing. Jiangsu, China yanhui iilhiilleinii mi@ejjueeeee li@nju.edu.cn
Nanjing, China
60202532001 20mma n ju. edu. cn
Tianyi Zhang
Diandong Liu
Xidian University
Shaanxi University of Science & Technology Xi an, China 221
Xi' an, China
221 11177110624Bstu..kee xidian, edu.cn
edu. on
ABSTRACT
Instruction fine-tuning is key technique for enhancing the performance of large language models (LLMs), but low-quality and redundant data often hinder its ef- fectiveness. Recent studies suggest that filtering a small amount of high-quality data for instruction fine-tuning can achieve faster and more efficient training per- formance. However, existing data filtering approaches predominantly depend on predefined evaluation models or manually designed metrics, without leveraging information from the target LLM itself. This limitation may result in mismatch between the filtering criteria and the actual requirements of the LLM being fine- tuned, thereby reducing the effectiveness of the fine-tuning process. these issues, we propose novel perspective: the hidden states of LLMs implic- itly reflect the quality of the training data. Based on this insight, we propose a novel data filtering method that extracts the hidden states that reflect the target LLM's perception the data as representative features, and builds a data classifi- cation model upon them, which outputs the Contrastive Perception Quality Score (CPQS) for dataset filtering. Our experiments are conducted in both general and downstream domains. ① In the general domain, our experiments show that train- ing on under 10% of the data from both the Alpaca_GPT4 and DeepSeck-PP R1 syn- thesized reasoning datasets enables our method to outperform models trained on the complete datasets. Moreover, it surpasses the performance of current state-of- the-art data-selection techniques. average performance gain exceeding 3.6% over leading data-selection algorithms across multiple benchmarks, including GSM8K, HvmanEval, and HumanEval- Plus.
address
downstream tasks, our approach delivers an
INTRODUCTION
Large language models (LLMs) Brown et al. 2020 Chiang et al. 2023 Yang et al. 2024a Zeng
al. 2024), such ChatGPT OpenAl 2023 Ouyang al. 2022a), have led groundbreak- ing shift in the realm of artificial intelligence in recent years. These models excel in understanding
and handling a wide array of complex language tasks. A critical factor behind their success is
struction tuning (Ding et al. 2023 Ouyang et al. 2022b [un et al. 2023 Yu et al. 2023), which enables models follow user instructions accurately and exhibit outstanding performance on mul-
tiple downstream tasks (Ren et al. (2024) Sun et al. 2025
Wang et al.) 2223a Zhou et al. (024).
"Corresponding author.

---

Published as a conference paper at ICLR 2026
During the instruction tuning process, high-quality training dataset is essential for effective fine- tuning. Early research on creating such datasets relied on expert-designed responses (Khashabi et al., 2020 Ye et al.] 2021: Wang al. 2022), but these efforts were limited by labor and cost constraints. More recent studies have used powerful teacher L.LMs to generate data (Lee et al. 2024 Li et al.) 2024a Wang et al. 2023b). The primary issue with these methods is that, in large-scale data generation, the quality of the generated data varies significantly, with both high-quality and low- quality data being produced. Zhou et al. (2023) propose the LIMA model with a new perspective to address this issue: using as few as 1,000 carefully chosen, high-quality instruction examples can substantially enhance model performance. This result suggests that developing practical algorithms to extract small, high-quality subset from large training datasets can lead to improved training outcomes.
Building on this idea, data filtering has become a popular area of research for efficient instruction
fine-tuning (Cao et al.) 2023b Chen et al. 2023a Chiang et al., [2023] Liu et al.) (024dd). On one
side, a main group of researchers has attempted use predefined reward models (Chen et al. 2024
Lu et al. 2024; Bukharin et al. 2024 Li el al. 2024b) to score data and filter it accordingly. Other
studies have analyzed data quality from multiple dimensions (Du et al. 2023] Lt 2024c Wu jtt al.) 2023] Yu et al. 2024) and selected data according to defined quality metrics. In summary, pre- vious research mainly relies on predefined evaluation models or metries for data filtering, without considering information from specific LLMs to be fine-tuned. This gap could lead to a mis- match between the evaluation criteria and the actual needs of the LLMs being fine-tuned, potentially impacting the success of the fine-tune fine-tuning process.
To address this gap, this study uses runtime information from large language models as features to enhance data representation. By working with feature vectors derived from both high-quality and low-quality data processed by LLMs, it constructs data classification model that helps select better- suited, higher-quality data for fine-tuning LLMs, making the process more effective and efficient. Specifically, our approach relies on two key ideas.
(a) Employ hidden states as LLM features: we extract the hidden states (i.e., neuron activa- tions) (Goloviznina & Kotelnikov 2024, WWng et al.] 2024a) of the target LLM as representative features, which encode the model's 's implicit evaluation of data quality. Leveraging them enables us to analyze training data quality from the LLM's own perspective.
(b) Label training data based on quality tiers: we build datasets with high-quality and low-quality labels (Wettig et al., 2024; Wen et al., (024].. enabling contrastive training that allows our CNN model, trained on LLM hidden states, more effectively capture and interpret the implicit evalua- tion differences that the L.M encodes regarding data quality.
To realize the above ideas, method is divided into the following four steps: @ We first construct an instruction fine-tuming dataset with varying performance, containing both "high-quality" and "low-quality" samples; @ then extract the hidden states of the target fine-tuning model for each instruction; @ based on these hidden states, we train a Convolutional Neural Network (CNN) model to identify whether the current testing sample is effective (i.e., of high quality) not; @ during the prediction phase, the CNN model analyzes the hidden states perceived by the LLM for each instruction, generating a prediction probability and classification result. The prediction probability classified as effective is referred to as CPQS, which serves as the criterion for dataset filtering.
Through extensive experiments, we validated that our algorithm performs excellently in data se- lection for general and downstream domain tasks. In the general domain, we tested using the dataset (Hartford & Com- putations 2025). The experimental results show that the selected data amount by our method was less than 10% of the original dataset, yet it outperformed models trained on the entire dataset. Addi- tionally, our algorithm is proven to surpass various stale-of-theaas algorithms on multiple LLMs. In downstream task domains, such as mathematical problems and programming tasks, the experimen- tal results show that our algorithm outperformed existing state-of-the-art algorithms by an average of 3.6 percentage points on benchmark tests like GSM8K, HumanEval (Chen et al. 2021). and HumanEval-Phin (Liu et al. 2024c) with the same data scale.
Alpaca Alpaca.GPT4 datasets (Taori et al., 2023) and the reasoning-deepsee
The main contributions of this paper can be summarized as follows:
2

---

Published as a conference paper at ICLR 2026
Projection onto logistie-regression hyperplane
Qua ity Rating
1
High-quality synthetic data
Low-quality synthetic data
WCX (scorc)
Figure 1: Synthetic-data scored by ALPAGA SUS
Figure 2: Quality Separation in Hidden-Star
Space: Projeticnrooecction w-x vs. PC1
Method We proposed an efficient and accurate data selection method based on the LLM's own con- trastive perception quality score, significantly enhancing instruction-tuning performance.
Study This paper presents extensive empirical studies that utilize two general fine-tuning datasets and two task-specific datasets. The results indicate that the proposed data selection method achieves optimal performance both general tasks and specific areas such as mathematics and programming.
MOTIVATION
We introduce our core idea: using LLM hidden states to extract signals of training data quality, illustrated with an initial experiment.
Recent studies show that LLM hidden states hold rich, actionable information. Xie et al. (2022) measured task specialization through variability within and between classes in hidden states; Serve- dio et al. (2025) evaluated factual accuracy from hidden-state signals; and Qian et al. 2025 used
hidden states to actively filter harmful inputs.
Motivated by these findings, we hypothesize that LLM hidden states encode data-quality signals. To test this, we conduct initial experiment and use models of different parameter scales to generate high- and low-quality data (the rationale behind this selection strategy will be discussed in detail in Section 3.Ip. Specifically,
GPT-4-produced Apaca_GPTS (Taori et al., 2023) as high- quality data, while low-quality data are obtained by regenerating samples with smaller models (c.g., Llama-3.2-1BB 1 Qwen2.5-1..3 1.een.5-1.5B---1 -Sstruutuctt). Following Chen al. (2024), we use DeepSeek- V3 (Liu et al. 2024a) to assign a quality score to each sample. As shown in Fig. I the high-quality set averages 4.73 (mostly 4.0) versus 3.73 for the low-quality set (mostly 3.0 4.5), indicating clear gap.
We select the top 500 scored samples from the high-quality set and the bottom 500 from the low- quality set. Using the last-layer hidden states of 2---225--7-B-ttuue as embeddings, we train a linear logistic regressor; stratified 5-fold CV yields AUC 1.00 (mean std), indicating linear separabilityy. For visualization, we project each embedding onto the hyperplanc normal w to obtain w-x, plot against PCI, and mark the decision boundary w-x + b = O; colors denote ground-truth labels and show clear separation along w.x (Fig. 2). Overall, the sets linearly separable in the model's hidden-state space.
Building these findings, discover that LLM hidden states differentiate high- from low- quality data. We train a CNN using hidden-state embeddings from both sets and leverage its outputs score new samples, providing prompt-insensitive evaluation of training data quality.
3

---

Published as a conference paper at ICLR 2026
(1 Dita Coastraction
Bllden States Extraction
Teddes sERKO
Generite dasa
Eabedlis
Clanr 30
LLM exditecture
Son bn EPQS Sample UX, Sample Samppe
(4) Prediction
Figure 3: Overall algorithm architecture diagram
PROPOSED METHODS
This section provides detailed description of our method. The core of our approach is to train an external CNN model by extracting the hidden states perceived by an LM on high-quality and low-quality general fine-tuning datasets for each data point. This model will be used to analyze the implicit evaluation of the training data within the hidden states of the LLM. The method consists of four key steps, shown in Figure 3
We will discuss each step in depth and analyze accordingly.
3.1 CONSTRUCTION OF HIGH- AND LOW-QUALITY DATASETS
Prior work has demonstrated that data synthesized by stronger models, such as GPT-4, tends to be of higher quality and can substantially improve the downstream performance of smaller models (Wang) al. 2023b Peng et (023). Motivated by this observation, we treat outputs from powerful mod- els (e.g., GPT-4) as high-quality data, while the low-quality set is synthesized using much smaller models. To construct the training data for the CNN, we randomly sampled 5,000 items from the A Alpaca_GPT4 dataset high-quality samples, capping the subset mitigate overfitting risks. The GPP4 dataset itself is a large-scale, general-domain fine-tuning corpus containing 52,000 entries, each represented triplet (Instruction, Input, Response), where Instruction specifies the task, Input provides auxiliary context, and Response is the SEPT-4-generater answer.
To construct the low-quality data samples, used two small LMs Llama-3.2- 3.2- 1---BBIItcct (Dubey jet al.) 2024) and 1..52..5-.5B-Istttte (Yang et al.] 2024b). Specifically, (1) from each Al- paca GPT4 entry, extracted the pair (Instruction, Input); (2) we then used the two models to generate the corresponding Response, forming new triplets (Instruction, Input, Response); and (3) we uniformly sampled 5,000 entries from each model's outputs, resulting in 10,000 items as the low-quality dataset.
Finally, we combine the high-quality and low-quality datasets to obtain the complete training set.
3.2 EXTRACTION OF HIDDEN STATES
For the collected training dataset, we concatenate the Instruction and Input parts of each entry and use this concatenated value as the "user" input to the model, while the Response part is used as the "assistant" input. This combined entry is then passed to the model to obtain the hidden states across all layers of the model. We retain only the hidden state corresponding to the Response part of each entry. This choice is made because the evaluation of the fine-tuning dataset's 's quality primarily depends the quality of the Response.
4

---

Published as a conference paper at ICLR 2026
3.3 TRAINING OF THE EXTERNAL CNN MODEL
The core idea of our algorithm is that the hidden states generated by the LLM contain an implicit evaluation of the quality of the training data. To analyze the LLM's quality assessment of the training data, we propose using an external model to interpret the hidden state vectors produced by the LLM. The external model uses a Convolutional Neural Network (CNN) architecture. We employ a simple CNN to learn information relevant to data-quality assessment from an LLM's hidden states (subsequent experiments show that this CNN performs quite well compared with simple models such as MLP), employing both 2D and 1D convolutions to capture detailed spatial and sequential patterns within the hidden-state vectors.
The CNN model further employs adaptive max pooling and fully connected layers to perform bi- nary classification, distinguishing between high- and low-quality data based on the LLM's hidden states. For efficient training, we optimize the model using the Adam optimizer with learning rate of 0.0001 and employ gradient accumulation alongside mixed precision training (AMP) to reduce memory overhead and accelerate convergence. The training process minimizes CrossEntropyLos. with periodic checkpoints saved to ensure robustness. To guarantee optimal performance, we track the loss trajectory during training and retain the best-performing model based on validation metrics.
During the training process, we use the hidden states of each sample obtained in the previous section, along with their corresponding positive and negative labels, for training. The positive-to-negatee sample ratio was set to 1:2, yielding a total of 15,000 samples. We chose this ratio because accelerates loss reduction, and with 15,000 training examples, the loss had already nearly converged. Inspired by contrastive learning, we framed the CNN's training as a binary classification task. This enables the model to distinguish between the different information perceived by the hidden states of the LLM for good and bad samples. In doing so, the CNN model can more accurately reflect the LLM's evaluation of the training data quality.
3.4 PREDICTION OF CONTRASTIVE PERCEPTION QUALITY SCORE
In the prediction phase, introduce the Contrastive Perception Quality Score (CPQS) to evaluate the training quality of each instruction corructtonnfolooiing sample. A higher CPQS indicates that the LLM assigns greater importance the data, implying better training effectiveness and higher quality. The calculation process is as follows:
1. For each instruction-followinn sample, we first concatenate the Instruction and Input the "user" input and the Response as the "assistant" input. The entry is then fed into the I.LM, from which we extract only the hidden states corresponding the Response part. These hidden states are passed into a pre-trained CNN to predict class probabilities. We focus on the probability of class 1, which indicates how likely the LLM regards the sample high-quality. This value, denoted as CPQS, serves as our data quality metric. The calculation is as follows:
CPQS: - P: = fxx),
where p; is the predicted probability for the i-th sample, representing the likelihood that the sample belongs to the positive class, and fx,) is the output of the LLM's hidden state vector for the i-th entry, processed by the CNN model to produce the probability that the sample belongs to class 1.
2. After calculating the CPQS for all samples, sort the entire dataset based on these probabilities in descending order and select the top K samples for further processing. The specific selection process is represented as:
Deelected topx (ICPQS.)N.)
where Dsalected is the set of the top K selected samples from the dataset based on their CPQS values, and topx denotes selecting the top K samples after sorting.
4 EXPERIMENTAL SETUP
4.1 DATASETS AND MODELS
We benchmark our method on four datasets- two seneral-domain and two downstream- and three 7 B- parameter open-source L.MMs.
5

---

Published as a conference paper at ICLR 2026
Training Datasets. ① General domain. (i) GPT4 (Taori al. 2023): 52,K instruc- tion-response pairs generated by GPT-4, widely used as a standard benchmark dataset; overall medium quality with relatively fluent but sometimes shallow responses. (ii) Reasoning- DeepSeek: 146,K bong-chain-reasoning samples distilled from the 300,K Dolphin-RI corpus (Hartford & Com- putations 2023) after filtering out sequences longer than 2, 048 tokens; considered high quality due to their complexity and coherence, particularly suitable for evaluating reasoning ability. @ Down- stream tasks. (i) GSM8K Cobbe et al. (2021): 7.5.K training and 1.K test elementary-mate word problems, designed to assess arithmetic and step-by-step reasoning. (ii) Magicoder-Evole Evol-Instruct- 110K (Wei et al. p024j: 110,K programming instructions covering wide range of languages and problem types, offering challenging benchmark for code generation and instruction following.
Models. (i) Ilama 2-7B (Touvron al., 2023): a base model with 7B parameters, pretrained for
general natural language understanding and generation. (ii) Llama 2-7B-Chat (Touvron et al.) 2023 a dialogue-tuned variant of I.lama 2-7B, optimized for multi-turn conversational scenarios. (iii Bren2.5-7B-Instruee
Yang et al. 2024b): instruction-tuned model designed for text and code generation, mathematical reasoning, and complex multi-step tasks, providing strong perfor- mance across diverse downstream benchmarks.
4.2 COMPARISON ALGORITHM
To validate our algorithm, we compared it with three state-of-the-art data selection methods that have received broad attention (ICLR 2024, ACL 2024, arXiv preprint):
1. ALPAGASUS: Chen et al. 2024 leveraged LLMs such as ChatGPT to automatically detect and filter out low-quality data. Specifically, it employs prompt-based scoring system where strong model evaluates instruction-response pairs and retains only those exceeding a predefined quality threshold.
2. MoDS: Du et al. (0233) proposed a data selection strategy based on the criteria of quality, cov- erage. and necessity. This approach sequentially applies reward model to extract high-qualic instances, clustering algorithms to ensure diverse instruction coverage, and target-model evaluation to select instances that address specific capability gaps.
3. Superfiltering: Li et al. (2024)) introduced a method that uses a smaller model to filter data by instruction-following difficulty before fine-tuning a larger model. By demonstrating that weak models share similar perceptions of data difficulty with strong models, this weak-to-strong paradigm significantly reduces the computational overhead of the filtering process.
4.3 IMPLEMENTATION DETAILS
We conducted our experiments on a platform equipped with two NVIDIA RTX 4090 GPUs. We adopted AoRA-based fine-tuning using the Llama-Factory framework (Zheng et al.. 2024). During supervised fine-tuning (SFT), we used bf16 6 precision, three epochs, learning rate of 5e-5, a batch size of 16, and a maximum sequence length of 2048. The LoRA scaling factor was a 8, and the rank was 16. For the deployment and inference of our model, we utilized vLLM (Kwon et al., 2023). During inference, we configured the temperature to O, maintained the precision at bfl6, and set the maximum sequence length to 2048.
4.4 EVALUATION METRICS
General Domain Evaluation Standards. We design evaluation metrics tailored to each data type. For Alpaca GPT4, we adopt three common metrics: ① Pair-wise Comparison. following Chen et al. (2024), where GPT-4o scores model outputs on Koala (1 (180), WizardL.M (218), Self-instruct (252), and Vicuna (80) across relevance and accuracy (1-10 scale), with two rounds to mitigate position bias and results categorized as Win/Tie/Loss: @ Open LLM Leaderboard, benchmarking on MMLU, ARC, HellaSwag, and TruthfulQA via the im-evaluation--haree
(Gao et al. 2024) (batch size 8);
and 3 Alpaca Eval, measuring GPT-4o win rate against text-davinci- 003 (Li et al. 2023). For the bessonnng-deepsee dataset, we evaluate GSM8K, Math_500, HumanEval, and GPQA, covering mathematical reasoning and code generation, using Im-evaluation -haneess- (Gao et al. 2024) for GSM8K and HumanEval, and EvalScope TTam 2023 for Math 500 and GPQA.

---

Published as a conference paper at ICLR 2026
Table Comparative evaluation data-sclection algorithms on GPT4 with varying sample
sizes across MMLU, ARC, TruthfulAA, HellaSwag, and AlpacaEval.
Size
Algorithm
MMLU
ARC
TruthfulQA
HellaSwag
Average
AlpacaEval
1k
Self
42.07
45.73
45.54
77.38
52.68
55.98
1k
Supertiltrrnng
41.54
46.67
44.59
77.33
52.53
55.87
MoDs
40.21
46.25
46.79
77.20
52.61
52.78
1k
Alpagasus
40.32
46.76
43.50
77.12
51.92
49.65
2k
Self
44.30
47.18
45.68
77.54
53.68
57.90
2k
Superfiltrring
42.42
47.61
45.77
77.50
53.32
57.02
MoDs
43.21
47.35
45.78
77.59
53,48
53.42
2k
Alpagasus
42,93
46.42
45.65
77.71
53,18
56,90
3k
Self
43.87
47.89
46.03
77.69
53,87
58.76
5k
Self
44.33
48.21
47.42
77.83
54.40
59.94
12k
Alpagasus
43.22
48.29
46.86
78.43
54.20
58,80
52k
Full
42.15
48.23
48.46
78.65
54.37
59.81
Evaluation Criteria for Downstream Task Domains. For downstream evaluation, target two domains mathematical reasoning and code generation. In mathematics, we use the GSM8K dataset (Cobbe al., 2021) set of 1,000 middle- and high-school arithmetic, algebra, and geom- etry problems- to measure problem-solving and reasoning skills. For code generation, we employ the 164-question HumanEval benchmark (Chen al. (22T) and its demanding extension, HumanEval-Pius (Liu et al. [2023] 2024c), which adds complex tasks to assess code accuracy, rea- soning, adaptability, and robustness across diverse inputs.
EXPERIMENTAL RESULT
5.1 GENERAL DOMAIN EVALUATION
In this section, we compare our data-selection algorithm with three state-of-the-art methods. Exper- iments are conducted Instructt to evaluate performance across models and datasets. + On Aipaca_GPT4, we train with subsets of 1K, 2K, and larger sizes, evaluating on MMLU, ARC-Challenge, TruthfulQA, HellaSwag, and AlaacaEval (Table I @ On Deepon,ng--DeepSeek, we use 10k, 20k, and 50k samples, bench- marking against the base model, Superfiltering, and ALPAGASUS; MoDS fails on larger subsets. Results are summarized in [ables
Dipaca_GPTT with Llama2-7B and Reasoning-DeepSeek with Qwen2.5-
Our algorithm outperforms state-of-the-art methods and achieves better results than full- dataset training the Aipaca_GPTV dataset with Llama2-7B, using less than 10% of the data. Table [ presents a performance comparison between our algorithm and other methods on models trained with filtered subsets of varying sizes from the Alpaca. GPT4 dataset. As shown in (data size = 1k), our approach consistently outperforms all competitors in
the first part of Table [ the low-data regime. With only 1k training instances, it attains macro-average accuracy of 52.68% over the four public benchmarks and an AlpacaEval win-rate 55.98%, exceeding the strongest baseline by 0.15 and 0.11 percentage points, respectively. As illustrated in the second part of Ta- ble (data size = 2k), doubling the budget to 2k further lifts the macro-averas to 53.68% and the AlpacaEvall score 57.90 57.90%, while still maintaining clear margin over all baselines. We addi- tionally conduct pairwise preference tests with GPT-40 on Vicuna-style prompts: the comparison in Figure shows that our 1k model already surpasses the model trained on the full 52k corpus. To assess scalability, we increase the subset size to 3k and 5k. As shown in the third part of Table performance improves steadily: 5k, the open-domain average reaches 54.40%, and the AlpacaF-- val win-rate climbs to 59.94%. Notably, the 5k model outperforms the Alpagasus ipagasuu-filtered filtered 12k model and surpasses the full 52k model on every metric, confirming the superior efficiency of our selection criterion.
Our algorithm also outperforms existing methods on the Reasoning-DeepSeee dataset with the Qwve22.577B-Instruee model, achieving better performance using less than 10% of the data compared full-dataset training. As shown in Table 2 despite the generally higher quality of the Reasoning-DeepSeed dataset, existing algorithms fail achieve outstanding performance, while

---

Published as a conference paper at ICLR 2026
(a) Self vs Superfiltering
Self MoDs
Vicuna
Vicune
T
Supiriikring wind
MoDu wiro
WoardLM
111
(c) Self vs Alpagasus
(d) Self (IK) vs Alpaca dataset (52K)
Seiwir
Vitura
Viture
Rowl
Kcel
WesroMM
107
WosrtM
Figure 4: Performance Comparison of Data Selection Methods on the Llama2-7B Model Using the Alpaca.GPT4 Dataset
Table 2: Performance of models trained with data filtering on the Reasoning- dataset
across GSM8K, Math 500, Hvm, and GPQA benchmarks.
Size
Model
GSM8K
Math_500
HumanEval
GPQA
Average
Base
76.27
73.40
64.02
30.30
61.00
10k
Self
85.37
72.40
67.68
36.36
65.45
10k
Superfiltering
81.43
70.40
62.20
31.82
61.46
20k
Self
85.14
76.20
64.02
34.85
65.05
20k
Superfiltering
76.19
71.60
63.41
28.79
60.00
50k
Self
84.99
74.20
64.02
36.87
65.02
113k
Alpagasus
84.84
66.52
64.63
28.18
61.04
146k
Full
85.06
70.60
57.32
30.71
60.92
our method consistently leads. Specifically, the model trained on our 10K filtered subset achieved an average score of 65,45 across mathematical and coding reasoning tasks, surpassing models trained on Superfitterng's 10K and 20K datasets, Alpagasus's 's 1 13K dataset, and even the full 146K dataset. Notably, the performance on Math Math initially declined compared to the base model. We attribute this to a limited maximum generation length setting of 8K tokens; our additional experiments con- firmed that increasing the token limit enhances performance significantly. Moreover, we observed that increasing the size of the filtered training dataset beyond certain point did not yield substantial improvements. For example, the model trained on the 10K dataset outperformed both the 50K fil- tered dataset and the full 146K dataset, with the 10K model surpassing the full 146K-trained model by 3.48 points. This indicates potential noise and redundancy in the full dataset, underscoring the effectiveness of our targeted filtering approach.
In addition to the evaluations mentioned, we applied our algorithm to select 10k samples from the 930k Tului--FT-Mixxure dataset (Appendix A.2), where it continued outperform competing methods. We also explored its performance under full-parameter fine-tuning, with results in Ap- pendix A. confirming its superiority. The iterative nature and performance-ifflunccin factors of our method are analyzed in Appendices A.3 and A.4 Further, we demonstrate the generalization of our method across different LLM sizes in Appendix A.5 compare its performance with models using different architectures for hidden-un indden-state state extraction in Appendix A.6 and present a comparison of performance and efficiency with other methods in Appendix A.7
5.2 DOWNSTREAM TASK EVALUATION
We evaluated our algorithm on downstream-tar datasets using Llama2-7B-CIl and Qwen2.5-7B. Instruct. Following the practice of fine-tuning pre-optimized models, tested GSM8K (Cobbe
8

---

Published as a conference paper at ICLR 2026
Table 3: Performance Evaluation of Models Trained with Different Data Selection Methods on the
GSM8K Dataset in the Mathematics Domain
Model
Original
Self
MoDs
Alpagasus
Superfiltering
Full
Llama 2-7B-Chat
24.56
25.25
23.05
23.12 76.27
17.06
35,56 69.83
Qwen
76.27
81.12
79.45
76.80
Table 4: Performance Evaluation of Models Trained with Different Algorithms on Llama 2-7B-Chat in the Code Domain (pass @
Original
Self
MoDs
Alpagasus
Superfiltering
EvaanEval
13,4
16.5
12.2
12.2
10.0
HumanEval-Pinn
11.6
13.4
10.4
9.1
Average
12.5
14.95
11
11.3
9.55
al. 2021) (math) and Magicoder-Evool-nstruee 10K (Wei et al. 2024 (code), filtering them to 500 and 1,000 samples respectively. Performance was assessed on GSM8K test, HumanEval, and HumanEval-Pluss For ALPAGASUS, GPT-4o-mini was used for data selection due to its higher efficiency and lower cost.
Our algorithm outperforms other algorithms in the field of mathematics. Table presents re- sults the GSM8K Dataset, where algorithm outperforms the others by 4.17 points Llama 2-7B-Chat and 3.61 points on Qwen 2.5-7B- IIIsrrucc.. Notably, the model trained on the full GSM8K dataset underperforms
Qwen 2.5-7B II-Instruct (score 76.27), while the model trained 500 lected data points achieves 81.12, surpassing the full dataset by 11.29 points. This demonstrates the effectiveness of algorithm in the field of mathematics.
Our algorithm outperforms other algorithms in the field of code. Tables and show re- sults with the Magicoder-Evol Eno---ossteett-ll---- outperforms others by 4.3 points on average, and models trained on 1000 data points from other algorithms performed worse than the original. In contrast. the model trained on 1000 points selected by our algorithm improved by 2.45 points. On Qwen 2.5-7B- Instruct (Table the model trained with our selected data outperformed others by 1.47 points on average. However, all models trained on the filtered Magicoder-Fivol-Inee its lower quality for this model. This demonstrates the effectiveness of our algorithm the code domain.
dataset. On Llama 2-7B-Chat (Table 4 our algorithm
10K dataset showed performance degradation, likely due
5.3 THE IMPACT OF DIFFERENT HIDDEN LAYERS ON MODEL PERFORMANCE
To investigate the impact of different hidden layers on data selection performance, we trained sepa- rate CNN models using hidden states from various layers of Qwen 2.5-7t--nntrrut. Specifically, extracted hidden states from the first 9 layers, the middle 9 layers, the last 11 layers, and the final layer. We then evaluated the filtering performance of these layer-specific models on the GSM8K dataset (Cobbe et al., 2021> (mathematics) and the
11K dataset (Wei
al. 2024) (code).
Utilizing hidden states from all layers yields the optimal data filtering performance. Table presents the comparison of CNN models trained on different hidden layer segments. Across both mathematical and coding tasks, the model trained with hidden states from all layers (Full) consis- tently achieves the highest performance. When utilizing subset layers, the optimal segment varies by task. On the GSM8K dataset, the early layers (first 9) perform best among the subsets with a score of 84.23. Conversely, in the code domain (evaluated via HumanEval and HumanEval- Pluss, the later layers (last 11) demonstrate superior subset performance, achieving average score of 76.55. These results indicate two main points: first, while later hidden layers capture critical task -pecific information (such for code generation), different tasks may rely on distinct layer representations; second, utilizing any individual subset layers still lags behind the comprehensive representation provided by all layers combined.

---

Published as a conference paper at ICLR 2026
Table 5: Performance Evaluation of Models Trained with Different Algorithms on Qwen 2.5-7B- Instruct in the Code Domain.
Original
Self
MoDs
Alpagasus
Supertiltering
EvaanEval
82.9
80.0
78.7
78.0
79.3
HumanEval- Eval-Plus
75.6
74.4
72.6
72.6
73.2
Average
79.25
77.2
75.65
75.3
76.25
Table 6: Comparison of CNN Model Performance Trained on Different Hidden Layer Parts of Qwen 2.5-7B-Instruch
Dataset / Metric
Full
Early (9)
Middle (9)
Last (11)
Final (1)
GSM8K
84.91
84.23
83.85
83.70
83.70
HvmanEval (pass@1)
80.0
75.0
77.4
79.3
78.7
HumannEal-PPie (pass@1)
74.4
70.1
71.3
73.8
72.6
RELATED WORK
Data Selection Strategies. The goal of instruction tuning (Liu al., 2024b Longpre et al. 2023
Sanh et al.] 2022 Wei et al. 2022) is to help LLMs better understand human task requirements. Early research primarily focused on building large-scale instruction datasets, but studies like LIMA have shown that only a small amount of high-quality data is needed during instruction fine-tuning achieve good results. Existing data selection methods be classified into four categories: indicator based methods, trainable LLM-based methods, powerful LLM-based methods, and small- model-based methods.
Indicator-based methods use metric system to identify multiple evaluation indicators to compre- hensively assess data quality (Cao et al. 2023a] Wei et al., 2023). Trainable LLM-based meth- ods treat the LLM as a trainable data selector, processing and assigning scores to each instruc- tion fine-tuning data for selection (Chen et al., 2023b et al., 2024c). Powerful LLM-based ap- proaches, such those using models like CharGPT, typically design prompt templates and leverage the model's capabilities to quantitatively evaluate the quality of data samples (Chen et al.) 2024| Liu ett al.) 2024d). Finally, amall-model-bias bbasdd methods involve using external small models to score the data or projecting the data samples into vectors with a small model for further processing and selection (Chen al. 2023a: Li et al. 2024b).
Performance Evaluation of LLMs. The evaluation of LLMs is typically done through automatic evaluation, human evaluation, and using LLMs evaluators. Automatic evaluation relies pre- defined criteria and quantitative assessment (Chen et al.) 2021: Hendrycks et al.) 2021 Wang et al.] 20245). Human evaluation focuses qualitative aspects like clarity, consistency, and factual aceu- racy, and is essential for quality assessment (van der Lee et al. 2021 Zheng et al. 2023). However, due its time and labor demands, using powerful LLMs (Chen et 2024 Huang al. 2024 to evaluate other LLMs has become popular approach in recent years.
CONCLUSION
This paper addresses the issue of low-quality and redundant data in LLM instruction fine-tuning. Based on the hidden states that reflect the target LLM's perception of the data, we build a data classification model and define CPQS as its output. Using CPQS as the criterion, our method filters high-quality data subsets, thereby improving the efficiency and effectiveness of instruction fine- tuning.
Experimental results show that our approach achieves superior performance with less than 10% of the original data compared to models trained on the full dataset. It also outperforms existing data selection techniques at equal data scales. In downstream tasks, such as mathematical problem solv- ing (GSM8K) and programming (HumanEval, HumanEval+), method provides 3.6% average performance improvement over current state-of-the--eeeee data selection algorithms. The code and data have been provided online at htt.pp///ggtttb..cmm//rrl/////// Tunnisgg
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors thank anonymous reviewers for their time and invaluable feedback to improve our study. This work is supported by the National Natural Science Foundation of China (Grant Nos. 62172202, 62272221, and 62572226).
REFERENCES
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Gretchen Krueger. Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse. Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL htt.ttttss/prrccedingge nww................. CC/ wttppp/pp////////aa/./// (15570005510444477771.11.B.aaa.
html
Alexander Bukharin, Shiyang Li, Zhengyang Wang, Jingfeng Yang, Bing Yin, Xian Li, Chao Zhang, Tuo Zhao, and Haoming Jiang. Data diversity matters for robust instruction tuning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Compu- tational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 3411- 3425. Association for Computational Linguistics, 2024. URL httts::/aaclanthollo or2/ findings wwwinngs-eenllp.. 195
Yihan Cao, Yanbin Kang, and Lichao Sun. Instruction mining: High-quality instruction data lection for large language models. CoRR, abs/2307.066900, 2023a. doi: 10.48550/ARXIV.23077 06290. URL httpp:::/aol..rr//////// 48550/48555//4
2307. 06290
Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. Instruction mining: Instruction data selection for tuning large language models. arXiv preprint arXiv:2307.06299, 2023b.
Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yifan Yanggong, and Junbo Zhao. Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning. CoRR, abs/2305, 092446, 2023a. doi: 10:4855/ARXXV.22350.222 URL 178550/aaxi110 2305 09246
https:://oi..org
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang. Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training bet- ter alpaca with fewer data. In The Twelfth International Conference on Learning Represen- tations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https: 7 //openreview. net htt///orrmmmii dVXgsJhvz
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavar- ian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plap- pert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji. Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Pe- ter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374 2021. URL https: www.xxvv.v.rrraoooo aas/ attp://bb//100.....0. 03374
Yongrui Chen, Haiyun Jiang. Xinting Huang, Shuming Shi, and Guilin Qi. Tegit: Generating high- quality instruction-tuning data with text-grounded task design. CoRR, abs/2309.054.4 2023b.
11

---

Published as a conference paper at ICLR 2026
doi: 10.4850/ARRXIV2330.00544 http:pps//e.....g/11.4444433 05447
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality. See https://vicuma Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168. 2021. URL
https::/aaxxi org/ hwagg//b//2111 22100.110.. 14168
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high nigh-quality instructional conversa- tions. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, De- cember 6-10, 2023, pp. 3029-3051. Association for Computational Linguistics, 2023. doi: URL
10.18653/V1/0233.EMM emrlp main 183
Qianlong Du, Chengqing Zong, and Jiajun Zhang. Mods: Model-oriented data selection for in- struction tuning. CoRR, abs/2311.15653, 2023. doi: 1044550ARXKI2311.556. URL
48550/ arxiv. 2311 15653
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloc Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Ander- son, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Ko- revaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lec, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang. Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Al- wala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The Ilama 3 herd of models. CoRR, 0tbb/22477.21775 2024. doi: 10.48500ARXI.2200..211 21783
URL
huttps://doi. org. 48550
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter. Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lin- tang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 07 2024. httpsttt:: 12608602
org/ records/
Valeriya Goloviznina and Evgeny Kotelnikov. I've got the "answer"! interpretation of Ilms hidden states in question answering. In International Conference on Applications of Natural Language to Information Systems, pp. 106-120. Springer, 2024.
Eric Hartford and Cognitive Computations. Hugging Face Dataset, 2025. URL
Dolphin R1: R1-Style Reasoning Dataset.
https: attps://hugqinn. http:///gggggggggaaaceecc co datasets/
2p0 licensed, 800000 samples: 300 DeepSeek-RI reasoning, 300 Gemini 2.0 flash thinking, and 000 000 Dolphin chat.
coonniiivve weegnittivecoopuraaiiioo hatps//olppinn22
12

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Confer- ence on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021.
net. umr id=dd jm 3mQ
Hui Huang,
Yingqi Qu, Jing Liu, Muyun Yang. and Tiejun Zhao. An empirical study of Ilm- as-a-judge for LLM evaluation: Fine-tuned judge models are task-specific classifiers. CoRR, doi: I04850AARXX.240000255.. URL htttp:://doo org//c. 48550/arXis 02033.0839
abs/2403.02839. 2024.
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. Unifiedqa: Crossing format boundaries with single QA sys- tem. In Trevor Cohn, Yulan He, and Yang Liu (eds.). Findings of the Association for Com- putational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACI, pp. 1896-1907. Association for Computational Linguistics, 2020. doi: 10. 1/020/1//220 BAN.DDD..FFNDIINNGGEEELL.LIIIT. EM.LP.111. URL htsps: orr/10. 18653/
v1 v1/2020. findings emnlp 171
Woosuk Kwon, Zhuohan Li. Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Jason Flimn, Margo I. Seltzer, Peter Druschel, Antoine Kaufmann, and Jonathan Mace (eds.), Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz. Germany, October 23-26, 2023, pp. 61 1 626. ACM, 2023. doi: 10.1 3600006.3613165 URL Uttps:ts:///oi. org 10. 100111114430000 3613165
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipalli, Michael W. Mahoney, Kurt Keutzer, and Amir Gholami. LLM2LLM: boosting I1ms with novel iterative data enhancement. In Lun-Wei Ku, Andre Martins. and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pp. 6498-652 Association for Computational Lin- guistics. 2024. doi: 10. 18653/v1/2000 http///222222 findingss-aeee acl. 388
htts:::: //doi. org/10.
Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. Selective reflection- tuning: Student-selected data recycling for LLM instruction-tuning, In Lun-Wei Ku, Andre Mar- tins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11 11-16, 2024. pp. 16189-16211. Associa- tion for Computational Linguistics. 2024a. doi: 10. 10.18653/V 1AASS/////OAAAIIIIINSAAAAAA
URL
https: //ooi or.grrrgg/.oo... / 10 163./6551lvv. 2024 f ind i ngs acl. 958
Ming Li, Yong Zhang, Shwai Hc, Zhitao Li, Hongyu Zhao, Jianzong Wang. Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. 14255-14273. Association for Computational Linguistics. 2024b. doi:
URL https: wt.poo/......... 186533
v1 v1/2024. acl ong. 769
Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning. In Kevin Duh, Helena GómezzAAorno, and Steven Bethard (eds.). Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pp. 7602-7635, Association for Computational Linguistics, 2024c. doi: 10. 11186663/112222AAAAALLLL. 10. 10.18653/v1/2022 http://0222..aaaloooooo.o..
421
Xuechen Li, Tianyi Zhang. Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of cnatrucctio-followiin models. hhttps:///i wttph//......ttttaaaa////////
paca__vaa 5 2023.
13

---

Published as a conference paper at ICLR 2026
Yunshui Li, Binyuan Hui, Xiaobo Xia, Jiaxi Yang, Min Yang, Lei Zhang, Shuzheng Si, Ling- -Hao Chen, Junhao Liu, Tongliang Liu, Fei Huang, and Yongbin Li. One-shot learning as instruction data prospector for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACI. 2024, Bangkok, Thailand, August 11-16, 2024, pp. 4586-4601. Association for Computational Linguistics. 2024d. doi: 1/216633/12224442--LLL long. 252
URL httpt::://doo.org//10.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Ya- coob, and Dong Yu. MMC: advancing multimodal chart understanding with large-scale instruc- tion tuning. In Kevin Duh, Helena Gómez-Adomo and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguis- lies: Human Language Technologies (Volume I: Long Pupers), NAACL. 2024, Mexico City, Mex- ico, June 16-21, 2024, pp. 1287-1310. Association for Computational Linguistics. 2024b. doi: 10066333/2222NNNAALLLLLLLL naacl- Monc.. 70
URL hutpps://dbl wwg. http:/....1153/111111
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https:// 0Cu7
openreview. net/ featt//orummidddde
Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. Evalu- ating language models for efficient code generation. In First Conference on Language Modeling, 2024c. /tLL/tttpss:/pperevviiee http///peeeeeeeeeeeeeeeee net ddt IBIBCMMeA
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? A comprehensive study of automatic data selection in instruction tuning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.ned 2024d. URL htttps://openreeeee BTKAeLqLMW
net fooummiiii
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. The flan collection: Designing data and methods for effective instruction tuning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). International Confer- ence on Machine Learming, ICML. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. www.mmrr.r..ss......mmmm. press w202/ 2eto2//ongpre2330 html
PMLR, 2023. URL
pettps://proeediie
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. #instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL
net/
URL Forum?id=pszewtye
OpenAl. technicll report. CoRR, htt.aaab.//333//////444 2023. doi: 1048855/ARXXIV22033.8777
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022a.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida. Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano,
14

---

Published as a conference paper at ICLR 2026
Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feed- back. In Sanmi Koyejo. S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- fion Processing Systems 2022, NewrlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022b. URL http: httpthhht/pppppeteatt. nis. wwtpppp.cpppppppppppppppppppeeep fi www.eeeeepppe. wetlee/pappeee https/paprr//22//aaaaee wwt.:taaattttooeeeeeeeeeeee html
Baolin Peng. Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with GPT-4. CoRR, abs/2304.032777, 2023, doi: 10448550/ARXIV.2300005 URL https:
7/ //doi.org//nn 2ttp//10.48555////x1111 03277
Cheng Qian, Hainan Zhang, Lei Sha, and Zhiming Zheng. HSF: defending against jailbreak attacks with hidden state filtering. In Guodong Long, Michale Blumestein, Yi Chang. Liane Lewin- Eytan, Zi Helen Huang, and Elad Yom-Tov (eds.), Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025 2 May 2025, pp. 2078- 2087. ACM, 2025. doi: 10.114/3701716337776633 URL https: /ttp::://cci.ooo
wwg.doi..orrrggooooo org/ 1145/
7711776.
Yi Ren, Tianyi Zhang, Xurong Dong, Weibin Li, Zhiyang Wang, Jie He, Hanzhi Zhang, and Licheng Jiao. Watergpt: Training a large language model become hydrology expert. Water, 16(21): 3075, 2024.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang. Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. In The Tenth International Conference Learning Representattions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Giovanni Servedio, Alessandro De Bellis, Dario Di Palma, Vito Walter Anelli, and Tommaso Di Noia. Are the hidden states hiding something? testing the limits of ties in llms. Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguis- tics (Volume 1: Long Papers). ACL 2025, Vienna, Austria, July 27 August 1, 2025, pp. 6089- 6104. Association for Computational Linguistics, 2025. URL htttps:/acclanthol.. org/2025. acl- 304/
capabili-
Lianshan Sun, Diandong Liu, Maoxue Wang, Yongyi Han, Yanqing Zhang, Biwei Zhou, Yi Ren, et al. Taming unleashed large language models with blockchain for massive personalized reliable healthcare. IEEE Journal of Biomedical and Health Informatics, 2025.
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David D. Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language mod- els from scratch with minimal human supervision. In Alice Oh. Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.). Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL orarapaaehaaaeaaaaoooooaooorrrrr (166611110091a333322222222131110
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An cnaruuctionfolloviinn llama model.
httts:://ithhn com wtt.a//ttttttttttaaa/aaa/aaaaaaa staabsstafford allacaa 2023.
ModelScope Team. Evalscope: Open-source evaluation toolkit for 1lms. https://githan www.:oddellsoooeeeeeellloloees 2023. GitHub repository.
15

---

Published as a conference paper at ICLR 2026
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee. Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan. Binh Tang. Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307 092888 2023. doi: 10448550ARRXIV.330...2222
URL
https://doi. www.ooggo......... 48550
09288
Chris van der Lee, Albert Gatt, Emiel van Miltenburg, and Emiel Krahmer. Human evaluation of automatically generated text: Current trends and best practice guidelines. Comput. Speech Lang., 67:101151, 2021. doi: 10.
URL
p.cs1.2020, 101151
Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. Huatuo: Tuning Ilama model with chinese medical knowledge. CoRR, habs/2304.06975. 2023a. doi: 10. 48550/AARXIV2334000777
230.. 06975
Yiming Wang, Pei Zhang, Baosong Yang, Derek F Wong, and Rui Wang. Latent space oh embedding enables output-free Ilm self-evaluation. arXiv preprint arXiv:2410. 13640, 2024a.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap. Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Ander- son, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal. Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen. Superr-naturalinstruutiio
Generalization declarative instructions on 1600+ NLP tasks. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.). Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 5085- 5109. Association for Computational Linguistics, 2022. doi: 10.18653/V1/2002.. 22A.00222..NNPPPPAAAAA 340. BtLLhttss:// //ttpp:://dd...o wrg/eogg//oo.. 186.3.1663/1880 202222222222222 emnlp main 340.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan L. Boyd-Graber. and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 14, 2023, pp. 18484-13500 Association for Computational Linguistics, 2023b. doi: 10.
URL ht :t org. /10. 186537
v v1/2023. Long. 754
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. CoRR, 0ao/22406001574, 2024b. doi: 1048550/ARXIV244600157777
URL / /tt.:/...........//////////..//. 4/550//02855//xxx11 www./////w............. 01574
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representattinss, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https://///......... htttps://openzeviies net forum?id= gE2rGCozdgR
16

---

Published as a conference paper at ICLR 2026
Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. Instructiongpt-4: A 200-instruction paradigm for fine-tuning minigpt-4. CoRR. abs/2308. 12067, 2023. doi: 10.48550ARXXI2230.22000 hRLthtts://doo.coo
405.00//55500..2....// 2308 12067
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Em- powering code generation with oss-instruct. In Forty-first International Conference on Ma- chine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL
net for https://openrevite http://rwwww...eeooooooooooooooo
Xiaoyu Wen, Chenjia Bai, Kang Xu, Xudong Yu, Yang Zhang, Xuelong Li, and Zhen Wang. Con- trastive representation for data filtering in cross-domain offline reinforcement learning. arXiv preprint arXivv2405.06119, 2024.
Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. Qurating: Selecting high- quality data for training language models. In Forty-first International Conference on Ma- chine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL
net httpss://opereeview oppprevieweeeeeeeeeeeeeeee
https: /
Shengguang Wu, Keming Lu, Benfeng Xu, Junyang Lin, Qi Su, and Chang Zhou. Self-evolved diverse data sampling for efficient instruction tuning. CoRR, abs/2311.08182, 2023. doi: 10. 48550/ARXIV.2311.0 1.08182. URL httthhtts//ddee /ttp:///oo...r/..////// org/10. 48550/arXX.. 2311.08182
Shuo Xie, Jiahao Qiu, Ankita Pasad, Li Du, Qing Qu, and Hongyuan Mei. Hidden state variability of pretrained language models can guide computation reduction for transfer learn- ing. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.). Findings of the Associ- ation for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, De- cember 7-11, 2022, pp. 5750-5768. Association for Computational Linguistics, 2022. doi: 18663//31..166111
10.1863/N1/2022 FINDINGS BEANADINNG--ENLPP22.. EMNLP.22. URL 2022. www.iiinnnnnnnnninn 422
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li. Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li. Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report. CoRR, 0bs/22407.106777 2024a. doi: 10.48550/ARXIV.2200 10671. URL https: doi. org/10. 48150/04555/rraa. 4255100.455//2xx1222222 10671
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15111 2024b.
Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. Crossfit: A few-shot learning challenge for cross- task generalization in NLP. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Nat- ural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 7163-7189, Association for Computational Linguisties, 2021. doi: http:///6o...rr/......./////////
10.18653/V1/2021. http://112222112MPPPP MMMNLP-MAIN.S URL emx.ppmainn.72
Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J. Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large language model as attributed training data generator: A tale of diversity and bias. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL /ttpttpppppppapp..ppptpp nipss.ippp... cc/ paper wawpaper_eleee/aaeeeaaeaeeaeeeea wwwes/ paper/ ae 00222//4//2299000550000020022222 and Benchmarkss html
17

---

Published as a conference paper at ICLR 2026
Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang. Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. Wavecoder: Widespread and versatile enhancement for code large language models by instruction tuning. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACI. 2024, Bangkok, Thailand, August 11-16, 2024, pp. 5140-5153. Association for Computational Linguistics, 2024, doi: doi. 10. 18653/v1/ w1//0//.2222222222 2000o..cl-oon.22555 1ong.
URL https://
Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin. Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang. Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao. Shuxun Yang. Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang. Xixuan Song. Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang. Yueyan Li, Yushi Bai, Yuxiao Dong. Zehan Qi. Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. Chatglm: A family of large language models from 1-000B to M-4 all tools. CoRR, abs/2406. 12793, 2024. doi: 10.48550/ARXIV22466 12793. http::::: /oo. wog.rrrgg..... 10. 48550/ .2w..244000 2406. 12793
Liammin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang. Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging Ilm-as-a-judge with mt-bench and chatbot arena. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurlPS 2023, New Orleans. LA, USA, December 10 http:/ppee_.feeepppeeeeeeeeeeeee and
16, 2023, 2023. URL http: /ttp:::/pppers..eoo nips cc wwtpppppp.ppppp..appeeeeppceeeee wwweeffilesppppppe
Benchmarks.her html
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguis- tics. UtLpttt:
.rq/ abs 2203. 12403.13377 13372
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettle- moyer, and Omer Levy. LIMA: less is more for alignment. In Alice Oh, Tristan Nau- mann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference tion Processing Systems 2023, NeurIPS 2023, New Orleans, IA, USA, December 10 16, 2023, 2023. URL http: http:::/papps....... n ps cwtpaper_ppepeeeeeeeeeeeeeee 1 welleesepeeeee http://pppr//222////mm 3c662674829e440066
Neural Informa-
Abst Con Coneerctt
Zhi Zhou, Jiang-Xin Shi. Peng-Xiao Song, Xiaowen Yang, Yi-Xuan Jin, Lan-Zhe Guo, and Yufeng Li. Lawgpt: A chinese legal knowledge-ehhaneed large language model. CoRR, abs/2406.04614, 2024. doi: 0044850/ARXX1..240004000 URL. https: : ///.i wtt.:.0
24C6. 04614
A APPENDIX
A.1 FULL-SCALE FINE-TUNING EXPERIMENT
At the Ik and 2k data scales, we performed full-parameter fine-tuning on the datasets produced by each selection method to assess their performance under more exhaustive training. The setup employed two H20 GPUs, a learning rate of le-5, a batch size of 64, and three training epochs. Results are summarized in Table
After full fine-tuning, every method achieved noticeably higher scores on all five benchmarks than with LoRA-only tuning, demonstrating that updating the entire set of model weights can further
18

---

Published as a conference paper at ICLR 2026
Table 7: Comparative evaluation of models trained with different data-sclection algorithms and
sample sizes (1k vs. 2k) on five benchmarks.
Size
Model
MMLU
ARC
Truthful-QA
HellaSwag
Avg.
AlpacaEval
Ik
Self
41.95
47.87
50.92
77.57
54,58
65,93
1k
Superfiltering
41.76
47.78
51.05
77.56
54.54
64.36
1k
MoDs
41.43
47.53
48.46
76.82
53.56
57.02
1K
Alpagasus
39.76
47.87
$2.08
77.54
54.31
55.14
2k
Self
42.06
48.25
53.47
77.66
55.36
69.73
2k
Superfiltering
42.14
48.63
51.71
77.97
55.11
69.69
2k
MoDs
39.94
46.67
53.10
77.67
54.34
54.10
2k
Alpagasus
41.66
47.78
51.42
76.66
54.38
64.16
unlock data value. Notably, the dataset selected by our algorithm led both average score and AlpacaEval both the Ik and 2k scales, confirming its clear performance advantage.
A.2 ADDITIONAL EXPERIMENTS DATASET FILTERING EFFECTIVENESS
We conducted additional experiments on the Llama2-7B model using the TULU-3-SFT-mixtur dataset, comparing results from filtering 930K samples down to 10K samples. As shown in Table our method outperforms other algorithms on both open- large model evaluation benchmarks and AlpacaEval (notably, the MoDs algorithm was excluded from comparison due to GPU memory limitations with larger sample sizes). Our approach achieved an average score of 53.27 on the open- ended evaluation benchmarks and 52.37 on AlpacaEval.
Table 8: Comparative Evaluation of Llama2-7B Models Trained with Different Data Selection Al- gorithms on TULU-3-SFT-mixte (Filtered to 10K Samples) Across MMLU, ARC (Challenge), TruthfulQA, HellaSwag, and AlpacaEval Benchmarks.
Model MMLU
ARC
TruthfulQA
HellaSwag
Avg.
AlpacaEval
Self
44.54
45.05
46.42
77.08
53.27
52.37
Superfiltering
42.98
45.56
43.49
76.87
52.23
45.90
Alpagasus
42.95
46.44
44.42
76.39
52.55
32.19
We further evaluated pairwise model comparisons using AlpacaEval to measure the win rates be- tween different approaches. demonstrated Table 9 our method shows significant advantages over other dataset filtering techniques.
Table 9: Pairwise Model Comparison Results (AlpacaEval)
Model Comparison
Win Rate (%)
Loss Rate (%)
Self Superfliting 52.25
47.74
Self vs Alpagasus 68.36
31.49
A.3 EXPERIMENT ON ITERATIVE MODEL TRAINING AND DATA SELECTION
We conducted experiments to assess the iterativeness of our algorithm. First, we trained CNN model using our algorithm the Qwwn2.5-7BB-nsttuee dataset. This trained model was then used to predict and rank the Alpaca_GPT4 dataset. Based on the ranking, we extracted the top 5,000 and the last 10,000 data samples. We then retrained another CNN model using this subset to further filter 1,000 samples from the Alpaca GPT4 dataset for additional training. As shown in Figure the performance of the newly trained model demonstrated significant improvement over the previous version. For example, the green sections in the figure represent the number of wins by the newly trained model, which show clear advantage across multiple datasets.
19

---

Published as a conference paper at ICLR 2026
Self1.0
Vicuna 13
20
Tie
Selr2.0 wins
Koala
Wiza MLM
98
Self-Instruct 42
113
97
Figure 5: Comparison of Model Performance After Two-Stage Data Selection.
A.4 PREFERENCES OF DIFFERENT MODELS FOR HIGH-QUALITY DATASETS
In this section, we explore whether different LLMs have distinct preferences for high-qualig, datasets. To this end, we trained the models on high-quality datasets selected from the GSM8K dataset and Magicoder_ Bertoodeer_vvlo_nnne Lyvo__nstrruut11100
Dataset using Llama 2-7B-Chat and Qwen 2.5-7B- Instruct. We then compared performance of these models when exchanging datasets. Specif- ically, we trained Qwen 2.5-7B- InIntrucct and Llama 2-7B- compared their performance with training on their own selected datasets.
each other's ' selected datasets and
As shown in Tables and after swapping datasets did not perform as well as those trained on their original datasets. For both Llama 2-7B-Chat and Qwen 2.5-7B-Instruct, the high-quality dataset considered by one model did not yield the same results when used by the other model. Therefore, our experiment shows that different models exhibit significant differences in selecting high-quality datasets, with each model having its own definition of what constitutes a thigh-quaiity dataset."
and IT whether in the mathematical or coding domains, the models trained
Table 10: Performance of Llama 2-7B-Chat and Owen2.557BB-Insiee Models Trained on Their Own and Swapped Datasets on the GSM8K Dataset (Cobbe al. 202T).
Training Method Llama 2-7B-Chat Qwen2.5-7B-Instree
Self Training Dataset Swapping
25.25
84.91
23.58
83.24
Table 11: Performance of Llama 2-7B-Chat and Qwen2.5-7B-Instruet Models Trained Their Own and Swapped Datasets on the HumanEval Dataset.
Training Method
HumanEval pass@1
HumanEval-Plus pass@1
Average
Llama 2-7B-Chat (Self)
16.5
13.4
14.95
Llama 2-7B-Chat (Swapped)
11.2
11.0
11.1
Qwen2255-BB-Instruue (Self)
80.0
74.4
77.2
Qwen2.5-7B-Instree (Swapped)
72.0
67.1
69.55
Furthermore, we explored whether large models of different sizes have distinct preferences for high- quality datasets. To this end, we selected larger model (such Qwen 2.5-32BBInstrrue and a smaller model (such as BirCPM4-1B), and conducted experiments on the Seasoning-DeepSSer dataset. Specifically, swapped the datasets selected by different models, retraining the models and evaluating their perfor- mance differences.
trained CNN for each model to predict and filter the dataset, and then
The experimental results, as shown Table 12 indicate that the models performed better when using their selected datasets. This true across the GSM8K, Math_500, HuanEval, and GPQA datasets, where the models consistently achieved better average scores. This further validates the differences in how each model defines and selects "high-quality data."
20

---

Published as a conference paper at ICLR 2026
Table 12: Performance of CNN Models Trained on Their Own and Swapped Datasets on Four Datasets: GSM8K,
HuanEval, and GPQA.
Training Method
GSM8K
Math 500
HuanEval
GPQA
Average
BitCPM4-IB (self)
38.06
33.00
54.88
28.79
38.68
BicCPM4-1B (swapped)
42.00
32.80
48.78
24.24
36.96
Qwen 2.5-32B (self)
85.52
81.40
56.10
46.46
67.36
Qwen 2.5-32B (swapped)
85.60
80.40
51.22
42.93
65.05
Table 13: Comparative evaluation of data-selection methods BitCPM4-1B using Reason- ing. DeepSekk subsets across GSM8K, 500, HuanEval, and GPQA.
Size
Method
GSM8K
Math 500
EvanEval
GPQA
Average
10k
Self
38.06
33.00
54.88
28.79
38.68
10k
Superfiting
39.35
30.20
52.44
24.24
36.56
20k
Self
37.00
32.00
53.66
29.80
38.12
20k
Superfliting
36.92
30.09
54.27
27.27
37.14
113k
Alpagasus
37.38
30.80
53.66
25.76
36.90
A.5 GENERALIZATION OF CPQS ACROSS ARCHITECTURES AND SCALES
To assess whether CPQS generalizes beyond the three LLMs reported the main paper, we further evaluated it on two additional model families different parameter scales: small BitCPM4-1B model and a large Bwen2.5-32B-Instruue model. In both cases, CPQS was used to filter the Rea- DeepSSek corpus into high-quality subsets of varying sizes (10k, 20k, etc.). We compared CPQS (self) against two strong data selection baselines, Superfliting and Alpagasus, and trained models under identical settings per architecture. We report performance on GSM8K, Math 500, HuanEval, and GPQA, along with the simple average.
Across both the 1B and 32B regimes, CPQS-selected selected data consistently outperformed all baselines at matched subset sizes. On BitCPM4-1B. 1B, CPQS delivered the best averages for 10k and 20k sub- sets, exceeding Superfliting and the larger 113k Alpagasus subset despite using fewer samples. On Qwen2.5 --2 CPQS likewise led at 10k and 20k, with stronger averages than Superfiiting and the much larger Alpagasus set. These results indicate that CPQS's selection criteria transfer across architectures and scales, and that quality outweigh quantity when curating reasoning- focused training data. The detailed experimental results are presented Table 13 and Table 14
A.6 ABLATION ON THE SELECTOR ARCHITECTURE: CNN vs. MLP vs. TRANSFORMER
To further validate the effectiveness our selector design, we conducted an ablation study compar- ing the CNN architecture used in our method with two alternative designs: multi-layer percep- tron (MLP) and a Transformer based selector.
We evaluated all three architectures on the Reasoning-DeepSSer dataset (10k samples selected by CPQS). As shown in Table 15 the CNN-based selector consistently outperforms both ML. and Transformer variants across GSM8K, Math_500, HuanEval, and GPQA benchmarks, achieving the best overall average. We attribute this performance advantage to CNN's strong locality bias and computational efficiency, which enable it extract hierarchical spatial correlations from hidden states and thus improve data-quality discrimination.
A.7 ADDITIONAL COMPARATIVE EVALUATIONS AND EFFICIENCY ANALYSIS
A.7.1 COMPARISON WITH RECENT DATA-SELECTION METHODS
We extend our comparative study by including several recent and representative data-selection ap- proaches: Select[T, DS2, and Deita, which widely regarded strong baselines for instruction- tuning data curation. All methods were evaluated on the Reasoning Reasoning-DeepsSek dataset using 10k selected samples and fine-tuning Dwen2.5--B-Inntrrun model under identical conditions (maxi-
21

---

Published as a conference paper at ICLR 2026
Table 14: Comparative evaluation of data-sclection methods on Qwen2.5-32B-Insten using Rea-
soning DeepSeek subsets across GSM8K, Math 500, HuanEval, and GPQA.
Size
Method
GSM8K
Math_500 500
HuanEval
GPQA
Average
10k
Self
85,52
81.40
56.10
46,46
67.36
10k
Superfiting
85.22
81.80
56.71
43.43
66.79
20k
Self
84.76
$1.60
53.66
46.46
66.62
20k
Superfiting
84.99
81.40
49.39
42.93
64.68
113k
Alpagasus
84.91
81.00
53.66
39.90
64.87
Table 15: Ablation on selector architectures (CNN, MLP, Transformer) using 10k CPQS-selected Reasoning-DeepSeeek samples. Metrics are reported on GSM8K, Math_500 HuanEval, and GPQA, with the simple average.
Size TOK
Method
GSM8K
Math 500
HvanEval
GPQA
Average
CNN
85.37
72.40
67.68
36.36
65.45
10k
MLP
87.04
72.80
60.37
29.29
62.38
10k
Transformer
84.15
73.20
67.07
30.81
63.81
mum output token length set to 8000). Table T6 reports results on GSM8K, Math_500, HuanEval, and GPQA, well the simple average.
As shown in Table [6] our method achieves the highest average performance among all compared baselines, indicating stronger ability identify samples that are most beneficial for the target model.
A.7.2 COMPUTATIONAL COST AND THROUGHPUT ANALYSIS
To provide transparent accounting of efficiency, we report GPU memory consumption and through- put (samples per second) under identical hardware constraints. All methods
executed NVIDIA RTX PRO 6000 GPU, Throughput was computed over 146,224 samples using total wall- clock processing time. Results are summarized in Table As shown in the table below, our method ranks just behind Alpagasus and Superfiting in terms of GPU memory usage, and second only to Superfliting in throughput.
22

---

Published as a conference paper at ICLR 2026
Table 16: Comparison with recent data-selection methods on Sertoning--eepSeee (10k selected
samples) using Qwen2.5-7B-nnstruce All methods are evaluated under identical generation settings
Size
Method
GSM8K
Math 500
HuanEval
GPQA
Average
10k
Self
85,37
72.40
67.68
36.36
65.45
10k
DS2
84.15
72.80
62.80
29.80
62.39
10k
Select|T
85.67
73.60
59.76
31.82
62.71
10k
Deita
82.56
76.00
62.80
30.81
63.04
Table 17: GPU memory usage and throughput under identical hardware (NVIDIA RTX PRO 6000).
Throughput is measured in samples per second over 146,224 samples. Best values per column
bold.
Method
GPU Memory (GB)
Throughput (samples/s)
Self
18
4.78
MoDs
24
1.40
Alpagasus
0
1.45
Superfliting
34.82
DS2
30
1.44
Deita
50
2.03
Select|T
26
1.13
23


Reviews:
[{"review_id": "gXkEBPZYGR", "reviewer": "Reviewer_eg3w", "summary": "This paper introduces CPQS-Tuning (Contrastive Perception Quality Score), a data filtering algorithm for instruction fine-tuning based on model self-perception. The core idea is to leverage the hidden states of large language models (LLMs) as implicit indicators of data quality. A CNN classifier is then trained on these hidden-state features to output a contrastive perception quality score (CPQS) for each training example, enabling the selection of high-quality samples. This work is particularly promising in the area of evaluating the quality of synthetic instruction data.", "questions": "1.It remains unclear whether CPQS can generalize well across different model architectures or parameter scales. Can the method be effectively applied to other open-source LLMs?\n\n2.Although the results show linear separability between high- and low-quality samples, the underlying mechanism of why hidden states reflect data quality is still largely empirical. I encourage the authors to explore this aspect more theoretically in future work.\n\n3.Since hidden layers contribute differently to performance (see Appendix A.4.1), it would be valuable to develop an automatic strategy for determining the optimal layer combination for CPQS extraction.", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.The paper takes an innovative perspective by using the model’s own hidden states as a signal of data quality, removing dependence on external evaluation models or manually crafted metrics. It would be interesting to see future work analyzing how these hidden-state signals correlate with human evaluation metrics to improve interpretability.\n\n2.Experimental results demonstrate that the proposed approach can achieve better performance with less than 10% of the data compared to full-dataset training, which substantially reduces training costs.\n\n3.The linear separability and layer-wise comparison analyses nicely support the claim that hidden states indeed encode discriminative semantic features of high- vs. low-quality data — this is a very interesting finding.\n\n4.The appendix includes additional experiments such as full-parameter fine-tuning, iterative filtering, inter-layer comparison, and cross-model preference studies, all of which strengthen the robustness of the method. I look forward to the authors releasing the project code after acceptance so that I can further explore this work in detail.", "weaknesses": "1.The method is relatively complex, as it requires extracting multi-layer hidden states and training an external CNN model, which demands significant GPU memory and computation.\n\n2.The cross-model experiment shows that using “high-quality data” selected by Qwen2.5–7B-Instruct to fine-tune LLaMA2–7B–Chat actually led to degraded performance. This indicates that each LLM has its own internal definition of “high-quality data,” implying that CPQS must be trained separately for each target model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DrlOMXettI", "reviewer": "Reviewer_vfku", "summary": "This paper trains a CNN to classify low-quality data and high-quality data with the hidden state of a model, before tuning the model on the selected high-quality data. It achieves state-of-art performance on multiple tasks.", "questions": "1. I see in the appendix A.4.2, you demonstrated that for similar sized models, it's better to use the model's own hidden states to do the classification. What if you use models of different size? For example, will a 13B model's hidden state be better than a 1B model's own hidden state?\n2. See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper has a novel idea\n2. The experiments can demonstrate the effectiveness of the proposed method\n3. The paper is written clearly", "weaknesses": "1. The paper lacks in-depth analysis of the method's improvement over baselines. For example, Alpagasus is the method to provide ground truth labels, but the paper's method performs even better than that. The paper provides no explanation to this.\n2. The paper lacks a justification of the CNN structure: It would be good to compare with a more natural transformer structure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZcNwViF66S", "reviewer": "Reviewer_jCwj", "summary": "This paper proposes a method to select data that is most informative for a specific target model, rather than data that is \"high quality\" in a general sense. The approach uses the hidden states of LLMs to train a two-way classifier on a synthetic dataset labeled with quality scores, and this framework is validated through experimental analysis.", "questions": "The synthetic \"quality\" dataset (and its good/bad labels) is shared across all experiments, even though a separate CNN is trained for each target LLM. This design implies that \"quality\" is a pre-defined, model-agnostic concept, determined by the generative source. If all model-specific CNNs are being aligned to this single, shared quality space, why do the final results show that different LLMs have different judgments of \"quality\"? This seems contradictory.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper’s core motivation and its corresponding experimental validation are internally consistent and logically presented.", "weaknesses": "1. The paper proposes a pipeline with data synthesis, CNN training and data evaluation, while completely omitting an analysis of computational cost, which is a critical metric for any work in the data efficiency domain.\n\n2. The comparative study is weak. For example, but not limited to:\n- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning\n- DataMan: Data Manager for Pre-training Large Language Models \n- IMPROVING DATA EFFICIENCY VIA CURATING LLM-DRIVEN RATING SYSTEMS\n- SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection\n- LESS: Selecting Influential Data for Targeted Instruction Tuning\n\nBesides, It notably lacks a comparison against random sampling, which is the simplest and one of the most essential baselines for data selection.\n\n3. The motivation is not novel. The idea of selecting `actual requirements of the LLM` is the fundamental premise behind all uncertainty-based and influence-based data selection methods.\n\n4. The proposed pipeline appears overly complex and is not clearly justified. See question.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JhOdwczldn", "reviewer": "Reviewer_ZLxp", "summary": "This paper proposes a novel method for filtering instruction-tuning data for LLMs. The authors identify that many available instruction-fine-tuning datasets contain low quality or redundant examples, and that existing filtering approaches often rely on external evaluation models or manually designed metrics, which may mis‐align with the target LLM’s actual needs. \nTo address this gap, they posit that the hidden states of the target LLM implicitly encode its perception of data quality. They extract hidden‐state embeddings from the LLM when processing candidate instruction examples, train a contrastive classification model to distinguish high vs low quality samples, and define CPQS to rank/filter the dataset. \nEmpirically, they show that using under 10 % of the data from large synthetic instruction dataset, their method outperforms models trained on full dataset; in downstream tasks, the CPQS-filtered subset achieves on average +3.6 % improvement over leading selection algorithms.", "questions": "The paper defines “high‐quality” vs “low‐quality” data via outputs from stronger vs weaker models (e.g., GPT-4 vs Llama-3.2-1B-Instruct). Could the authors clarify how this quality definition correlates with user‐instruction usefulness or downstream task gain? Have you measured human‐annotated instruction usefulness to validate this proxy?\n\nHow robust is the CPQS classifier to architecture shifts? For example, if the target fine-tune model is of a significantly different architecture (or size) from the one used to extract hidden‐states, how does performance vary?\n\nCould the authors provide more detailed compute cost comparisons (selection + fine-tuning) versus baseline filtering algorithms (e.g., IFD, Superfiltering) under identical hardware settings, including GPU-hours, memory, latency?\n\nHave you analysed the selected subset in terms of domain/task coverage, instruction style diversity, difficulty distribution, or edge‐cases (e.g., dialogue vs code vs reasoning)? Specifically, is there a risk that CPQS selects only “safe/easy” instructions that align with the model’s hidden-state biases?\n\nIn downstream tasks (GSM8K, HumanEval, HumanEval-Plus), the average +3.6% improvement is promising — could you provide per‐benchmark breakdowns (wins vs losses) and variance across random seeds to help assess stability?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Introduces an innovative strategy: leveraging the target LLM’s own hidden‐state representations to capture its perception of data quality — this internalizes the filter criterion to the fine-tune target rather than relying on external proxies.\n\nDemonstrates significant empirical efficiency: using under 10% of data and still achieving (or exceeding) full‐data performance is compelling for resource‐constrained tuning scenarios.\n\nThe pipeline is reasonably practical — hidden‐state extraction + classifier + ranking is implementable and does not require extensive external hardware (relative to training giant filter models).\n\nAddresses a timely problem: as LLM fine‐tuning scales up, data redundancy and dataset cost are real bottlenecks; effective filtering helps reduce compute/data cost.", "weaknesses": "The training data for classifier (high/low quality) comes from synthetic sources (GPT-4 vs weaker models) rather than human annotated “instruction usefulness,” which may bias the filter toward model‐generated signal rather than actual instruction utility for users.\n\nThe evaluation, while covering several benchmarks, is limited in scope: primarily to specific datasets (Alpaca_GPT4, DeepSeek-R1) and certain LLM sizes; less evidence for very large scale (>30 B) or for domain‐specific instruction sets.\n\nThe cost/efficiency claims are somewhat under‐detailed: while “under 10% data” is quoted, actual compute/GPU‐hours, hyper‐param parity, training seeds, variation across runs are less fully described, which may reduce replicability confidence.\n\nThe method introduces potential bias in data selection: by relying on hidden-state classification, it may inadvertently favour samples that “look good” to the model’s internal representation (which may favour certain styles or tasks) and down-select others (e.g., rare domains, under‐represented instruction types). The paper does not deeply analyse coverage/diversity of the selected subset or risk of bias.\n\nWhile the hidden‐state classifier is shown to separate high vs low quality (AUC=1.00 in logistic regressor example), that may reflect a somewhat contrived two‐class setup (strong model vs weak model generation) rather than realistic open filtering scenarios; the real generalization beyond this setting may be less strong.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)