ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning
Authors: Xiaohan Qin
Xiaoxing Wang
Ning Liao
Cancheng Zhang
Xiangdong Zhang
Mingquan Feng
Jingzhi Wang
Junchi Yan
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Supervised Fine-tuning
Data Selection
Primary area: foundation or frontier models, including LLMs

Abstract: Data quality plays a critical role in enhancing supervised fine-tuning (SFT) for large language models (LLMs), and token-level data selection has emerged as a promising direction for its fine-grained nature. Despite their strong empirical performance, existing token-level selection methods share two key limitations: (1) requiring training or accessing an additional reference model, and (2) relying solely on loss information for token selection, which cannot well preserve semantically important tokens that are not favored by loss-based metrics. To address these challenges, we propose **ssToken**,  a **S**elf-modulated and **S**emantic-aware **Token** Selection approach. ssToken leverages readily accessible history models to compute the per-token loss difference with the current model, which serves as a self-modulated signal that enables the model to adaptively select tokens along its optimization trajectory, rather than relying on excess loss from an offline-trained reference model as in prior works. We further introduce a semantic-aware, attention-based token importance estimation metric, orthogonal to loss-based selection and providing complementary semantic information for more effective filtering. Extensive experiments across different model families and scales demonstrate that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning, while their integration—ssToken—achieves synergistic gains and further surpasses prior token-level selection methods, delivering performance improvements while maintaining training efficiency. 
Source code is available at https://github.com/jianke0604/ssToken.

Published as a conference paper at ICLR 2026
SSTOKEN: SELF-MODULATED AND SEMANTIC-AWARE TOKEN SELECTION FOR LLM FININTUNING
Xiaohan Qin Xiaoxing Wang' Ning Liao', Cancheng Zhang', Xiangdong Zhang',
Mingquan Feng'. Jingzhi Wang's Junchi Yan Schooo of AI, Shanghai Jiao Tong University, ?Shanghai Innovation Institute
ABSTRACT
Data quality plays critical role in enhancing supervised fine-tuning (SFT) for large language models (LLMs), and token-level data selection has emerged as promising direction for its fine-grained nature. Despite their strong empirical performance, existing token-level selection methods share two key limitations: (1) requiring training or accessing an additional reference model, and (2) relying solely on loss information for token selection, which cannot well preserve seman- tically important tokens that are not favored by loss-based metries. To address these challenges, we propose ss soToeen, Self-modulated and Semantic-aware To- ken Selection approach. ssToken leverages readily accessible history models to compute the per-token Ioss difference with the current model, which serves as a self-modulated signal that enables the model to adaptively select tokens along its optimization trajectory, rather than relying excess loss from offline- trained reference model as in prior works. We further introduce semantic-aware, attention-based token importance estimation metric, orthogonal loss-based se- lection and providing complementary semantic information for more effective fil- tering. Extensive experiments across different model families and scales demon- strate that both self-modulated selection and semantic-aware selection alone out- perform full-data fine-tuning, while their gistic gains and further surpasses prior token-level selection methods, delivering performance improvements while maintaining training efficiency. Source code available at https: //g/thtttb///////////////// com/ (ammm//iaaee000000 /ss4///ssse Tokeen.
syner-
INTRODUCTION
Large language models (LLMs) have achieved remarkable progress in recent years (Wu al., 2025; Xiong al., 2025; Yang al., 2026), and supervised fine-tuning (SFT) (Wei et al., 2021; Ouyang et al., 2022; Li et al., 2025b) has emerged as a crucial stage in their modern training pipeline, en- capability and practical utility. While early efforts primarily focused on scaling up instruction-tuning corpora, sometimes with millions of samples (Longpre et al., 2023; Chung et al., 2024), recent findings have consistently demonstrated that data quality outweighs data quantity in SFT (Chen et al., 2023a; Zhou et al., 2024; Fu et al., 2025), as small but carefully cu- rated datasets often yield stronger downstream performance than much larger but noisier collections. Therefore, identifying and prioritizing subsets of data that contribute most to model improvement from massive candidate data pools has become key challenge LLM supervised fine-tuning.
hancing
Recent studies (Lin et al., 2024; Simoulin et al., 2025; Li et al., 2025a) have shown that even after rigorous sample-level filtering, high-quality datasets still contain substantial token-level noise that degrades training outcomes. Non-task tar-----laleed patterns or phrases are often redundant or uninfor- mative, and continuing to fine-tune on them offers limited benefit while potentially undermining downstream task performance. This has motivated growing interest in token-level data selection, which aims to refine training datasets at a finer granularity by filtering individual tokens rather than
"Correspondence author. This work was partly supported by National Natural Science Foundation of China (No. 92370201) and Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Edu- cation of China, JYB2025XDXM411.

---

Published as a conference paper at ICLR 2026
Are other metricca indicate Importence?
Do really seed veference model token selection?
Example
semantic-
modulated
Tealning Stepe
User: Who Kobe Bryant?
Assistant: Uoooee, "Bryont', "is'. 'a', "Iooous "msericon 'basketball'
prompt
response
Cacens Lose
AttuScocn(s)
Zinnt Apoace
per-token
AC Rese f forediou))
reference model (nee required)
Lesrring dree "uuture Inse"
Takens
current model
fron
'Bryant
history model
AC' Datroopentive
basketball
Normalize
The overall framework of ss ssToken, which consists of two key components: (i) the self- modulated token selection strategy and (ii) the semantic-aware token importance estimation. We also provide a comparison with prior reference-modell-oase
Figure/ 1:
methods.
entire samples. Importantly, token-level approaches are orthogonal to domain-level and sample-level strategies, thereby broadening their applicability across diverse data selection scenarios.
Despite its advantages, we observe that existing token-level selection methods (Lin et al., 2024; Pang et al., 2025) share two key limitations: (1) requiring training or accessing an additional reference model, and (2) relying solely on loss information for token selection, which cannot well preserve semantically important tokens that are not favored by loss-based metrics. Specifically, recent ap- proaches typically prepare a task-related high-quality dataset and fine-tune either the base model or a smaller proxy model on it to serve as the reference, or alternatively adopt a more powerful model with the same tokenizer guided by prior knowledge. Excess loss is then computed against this reference model to identify task-related tokens that are important for improving downstream performance and instruction-follooing capability, as well as learnalee tokens whose loss steadily decreases. However, directly adopting a stronger model is not always practical, and training an extra reference model incurs additional time and resource costs. Moreover, (Pang et al., 2025) has shown that the performance of the reference model itself has a substantial impact on the effectiveness of to- ken selection. On the other hand, while leken- losses reflect the model's 's prediction uncertainty, they do not necessarily capture the semantic importance of tokens within their specific context. Fre- quent yet semantically uninformative tokens may yield similar excess loss values as task-critical ones, making loss-only selection prone to discarding informative content.
In this paper, we propose soToeen (Self-modulated and Semantic-aware Token Selection), a new token-level data selection method designed to address the two limitations discussed above. The core role of a reference model is to identify teskerelated and learnable tokens while discarding noisy or already well-learned ones. Instead of preparing additional curated dataset and training separate reference model, we view the current model itself as a natural teacher: as training progresses, its improvements over its own history provide reliable signals for token selection. As shown in Fig. 1, relative to the history model (e.g., the base model before SFT), the current model has already learned certain task-related patterns and achieved stronger performance, which can be viewed a reference. This perspective aligns with the principle of self-modulated learning, where the model adaptively regulates tokens that remain informative and learnable along its trajectory. Intuitively, if the current model achieves a notable loss reduction on a token compared with the history model, that token is unlikely to noisy or already lell-lea but rather represents learnable and informative content. Furthermore, the current model improves, the history model can be updated adaptively (c.g., via EMA), providing more stable guidance than relying on a fixed reference model.
On the other hand, for well-pretrained model, the attention matrix over input samples inherently encodes rich semantic information, which we regard as a complementary signal beyond loss for assessing token importance. To leverage this property in the fine-tuning setting, design an attention- based semantic-aware selection metric that identifies tokens more semantically salient in context and more beneficial for enhancing folllwiionnfollooiing capability. Extensive experiments
2

---

Published as a conference paper at ICLR 2026
show that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning, while their integration - synergistic gains and further surpasses pre- vious token-level selection methods. Overall, our contributions can be listed as follows:
1) Self-modulated token selection paradigm. We replace costly or even unavailable reference models with self-modulated strategy that leverages the current model's improvements over its history, thereby avoiding additional time and resource costs. Unlike a fixed reference model, the history model can be updated throughout training, providing more stable and long-term guidance.
2) Semantic-aware token importance estimation metric. We introduce an attention-based met- ric that leverages the rich semantic information embedded in attention matrices, offering a com- plementary signal to loss-based selection. The two signals are orthogonal, and their integration leads to mutually reinforeing effects. Additionally, we design a lightweight implementation that is compatible with efficient attention mechanisms such as FlashAttention, thus introducing negligible computational overhead.
3) Strong empirical performance across various model families and scales. We conduct ex- tensive experiments on models ranging from to 14B parameters across multiple benchmarks. Results consistently show that our proposed ssToken not only surpasses full-data fine-tuning base- lines by up to 4.3 but also outperforms previous token-level selection methods by up to 2.8% while maintaining training efficiency.
2 RELATED WORK
LLM Data Curation. The core objective of data curation for LLMs is to maximize training perfor- mance and efficiency by enhancing the quality and scale of training data. This encompasses wide range of strategies, including acquiring data through web crawling (Penedo et al., 2024; Yu et al., 2025) or synthetic generation (Wang et al., 2022; Li et al., 2023; Long et al., 2024; Han et al., 2025; Wang et al., 2025a), filtering (Gao et al., 2020; Penedo et al., 2023) and deduplication (Lee et al., 2021; Kandpal et al., 2022), rebalancing across domains (Xie et al., 2023; Liu et al., 2024), selecting subsets from given data pool (Liu et al., 2025; Wang et al., 2025b; Fu al., 2025), and ranking data to enable curriculum learning (Lee et al., 2023; Chen et al., 2023b). In this work, we focus on data selection for LLM supervised fine-tuning.
LLM Data Selection. Data selection is a critical component of the LLM data curation pipeline, aiming to further refine filtered and deduplicated high-quality datasets improve training perfor- mance and efficiency (Raffel et al., 2020; Tan et al., 2023; Bukharin & Zhao, 2023; Yu et al., 2024). Sample-level selection has been extensively explored by recent studies (Pang al., 2024; Li al., 2024; Fu et al., 2025), and some approaches (Zhang et al., 2024; Wang et al., 2025b) can achieve nearly lossless performance using less than 10% of the original data for fine-tuning. Recently, RHO- (Lin et al., 2024) highlights that even carefully curated datasets at the sample level still contain token-level noise and uninformative content. They are the first to explore token-level data selection for LLM training and report substantial gains over full-data baselines. TokenCleaning (Pang et al., 2025) further optimizes token-level selection in the SFT setting. However, these methods all share the two limitations discussed in the introduction. In contrast, our proposed soToeen avoids the need to train or access an extra reference model and leverages attention basedd signals to complement loss information with additional semantic cues.
3 METHODS
3.1 PRELIMINARIES: REFERENCE-MODEL-BER
TOKEN SELECTION
Given a data pool D with N samples, let x = { 22... + E D denote one sample with se- quence length L. Training a large language model under the next-token prediction paradigm can be formulated as minimizing the negative log-likelihood of observed tokens in the dataset. The loss on a single sample æ is defined as
1 Σ
Cg(x)
log Pa xzs);
(1)
T
IET
3

---

Published as a conference paper at ICLR 2026
where 8 denotes the model parameters, z; is the i-th token, and I2z; { :.:.....:....................... -h-1))
denotes the preceding context. The index set I S {1...., ,,,, L} specifies which tokens are supervised. For pre- training. I typically includes the entire sequence; for SFT, I only covers the response tokens, while prompt tokens serve as conditioning context but are not directly supervised.
RHO-1 (Lin al., 2024) observes that when continuing training on well-pretrained LLM,, token losses can be categorized into four types based on their trajectories: persistent high loss (H-H), increasing loss (L,+H), decreasing loss (H-+L, and consistently low loss (L-L). It prioritizes task- related tokens that the model can stably learn (H-+L) rather than those already mastered (L->L) or those that remain persistently difficult or noisy (H-*H and L-+H). To this end, a reference model is obtained by further training the base model on a high -uallit task-related dataset. Token-level Excess Loss (EL) is then computed with respect to this reference model to score and filter tokens. Specifically, the excess loss for token is defined as:
Por (2i (ee))
EL(,,) Co(x;) Ce (t) log
Pe (2) T<())
Here, EL;x;) measures the extent to which the prediction probability of token z; is expected to improve in future training. A larger value indicates that the model is more likely to achieve greater predictive gains on this token. They then rank tokens the data pool based on El. and their designed scoring strategies, followed by token selection according to predefined selection ratio. Reference- model-based methods have been empirically shown to deliver substantial improvements over full- data baselines, underscoring the promise of token-level selection while also highlighting the need for more practical and flexible approaches.
3.2 SELF-MODULATED TOKEN SELECTION
As discussed in Sec. 1, although reference-moded formance, directly adopting a stronger model with the same tokenizer is not always practical, and training a reference model incurs additional time and resource costs. We therefore reinterpret token selection as self-modulated process: rather than depending on an external reference, the model leverages its own training trajectory to progressively determine which tokens remain informative and learnable, thereby aligning token selection with its evolving capability.
token selection demonstrates strong per-
Specifically, the history model can be defined as the well-pretrained base LLM before SFT, or up- dated using checkpoints saved during training. Accordingly, we propose Retrospective Excess Loss (REL) as self-modulatee signal that quantifies model progress over its history, serving as a replacement for the EL defined in Sec. 3.1:
(x (x.) (oc.)-Co(33) Lo(x,)) logg Po(x; xes))
IP Po.. (5; x<()
This formulation operationalizes the self-modulated principle: as the current model improves over its historical states, tokens that continue to yield notable gains are adaptively prioritized, whereas noisy or already well-learned ones down-weighted. In this way, token selection becomes both reference- free and dynamically aligned with the trajectory of model training. Conceptually, if the current model assigns substantially higher prediction probability to token x; than the history model, x; is more likely to represent content that remains learnable at the current stage, rather than tokens already mastered or inherently noisy.
Moreover, Pang et al. (2025) points out that the capacity of the reference model significantly affects the quality of selected tokens and the final model performance. For training on large-scale datasets, however, providing stable long-term guidance becomes particularly important. Unlike reference models, which are often difficult to update continuously, the history model can be easily iterated through training checkpoints, for example via exponential moving average (EMA):
(Optional) 0 o + (1-a)00'
where a controls the smoothing factor. In this way, the history model retains information from past parameters while remaining adaptive, thereby enhancing its potential to provide stable guidance for large-scale, long- morizon training.
Although modulmmddlated token selection eliminates the need to train or access a separate refer- ence model, the history and current models share identical parameters at the beginning of training.
4

---

Published as a conference paper at ICLR 2026
which makes early-stage token selection nearly random. Moreover, similar to prior approaches, solflmodulaeed selection still relies solely on loss information, potentially overlooking tokens that are less favored by loss-based criteria but highly relevant to the task. This limitation motivates us to explore new signals beyond loss that can better capture token importance and task relevance, enabling more comprehensive token selection strategy.
3.3 SEMANTIC-AWARE TOKEN IMPORTANCE ESTIMATION
For rell-pretraine LLM, the attention matrix encodes rich semantic information. When answer- ing task-related questions, a token may exhibit persistently high prediction uncertainty (i.e., high loss), yet its attention signal can still play an essential role in predicting subsequent tokens and thus should not be discarded. Motivated by this, we aim to design an attention- estimation metric that complements loss-based selection with additional semantic signals. Although prior work has proposed estimating token importance using the sum of attentions from other tokens to the target token (Luo et al., 2020; Adnan et al., 2024), directly applying this approach under the next-token- mexx-token-prediction paradigm with a causal mask introduces additional positional bias, where tokens at different positions receive highly uneven attention. We observe that in the SFT setting, only response tokens need to be selected and used for gradient computation, and all response tokens consistently attend to fixed-length prompt. This motivates simple yet effective idea: measuring the total attention each response token assigns to the prompt tokens. Intuitively, since the prompt en- codes the task description or instruction be followed, the extent which a response token attends to the prompt naturally reflects its task relevance and cnsructioonfollowiing importance, making it reasonable indicator of token importance.
token importance
Specifically, we focus on the attention matrix at a fixed layer I during the forward pass of the model. For each attention head h E {1,..., ,..,,, H} in the decoder, we denote its attention matrix as ((h) E Rhog x Lon, where A!! represents the attention weight from token (query) to token j (key), and Ligg Lprompt + Liesp the total sequence length. Let Iprompt {0, ........ Lrrompt - 1} and Tresp promptt ................................ 11} denote the index sets of prompt and response tokens, respectively. To compute the attentions from response tokens to prompt tokens, we extract the following submatrix:
A (h)
Appmppp == A (1) [resp, Zprompt]
Rebay x X-ynaree
(5)
By summing over the prompt dimension, we obtain the attention scores of all response tokens in the sequence:
AttnScore (1) (h)
prompt E TR Lan
resp pproppt
Subsequently, we average the attention scores obtained from all heads h {l...., H). which serves as token importance estimator complementary to loss information. In summary, the final attention score for response token x; computed as:
(h) + +M,
1
Σ AttnScore(x:) = H 1=1
1
prompt softmax
Vde
where M, denotes the causal mask. An important design question is which layer's attention matrix should be used to compute the attention scores. In Sec. 4.3, we conduct ablation studies compar- ing early, middle, and deep layers, and conclude that using deeper layers generally yields better results. This finding aligns with prior studies (Aljaafari et al., 2024; Zheng et al., 2024; Rocchetti & Ferrara, 2025), which suggest that semantic representations become increasingly abstract across layers: shallow layers (closer to the input) primarily capture syntax, local structures, and surface- level patterns such as positional relations, bracket matching, and syntactic cues, while deeper layers (closer to the output) focus more on semantic abstraction, high-level concepts, and task-relevant global information, which are typically more influential for instruction following.
Moreover, prior works in other domains that rely on attention scores for analysis (Chen et al., 2024; Ye et al., 2025) are often incompatible with efficient attention implementations such as FlashAtten- tion (Dao et al., 2022). To avoid this, we design lightweight solution: during the forward pass, we use a hook to store the hidden states of the target layer and then perform a simple recomputation of that layer retrieve its attention matrix. This design eliminates the need output full attention matrices during the complete forward pass, thereby making our algorithm fully compatible with efficient attention mechanisms like FlashAttention and ensuring training efficiency.
5

---

Published as a conference paper at ICLR 2026
3.4 OVERALL FRAMEWORK OF SSTOKEN
After obtaining both the retrospective excess loss and the attention score for each token during training, combine them to perform multi-dimensional token selection. Specifically, due to the Transformer's 's computation mechanism and our definition of attention scores, the attention score is naturally bounded within [0, 1]. For REL, we normalize all response tokens within each sample as
REL(2;) miny REL(x,) Normalize (REL(( max, RHI(x,) min, REI(2,))
which maps the values into the range [0, 1J. We then introduce balance coefficient 7 E weight the two signals and compute the final score for each token:
[0, to
Score(z;) = 7 Normalize(REL(())) + (1 (1-7) AttnScore(z:)..
Here, 7 controls the relative contribution of loss-based and attention-based signals in token selection. When y 0 or = 1, ssToken degenerates into purely attention-basse selection or purely loss- based selection, respectively. Our ablation studies in Sec. 4.3 reveal that appropriate weighting consistently outperforms using either signal alone, and the best performance across benchmarks typically emerges when y takes intermediate values rather than the extremes (i.e., 0 or 1). Based on experimental validation, we set 7 0.5 as the default choice.
Following (Lin al., 2024; Pang et al., 2025), we perform token selection with fixed ratio p. Specifically, the loss function in Eq. is modified
Co(x)
Lresp Σ (osss) log Pog mee))
(10)
where Leesp p denotes the number of response tokens selected from the sample according the top-p fraction of (c; The indicator function (x;) is defined as
[1, if xr is among the top-p tokens ranked by Score(x;),
1x(x,)
(11)
0 otherwise.
Based on prior empirical evidence (Lin et al., 2024; Pang et al., 2025), setting p = 0.6 generally effective for retaining task-relevant tokens while filtering out less informative ones. We further provide a detailed ablation study on o in Sec. 4.3 and explore an adaptive strategy for adjusting p in Sec. 4.4. The complete algorithmic workflow of ssToken is presented in Appendix D.
4 EXPERIMENTS
4.1 PROTOCOLS
Data Preparation. Following the setup in (Pang et al., 2025), we construct our experimental data pool by sampling 50k examples from collection of five widely used SFT datasets (300k samples in total): Flan_v2 (Longpre et al., 2023), OpenAssistant (Köpf et al., 2024), Stanford Alpaca (Taori al., 2023), Dolly (Databricks, 2023), and WizardLM (Xu al., 2023).
Base Models. We adopt four widely used open-source LLMs of varying sizes, ranging from 3B to 14B parameters: LLaMA-3.2-3B (Dubey al., 2024), LLaMA-3.1-80 (Dubey et al., 2024), Qwen- 2.5-7B (Hui et al., 2024), and Qwen-2.5 2.5--1.1 (Hui et al., 2024). These models are fine-tuned on our curated data pool and compared against different token selection methods.
Baselines. For each model, we evaluate several strategies for comparison: (1) Full-data fine-tuning: training on the entire pool of 50k high-quality samples using supervised fine-tuning. (2) Uniform Random: selecting fixed proportion of p tokens uniformly at random from the data pool for train- ing. (3) RHO-1 (Lin et al., 2024): selecting the top-p tokens within each sample based on excess loss. (4) TokenCleaning (Pang et al., 2025): including Fixed-model Cleaning, which selects the global top-p tokens from the entire data pool according to excess loss, and Self-evolving Cleaning, which partitions the data pool and iteratively updates the reference model while selecting global top- tokens from each partition. In our experiments, we reproduce the methods based on their released code and report the best performance between the variants.

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison of different token selection methods across various models and benchmarks. The best scores are highlighted in bold.
Methods
TriviQQA
TruthfulQA
MMLU ARC-C
ARC-E
TyDIQA
Wino
HS
LogiQA
AGlEval
AVG
Base model:
LLaMA-32-31
BASE
50.85
39.37
56.20
42.15
74.49
19.05
69.06
55.21
29.49
30.04
46.59
FULL
52.50
43.97
57.34
45.56
76.42
39.48
70.36
56.75
29.88
32.11
50.35
REO-1
53.14
44.35
56.78
45.31
76.98
50.47
70.17
56.67
29.80
32.70
51.65
TOXENCLRANING
53.09
44.79
57.17
45.22
77.06
51.40
69.93
56. 699
30.57
32.81
51.87
SSTOKEN
54.04
47.42
57.02
70.56
56.19
32.87
33.97
52.50
Rase L.aMA-3.1-88
HASE
61.51
45.22
65.
51.45
81..7
20.50
73.56
60.01
27.94
35..3
52..22
51.69
54.64
RANDOM
63,23
48.75
65.68
54.27
63.38
54.13
74.39
60.32
30.41
37.41
57.42
RHO-1
65.39
$2.80
65.50
56. 23
83.88
57.83
76.48
62.46
31.80
37.77
50.01
TOXENCLEANING
65
56.06
SSTOKEN
66.33
55.
66.
56.29
83.12
69.228
75.77
61.
34. 1.0
397.97
64h 94
Base model:
Qwen-225-77B
26.67
54h
70.77
FULL
56,20
50.51
74.22
50.26
81.19
63.59
73.09
50.85
3...57
58.85
59.73
RANDOM
52.08
$1.86
74.18
48.21
79.84
63.09
73.32
60.07
28. 28.33
56.25
58.72
46.45
47.33
72.61
TOXENCLEANING
55.57
46.14
73.88
48.72
80.17
68.17
73.21
58.55
27.31
56.17
58.53
SSTOKEN
56.73
50.75
74.18
50.51
$1.48
69.74
72.03
60.30
31.03
57.18
60.48
Base model:
Owen-2,5-148
BASE
79.80
82.32
75.06
63.36
39.
60.55
FULL
66.36
$1.77
79.75
58.40
82.83
64.59
74.98
63.33 63.37
36.87
66.00
64.50
56.31
63.30
64.82
RHO-1
63,28
52.14
79.62
50.57
83. 71
71.21
75.85
63.03
35.33
65.23
64.88
TOKENCLEANING
66.42
51.87
79.66
$5.97
63.08
69.37
76.56
62.33
38.58
64.88
64.87
SSTOKEN
56.22
02:
Consistent with the setup in (Pang al., 2025), for methods requiring a reference model, the ref- erence is trained on a 10k high-quality subset of the data pool, which is obtained through further filtering with the new powerful sample-level selection method DS (Pang et al., 2024). The to- ken selection ratio is predeternined, and detailed settings as well as related ablation studies are provided in Sec. 4.2 and Sec. 4.3.
We evaluate method diverse suite of benchmarks that cover different aspects of LLM capabilities, including MMLU (Hendrycks et al., 2021), TriviaQA (Joshi et al., 2017), TruthfulQA (Lin et al., 2021), ARC-Easy (Clark al., 2018), ARC-Challenge (Clark et al., 2018),
Evaluation.
TyDiQA (Clark et al., 2020), Winogrande (Sakaguchi et al., 2021), HellaSwag (Zellers et al., 2019), LogiQA (Liu al., 2020), and AGIEval (Zhong et al., 2023). Together, these benchmarks provide comprehensive assessment of factual knowledge, reasoning ability, and cross-lingual generalization. Evaluations conducted using the standard 1m-eva -ha rness' s (Gao al., 2024) framework, and additional training and evaluation details can be found in Appendix B.
4.2
MAIN RESULTS
We conduct experiments on four different base models with five token selection strategies (including our proposed ss soooeen) and evaluate their performance across ten -domanl benchmarks. The detailed results are presented in Table The balance coefficient y is set 0.5 across all model sizes. The token selection ratio is set to 0.6 for -LaMA-3.2-31 LLaMA-3.1-888 and Qwen-2.5-7B, and 0.8 for 2w5n-22.5-14B For the same base model, all token selection methods are evaluated under the same ratio. Detailed ablation studies can be found in Sec. 4.3.
As shown in Table 1, sokke achieves the best average performance across all four base models compared to other existing token selection methods. Relative to the full-data fine-tuning baseline, soToeen improves the average performance by 4.3%, 3.4%, 1.3%, and 2.1 % on LLaMA-3.2-3B LLaMA-3. 1-B, Qwen-2.5-7B, and Qwen-2.5-14B. respectively. We also observe that although prior token selection methods RHO-1 and TokenCleaning show strong performance on LLaMA-3.2. 3B and LLaMA-3.1-8B their performance the Qwen family is only marginally better than or
hhtpp:://gi thub. com/ wwt.om//wwetteer.... 1mmeerr//mmmeaaaaaaaa uat on-harnesse

---

Published as a conference paper at ICLR 2026
sometimes even worse than that of full-data fine-tuning. In contrast, our proposed ssToken consis- tently delivers strong and stable gains across different model sizes and families, further demonstrat- ing its effectiveness and generality.
Furthermore, we find that the impact of token selection varies across different benchmarks. For knowledge-intensing tasks such MMLU, ARC- ARC-Challenge, and ARC-Easy, which heavily rely on pretraining, token selection during SFT does not lead to notable performance improvements. In con- trast, for benchmarks like TyDiQA, TriviaQA, and AGIEval that target downstream tasks or require stronger instructiin-folloiing ability, SFT typically yields substantial gains for the base models, and token selection often provides further improvements. Notably, ssToken outperforms previous token selection methods on nearly all QA tasks across the four models. Since QA tasks generally demand stronger catructioo-followwii component of ssToeen, which complements loss information with rich semantic cues and effectively guides the selection of iesleat tokens.
capability, we attribute this advantage to the attention-based
Performance vs. Training Time. In Fig. 2, we fur- ther compare the performance and total training time (including model loading and reference model train- ing) of different token selection methods against the full-data fine-tuning baseline across various model sizes. is worth noting that existing token selection approaches are primarily designed to enhance per- formance rather than reduce training cost. Specifi- cally, while the filtered-out tokens are masked dur- ing loss computation, they still participate in the for- ward pass, and thus these methods do not yield re- ductions in training time compared to full-data fine- tuning. shown in the figure, while RHO-1 and TokenCleaning achieve notable performance gains over the full-data baseline, their reliance on a refer- ence model incurs substantial additional time and re- source overhead. In contrast, ssToken does not need to train extra reference model, and its lightweight design for obtaining attention matrices introduces marginal computational cost. Consequently, com- pared with the full-data baseline, ssToken delivers significant performance improvements with only a marginal increase in training time.
140
Performance
64
TakesTeasing
18
62
82
12
50
10
20
20
100
120
Total Training Time (minutes)
Figure 2: Average performance vs. total training time across different methods.
4.3 ABLATION STUDIES
Our proposed ssToken involves key hyperparameter, the balance coefficient y, which controls the relative contributions of bass-based and iitention-basee information. In addition, similar to previous token selection methods, soToken also requires a predefined ratio for selecting tokens. We there- fore conduct experiments these two hyperparameters across models of different sizes, aiming to investigate their general effects on model performance and to provide practical guidance for their appropriate settings.
Ablation studies on the balance coefficient 7 As shown in Fig. 3a, compared with the full-data fine-tuning baseline, both self-modulated token selection (7 1) and semantic-aware token selec- tion (y 0) achieve superior performance, while combining the two signals in proper proportions yields synergistic effects and further improvements. This validates our hypothesis that loss-based and attention-based information capture complementary features: the latter provides additional se- mantic cues beyond what is reflected in loss values.
From the results, we observe that 7 0.5 achieves the best performance for models with 8B and 14B parameters, and ranks second for the 3B model (LLaMA-3.2-3B) slightly behind 7 = 0.75. A closer look at the detailed results in Appendix C. shows that the advantage of 0.75 on LLaMA-3.2-3B mainly comes from its strong performance on TyDiQA and TruthfulQA. In contrast, although 0.5 is marginally weaker than 7 0.75 on LLaMA- performance across benchmarks. Considering these observations, together with its optimal results on larger models (8B and 14B), recommend setting 7 0.5 the default choice.
it achieves more balanced
8

---

Published as a conference paper at ICLR 2026
Average Performance
148 Baseline
Average Performance
65.0
Model
62.3
66.
Model
XBI Baseline
25.0
12.3
3B Model
10.d
Baseline
(Rase model)
(Full-
0.75
0.5
0.25
0
0.0
0.2
0.4
0.6
0.8
1.0
(pare-lous)
Balance Coefficient y (pure-aito)
Token Selection Ratio p
(a) Ablation study on the balance coefficient 7/
(b) Ablation study on token selection ratio p
Figure 3: Ablation studies on y and p. (a) Varying the balance coefficient 7 with dashed lines in- dicating the full-data fine-tuning baselines for different model sizes. (b) Varying the token selection ratio p. Both plots report average performance across model sizes (3B, 8B, 14B).
Ablation studies on the token selection ratio p. As shown the Fig. 3b, although the SFT data we use has already been curated through strong data cleaning pipelines, applying token-level selection with an appropriate ratio still yields further improvements over full-data fine-tuning across different model sizes. For LLaMA-3.2-3B and ALaMA-3.1-88B the best results are achieved at p 0.6, consistent with the ablation findings reported in (Pang et al., 2025). However, for Qwen-2.5-14P5 observe that not only ssToken but also other token selection methods (Random, RHO-1, Token- Cleaning) reach their peak performance at 0.8 (detailed results on this comparison are presented in Appendix C. We hypothesize that this may stem from differences in pretraining data distribu- tions, which affect the capability boundaries of different model families; in addition, larger models may better capture more challenging rask-related patterns that smaller models fail to recognize. In summary, the choice of p should be guided by both model capacity and data quality. In line with prior works (Lin et al., 2024; Pang et al., 2025), our ablations suggest that p = 0.6 is generally a robust choice, while slightly larger values may yield further improvements in certain settings.
Ablation Study on Attention Layer Depth. In our semantic-aware token importance estimation, require the attention matrix from specific layer of the forward pass, and the choice of layer can influence the effectiveness of token selection. To investigate this, we conduct experiments on LLaMA-3.2-3R and Qwen-2.5-7B by extracting attention matrices from shallow, medium, and deep layers, respectively. The results are reported in Table 2, with the balancing coefficient 7 fixed at 0.5 and the token selection ratio p set to 0.6.
Across both model families, several consistent patterns emerge. First, for knowledge-intenn -itteneeee benchmarks that rely pretraining (e.g., MMLUU, different attention layers do not exhibit a strictly dominant winner. In contrast, for QA-style tasks (e.g.. TruthfulQA, TyDiQA) and logical/common- sense reasoning benchmarks (c.g., LogiQA, HellaSwag), deep-layer attention consistently performs best. More concretely, on LLaMA-3.2-BB. deep-layer attention achieves the best score on 7 out of 10 tasks, and on Qwen-2.5-7B, it achieves the best score on 6 out of 10 tasks. both models, the deep-layer variant also achieves the highest overall average score.
As discussed in Sec. 3.3, this pattern aligns with observations from prior interpretability and probing studies (Aljaafari et al., 2024; Zheng et al., 2024): shallow layers primarily encode local syntactic or positional features, whereas deeper layers capture more abstract semantic structure and telk-atl dependencies. Overall, these results demonstrate that using deep-layer attention provides stable and semantically meaningful signal for token selection, and performs well across model families and task types without the need for task-specific tuning, We therefore adopt deep-layer attention as the default configuration in ss soTeen.
4.4 EXPLORING ADAPTIVE TOKEN SELECTION RATIO
In line with prior token-level selection methods, Token predefined token selection ratio p as the threshold for filtering informative tokens. While Sec. 4.3 presents a comprehensive study of different fixed values of p, the optimal setting may vary across model families, training stages, or

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study on attention layer depth on LLaMA-3.2-2 -32--BB and Qwen-2.5-7B. Results are reported on ten benchmarks, with the best score among ss Token variants highlighted in bold.
Methods
TriviaQA TruthfulQA
MMLU
ARC-C
ARC-E
TyDiQA
Wino
HS
LogiQA
AGlEval
AVG
model:
LLaMA-322-3
30:37
74.49
29.49
FULL-DATA
52.50
43.97
57.34
46.36
76.42
39.48
70.56
06.75
20.50
32.11
50.35
SHALLOW (LAYER 1)
54.36
45.12
56.25
45.48
77.46
48.52
70.09
53.76
31,80
33.43
51.63
MEDIUM (LAYER 14)
47.04
07.43
77.23
21.35
DEEP (LAYER 28)
51.04
47
37.02
45.82
76.12
51.04
70.17
56.19
32.87
33.97
52.50
Base model:
Qwen-2,5-7B (28 layers)
56.31
59.98
FULI-DATA
36.20
04.61
74,22
00:22
81.19
63.50
73.08)
59.86
31.57
56.85
30.73
SHALLOW (LAYER 1)
55.17
46.25
74.26
49.23
81.10
68.70
72.69
58.32
27.90
57.06
59.20
MEDIUM (LAYER 14)
48.40
73.78
72.74
58.31
DEEP (LAYER 28)
36.73
50.75
74.18
50.51
81.48
69.74
72.93
60.30
31.03
57.18
60.48
sample quality. This motivates us to explore whether adaptively adjusting p during training can offer further benefits beyond manually chosen constant. To this end, we introduce a simple training- step-based decay schedule:
Ps = Pmin + (Pinax Pmnin) (1- 11-4)
where Pmax and Pmin are fixed hyperparameters but offer greater flexibility in setting, as they do not require rigorous ablation experiments determine like the fixed selection ratio. The rationale behind this design is intuitive. At the early stage of training, model predictions are still unstable, and using a relatively large p (e.g., s=0.8) avoids prematurely discarding potentially useful tokens when the semantic and loss-based signals have not yet fully matured. As training progresses and the model becomes increasingly capable of identifying informative patterns, a gradually decreasing encourages the model to concentrate more high-value tokens. In the late stage, smaller p (e.g., 20.4-0.5) sharpens this focus and helps the model refine the most semantically and instruction- relevant components of the data. This decay process naturally creates online curriculum in which the effective training subset shrinks as model capacity improves.
We evaluate this adaptive schedule on LLaMA-3.2 3-2--BB (with B 1), and the detailed results presented in Table 6 in the Appendix. The adaptive strategy consistently outperforms all predefined fixed values of & considered in our ablations (i.e., P 0, 0.2, 0.4, 0.6, 0.8, 1.0), highlighting the potential of dynamically adjusting and pointing to adaptive token selection as a promising direction for future work.
CONCLUSION. LIMITATIONS AND FUTURE WORK
In this paper, introduced ss soTeen,, a self-modulated and semantic-aware token selection frame- work for LLM supervised fine-tuning. By formulating token selection as a self-modulated process, ssToken leverages retrospective excess loss historical model states adaptively prioritize kens that remain informative and learnable, thereby eliminating the need for training an additional reference model. Moreover, by leveraging an attention-based score tailored for the SFT setting, ssToken provides a semantic complement to loss- boss-based information. Experiments demonstrate that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning, while combining the two signals yields synergistic gains and further improvements. Extensive re- sults across different model families and scales show that ssToken consistently surpasses both full- data baselines and prior token selection methods, achieving performance improvements while main- taining training efficiency.
Limitations and Future Work. Similar to previous token selection methods, ssToken requires predefined token selection ratio p to filter informative tokens. In See. 4.4, we explore a simple seiniing-ttage-aaee adaptive schedule for P and empirically verify its potential. Beyond adjust- ing according to training progress, another promising direction is to make dependent on model behavior or data characteristics such as token-level loss dynamics sample quality-there en- abling truly data-driven and aodel-aware token selection. We leave the development of such more flexible adaptive mechanisms for future work.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We are committed to conducting this research in line with responsible and ethical Al practices. All experiments are performed on publicly available datasets that widely used in the community, and no private or sensitive user data is involved. The proposed method, ssToken, aims to improve the efficiency and performance large language model fine-tuning without introducing additional risks beyond those already associated with LLMs. Nevertheless, we acknowledge the dual-use nature of language technologies and encourage responsible application of our method in accordance with established community guidelines for safe and ethical Al development.
REPRODUCIBILITY STATEMENT
All base models, datasets, and evaluation tools used in this work are publicly available and widely adopted in the community. As detailed in Sec. provide clear descriptions of the models, data sources, and evaluation benchmarks employed. Comprehensive training details, including learning rate schedules, batch sizes, and optimization settings, are further specified in Appendix B. In ad- dition, all hyperparameters analyzed in the paper have been thoroughly examined through ablation studies ensure the robustness of our conclusions. To further facilitate reproducibility, we have anonymously released the complete source code at https: asToken.
com/ /1anke0604/
REFERENCES
Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant Nair, Ilya Soloveychik, and Pu- rushotham Kamath. Keyformer: Kv cache reduction through key tokens selection for efficient generative inference. Proceedings of Machine Learning and Systems, 6:1 14-127, 2024.
Nura Aljaafari, Danilo Carvalho, and André Freitas. The mechanics of conceptual interpretation in gpt models: Interpretative insights. arXiv preprint arXiv:2408. 1/827, 2024.
Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning. arXiv preprint arXiv:2311.1472 2023.
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision- language models. In European Conference on Computer Vision, pp. 19-35. Springer, 2024.
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. arXiv preprint (tt::::00000700 2023a.
Mayee Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. Skill-it! data-driven skills framework for understanding and training language models. Advances in Neural Information Processing Systems, 36:36000-36040, 2023b.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned lan- guage models. Journal of Machine Learning Research, 25(70):1-53 2024.
Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski. Vitaly Nikolaev, and Jennimaria Palomaki. Tydiqa: A benchmark for iiformmtioon-cekinng question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 8:454 470, 2020.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: 1803.000355555 2018.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in neural information processing systems, 35: 35:16344-16359 2022.
11

---

Published as a conference paper at ICLR 2026
Databricks. Free dolly:
Introducing the world's first truly open instruction-tuner 11m, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.22 1783, 2024.
Yanjun Fu, Faisal Hamman, and Sanghamitra Dutta. T-shirt: Token-selective hierarchical data lection for instruction tuning. arXiv preprint arXiv:2506.013/7 2025.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101 00000000.00....00000000000020002 2020.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter. Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf. Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL https: zewoeonooooo..o.oo org/ www.ododo.rgrrccrdsss..s.....o rewordseccc///eeeeeeeeeoeee
Guangzeng Han, Weisi Liu, and Xiaolei Huang. Autributes as textual genes: Leveraging llms as genetic algorithm simulators for conditional synthetic data generation. arXiv preprint arXiv:2509.0204 02040, 2025.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.088
2021.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)33 2022.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Mandar Joshi, Funsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv: httv://75.03300 03351, 2017.
Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pp. 10697-10707. PMLR, 2022.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Due Nguyen, Oliver Stanley, Richárd Nagyfi, al. Openassistant conversations-ddmooratiiza large language model alignment. Advances in Neural Information Processing Systems, 36, 2024.
Bruce W Lee, Hyunsoo Cho, and Kang Min Yoo. Instruction tuning with human curriculum. arXiv preprint arXiv:2310.09511 2023.
Katherine Lee, Daphne Ippolito. Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. arXiv preprint
2021.
Ming Li, Yong Zhang. Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. arXiv preprint arXiv:2402.00533) 2024.
Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang. Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, et al. Attention illuminates IIm reasoning: The segplan-and-achhol rhythm enables fine-grained policy optimization. arXiv preprint arXiv:2510.1333 2025a.
Yang Li, Jiale Ma, Yebin Yang, Qitian Wu, Hongyuan Zha, and Junchi Yan. Generative modeling reinvents supervised learning: Label repurposing with predictive consistency learning. In Forty- second International Conference Machine Learning, 2025b.
12

---

Published as a conference paper at ICLR 2026
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large lan- guage models for text classification: Potential and limitations. arXiv preprint arXiv:2310.07849, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07955 2021.
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang. Jian Jiao, Nan Duan, et al. Rho-l: Not all tokens are what you need. arXiv preprini arXiv:2404.077 07965, 2024.
Fengze Liu, Weidong Zhou, Binbin Liu, Zhimiao Yu, Yifan Zhang, Haobin Lin, Yifeng Yu, Bingni Zhang, Xiaohuan Zhou, Taifeng Wang, et al. Quadmix: Quality-diversity balanced data selection for efficient I1m pretraining, arXiv preprint arXiv:2504. 16511, 2025.
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. arXiv preprint arXiv:2007.088 /24, 2020.
Qian Liu, Xiaosen Zheng. Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang. and Min Lin. Regmix: Data mixture as regression for language model pre-training. arXiv preprint ar 24X.02440..01992 2024.
Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang- On llms-driven synthetic data generation, curation, and evaluation: A survey. arXiv preprint 1xXiv:2406.15126 2024.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoe V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In International Conference on Machine Learning, pp. 22631-22648, PMLR, 2023.
Xiao Luo, Haoran Ding, Matthew Tang, Priyanka Gandhi, Zhan Zhang, and Zhe He. Attention mechanism with bert for content annotation and categorization of pregnancy-related questions on a community q&a site. In 2020 IEEE International conference on bioinformatics and biomedicine (BIBM), pp. 1077-1081. IEEE, 2020.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Jinlong Pang, Jiaheng Wei, Ankit Parag Shah, Zhaowei Zhu, Yaxuan Wang, Chen Qian, Yang Liu, Yujia Bao, and Wei Wei. Improving data efficiency via curating Ilm-driven rating systems. arXiv 2024.
preprint arXiv:2410.1083
Jinlong Pang, Na Di, Zhaowei Zhu, Jiaheng Wei, Hao Cheng, Chen Qian, and Yang Liu. To- ken cleaning: Fine-grained data selection for Ilm supervised fine-tuning. arXiv preprint arXiv:2502.01965 2025.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei. and Julien Launay. The refinedweb dataset for falcon Il:: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.011 16, 2023.
Guilherme Penedo, Hynek Kydlíček, Anton Lozhkov, Margaret Mitchell, Colin A Raffel, Leandro Von Werra, Thomas Wolf, al. The fineweb datasets: Decanting the web for the finest text data at scale. Advances in Neural Information Processing Systems, 37:3081 1-30849, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 2020.
13

---

Published as a conference paper at ICLR 2026
Elisabetta Rocchetti and Alfio Ferrara. How instruction-tumi -uning imparts length control: A cross- lingual mechanistic analysis. arXiv preprint 1.XIv:2509.020733 2025.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
Antoine Simoulin, Namyong Park, Xiaoyi Liu, and Grey Yang. Memory-efficient fine-tuning of transformers via token selection. arXiv preprint arXiv:2501. 18824, 2025.
Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moviing-ne-sampll-out. Advances in neural information processing systems, 36: 18251-18266 2023.
Rohun Taori, Ishaan Guirajani, Ting Zhang, Yann Dubois, Xiaodan Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloinng llama model. https:
//github. http:/ttssuaabbbbba sttnfornnddrroro/ caaisddappppce alpaca, 2023. GitHub repository.
Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, et Reverse-engineered reasoning for open-ended generation. arXiv preprint arXiv:2509.0611 2025a.
Shaobo Wang, Xiangqi Jin, Ziming Wang. Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, et al. Data whisperer: Efficient data selection for task-specific Ilm fine-tuning via sho-shot in-context learning. arXiv preprint arXiv:2505.12212 2025b.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212. 105.0000,, 2022.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannanch Hajishirzi. How far can camels go? exploring the state instruction tuning on open resources, 2023.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc Le. Finetuned language models are zero-shot learners. arXiv preprint 1.Xi:22009.0152, 2021.
Huaijin Wu, Lianqiang Li, Hantao Huang, Yi, Jihang Zhang, Minghui Yu, and Junchi Yan. Hshare: Fast I1m decoding by hierarchical key-value sharing. In The Thirteenth International Conference on Learning Representations, 2025.
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoe V Le, Tengyu Ma. and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. Advances in Neural Information Processing Systems, 36:69798- 69818, 2023.
Hao Xiong, Yebin Yang. Huaijin Wu, Xiaoqiu Zhong, Yehui Tang, Zhuo Xia, Xiaoxing Wang, and Junchi Yan. Reinvent the operation not the architecture: Quantum-inspire high-order product for compatible and improved Iims training. In Proceedings of the 31st ACM SIGKDD Conference Knowledge Discovery and Data Mining V. 2, pp. 3356-3365, 2025.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprini 2r3iv:2304. 12244, 2023.
Yebin Yang, Huaijin Wu, Fu Guo, Lin Yao, Xiaohan Qin, Jingzhi Wang, Debing Zhang, and Junchi Yan. Jtok: On token embedding another axis of scaling law via joint token self-modulation arXiv preprint arXiv:2602.00880 2026.
Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. Fit and prune: Fast and training- ffeee visual large language models. In Proceedings of the AAAI Conference on
token pruning for Artificial Intelligence, volume 39, pp. 28128221336, 2025.
14

---

Published as a conference paper at ICLR 2026
Shi Yu, Zhiyuan Liu, and Chenyan Xiong. Craw4llm: Efficient web crawling for Ilm pretraining.
arXiv preprint arXiv:2502.13347, 2025.
Simon Yu, Liangyu Chen, Sara Ahmadian, and Marzich Fadace, Diversify and conquer: Diversity- centric data selection with iterative refinement. arXiv preprint arXiv:2409./1378, 2024.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- chine really finish your sentence? arXiv preprint
2019.
Xiaoyu Zhang, Juan Zhai, Shiqing Ma, Chao Shen, Tianlin Li, Weipeng Jiang, and Yang Liu. Spec- ulative coresct selection for task-specific fine-tuning. arXiv preprint arXiv:2410.01296. 2024.
Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. Attention heads of large language models: A survey. arXiv preprini arXiv:2409.03752. 2024.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: human-centric benchmark for evaluating foundation models. arXiv preprint 233.:2200.003644 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
15

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
During the preparation of this manuscript, we employed large language models solely to aid or polish the writing. The ideas, algorithmic design, code implementation, experiments, and overall manuscript writing were entirely carried out by the authors.
B EXPERIMENT DETAILS
B. DATA POOL
Following (Pang et al., 2025), we construct data pool from five commonly adopted SFT datasets, which are either manually curated or generated with the assistance of powerful LLMs. These datasets vary significantly in terms of prompt style, annotation quality, average length, and task focus, thereby ensuring that the resulting pool is both representative and diverse. To maintain con- sisteney across sources, we standardize all datasets using the template introduced by Wang et al. (2023), which explicitly marks conversational roles with < User| and < Ass issantt tags.
B.2 EVALUATION BENCHMARKS
We evaluate models on a wide range of benchmarks that capture different aspects of LLM capabil- ity, again following (Pang et al., 2025). For datasets such as MMLU, LogiQA, ARC-C, ARC-E, Winogrande, AGIEval and HellaSwag, we report accuracy as the evaluation metric. In particu- lar, for TruthfulQA, which is formulated scores only the answer assigned the highest probability by the model. For TriviaQA, we use the exact march_ metrie. For TyDiQA, use the 1-shot Fl score. All evaluations are performed using the official 1m-evy laaaluattt ceciionhhee -aarnearsssss toolkit, with the default setting of evaluating on the full benchmark sample set.
multiple-choice task, adopt the MC2 metric that
B.3 TRAINING DETAILS
Following the setup in (Pang al., 2025), we adopt the LoRA technique (Hu et al., 2022) with
rank 64 and a scaling factor of 16. Training is conducted for one epoch with a total batch size of 48 and learning rate of 1 14-4, The maximum input sequence length is set to 2048 tokens. All experiments are conducted on eight NVIDIA H200 GPUs. For the training time reported in Fig. 2, present the complete runtime of each token selection method, including both model loading and the training of the reference model. Due to the relatively small size of the SFT data pool, observe that enabling the optional EMA-based updating of the history model does not yield noticeable performance gains, while model merging introduces additional computational overhead. Therefore, in all experiments we use fixed base model as the history model. We hypothesize that adaptive updating of the history model may demonstrate greater potential in harge-hoiizon training scenarios.
For computing REL in Eq. 3, we do not load the history and current models simultancously nor perform two forward passes. Instead, we adopt a lightweight two-stage implementation: we first run the history model once over the training corpus to cache token-level losses, and during SFT the current model simply reads these cached values and computes REL via single tensor subtraction. This preprocessing step requires only a few minutes and avoids any increase in peak GPU memory, making soken incur only marginal overhead compared with full-data fine-tuning.
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 DETAILED ABLATION ON 7 AND p
In Sec. 4.3 of the main text, we presented the final results briefly summarized in Fig. 3. In this section, we provide the complete experimental results of sooeen with respect to 7 and p. reported in Table. 3 and Table. 4. In addition, Table. compares the performance of different token selection methods on Qwen-2.5- 14B under two set- tings, p 0.6 and p 0.8.
ablation study on the hyperparameters and p. with
16

---

Published as a conference paper at ICLR 2026
Table 3: Performance of ssTOKEN with different balance coefficient 7 across different base models and various benchmarks.
TriviaQA
TruthhiQQA
TyDiQA
LagiQA
Base model: L.
PULL
52.50
43.97
57:34
45.56
76.42
33.48
70.56
55.75
29.50
32.11
50.33
SSTOKEN 1.00
51.04
48
06.34
43.26
74.06
06.U77
68.51
51.78
34.87
34.30
52.02
SSTOKEN
SSTOKEN ( 0.5)
64.04
47-42
07.02
46.82
76.422
51.04
70:17
56.
322.87
33.97
52,50
SSTOKEN 0.25)
51.16
06.02
57.51
15.82
77.36
49.11
70.72
55.75
91.31
32.87
52.14
0.0)
Base model:: LLaMA-3.1-888
46
FULL
65,21
09.08
65.75
54.09
83.59
54.64
74.74
60.40
29. 29.50
33.007
37.49
SSTOKEN 1.0)
65.50
56.51
64.70
53.33
82.07
60.00
74.90
56.78
32.87
38.83
58.61
SSTOKN
76
SSTOKEN (7 0.5)
60.33
55.56
65.32
55.29
83.12
59.28
75.77
341.10
37.97
39.11
SSTOKEN 0.25)
65.54
53.57
65.63
55.20
83.25
58.40
76.
60.24
32.72
37.19
58.75
0.0)
Base model:
Qwen-2.5-1488
BASE
39.12
58.45
79.50
55.80
82.32
25.89
75.06
63.36
38.10
64.27
60.55
66.36
79.75
36.87
SSTOKEN 1.0)
64.41
52.62
79.69
56.38
82.45
69.31
75.61
63.01
37.64
66.18
61.63
SSToKEN 0.75)
66.64
52.15
77.93
56.40
83.21
70.91
75.69
63.10
43.32
66.30
65.57
STOKEN 0.5)
06.55
02.
79.97
56.22
71.94
76 14
62.99
42.70
65.84
SSTOKEN 0.255
67.01
53.17
99.90
56.55
82.32
72.30
76.85
62.91
43.16
66.10
65.83
SSTOKEN 0.0)
66.16
54.06
79.92
56.14
83.12
79.53
76.01
37.94
66.11
65.62
Table 4: Performance of ssToken under different token selection ratios across different base models and various benchmarks.
Methods
TriviqQA
TruthfulQA
MMLU
TyDiQA
Wino
LegiQA
Base model: LLaMAA332-33
(p
50.85
56.20
19.05
PULl (o
52.50
43.97
57.34
45.56
76.42
39.48
70.56
35.75
29.50
32.11
50.35
SSTOKEN
0.2)
51.70
51.82
55.63
44.20
73.65
47.50
70.48
54.32
33.03
32.91
51.72
sSTOKEN
50.70
sSTOKEN (p 0.6)
54.01
47.42
57. 02.02
45.82
76. 12
51.04
70.17
56..9
32.87
33.97
52.50
SSTOKEN
53.99
57.40
45.65
77.57
50.10
70.24
56..28 26
31.95
02.29
BASE (p -00 0)
61.51
45.22
B5.08
51.45
81.57
120.00
73.56
00.01
27. 56
35.33
0/2.22
FULL
65.21
49.03
65.75
54.69
83.50
54.64
74.74
60.40
29.50
37.07
57.49
syTOKeN 0.20
68:00
57:25
05:62
80.93
74.98
3:2.1
38.99
sSTOKEN (p 0.4)
66.70
35. 12.12
64.98
53.41
82,15
57.98
75.14
57.38
36. 25
37. 11
58.65
SSTOKEN
66.33
34.10
SSTOREN (p 0.8)
65.60)
52.13
65.22
54.95
$3.38
61.69
73.22
61.77
33.18
37.25
59.04
Base model:
2wwen22.5144p
BASE
50.42
58.45
79.80
82.32
28.50 (9.
75.06
63.36
38.10
64:27
60.55
FULL (o
68.36
53.77
79.75
82. $2.83
64.59
74.99
63.37
36.87
66.00
61.50
SsTOKEN (p 0.2)
68.71
53.88
78.57
56.66
82.49
61.34
75.61
59.41
33.00
641.33
63.40
sSTOREN (p 0.6)
64.07
52.90
70.58
56.57
83.63
71.94
75.85
62.62
38.71
65.46
65.16
SSTOREN RN 0.8)
66.55
32.96
79.97
06.22
82.49
71.94
76.24
62.98
42.70
66.30
05.84
Table 5: Comparison of token selection methods on Swen-225514PP at two ratios, p= 0.6 and p 0.8.
Metbods
TriviaQA TruthfulQA
MMLU
ARC-C
ARC-E
TyDiQA
Wino
HS
LagiQA
AGiEval
AVG
Base model: Q Qwen-2.5-140 2,5-14B
BASE
59.42
58.45
79.80
35,80
82.32
28.89 89
75.06
63.36
35.10
64.27
80.56
RANDOM RANDOM 0.8)
65.91
53.57
79.85
36.31
83.50
63.32
75.22
63.33
33.33
66.10
66.03
63.91
53.86
79.02
36.31
82.87
65.32
75.53
63.30
39.02
66. 06.19
61.02
RHO-1 (p=
62.16
51.71
79.55
36.06
82.91
69.50
76.48
62.65
35.33
65.07
66.14
64,80
ToKENCI FANING (p
65.58
51.35
79.70
35.80
82.40
70.02
76.56
62.34
38
65.
64.77
TOKENCLEANING (p= 0.5)
66.12
51.87
79.00
35.97
$3.08
69.37
70.56
62.33
38.56
64.86
64.87
SSTOKEN (6))
61.07
52.90
79.88
36.57
83.63
71.94
75.85
62 02.62
38.71
65.46
65.16
17

---

Published as a conference paper at ICLR 2026
C.2 DETAILED PERFORMANCE WITH ADAPTIVE P
In Sec. 4.4, we introduce training-stage-Due based strategy that adaptively adjusts the token selection ratio Experiments on LLaMA-3.22 show that this dynamic schedule achieves consistently better performance than all manually specified fixed values of p examined in our ablation study (i.c., E 0, 0.2, 0.4, 0.6, 0.8, 1.0). Table 6 reports the detailed results across the ten benchmark tasks.
Table 6: Comparison of ss Token with fixed and adaptive token selection ratios p LLaMA-3.2-3B
Methods
TriviaQA
TruthfulQA
ARC-C MMLU
ARC-E
TyDiQA
Wino
BS
LogiQA
AGIRval
AVG
BASE (p=
30.85
39.37
36.20
12.15
74.19
19.05
69:06
55.21
29.19
30.04
16.39
FULL (p 1)
52.50
43.97
52.34
45.56
76.42
39.48
70.56
$5.75
29.50
32.11
50.35
SSTOKEN
51.70
58.82
55.63
14.20
73.65
47.50
70.48
54.32
33.03
32.91
51.72
SSTOKEN (p
51.79
50.70
55.73
SSTOKEN (p mp==0.00 0.6)
54.04
47.42
57.072
45.82
76.42
51.04
70.17
56.19
32.87
33..97
52.50
SSTIKEN (p 0.8)
53.99
41.72
37.49
45.05
77.57
30.10
70.24
56.26
31.95
33.88
52:21
(ADAPTIVE
53.60
55.91
C.3
FINE-TUNING RESULTS
While main experiments in ssToken are performed under the LoRA fine-tuning regime to ensure consistency with prior token-level selection methods (Pang al., 2025), the method itself is agnostic to the underlying parameterization strategy. ssToken depends only on token-level loss signals and deep-layer attention distributions, which remain identical regardless of whether the model trained with LoRA adapters or via full-parameter updates. To verify this generality, we further conduct full- faramperreeeer supervised fine-tuning experiments on LLaMA-3.2-3 evaluating ssToken alongside several representative baselines under matched training configurations.
For full-pane hall-parameter SFT, we reduce the learning rate Ie-5 ensure stable optimization, while keeping all other hyperparameters consistent with the LoRA setting. The complete results are sum- marized in Table 7.
Table 7: Full-parameter fine-tuning results of L.AMA-3.2-3B across ten benchmarks. The best score in each column highlighted in bold.
Methods
TrivinQA
TruthfulQA
MMLJ
WinoGrande
HellaSwag
AGIEval
1.aMA-3.2-38
BASE
50.85
30:37
56,20
42.15
74.00
19.05
63.36
56.21
29.49
30.04
46.59
FULL--AAA DATA
51.11
40:25
56.54
43.60
76.81
48.40
70.48
66.71
30.66
32.73
60.63
RANDOM
50.00
41.22
66.99
43.11
76.54
45.26
70.17
56.22
31.41
31.68
60.24
RNO-1
42.85
56.74
77. 77.008
70.01
56.29
31.06
33.30
TOKENCI
56.53
SXTOKEN
44.37
56.71
45.52
70.32
55.99
33.00
33.44
As shown in Table and Table 7, ssToken consistently outperforms all baselines in both LoRA and full-parameter SFT settings, with highly similar improvement trends across the two fine-tuning regimes. This demonstrates that the effectiveness of ssToken is not tied to any specific parameteri- zation strategy and remains stable across different training configurations.
C.4 MULTI-EPOCH TRAINING AND EMA-ENHANCE SSTOKEN
To further evaluate the behavior of ssToken beyond the standard single-epoch SFT setting, we con- duct additional experiments under multi-epoch supervision and examine the effects of the optional EMA-based history model update. These experiments are performed on LLaMA-3.2-3B using a reduced learning rate avoid rapid overfitting. Some tasks (e.g., MMLU, ARC-Challenge, Hel- laSwag) converge rapidly within a single epoch and show limited gains in subsequent epochs. In contrast, other tasks exhibit steady improvements across multiple epochs, making them better suited strategies. We report results for three
for revealing the behavioral differences among representative tasks (TruthfulQA, LogiQA, and TyDiQA) in Table 8.
18

---

Published as a conference paper at ICLR 2026
Table Effect of EMA for ssToken across epochs on TruthfulQA, LogiQA, and TyDiQA The best score in each column for each dataset is highlighted in bold.
Method
Base
Epoch 1
Epoch
Epoch 3
TruthfulQA
Full-Data
39.37
42.80
43.06
42.86
Rho-1
39.37
42.62
43.17
43.50
TokenCleaning
39.37
42.89
43.46
44.67
ssToken (w/o EMA)
39.37
43.73
44.82
45.30
ssToken (w/ EMA)
39.37
43.41
45.13
45.72
LogiQA
Full-Data
29.49
29.19
30.57
31.03
Rho-1
29.49
31.18
33.03
33.79
TokenCleaning
29.49
31.49
32.57
32.26
ssToken (w/o EMA)
29.49
33.49
34.10
34.25
ssToken (w/ EMA)
29.49
33.72
34.23
34.50
TyDiQA
Full-Data
19.05
30.27
35.33
35.12
Rho-1
19.05
39.73
43.88
47.23
TokenCleaning
19.05
41.18
46.21
46.79
ssToken (w/o EMA)
19.05
41.83
45.72
47.50
ssToken (w/ EMA)
19.05
42.14
46.08
48.30
As shown in the results, for TruthfulQA, full-data SFT essentially saturates after the first epoch, whereas all token-selection methods continue to yield stable gains across additional epochs. Among them, ssToken consistently maintains the strongest performance across epochs.
Comparing soToeen with and without EMA, we observe that while the difference at epoch is small, EMA consistently outperforms the non-EMA variant by epoch 3 across all three tasks. This suggests that, compared with using a fixed history model, an EMA-updated history model provides more stable and informative guidance during long-hone long-horizon training. These results further support the potential of ssToken in multi-epoch and comtinual-leaming scenarios.
D ALGORITHMIC FRAMEWORK OF SSTOKEN
In this section, we summarize the overall algorithmic framework of ssToken in Algorithm high- lighting its two key components: self-modulated token selection and semantic-aware token impor- tance estimation.
E VISUALIZATION OF REL DYNAMICS DURING TRAINING
To better understand how self-modulation emerges during optimization, visualize the evolution of Retrospective Excess Loss (REL) throughout training. For each training batch, we compute maximum REL value across the response tokens of every sample and then average these values across the batch. The resulting learning curves are shown Fig. 4: Fig. 4a illustrates sokoeen without EMA updates, and Fig. 4b depicts the EMA-enhanced variant.
As shown in the figure, the batch-averaged REL increases sharply within the first 100 training steps, after which it continues to rise more gradually with small fluctuations. This indicates that token- level losses quickly develop sufficient separation, allowing REL-based token selection to become effective very early stage of training.
Moreover, the presence of the attention- based token selection signal further stabilizes etrly-stage token selection, helping the model focus instraction-relevant tokens before REL fully converges.
19

---

Published as a conference paper at ICLR 2026
Retrospecctvv Eacess Loss (TEL)
Retroppective (RELL)
Ama
12.00
Sleg
Slep
(a) REL dynamics of ssToken without EMA up- dating.
(b) REL dynamics of ssToken with EMA updat- ing.
Figure 4: Visualization of the evolution of Retrospective Excess Loss (REL) during training
This complementary effect accelerates the emergence of meaningful REL values and contributes to the rapid performance improvement observed in the early training phase.
Algorithm 1 ssToken: Self-modulated and Semantic-aware Token Selection
Input: Dataset D, base model 8 token selection ratio p. balance coefficient Ty.
2: Initialize history model Onis +-
3: for 1 to T do
4: Sample a sequence æ from 'D
5: Compute REL(x;; for each token via Eq.
6: Compute attention score Att)(x;; via Eq. 7
7:
Normalize REL(x;; via Eq.
8:
Calculate combined score:
Score(z;)
Normalize(RHI((:)) + - 7) AttnScoree
9: Select top-p tokens by Score(x;)
Update ( with selected tokens
11:
Optional: Update Oni (e.g., EMA, Eq. 4)
end for
13:
Return: Fine-tuned model gt
F CASE STUDIES OF TOKEN SELECTION
In this section, we present several case studies to illustrate the outcomes of different token selection strategies. For each sample, the tokens selected by given method are highlighted in blue. In particular, for SsTOKEN, we additionally use orange to mark tokens that are primarily selected due their high attention scores, distinguishing them from those selected mainly by loss-based signals. These visualizations provide an intuitive understanding of how various approaches behave and highlight the complementary role of semantic-aware signals in our method.
20

---

Published as a conference paper at ICLR 2026
Sample
User:
Write a response that appropriately completes the request.
### Instruction:
Name one type of organic compound.
### Response:
One type of organic compound is an alkane, which is saturated hy- drocarbon that contains single bonds between carbon atoms.
Assistant (RHO-1):
Assistant (TokenCleaning): One type of organic compound an alkane, which satu- rated hydrocarbon that contains single bonds between carbon atoms.
Assistant (ss (ssToken); One type of organic compound an alkane, which is a saturated hy- drocarbon that contains single bonds between carbon atoms.
Sample 2
User: Provide instructions on how to accurately perform a bicep curl.
Assistant (RHO-1): 1. Stand with your feet shoulder -width apart and grab a dumb bell in each hand with your palms facing forward. 2. Keep your elbows close to your body and slowly lift the weights towards your shoulders. while exhaling Remember keep your wrists straight and engage your biceps to lift the weight 3. Pause for a second at the top of the curl, squeezing your b iceps. 4. Slowly lower the weights back down the starting position, while inhaling for the desired number of reps. 6. To avoid injury, avoid swinging your arms or us- ing momentum to lift the weights Keep your core engaged and focus on isolating your iceps for maximum benefit.
Repeat the exercise
Assistant (TokenCleaning): 1. Stand with your feet shoulder -width apart and grab dumb bell in each hand with your palms facing forward 2. Keep your elbows close to your body and slowly lift the weights towards your shoulders while exhaling Remember keep your wrists straight and engage your biceps lift the weight 3. Pause for second the top of the curl, squeezing your b iceps 4. Slowly lower the weights back down to the starting position, while inhaling. 5. Repeat the exercise for desired number of reps. 6. To avoid injury, avoid swinging your arms using momen- tum to lift the weights Keep your core engaged and focus on isolating your iceps for maximum benefit.
Assistant (ssToken): 1. Stand with your feet shoulder-width apart and grab a dumb bell in each hand with your palms facing forward. Keep your elbows close your body and slowly lift the weights towards your shoulders. while exh aling, Re- member to keep your wrists straight and engage your iceps to lift the weight. 3. Pause for second at the top of the curl squeezing your biceps. 4. Slow ly lower the weights back down to the starting position, while inhaling. 5. Repeat the exercise for the desired number of reps. 6 To avoid injury avoid swinging your arms or using momentum to lift the weights Keep your core engaged and focus on isol ating your biceps for maximum benefit.
21

---

Published as a conference paper at ICLR 2026
Sample
User: In a restaurant kitchen, a chef uses food processor to prepare a sauce. The recipe calls for 2 cups of chopped tomatoes, I cup of diced onions, and 1/2 cup of minced garlic. The food processor has a maximum capacity of cups. If the chef wants to make enough sauce for 6 servings, how many times must the ingredients be processed in the food proces- sor? Hint: Assume that the food processor can process the entire recipe at once.
Assistant (RHO-1): The recipe requires a total of ents. To make enough sauce for 6 servings, the chef needs to prepare 6 * 3.5 = 21 cups of in- gredients in total. Since the food processor can process up to 4 cups of ingredi- ents at time, the chef needs to process the ingredients 21 / 4 5.25 times. How- ever, since the chef cannot process ingredients a fraction of a time, they will need to pro- cess the ingredients 6 times to make enough sauce for 6 servings.
+ 0.5 3.5 cups of ingredi-
Assistant (TokenCleaningl) The recipe requires a total of 2 1 0.5 = 3.5 cups of ingredi-
ents. To make enough sauce for servings, the chef needs to prepare 6 $33.5 = 21 cups of in- gredients in total. Since the food processor ents at a time, the chef needs to process the ingredients 21 / 4 = 5.25 times. How-
process up to 4 cups of ingredi-
ever, since the chef cannot process ingredients fraction of time, they will need to pro- cess the ingredients 6 times to make enough sauce for 6 servings.
Assistant (ssToken): The recipe requires a total of 2 + 1 + 0.5 3.5 cups of ingredi- ents. To make enough sauce for 6 servings, the chef needs prepare 6 * 3.5 21 cups of in- gredients in total. Since the food processor can process up to 4 cups of ingredi- ents a time, the chef needs to process the ingredients 21 / 4 5.25 times. How- ever, since the chef cannot process ingredients a fraction of time, they will need to pro- cess the ingredients times to make enough sauce for servings.
Sample 4
User: Using a deterministic algorithm, please generate a set of 3-6 integers, all of which must fall within the inclusive range of 10 to 100. These integers must satisfy the following conditions: the sum of all numbers in the set must be greater than or equal to 200, the dif- ference between the largest and smallest numbers must be greater than or equal 50, and no two numbers in the set may have a common factor greater than 1. Please provide the resulting set of integers.
Assistant (RHO-1): Here is set of integers that satisfies the given conditions: 13, 31, 97, 49 The sum of these numbers is 190, which is less than 200, so we can add an- other number tions. The sum of the numbers is 21 1, which is greater than or equal to 200. The differ- ence between the largest and smallest numbers is 78, which is greater than or equal to 50. Fi- nally, no two numbers in the set have a common factor greater than 1.
the set. 11, 29, 89, 47, 35 This set satisfies all the given condi-
Assistant (Token Cleaning): Here is a set of integers that satisfies the given conditions: 13, 31, 97, 49 The sum of these numbers is 190, which is less than 200, so other number to the set. 11, 29, 89, 47, 35 This set satisfies all the given condi- tions. The sum of the numbers is 211, which is greater than or equal to 200. The differ- ence between the largest and smallest numbers is 78, which is greater than or equal to 50. Fi- nally, no two numbers in the set have a common factor greater than 1.
add an-
Assistant (ssToken): Here is set of integers that satisfies the given conditions: 13, 31, 97, 49 The sum of these numbers is 190, which is less than 200, so we can add an- other number tions. The sum of the numbers is 211, which is greater than or equal to 200. The differ- ence between the largest and smallest numbers is 78, which is greater than or equal to 50. Fi- nally, no two numbers in the set have a common factor greater than 1.
the set. 11, 29, 89, 47, 35 This set satisfies all the given condi-
22


Reviews:
[{"review_id": "bcQE7OkZXR", "reviewer": "Reviewer_Fmnc", "summary": "This paper introduces ssToken, a token-level selection algorithm for fine-tuning Large Language Models (LLMs). Unlike previous token-level selection methods that rely on a reference model, ssToken employs a historical model as a proxy to dynamically compute the loss difference with the current model. The authors also develop an attention-based metric to assess token importance. These two metrics are then combined to select important tokens. Experiments conducted across various models and benchmarks demonstrate the effectiveness of ssToken.", "questions": "See **Weaknesses**", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. ssToken does not require a pre-trained reference model, making it more cost-effective for token selection.\n\n2. In addition to the loss-based metric, the authors also developed an attention-based metric to evaluate token importance.\n\n3. The experiments are comprehensive and demonstrate the strong performance of ssToken.", "weaknesses": "1. There is a contradiction between Equation (2) and Equation (3) that requires further explanation. In Equation (2), tokens are ranked highly if the current model’s loss on the token is large. In contrast, Equation (3) prioritizes tokens for which the current model’s loss is small. The authors should provide more insight into the rationale behind this difference.\n\n2. Is the calculation in Equation (3) performed per training step or per epoch? How does the frequency of this calculation affect token selection performance?\n\n3. Could the authors provide some theoretical justification or analyses for why using the current model (rather than a fixed reference model) for loss calculation leads to better performance?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H9wE2p1pj1", "reviewer": "Reviewer_P6EM", "summary": "This paper proposes ssToken, a self-modulated and semantic-aware token selection method for LLM supervised fine-tuning. It addresses limitations of existing methods (needing extra reference models and relying only on loss) by using historical models to compute Retrospective Excess Loss (REL) and an attention-based semantic metric. Experiments on 3B-14B models (LLaMA, Qwen) across 10 benchmarks show ssToken outperforms full-data fine-tuning (up 4.3%) and prior token methods (up 2.8%), maintaining efficiency. It has a limitation of needing manual ρ setting, with future work on adaptive ρ.", "questions": "1. You used manual token selection ratio (ρ) – did you test any initial adaptive ways to adjust ρ (e.g., linking to model loss progress)? If not, what’s your hypothesis for designing such a mechanism?  \n2. You mentioned EMA-updatable history models but used fixed ones – what EMA hyperparameters (e.g., α) did you test? Do you plan to try EMA in longer training where fixed models may lag?  \n3. You chose deeper attention layers for semantic selection – could you test if layer choice differs by task (e.g., QA vs. reasoning) to strengthen the method’s generality?  \n4. You noted compatibility with FlashAttention – can you quantify the computational cost of your attention calculation for sequences longer than 2048 tokens?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It creatively combines self-modulated signals (via the model’s historical trajectory for REL) and semantic-aware attention metrics, solving prior reliance on external models and loss-only selection.\n2. It has rigorous methodology with grounded components and lightweight implementation, plus sufficient validation across models/benchmarks and ablation studies ensuring quality.\n3. It follows a clear problem-method-result structure, with precise technical definitions and transparent limitations, enabling easy understanding.\n4. It boosts LLM fine-tuning performance without extra costs, advancing semantic-informed self-evaluative data selection with academic and practical value.", "weaknesses": "1. It relies on manual tuning of the token selection ratio ρ, and without an adaptive mechanism to adjust ρ based on model capacity or data quality, it adds overhead for practitioners and limits generalization across diverse model families or domains.\n2. The optional EMA-based update of the history model is not fully explored—experiments only use a fixed base model as the history model, leaving unaddressed whether adaptive history model updates could bring more stable guidance in large-horizon training scenarios.\n3. It lacks in-depth analysis of why deeper attention layers outperform shallow ones in semantic-aware selection beyond citing prior studies, and no discussion on how layer selection might vary across different task types (e.g., QA vs. reasoning) limits methodological completeness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "52FTTlPYSB", "reviewer": "Reviewer_D1M4", "summary": "This paper proposes ssToken, an adaptive token selection approach to address two major issues in existing traditional approaches: (i) adopting a stronger model as the reference model for the token selection is not practical, and (ii) the token-level losses do not caption the semantic importance of tokens in the given contexts.  The proposed method doesn't rely on an additional reference model and using its history to improve the token selection and introduces a new semantic-aware token importance estimation metric. The empirical evaluation shows that the proposed method achieves the siginificant improvemnts in existing approaches.", "questions": "In Table 1, the performance of TokenCleaning is lower reported. In their paper's Table 1, *Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning*, the AVG score of LLaMA-3.2-3B is 53.00 (for self-evolving clearning). It seems that the difference is caused by the difference in the evaluation set; so, I have a few questions: \n1. Why is a different evaluation dataset, instead of the same one used in TokenCleaning's Table 1, used in this submission? \n2. How do you choose the evaluation set? Is it common to use a different evalaution set in this field? I noticed that all baseline approaches in their own papers (e.g. TokenCleaning, Pho) are using different evaluation sets (and even training set or models for some baseline papers). How can I really know which approach is better when there is no standard way to compare different methods?   \n3. Also, should the LLM generation process have some randomness? As the next token prediction is based on a sampling procedure instead of simply taking the largest possible token. However, I found all three of this submission,  the TokenCleaning paper, and the Pho paper, didn't report the confidence interval in the main experimental result table. It seems to be a common rule in this field but I don't understand that.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "This paper is well written. It clearly presents the current disadvantages of traditional approaches and those statements are convincing. \n\nThe experiental results are solid. It has covered multiple modern LLMs including LLaMA-3.2 & 31 and Qwen-2.5 in different scales. Also, the results are evaluated in multiple benchmarks.  \n\nThe ablation studies cover all necessary components.", "weaknesses": "I am not in this field so I am afraid that I may not be able to identify any key weakness of this work. So, I set my score 8 (a clear acceptance). I do have a few questions just for clarification (see Questions section). They do not affect my scores.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g8woPKGdrS", "reviewer": "Reviewer_RRS3", "summary": "The paper proposes ssToken, a token-level data selection method for SFT that (i) removes the need for an externally trained or stronger reference model and (ii) augments loss-based selection with a lightweight semantic-aware signal derived from attention. \n\nInstead of computing “excess loss” against a fixed reference model, the authors consider Retrospective Excess Loss (REL) computed against a history model, which is futher combined with attention-based token importance, another measurement of token weight by summing up attention scores over all prompt tokens. Lastly they consider the convex combination of the two signals and only train on top-x fraction of response tokens.\n\nEmpirically, they showed ssToken outperforms both full-data SFT and prior token selection baselines across various base models and benchmarks.", "questions": "1, How many extra forwards/backwards per step are required to compute REL and the attention metric? Is REL computed with an additional pass through the history model each step/batch, or are there cached losses? \n2, You mention an optional EMA but ultimately fix the base model as history due to small data. For larger/longer SFT, do you expect EMA to stabilize selection and reduce early randomness? Can you share a small-scale ablation to illustrate the trend?\n3, In prompt based attention approach, what happens if the response context is long? Will position bias become a problem? Does your analysis cover this regime?  \n4, Which exact layer index do you use across models? Do model families disagree on which depth is best?\n5, Have you considered head-weighted aggregation (e.g., entropy-based or gradient-based head importance) rather than simple averaging?\n6, can you justify why omitting response tokens' contributions in the attention approach? \n7, Do the conclusions change for full-parameter SFT vs LoRA? Can you add ablation studies to support this?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1, The paper proposed an innovative prompt-focused attention metric\n2, Details are well discussed, such attention-layer’s preference, hooks+recompute approach for FlashAttention compatibility, \n3, The paper is well structured (motivation → method → ablations → results) with comprehensive figures and explicit algorithmic details\n4, The topic of token-level data selection is an increasingly important topic for better data-efficiency. The methods introduced in the paper are easy to adapt and implement.", "weaknesses": "1, While the author claims the attention based measurement is lightweight and no reference training is needed, it still helps to quantify the additional costs of 1) forward passes to compute REL using history model (2) attention recomputation, and especially when scaled to larger models and longer contexts\n2, The paper notes that if history agrees with the current model at the beginning then the selection is “nearly random”, and suggests EMA, but eventually admits that EMA doesn’t yield gain and leads to additional overhead. What’s the take away on this discussion? The authors are expected to explain why this is the case, and whether this is due to small SFT. Does adaptive history update help on longer training horizons or larger corpora?\n3, The attention-based approach focuses exclusively on attention from prompt, and overlooks contributions from other response tokens. This omission can be problematic in reasoning-heavy tasks such as mathematical problem solving, where the model must follow a chain of logic across intermediate steps. In such cases, the next step often depends more on information contained in previous reasoning steps than on the original prompt. Ignoring attention flows among response tokens may therefore hurt model’s ability to capture these intra-response dependencies.\n4, A more concerning point is that all experiments were performed with LoRA, which is a non-standard approach for fine-tuning LLMs. This leaves the audience wondering if the method works for full tuning and or even pretraining.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "57GwFhqRPx", "reviewer": "Reviewer_4Xmz", "summary": "This paper proposes ssToken, a token-level data selection method for supervised fine-tuning (SFT) of large language models. The key contributions are: (1) a self-modulated selection strategy using retrospective excess loss (REL) that eliminates the need for training a separate reference model, and (2) a semantic-aware attention-based metric that complements loss-based selection. Experiments across models ranging from 3B to 14B parameters demonstrate improvements over full-data fine-tuning and prior token selection methods.", "questions": "1. Can you provide some analysis of when attention to prompt is vs. isn't a good indicator of token importance?\n2. Can you show learning curves of REL values over training to demonstrate when self-modulation becomes effective?\n3. Have you tried adaptive selection ratios (e.g., selecting different ρ per sample based on quality)?\n4. What happens if you use ssToken for continual learning or multi-epoch training?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Well-motivated approach: The paper clearly identifies two practical limitations of existing token selection methods (reference model requirement and sole reliance on loss) and proposes targeted solutions.\n2. Novel self-modulated paradigm: Using the history model instead of a separate reference model is elegant and practical. The formulation of REL is intuitive and eliminates additional training costs.\n3. Complementary signals: The integration of attention-based semantic information with loss-based selection is well-motivated. The paper demonstrates that these signals are indeed orthogonal and mutually reinforcing.\n4. Comprehensive experiments: Evaluation across multiple model families (LLaMA and Qwen), sizes (3B-14B), and 10 benchmarks provides good coverage. Ablation studies on γ and ρ are thorough.\n5. Practical implementation: The lightweight attention extraction design compatible with FlashAttention is a thoughtful engineering contribution.\n6. Consistent improvements: ssToken shows stable gains across different settings, which is valuable for practical adoption.", "weaknesses": "1. The reference models in baseline methods are trained on only 10k samples (20% of data), which may not provide fair comparison. A stronger reference trained on more data might close the gap.\n2. The claim of \"maintaining training efficiency\" (Fig. 2) might be misleading since tokens still participate in forward pass. The computational overhead reduction is minimal.\n3. The paper acknowledges that \"history and current models share identical parameters at the beginning of training, which makes early-stage token selection nearly random\" (line 210-212). However, there is no analysis of how many steps are needed before REL becomes effective, nor any experiments showing performance with different initialization strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)