Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Authors: Yuchen Yang
Wenze Lin
Enhao Huang
Zhixuan Chu
Hongbin zhou
Lan Tao
Yiming Li
Zhan Qin
Kui Ren
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Fine-tuning
Token-level Data
Primary area: optimization

Abstract: Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

EXPLAINABLE TOKEN-LEVEL NOISE FILTERING FOR LLM FINE TUNING DATASETS
Yuchen Yang" :1.2, Wenze Lin 33, Enhao Huang! Zhixuan Chu Hongbin Zhou', Lan Tao*, Yiming Li,, Zhan Qin Kui Ren
The State Key Laboratory of Blockchain and Data Security, Zhejiang University Hangzhou HighTech Zone (Binjiag)Bloocchaiin and Data Security Research Institute, China TTinnhuaa University Allibbba Group *Nanyang Technological University
ABSTRACT
Large Language Models (LLMs) have remarkable advancements, achieving state-of-the-at results in diverse applications. Fine-tuning, an important step for adapting LLMs specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current tne-ttming datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance). which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly optimize the performance fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.
INTRODUCTION
LLM technology An et al. (2024); Nam el. (2024); Kambhampati et al. (2024) has developed rapidly in recent years, possessing powerful reasoning capabilities and enabling widespread application across various downstream tasks Thirunavukarasu et al. (2023b); Wen et al. (2024); Guo et al. (2024). Meanwhile, to enhance the performance of LLMs in specific downstream tasks, developers usually train the base models, i.e., general purpose LLM such as Llama Touvron et al. (2023) and Deepseek Guo al. (2025), using relevant datasets, making adjustments to the model parameters. This technique, known as fine-tuning, is widely used in practical applications Wang et al. (2022); Lin al. (2024a).
However, existing fine-tuning datasets do not align fully with the -oken-by-token optimization process of LLMs. While LLM fine-tuning involves computing a loss at each token and updating model parameters accordingly, most fine-tuning datasets are designed at the sentence level, providing label sentences as the target output. Since not all tokens (in the label sentence) are valuable for performance improvement Lin al. (2024b); Peng al. (2023), training in the entire label sentence can possibly introduce token-level noise and misguide the direction of convergence, ultimately reducing performance of fine-tuned LLMMs in the target downstream task.
Current research lacks the capability to optimize Mainstream data optimization methods fall into two categories: data filtering Li al. (2019); Goyal
datasets at the token level for LLM fine-tuning tasks.
Equal contribution Corresponding Author
Correspond zhixuanchu @ edu.n

---

et al. (2024) and data augmentation Dai et al. (2025); Ding al. (2024). All of these approaches operate at the sample level and thus fail to further eliminate token-level noise. Some existing studies have explored the differences between token-level and sentence-level data from various perspectives, such as pretraining Lin et al. (2024b), human preference optimization Zeng et al. (2024); Yoon et al. (2024), and knowledge distillation Wei et al. (2024); Cui et al. (2025). However, these works are often limited to specific scenarios (e.g., pretraining with lower data quality requirements or direct preference optimization (DPO) Rafailov et al. (2023) that relies on prior knowledge of labeled text pairs) or do not sufficiently investigate the value differences between tokens (e.g., as in some knowledge distillation approaches Peng et al. (2023); Cui et al. (2025)), rendering them unsuitable for fine-tuning dataset optimization.
Achieving token-level dataset optimization for fine-tuning necessitates filtering out tokens in output labels that do not contribute to final performance, which is a non-trivial task. Firstly, no existing research clearly elucidates the relationship between individual tokens within these labels and fine- tuning effectiveness. Although some explainability studies can identify connections between tokens in the input content and the correct generation of label sentences during the reasoning process Chu al. (2024); Zhao et al. (2024), they cannot explain the value of specific tokens in label sentences for fine-tuning tasks. Secondly, fine-tuning performance depends on both the base model's pre-existing knowledge and the specifies of the target task. When filtering noisy tokens, it is essential consider both the base model's understanding of the data and that data's 's relevance to the downstream task Liu et al. (2022b); Han al. (2024). Therefore, filtering noisy tokens from fine-tuning datasets requires comprehensive consideration of fine-tuning task requirements, rather than reliance on single assessment criterion.
Motivated by the preceding discussion, we propose an explainable token-level data filtering method XTF. This method aims to assess the value of token-level data within fine-tuning datasets and filter out noisy tokens by considering the specific characteristics of LLM fine-tuning, XTF consists of three phases. In the first phase, we decompose the contribution of data to the fine-tuning effect into three attributes (reasoning importance, knowledge novelty, and task relevance) to reduce the complexity of token value analysis. Concurrently, we define the criteria for identifying noisy tokens based these attributes and provide theoretical justification in Appendix A. In the second phase, while considering the two key factors essential to fine-tuning: base model and task dataset, we design scoring mechanisms for the three attributes with controllable computational costs: 1) Reasoning Importance: We concatenate the input and label output, then compute attention scores for each token using the base model. A lower attention score indicates lower reasoning importance. 2) Knowledge Novelty: We introduce the probability of correct token prediction (PCP) to quantify the novelty of knowledge learned from the fine-tuning dataset. A higher PCP indicates lower knowledge novelty. 3) Task Relevance: We assess task relevance using embedding vectors generated by the base model for context-free inputs. The task domain is approximated by the average embedding of data samples, and token relevance scores are determined by their distance from the domain center. A larger distance implies lower task relevance. We present detailed scoring procedures in Section 3.2. In the third phase, we identify noisy tokens based on the statistical results and mask the gradients associated with these tokens during training to enhance the performance of the fine-tuned LLM. We adopt a conservative strategy to ensure the filtered tokens align with the criteria established in the first phase.
We conduct extensive experiments on datasets across 3 different downstream tasks and 7 representative LLMs. As shown in Figure 1, XTF achieve up to 13.3% and 13.7% accuracy optimization on math task and medicine task, respectively, demonstrating its effectiveness on noise filtering and fine-tuning enhancement. In code generation, our method achieves performance optimization of up to 5.6%. 5.6%, and 6.3% on pass@1 1, pass@5, @5, and pass@) 10, respectively, demonstrating that XTF is also effective in complex tasks involving multi-chance generation.
Our main contributions are three-fold:
We reveal the research gap of token-level data optimization for LLM fine-tuning.
We explore solutions for filtering token-level noise in fine-tuning datasets via three decomposed attributes: reasoning importance, knowledge novelty, and task relevance, proposing XTF.
We conduct extensive experiments of XTF across multiple representative LLMs and downstream tasks, verifying its superior performance on fine-tuning optimization, demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.
2

---

BBLLRA)
Lianna
spadi
Mitral-71
Midral-7p 32
Lama-118
Midraa-77HLLRAA
24
UP PLLmma118
DeepSeck:
Deepheeh-1.511
DeepSeck 14B1LRRA
DeepSerk-77i
DespSicck (BLLARA)
DeepSteke711
DeepScci--BLLLeRR
(a) Math task
(b) Medicine task
Figure 1:
The accuracy performance of our method on different LLMs. The results show that our method can significantly improve the final performance of fine-tuned LLMs across almost every case.
BACKGROUND AND RELATED WORK
2.1 LARGE LANGUAGE MODELS FINE-TUNING
Fine-tuning large models is widely recognized as key technique enhance their performance downstream tasks Wang et al. (2022); Han et al. (2024); Lin et al. (2024a). Numerous task-oriented models have been developed by fine-tuning high-performance base models, such Llama- Llama-Math Muñoz et al. (2024) and Llama-Finance Cheng et al. (2024). The quality of the fine-tuning dataset critical factor that determines the effectiveness of this process Zhou et al. (2023); Kuramoto & Suzuki (2025). Therefore, optimizing data is essential for enhancing fine-tuning outcomes.
2.2 EXPLAINABILITY FOR LLMS
Explainability methods for LLMs aim to explain the decision-making mechanisms and behavioral logic of these models Zhao et al. (2024); Chu al. (2024). However, current techniques predominantly focus on the reasoning process Huang et al. (2023); Kang et al. (2024); Singh et al. (2024), while research explaining the relationship between tokens and fine-tuning outcomes remains limited.
2.3 TOKEN-LEVEL TRAINING
Several pioncering works have explored token-level data refinement training, broadly categorized into three domains: data distillation, direct model training, and human preference training. Data distillation approaches Wei et al. (2024); Cui et al. (2025); Liu et al. (2022a) primarily focus on the performance differences observed when student model learns from token-level outputs (i.e., logits) versus sentence-level outputs, rather than specifically differentiating the value among individual tokens. Direct model training, including studies training small-scale transformer models Peng et al. (2023) and LLM pretraining Lin et al. (2024b), utilizes changes in loss values to guide the selection of valuable tokens. We observe that these methods operate under an implicit assumption: that high-qualit datasets are entirely noise-free and thus capable of correctly guiding token selection. This assumption rarely holds true in fine-tuning tasks, as base models often already exhibit strong performance, making it challenging to optimize dataset quality sufficiently to satisfy this noise-free free criterion. Furthermore, existing works Peng et al. (2023); Lin et al. (2024b) do not demonstrate that high-quality datasets are, in fact, approximately devoid of noisy tokens, which is critical premise for these methods. Human preference training often involves specific optimization frameworks that rely prior knowledge of labeled text pairs Zeng al. (2024); Xia et al. (2024); Yoon et al. (2024) and the construction of specialized token-level reward models; however, this approach can be challenging generalize to broader application scenarios.

---

Sentence-level
1004100
Input tokens
Output tokens
Reasoning
Knowledge novelty
Task relevance
importance
Format data
Domsin vector
Input ids
LLM
LLM
Semantic distance
Full-sequence
Token prediction
tabels
100 1000 100- 1100--00--10 Lodeie
Fine-tuning
token moise
Without token moise
score
score
score
Fine-tuning
Quantile method
Prob threshoki
Multi-Ostu
Poor performance
Good performance
Token-level
The pipeline of token-level data filtering comprises three steps. In the first step, we preprocess the dataset based regular format function. In the second phase, get the sentence- level data item and assess three scores, i.e., attention score, PCP score and relevance score, for the tokens of output label, suggesting the selection of noisy tokens. In the third phase, we mask the noisy tokens and fine-tune the target LLM.
Figure 2:
METHODOLOGY
3.1 WHICH DATA ACTS NOISE FOR FINE-TUNING?
Due to scale differences, it is inherently difficult to intuitively assess the impact of token-level data on the final fine-tuning outcome. Therefore, we attempt to explain the contribution token data to fine-tuning from three attributions through theoretical analysis.
fine-tuning process can be conceptualized as an alignment between a high-performance base model and task-specific dataset. Consequently, the performance of the fine-tuned model should be influenced by three factors: the cognition of the base model, the knowledge in the task dataset, and the contradiction between the base model and the task dataset. When we aim to mask token from the label sentence, we can assess its potential impact on the fine-tuning result from these three perspectives. Specifically, we propose three attributes that positively influence the fine-tuning process: for the cognition of the base model, we extract reasoning importance; for the discrepancy between base model and task dataset, we extract knowledge novelty; and for the knowledge the task dataset, we extract task relevance. These attributes represent the following meanings:
Reasoning Importance (RI): whether the presence absence of this token significantly affects the base model's inference results;
Knowledge Novelty (KN): whether the presence of this token is novel to the base model;
Task Relevance (TR): whether the presence of this token is related to the objective of the task dataset.
not feasible to consider all three properties simultancously and assign a composite score, as there is no clear basis to determine their interrelationship or any hierarchical order. However, we can still use these three properties identify which tokens are noise. Specifically, we find that if token completely lacks any of the three attributes, it can be considered noise. This is intuitive; for example, if token is entirely unrelated to the task objective (lacking the TR attribute), then even it may influence the base model's subsequent generation results (RI) represent prediction that the base model has not yet learned (KN). it does not contribute to the fine-tuning task. We elaborate on the analysis of the three derived attributes and formally prove the correctness of this judgment through logical reasoning in the Appendix A.
Formally, the teken-level noise in the fine-tuning tasks be represented as:
D paise (DRr)) U (DKNL) U (Drn
(1)
4

---

where (DRi)), (DKN.) and (DrRi) respectively represent data lacking reasoning importance, knowledge novelty, and task relevance.
3.2 How TO ASSESS TOKENS?
After obtaining the three attributes and defining noise data, we employ a parameterized scoring mechanism to separately assess the three attributes and to identify noise tokens in the dataset. In alignment with real-world requirements, the choice of scoring method is expected to meet two requirements: 1) Controllable computational cost: Since the dataset contains vast number of tokens, the assessment method should not impose excessive computational overhead. 2) Joint consideration of the base model and the data: The assessment of the three attributes must not be separated from the essential elements of the fine-tuning task itself. Consequently, we adopt three reasoning-level explainability methods assess the three attributes separately.
Attention Score for RI. We employ the base model's 's attention scores to assess reasoning importance. Attention is a mechanism that adaptively learns the contextual importance of tokens during the pretraining of the base model Vaswani et al. (2017). Existing work has demonstrated that masking low-attention tokens during the LLM reasoning process can even enhance generation quality Gupta et al. (2021). Therefore, using attention scores to assess reasoning importance aligns with our needs. We input the entire text (including both the input sentence and the output label sentence) into the model and compute the attention scores. We formulate the reasoning importance score SRI for the k-th token in the output label sentence as:
A
k)
(2)
where 9 denotes the parameters of base model, .A is the function to compute attention value, O represents the input tokens and output label respectively, and ly is the length of input tokens. This approach can be applied to explain the tokens the output label and considers the base model's understanding of the data throughout the answer generation logic.
PCP Score for KN. For knowledge novelty, we adopt the base model's probability correct token prediction (PCP). Intuitively, a lower probability indicates reduced confidence in predicting the token, suggesting that it is more likely to contain knowledge that the model has not yet acquired. We formulate the knowledge novelty score SKN as:
SKN 1- P (Ox I [Oa, On...,0--1)).
Distance Score for TR. For relevance scoring, we calculate semantic distance based on the base model's 'ss embedding layer. First, we feed each data item from the dataset into the base model in its entirety and obtain the average value of the embedding layer vectors as the domain vector for the fine-tuning task. Then, we collect all tokens appearing the dataset and compute their context- free embedding vectors. Finally, we measure the distance between these token vectors and the domain vector, using this distance as the relevance score. The specific formulas for the task relevance scores follows:
V(Domiin) Σ (E(0,,rrp.)) nu Srn(Ox) = 1-Normalize (D (E())), V(Domain))).
where & denotes the function to get the embedding vector of inputs, erpw and new represents the expert words and its number, respectively. D is the function compute the distance of two vector.
3.3 HOW TO FINE-TUNE BASED ON SCORES?
In this section, will describe how to correctly filter noisy tokens using conservative strategy and ignore them during the fine-tuning process through gradient masking.
Token Filtering. After obtaining scores for each dimension, need to filter the noisy tokens based on the scores. As shown in Figure 3, the distributions of scores across different tasks share the same features, requiring to design adaptive mechanism for each attributes. Specifically, the
5

---

Medicine
SRt Deepseek
(b) SKN Deepseek
(c) STR Deepseek
Figure 3: Distribution of the three scores across different datasets on Deepseek- 1.5ssekk-1.5.... The reasoning importance score is distributed across some fixed values, the knowledge novelty score has somewhat uniform distribution, and the task relevance score's 's distribution exhibits clustering features. More distribution figures of different L.Mss are shown Appendix D.1.
0.25
025
0.25
6-122
Fitrrd Rais
Fillered Raj
0.18
0.16
0.15
0:10
0.10
0.10
0,000
0,0150
2.1125
0.111
0.00
0.00
0.00
Onl R3
Ahor XN
Abor TR
Aftor KiN TR
Only
Attr
Aftr TR
After RI TR
Only T
After RI
Afr KN
Aftor RI&
(a) Filtering
(b) KN Filtering
(c) Filtering
Figure 4: Complementarity among attributes using Deepseek-1.5D and GSM8K. Using RI as example, Only RI represents the percentage of tokens that can be filtered using only RI. After KN represents the percentage of tokens that RI can filter after KN applied. The reduction tokens when comparing After KN with Only RI indicates the tokens on which KN and RI overlap.
reasoning importance scores exhibit extreme distribution where many scores share identical values and demonstrate significant differences after normalization. We directly apply the quantile method (Interquartile Range) filter tokens with extremely low scores. mathematical expression:
Q1, Q3 = Quantile(Szz(()), 75)).
TQR= = Q - -
(Q3-Q1)
O₂ E (DRIL) ) if (Rr(Or) < Q1 IQR.
where (O) means all the reasoning importance score of tokens in output label O. The knowledge novelty scores display a uniform distribution, which makes it difficult distinguish low scores. Therefore, we adopt a heuristic threshold. We consider tokens with a PCP higher than 95% as only containing knowledge without novelty and treat them as noise. mathermatiaa expression:
O₂ E (DRNL) if (kx(O2) 0.05.
(6)
Finally, the task relevance scores exhibit cluster characteristics, for which we employ the Multi-Otsu method Liu & Yu (2009) to partition the scores. Since the cluster with the smallest mean value typically consists of space replacement symbols, we filter out the tokens in the cluster with the second smallest mean value. mathematical expression:
O₂ (Dtr.) if (TR(Ox) E M(STR) 2nd
A is the Multi-Otsu method and M(STR) 2nd denotes the cluster with the second smallest mean value. The detail of Multi-Otsu method is shown in Appendix B.
Threshold Analysis. XTF adopts an aggressive filtering strategy, namely, it takes the union of three types of filtered tokens. The premise that supports this strategy is that the filtered tokens are assumed to be "completely" devoid of the corresponding attributes. Therefore, tokens with significantly low scores should be selected for filtering, which is also why conservative threshold chosen. Meanwhile, within the XTF framework, filtering attributes across multiple dimensions provide complementary effects, compensating for noise omissions introduced by the loose threshold.

---

Figure 4 illustrates the complementarity among attributes: the overlap of tokens filtered by different attributes is not higher than 58.3%. This strategy is based on a simple yet useful heuristic: ambiguous noise that is difficult distinguish from one perspective can be clearly identified from another perspective. When noise is jointly evaluated from multiple perspectives, a complex problem is transformed into several simpler problems. To further explore the results of token filterin,, we have provided specific examples of token filtering in Appendix F.
Training the Model. Here we describe how to optimize fine-tuning based on token filtering results. As shown in Figure 2, when processing the data, because the fine-tuning task focuses solely on the correctness of the output, we mask the input tokens by assigning them a default value (often [-100]) and thereby exclude them from gradient computation. After identifying noisy tokens in the output labels, we mark them with the this default value and use the resulting data for fine-tuning. Given the noisy token list N, the loss function Lr for learning data item can be expressed as:
CF =
log P(O I++ [Ou, O1...., OPEN
Σ
...... Ok-1l).
(8)
4 EXPERIMENTS
4.1 EXPERIMENT SETTINGS
Dataset, We select three representative downstream tasks to evaluate the fine-tuning performance, including two mainstream tasks: math, and code (widely used to evaluate LLMs Guo et al. (2025); OpenAl et al. (2024); Team et al. (2024)), as well as an important specialized tasks: medicine Thirunavukarasu
al. (2023a); Alberts et al. (2023). For the math task, we employ the GSM8K Cobbe et al. (2021) for fine-tuning and evaluation. For the code task, we fine-tune on the Codefixercise AI (2025) and evaluate using HumanEval Chen et al. (2021). For medicine tasks, we employ the PubMedQA Jin et al. (2019) for fine-tuning and evaluation. We also conduct additional experiments based on the NuminaMath-CC et al. (2018) datasets, but due to space limitations, these results are presented in the Appendix C.I.
Li et al. (2024), MATH-500 Hendrycks et al. (2021), and FIQA Maia
LLMs. We select 7 different base LLMs of varying scales from three outstanding model families: DeepSeck Guo et al. (2025), Llama Touvron al. (2023): Grattafiori et al. (2024) and Mistral Jiang et al. (2023). Specifically, for the Deepseek family, we choose DeepSeck- ---dddiiledd-ween 1.5B, 7B and 14B Bai et al. (2023). For the Llama family, we select Llama-3.2-11 1B, 3B and Llama-3. 1-BBB. For the Mistral family, we select Mistral-v0.1-7B.
Baselines. We consider 3 regular LLM implementations and 4 data enhancement methods to demonstrate the effectiveness of our XTF. Specifically, for regular LLM implementations, adopt clean accuracy (CA), i.c., using the original base model, normal fine-tuning (Normal) and more epochs (x2 epochs). Data enhancement methods can filter the data noise from different perspectives. Specifically, data filtering (DF) Li et al. (2019) filters noisy data at the sample level, data augmentation (DA) Dai et al. (2025) augments more data enhance the model's 's robustness, thus resist noise, selective language model training Lin et al. (2024b) trains the token-level data selectively based on changes in the loss value, and token cleaning (TC) Pang et al. (2025) performs fine- men-grained data selection for LLM supervised fine-tuning. More details are shown in Appendix B.2.
Metrics. We primarily use accuracy to evaluate the performance of LLMs on specific tasks. For tasks with a standard answer, we adopt a zero-shot form to pose the questions and use judge model Anthropic (2025) to determine the correctness of the responses. This evaluation method is less influenced by the format of the prompt and the model's response style. providing a clear and intuitive reflection of the fine-tuning effect. For the code completion task, we assess performance using @ass@1 1, pass@ and p 10 metrics, which are specifically used to evaluate code generation tasks Kulal et al. (2019).
Hy Hyperparameter: We conduct experiments based on existing work and strictly control the fairness of the results through hyperparameters, as detailed in Appendix B.3.
7

---

Table 1: Result of main experiment. We show the accuracy of LLMs across different fine- tunettunng methods. Best results are marked in bold and the second best results are marked with underline.
MATH: Fine-tuning and evaluate models on GSM8K
Model
[0]
LoRA
CA
Normal
x2 Ep
DF
DA
SLM
TC
XTF
Llama-3.2
IB
x
2.8
4.3
6.8
7.6
2.4
5.9
6.8
8.7
Llama-3.2
3B
X
3.9
25.8
33.4
36.9
27.1
38.8
38.4
40.5
Llama-3.1
8B
4.6
54.0
55.4
52.7
55.4
60.3
58.9
58.7
1lamm-3.1
8B
4.6
33.7
32.7
37.0
33.7
37.9
38.4
37.1
Mistral
X
15.0
16.1
21.3
22.6
29.1
Mistral
7B
23.4
20.7
25.6
21.8
21.3
20.7
25.6
1.5B
X
17.6
42.9
45.5
47.0
37.4
37.3
38.5
56,2
Deepseck-distee
bestilleedqwer
7B
37.9
63.0
68.1
65.5
56.5
63.8
61.9
69.3
7B
37.9
61.3
60.4
67.9
62.0
68.2
70.0
71.8
Deepseck-distilled
14B
34.5
47.6
50.3
52.4
50.5
49.3
52.1
60.3
Average
16.0
37.1
39.0
41.4
36.5
40.5
41.0
45.7
MEDICINE: Fine-tuning and evaluate models on PubMedQAA
Model
[0]
LoRA
CA
Normal
x2 Ep
DF
DA
SLM
TC
XTF
Llama-3.2
IB
x
2.9
15.5
15.6
15.4
18.3
13.1
12.4
18.5
Llama-3.2
3B
X
6.6
35.5
33.7
29.8
37.9
36.1
36.4
36.5
1lama-3.1
8B
X
4.9
13.6
14.0
18.4
14.7
22.9
22.9
21.3
Llama-3.1
8B
4.9
24.2
22.3
34.3
31.4
26.3
27.1
37.9
Mistral
7B
20.0
26.2
18.7
32.0
Mistral
7B
15.5
18.4
15.6
13.1
17.5
16.9
21.3
Deepseck-distilled-eee
1.5B
X
39.8
44.6
45.4
41.2
49.7
48.3
51.2
50.8
Deepseck-distin seeeek-ddssilled-qeeeeee
7B
42.7
50.5
42.1
55.4
52.7
51.3
51.6
55.6
neepsek--dsttiledd-e
7B
42.7
35.9
37.8
33.6
38.4
39.0
37.4
41.7
Deepseck boncck--isslled--eee -wwenn
14B
44.6
48.5
43.4
51.3
47.0
53.9
54.6
55.7
Average
20.6
30.4
29.9
31.4
32.7
33.5
33.7
37.1
4.2
MAIN EXPERIMENT RESULTS
We employ significant number of fine-tuning and three representative downstream tasks in our main experiment. By conducting fine-tuning experiments on different models using unified hyperparameters and datasets, and comparing the performance of the fine-tuned models, we can assess the effectiveness of the fine-tuning methods. During the training process, we employ training set, validation set, and test set prevent issues such as overfitting caused by varying convergence speeds. Specifically, the model parameters that perform best on the validation set are retained as the final model parameters and tested on test set. The reported results are test set accuracy.
Math. Math is an important downstream task in LLM research. It is widely adopted as the LLM benchmark, and the performance math task reflect the logic ability of L.MSs. As shown in Table 1, XTF has average 8.6% higher accuracy than normal fine-tuning and 4.3% higher accuracy than the best baseline DF. all 10 cases, XTF obtains best results and 2 second best results. In particular, when fine-tuning the Deepseek sibeeee--dsstille--eeee 1..5e 1.5B with all parameters, XTF achieves 13.3% higher accuracy than normal fine-tuning and 9.3% higher accuracy than the best baseline DF. Due to space limitations, additional experiments math tasks (NuminaMath-CoT and MATH-500) will presented
Appendix C.I.
Medicine. Medicine a promising application area for LLMs, and L.M researches on pharmaceuti- cal downstream tasks Thirunavukarasu al. (2023a); Alberts al. (2023) have already achieved widespread influence. As shown in Table 1, XTF has average 6.7% higher accuracy than normal fine-tuning and 3.4% higher accuracy than the best baseline TC. In all the cases, XTF obtains 6 best results and 4 second best results. When fine-tuning the Llama-3. 1-8B with LoRA, XTF demonstrates 13.7% higher accuracy than normal fine-tume tne-tuning and 3.6% higher accuracy than the best baseline.
Code. The code task is a more challenging task, and we provide pass 1, pass and pass 10 results. Due to the generally lower accuracy code tasks, the gap between XTF and the baseline is
8

---

50
Method
Normal
GA
-12
125
30
105-4hr
6.1
401
10
-4.134
Liana- 18
Liana- 3B
Liana 58
Limas SBoaya
Mistall 78
Mistrs 7B-bra
Deapaak 7B
Desponak: 7B
Comparets 14Boora
Deaptaok
+12
40
CA
130
+1.5-177
-1a
+18
XTF
01
20
-1.
Laaaa-10 18
Lamaa 30
Lamg 4B
Liama 4B fow
Mistre- 7B
Mittral TB4
Destannle 1.5
Despptsee-70 7B
Deppeek- Deepseek-BB-oor 7B-lor
Dempreek 14B-lowd
50
Method
+3
GA
x2 120
Pass
Normal
30
11.8
.5.
10
Liama-18
Liama-B
Liama-l
Liana
Mombal78
Mosell-
Devppnnk
Desponn-77
Deppwekk 7B ieom
Despanet: 14B ion
Figure 5: The results code task. We show the results of pass @ I, pass @5 and pass @ 10 respectively.
significant in other experiments. As shown in Figure 5, XTF generally exhibits better results than normal fine-tuning. This difference increases when the LLMs are given more generation chances (from pass@) pass@ 10). In certain cases, normal fine-tuning decreases accuracy, indicating the harmfulness of noisy tokens, while XTF still shows positive performance. We find that when fine-tuning larger-scale models, the effect of noise filtering is more pronounced than when fine-tuning smaller-scale models. This phenomenon aligns with the claim we propose in Section 3:1: finding noise in data should consider the base model's knowledge. Since larger-scale models generally possess stronger performance, we believe that stronger base model performance can better leverage the XTF method.
4.3 ABLATION STUDY
In this section, we conduct an ablation study on three noise filtering attributes. We selectively use the attributes to filter the noisy tokens and train models from different series.
As shown in Table 2, XTF consistently demonstrates the best performance compared to other settings, suggesting that all the attributes are necessary for better noise filtering. the same time, we observe that in mathematical tasks, using the combination of RI and KN to filter tokens consistently superior to other combinations, whereas this is not the case in the Medicine task. Additionally, in the Medicine task, the optimal combination of attributes differs across models. These phenomena suggest that the relative effectiveness of the three attributes can vary depending on the model and task type, which diseussed in Section 3.1.
aligns with understanding of
In addition, we also conduct ablation study against the threshold (for token filtering) selection, which detailed in the Appendix C.2.

---

Table 2: Ablation study of XTF across different settings. DS, LA and MS denotes Deepseek, Llama and Mistral employed in this experiment. Ma and Me denotes math task and medicine task respectively. means the corresponding noise has been filtered while means not. Best results are marked in bold and the second best results are marked with underline.
Case
DRIL
DRNI
DrRL
DS(Ma)
LA(Ma)
MS(Ma)
DS(Me)
LA(Me)
MS(Me)
Avg
Zero
42.9
25.8
15.1
44.6
35.5
20.0
30.7
X
44.0
28.3
16.6
44.7
36.1
22.3
32.0
X
48.1
28.4
20.2
46.8
36.2
27.1
34.5
III
X
45.3
30.1
17.7
45.7
35.5
25.4
33.3
IV
X
X
48.3
32.2
23.9
47.8
36.4
28.1
36.1
X
49.2
34.1
27.7
47.1
27.6
47.3
32.9
22.6
48.3
36.1
30.9
36.3
XIF (Ours)
X
X
56.2
40.5
29.1
50.8
36.5
32.0
40.1
5 DISCUSSION
Our proposed XTF effectively enhances LLM fine-tuning, but it still has some limitations. First, regarding computational cost, XTF incurs inference-level computational overhead (detailed in the Appendix D), which performs better compared to existing methods that train a reference model. However, it still imposes significant burden when dealing with large models. model could be provided to identify noise for large-scale base models, it would significantly reduce the cost token scoring. Additionally, we believe that more attributes can be explored for filtering noise, and these attributes be assessed from multiple perspectives. Such work would provide an effective complement to the application XTF real- wolld scenarios.
small distilled
CONCLUSION
In this paper, we investigated the influence of training data on fine-tuning performance at the token- level. We explored solutions for filtering token-level noise to optimize fine-tuning datasets using three decomposed dimensions: reasoning importance, knowledge novelty, and task relevance, subsequently proposing XTF. We conducted extensive experiments on datasets across 3 different downstream tasks and representative LLMs. XTF achieved up to 13.3%, 13.7% and 6.3% accuracy optimization on the math task, medicine task and code task, respectively, and outperformed all the baselines overall, demonstrating its effectiveness in noise filtering and fine-tuning enhancement.
ACKNOWLEDGEMENTS
This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 62502435, 625B 1032, 62441238, U2441240), the Zhejiang Provincial Natural Science Founda- tion (No. LQN26F020002). the "Pioneer" and "Leading Goose" R&D Program of Zhejiang (No. 2024C01 169), and the Ceopeng-Ascente Science and Education Innovation Excellence/Incubttion Center.
REFERENCES
Codefuse AI. Codeexercise Codeexercise-pythhnn pyttie-yphhhnntt 2025.
attps:://aagiingaace.co http:///aa.aaa.c//aaaaaaaaeeeeee
co/
codefuse-ail wttp:/aaeeeeeeeeeeeeeeeeeeeee-ee
Ian L Alberts, Lorenzo Mercolli, Thomas Pyka, George Prenosil, Kuangyu Shi, Axel Rominger, and
Ali Afshar-Oromich. Large language models (1lm and chatgpt: what will the impact nuclear medicine be? European journal of nuclear medicine and molecular imaging, 50:1549 2023.
Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. Make your I1m fully utilize the context. NeurIPS. 2024.
Zachary Ankner, Cody Blakeney, Kartik Sreenivasan, Max Marion, Matthew L Leavitt, and Mansheej Paul. Perplexed by perplexity: Perplexity-bane based pruning with small reference models. ICLR, 2024.
10

---

Anthropic.
2025.
httpss//ww..antrrooicc.ccmmmm/e
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023,
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03377 2021.
Daixuan Cheng, Shaohan Huang, and Furu Wei. Adapting large language models via reading comprehension. In The Twelfth International Conference on Learning Representations, 2024. URL https::/<pperevve
net/ / forum?id 1i
Zhixuan Chu, Yan Wang, Longfei Li, Zhibo Wang, Zhan Qin, and Kui Ren. A causal explainable guardrails for large language models. In CCS, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint 1rXiv:2110.14111 2021.
Xiao Cui, Mo Zhu, Yulei Qin, Liang Xie, Wengang Zhou, and Houqiang Li. Multi-level optimal transport for universal cross-tokenizer knowledge distillation on language models. In AAAI, 2025.
Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Fang Zeng, Wei Liu, et al. Auggpt: Leveraging chatgpt for text data augmentation. IEEE Transactions Big Data, 2025.
Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Luu Anh Tuan, and Shafiq Joty. Data augmentation using Ilms: Data perspectives, learning paradigms and challenges. In ACL, 2024.
Sachin Goyal, Pratyush Maini, Zachary Lipton, Aditi Raghunathan, and J Zico Kolter. Scaling laws for data filtering-dan data curation cannot be compute agnostic. In ICCV, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint arXiv:2407.22 00.4477.277.,. 2024.
D. Guo, D. Yang, H. Zhang, and J. Song. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint 1rxiv:2501.02222, 2025.
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: survey of progress and challenges. arXiv preprint arXiv:2402.00 6860, 2024.
Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, and Jonathan Berant. Memory--fficient transformers via top-k attention. arXiv preprint arXiv: 106.09996,,,99 2021.
Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: comprehensive survey. arXiv preprint arXiv:2403.14600 2024.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03877 2021.
Shiyuan Huang, Siddarth Mamidanna, Shreedhar Jangam, Yilun Zhou, and Leilani H Gilpin. Can large language models explain themselves? study of llm-generated self-explanations. arXiv preprint arXiv:2310.1 1207, 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, et al. Mistral 7b. arXiv:2310.06825, 2023.
arXiv preprint
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering, 2019.

---

Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. Position: Lims can't 't plan, but can help planning in lmm-modulo frameworks. In ICML, 2024.
Sungmin Kang, Gabin An, and Shin Yoo. A quantitative and qualitative evaluation of IIm-based explainable fault localization. 2024.
Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy Liang. Spoc: Search-based pseudocode code. volume 32, 2019.
Toshiki Kuramoto and Jun Suzuki. Predicting fine-tuned performance on larger datasets before creating them. COLING, 2025.
Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9);9, 2024.
Ming Li, Yong Zhang. Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou, Superfilterig: Weak-to-strong data filtering for fast instruction-tuning 2019.
Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. Data-efficient fine-tuning for llmebased recommendation. In SIGIR, 2024a.
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, Weizhu Chen, et al. Not all tokens are what you need for pretraining. In NeurIPS, 2024b.
Chang Liu, Chongyang Tao, Jiazhan Feng, and Dongyan Zhao. Multi-granularity structural knowledge distillation for language model compression. ACL, 2022a.
Dongju Liu and Jian Yu. Otsu method and k-means. In HIS, 2009.
Haokun Liu, Derek Tam, Mohammed Mugeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. NeurlPS, 2022b.
Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. Www' 18 open challenge: financial opinion mining and question answering. In WWW, 2018
J. Pablo Muñoz, Jinjie Yuan, and Nilesh Jain. Shears: Unstructured sparsity with neural low-rank adapter search. In NAACL, 2024.
Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. Using an Iim to help with code understanding. In ICSE, 2024.
OpenAI, Josh Achiam, and Steven Adler et al. Gpt-4 technical report, 2024. URL https:// // arxiv. org.orrggaa.. abs /2003. 08774.
Jinlong Pang, Na Di, Zhaowei Zhu, Jiaheng Wei, Hao Cheng, Chen Qian, and Yang Liu. Token cleaning: Fine-grained data selection for Im supervised fine-tuning. In ICML, 2025.
Keqin Peng, Liang Ding, Qihuang Zhong, Yuanxin Ouyang, Wenge Rong, Zhang Xiong, and Dacheng Tao. Token-level self-evolution training for sequence-to-sequeee learning. In ACL, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS, 2023.
Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models. arXiv preprint 11161 2024.
12

---

Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00111 001188 2024.
Arun James Thirunavukarasu. Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models medicine. Nature medicine, 2023a.
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez. Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 29(8): 1930-1940) 2023b.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS, 2017.
Chaozheng Wang, Yuanhang Yang, Cuiyun Gao, Yun Peng, Hongyu Zhang, and Michael R Lyu. No more fine-tuning? an experimental evaluation of prompt tuning in code intelligence. In ESEC/FSE, 2022.
Jingxuan Wei, Linzhuang Sun, Yichong Leng, Xu Tan, Bihui Yu, and Ruifeng Guo. Sentence-level or token-level? comprehensive study on knowledge distillation. In IJCAI, 2024.
Qingsong Wen, Jing Liang, Carles Sierra, Rose Luckin, Richard Tong. Zitao Liu, Peng Cui, and Jiliang Tang. Ai for education (ai4edu): Advancing personalized education with 11m and adaptive learning. In SIGKDD, 2024.
Han Xia, Songyang Gao, Qiming Ge, Zhiheng Xi, Qi Zhang, and Xuanjing Huang. Inverse-q#: Token level reinforcement learning for aligning large language models without preference data. In EMNLP, 2024.
Eunseop Yoon, Hee Suk Yoon, SooHwan Eom, Gunsoo Han, Daniel Wontae Nam, Daejin Jo, Kyoung- Woon On, Mark A Hasegawa-Johnson Sungwoong Kim, and Chang D Yoo. Tler: Token-level continuous reward for fine-grained reinforcement learning from human feedback. In ACI, 2024.
Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Token-level direct preference optimization. 2024.
Haiyan Zhao, Hanjic Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1-38, 2024.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. In NeurlPS, 2023.
A THEORETICAL FOUNDATIONS OF XTF
Executive summary. We provide a complete, assumption cramppiionntaanpareeet and parameteriza- tion- con-invariant analysis of how token filtering improves the alignment (in Riemannian sense) between the training direction and the "ideal" gradient direction. We (i) clarify the geometry and regularity of the Fisher information (via damping), (ii) decouple the analysis from any specific optimizer by working with an arbitrary symmetric positive-definite (SPD) preconditioner M (with M = FA recovering natural gradient and M recovering Euclidean SGD), (ii)) make precise the statistical assumption on the selector (and quantify the effect of selection bias), (iv) replace global smoothness by local smoothness, and (v) tighten the treatment of high- conh-conieeecee (KN) tokens with a clean and fully rigorous bound. Throughout, all random variables are defined on a common probability space; expectations are with respect the distributions explicitly indicated.
13

---

A.1 PRELIMINARIES AND GEOMETRY
Contexts and model. A token-level context is c t== (x, hei) and the associated label (gold token) L. Let pa(L denote the model's conditional distribution parameters 0, and let
Φe (c,tt) Ve log
be the score of the pair (c, I).
Teacher-forcing context law. Filtering will remove some token-level losses but, under teacher forcing, it does not alter the forward contexts c. We therefore fix a baseline context distribution
(cc) (the teacher-forcing context law; e.g. the empirical distribution over contexts in the corpus),
All expectations below that involve p- c) are taken with Q first.
Damped Fisher information.
Define the (damped) conditional Fisher information
(0) = Be- (o t) []] + AI,
A 0.
(10)
We will assume (0) for all under consideration; see Assumption 1. Given any SPD matrix (()) > 0 (to be specified below), we write
U) = u (()e, ||u||a-I = V //TMMN (()-
The geometric viewpoint will be stated for a general preconditioner M; the natural gradient corre- sponds to the special choice M = Fx.
Ideal risk. Let (c, t) denote the (unknown) ideal task distribution over (c, t), with marginal ((c)
under teacher forcing. The ideal per-token KL risk is
(0) Baoew.. log c)] = KK(p= 11 po) + const.
(11)
Since p. does not depend on 0,
V (0) = - -E(e))) [oolc, t)]
(12)
For any SPD M, we write the (preconditioned) gradient direction as (.),(0) (M(0- for M Fx this is the damped natural gradient.
A.2 DATA MODEL, SELECTOR, AND ASSUMPTIONS
Assumption 1 (Damped Fisher and teacher forcing). (1) Q(c) is fixed by teacher forcing and independent of filtering. (2) There exists À 2 0 such that F) (0) for all in a neighborhood of interest (e.g. along the optimization trajectory).
Assumption 2 (Mixture model of labels). There exists e E [0, 1) and distributions Peore. Pucise on (c, t) such that
Ptrain (c, -1--a a) Peore (c, t) + (noiso t), with Peore p..
p. (c, (13)
Assumption 3 (Selector quality & independence within components). Let 2(c,t) E (0, 1] be the indicator that the token is kept by the filter. Define the error rates
Pr|Z = 0 (c,t) Peorel.
Pr|Z (c, Pmoise].
(14)
We assume the selector has non-trivial skill: < independence:
We consider two flavors of conditional
Strong (MAR within components). Given the latent component label G {core, noise), Z is independent of (c, t):
1 (c, | G.
Weak (bounded selection bias). There exist Pr: Pn 2 0 such that || [[oo(c,t) t) G core, Z = 1] - (c t) G = core]]
Ar S Po (9cooe(()) (f) (15)
Elpo(c, () G==noise Z=1 E(co((, 1) G noise]
lar Pn gcore (0) 11 (16)
14

---

Assumption 4 (Teacher forcing invariance). Filtering removes token losses but does not change the
context law ((c) in the forward pass. Consequently, (0) in equation 10 is unaffected by filtering.
Assumption 5 (Local smoothness of L.). For each 0 there exist and L L(0,r) such that, for all with ||u]|2 S
( uu I L.(0) + VL. u 2 11u11
(17)
Assumption 6 (Score-norm control for high-confidence tokens). Assume the logits zo(c) E R are coordinatewise L,.Lipsshitz in 0, i.e. (/)zz(l)) I I,- for all k, uniformly in If Fx = μI for some μ > 0, then for all (c,t)
2Lg v
(1 - po(t c)).
(18)
Assumption 7 (Weak incoherence of noise). Let
Icore == (E(c.t)>peondd (c, t)), There exists SM E [0, 1) such that for all 0
Inoise (0) = E(ect) (c,
(geore (0), Inoise (0)) AS-1 I SM [goore (0) 1-
(19)
Remark 1 (Proof Assumption 6). For softmax po(t | c) = softmax(2z (c))) the score decomposes
as #b(c, t) = >x= (I(k- = t (k c)) (c)a(c)k- By triangle incquality and ()zz(())) S L..
| s L [ikk t po c)l 2L, (1-po(t c)).
If (x) ž μI, then ||llle-' S 1r 1/2 1/21|1/|2 for all v, which yields equation 18.
A.3 DISTRIBUTIONS USED BY SGD AND GRADIENT IDENTITIES
Let D be distribution on (c, E); write
g(0; D) = BEce,)yoloo(c, 0)
We denote by Perain the unfiltered training distribution equation 13, and by psi the filtered training distribution induced by keeping only tokens with Z = 1 and renormalizing.
Lemma (Filteiig-induced mixture and gradient identities). Let a:= 1 - E, b==g and
Zhi = a(1 - ax) + b.3.
(20)
Then
g(0; (train) = a goore (0) + b Inoise (0),
(21)
a(1 a) goore (0) + 6B gnoise (0)
(Strong MAR):
g(0; PHI)
(22)
2A1
g(0: pm)) = a(1 ax) georn (0) + b3 y Pooise
(Weak bias):
(23)
Zni
where geore := Eido G = 7=1] and sel is defined analogously, Moreover,
VoL. Score (0).
(24)
Proof. equation 21 is linearity of expectation applied to equation 13. For equation 22, under Z 1 (c,
B[go(c,tI((Z-
G= corc] (1- E(go G core] = (1 a.) Jcore:
and similarly for noise; after renormalization by Zn obtain equation 22. If independence is replaced by the weak bias bounds equation 15-equation 16, the same computation yields equation 23. Finally, equation 24 is the usual score-matching identity since p. does not depend on 0.
15

---

A.4 ALIGNMENT AND ITS IMPROVEMENT UNDER FILTERING (GENERAL SPD M)
Given an SPD M() > 0, define the M M-alignment of any preconditioned direction ğ with the ideal direction by
(V),. (m (0 M)
(25)
In particular, if use the M- --preconditioned gradient of distribution D, i.e. 5(0; D) M ((0) D), then by equation 24 and equation 25
Am(O0:(; = (- M M- Scores -M -M- M geore M 900 D).
(26)
Lemma (Explicit alignments under unfiltered and filtered sampling). Under Assumptions 2-4, for E, and Zi as in equation 20, we have
M (0) = AM (-: (((aa))))))))
= =a llggoorel + 6 (goore Iuoise) As (27)
(Strong MARJ: ( = AM (s:ā@((P)Pi) a(1 -0 a)
69
||9core 11ar-1 +
(goore: Inoise
Zil
ZHI
(28)
Under the weak-bias alternative in Assumption 3, the same formulas hold with the replacements Icore b geet and Iuoise + >g in the numerators.
Proof. Plug equation 21 and equation 22 into equation 26.
Theorem (Filtering strictly improves M Mallignment (strong MAR case)). Under Assumptions 2 b E, Zil a(1 a) 63, the improvement in alignment is
and 7, with
All ( Atrain (f) ab (1 B) (119core (0)1: (geore (6), Inoise (0)) (0)) (29)
Zn
In particular, if and equation 19 holds with Cas then
ab (1 a B) (1 Car)
A (0) A
train
(0) 2
|19care (0) (f)) > 0.
(30)
Za
Proof. Subtract equation 27 from equation 28 and note a(1 a1 ab(1 -
3)
63
63
ab(1 (1 - - -3)
+ b
-
7
Za
7
7
Zn
This simplifies the difference to equation 29, The lower bound equation 30 follows from equation 19.
Theorem (Robust alignment gain under bounded selection bias). Under Assumptions 2-4, 7, and the weak-bias bounds equation 15 15-equation 16, the alignment improvement satisfies
ab (1 B) (1 SMr)
|| 1|2-- a(1 a)pe + 63pa
Al (0) Add (0)
||oor M-1-
Z
(31)
Hence the net gain remains positive whenever
ab(1 3) (ar) > a(1 -a)pe + 63pm-
Proof. Repeat the proof of Theorem 1 using equation 23. Add and subtract the strong-Mo -MAR numerators, then apply equation 15 equation 16 and Cauchy-Schwarz deviation terms by geore 1 - and Pn ll | - respectively.
Schwarz in (-+) to bound the
Remark 2 (Edge cases), If g (no noise) or a + = (random selector), then equation 29 yields All AM i.e. gain; if ÇM 1 (perfect coherent noise), the lower bound equation 30 is zero.
16

---

A.5 ONE-STEP DECREASE IN L.: GENERAL PRECONDITIONERS AND EUCLIDEAN SGD
Lemma 3 (One-step decrease for M s/ppreconditioned steps (local version)). Let g be any direction and consider the update 0+
79.. Under Assumption 5 with radius
0, if 1 & r, then
(++))) & (0) " (V (0). IMM + 1 n² 1
(32)
In particular, for 0(: D) = -- D) we have
(0+) I (0) AM (0:3(:(D) + 1 D)12
Proof. By equation 17,
(0t) S (0) + (O) (0+ - 0) + =11 - -11-
With 0+ - and V W.C. = M -- VC.. we obtain V = -1 VC, MM
-n (V M C..
Corollary 1 (Filtered vs. unfiltered one-step improvement). Consider the two updates with common step size 7 > 0:
A1 = 17 (r
ttain = 0 - :
If 1 max (17( PAI) 11,, (oraii) (aaeeainnlll |12] S r, then
L 1/ (1 110 Phrain))13) (33)
(0 L. (0tmam)) S - AAl (0) Atrain (0)) + 2
In particular, whenever
2( (0) Atrrn(())
0 0< S min
(34)
L(lg(: pai)1l + 199 Ptrain) max (10(:
(o000:PPallla Pa)) ( Perain) Perain)
we have C. (0 < C. attr. with strict inequality if the alignment gain in equation 30 (or
equation 31) is strict.
Remark (Fuclidean SGD and Adam). Lemma 3 and Corollary hold for any SPD M; taking M = 7 yields the standard Euclidean form
(. ng) < (0) ( L., g) + 1191
and the same alignment improvement conclusions with M-1--ineer products replaced by Fuclidean inner products. Taking M = FA recovers the damped natural gradient geometry.
A.6 HIGH- CONFIDENCE (KN) TOKENS: NEGLIGIBLE ALIGNMENT CONTRIBUTION
We show that high-confidence tokens (KN) have vanishing contribution to alignment in the Fisher geometry, with a quantitative bound relying on Assumption 6.
Lemma 4 (KN tokens have vanishing Fisher contribution). Let SKN C Supp(Ptrain) be any measur- able set such that (t c) 2 1 - for all (c, E SKN. with d E (0, 1). Then, under Assumption
2hz Vp
(ec,t) T (c, (c, E SKN]]]
8 Perain (SKN).
(35)
FT!
Consequently, using Cauchy-Schward in (s)))
21z Vp
Ar, (O:r(:)))aim)) Ap, (0:0(: Ptrain \ \SKN)
5
8 Pirain (SKN) (f)) (36)
Proof. By Jensen and equation 18,
21 [1 1e111ptt (t c)) <
2h 8 Ptrain (SKN).
Bio Sscul
< |l 1 <
Vp
The alignment bound follows from equation 26 and Cauchy-Schwarz.
Remark 4 (From Fisher to general M). If M > 0 is any SPD matrix, then for all v, ||0 full-=1
| llo || Thus Lemma transfers to (+ -) up to the condition factor
1 op-
17

---

A.7 COMBINING TR/RI WITH KN
Let STRY U SRIL denote the subset labeled low relevance/impoottacce by the selector (mass under
Pursin). and let SKN, denote the subset of high-confidence tokens with po(t c) 2 1 8.
Combining Theorems 1-2 with Lemma 4 (for M FX), the net alignment improvement from filtering STR. Shy, U SKN, satisfies
(1 - E)E(1 a 3)(1 SFa)
AP, (0) - (0) 2
II (0)1
Zm
2
Perain (SKNL) (()
e) a)pe + EBPp
Meore (0))
(37)
ZAI
where the last (bias) is absent under strong MAR. For sufficiently small ó and small selection bias (Pe, Pa), the positive TR/RI term dominates and the one-step KL improvement of the filtered update is strictly larger (Corollary 1).
A.8 CONSEQUENCES FOR THE MODIFIED LOSS LF
Under teacher forcing (Assumption 4), excluding tokens identified by the selector from the token sum exactly replaces the expectation with respect to Ptrain by the renormalized expectation with respect to pat in the expression for the stochastic gradient (or its M-preconditioned variant). Therefore Theorems 1-2 and Corollary apply directly to the SGD (or M-preconditiioeed SGD) dynamics for Lp.
A.9 ADDITIONAL REMARKS AND CAVEATS
1. Estimator independence & selection bias. Equation equation 22 relies the strong MAR-within-compoee assumption in Assumption 3. When MAR fails, equation 23 and Theorem 2 quantify the degradation through (Pe.Pu).
2. Geometry and optimizers. All alignment statements given for an arbitrary SPD precondi- tioner M. The Fisher choice M = Fx confers parameterization invariance (up to damping), while M / matches Euclidean SGD; Adam/K -BC correspond other choices of M. Corollary 1 therefore bridges theory and practice without further assumptions.
3. Smoothness is local. We use local smoothness (Assumption 5) within radius around 0 to control the second-order term. The step-size condition equation 34 ensures 0+ remains this local region.
4. On variance claims. In general, filtering with renormalization does not guarantee a universal reduction in gradient covariance; the effect is data- and selector-dependent. Our theoretical results deliberately refrain from claiming such a reduction. Empirically, we observe reduced gradient dispersion when removing KN tokens, rigorous sufficient condition (e.g. stratified second-moment dominance) leads to provable variance reduction, but this orthogonal the alignment results developed here.
5. Parameter estimation. The selector ROC (a, () and the mixture mass can be estimated on a small annotated split; ÇM can be estimated from held-out data via (geores Inaise) M-1 (ateeee)-///ll/orre//e//e M-1- These estimators make the alignment gain equation 29 em- pirically checkable.
6. Notation. We write ZnI for the filtering normalizer equation 20 to avoid confusion with the Fisher matrix Fx.
A.10 SOLT-CONTAINED PROOFS OF KEY AUXILIARY FACTS
Lemma (Ideal gradient equals negative core expectation). Under teacher forcing with Prore = P+. have VoL.()) Gcore (f).
18

---

Proof. By definition, (0) = (e,t))pp. |- log c)], so VoL.(0) E (c.t) VV log po c) Ycore (0).
Lemma 6 norore-norm bound in Fisher geometry). Under Assumption 6, for all (c,t). [] 1 < 2h
vp (1 pa(t c)).
Proof.
Shown in Remark
Lemma 7 (Alignment identity). For any SPD M > O, (VMC., -M- D) Goore M -f(g;;;)))
Proof. (M-IVL., -M- =- VLJ -M goore M-1 by Lemma 5.
SUMMARY OF THE STRENGTHENED GUARANTEE
Under Assumptions 1-7, token filtering induces renormalized training distribution pm such that, for any SPD preconditioner M,
A (0) Abraii (0) (1 8)8 8 (1 a (3 (1. 1 (geore: Inoise) M-1).
Za
and hence, under the incoherence condition (M < 1, the alignment gain is strictly positive and explicitly bounded by equation 30. By the local one-step bound (Corollary 1), for sufficiently small step sizes, single filtered update yields strictly larger decrease of the ideal risk L. than the unfiltered update. Finally, KN tokens (high confidence under the base model) have vanishing Fisher contribution with an ((5) bound (Lemma 4), so removing them incurs negligible alignment cost while empirically stabilizing optimization. All statements specialize the Fisher geometry (M = Px)) for parameterization-innaaall conclusions and to M for Euclidean style updates.
B
MORE DETAILS
B. OTSU METHOD
Formulation. For histogram with gray levels in the range [0, 1-L11], let threshold divide the pixels into two classes Co (0 to t) and C₁ (f + to L 1). The between-class variance of (t) is defined as:
of = wn (t) po(())
where:
7--1
(l()) Σ (())
E p(i) are the class probability weights (p(i) is the probability
1=0l of gray level i)
1=t1
(o(() 2
Σ ip(1), ml)) w)(()
Σ ip(i) are the class means
The optimal threshold t is determined by maximizing the between-class variance:
arg max (t) OStcL
Multi-Otsu Extension. For partitioning into & classes requiring thresholds t, t2. ... the objective function becomes:
19

---

ob
Σ Wm (Mm uT)²
where frr the global mean. The optimal threshold combination is obtained by maximizing the total between-class variance.
B.2 BASELINES SETTING
Here we describe all baseline methods in detail and explain their roles in our experiment.
Regular LLM Implementations:
CA: This represents the performance of the original base model, ensuring that our fine-tuning is correct and effective.
Normal:
This gives the performance of regular fine-tuning, serving as direct reference to observe the influence not applying token-level noise filtering.
More Epochs ( (x2 Epochs): This method uses double the training epochs for fine-tumi fine-tuning, eliminating bias caused by recording experimental results before complete convergence.
2) Data Enhancement Methods: Since our XTF is essentially a dataset enhancement method that optimizes fine-tuning performance before training the model, we adopt mainstream data enhancement methods as baselines: data augmentation (DA) and data filtering (DF).
DA: We the state-of-the-art data augmentation method, which leverages high- performance LLMs to optimize the dataset Dai et al. (2025). Specifically, we employ Claude-3.5-sonnet Anthropic (2025) in experiment.
DF: Recent studies have shown that removing data items with high perplexity can improve training performance Li et al. (2019); Ankner et al. (2024). Thus, filter out the top-5% perplexity data items in this baseline experiment.
SLM: This is novel method that can effectively filter noisy tokens in pre- pre-training tasks. Vanilla SL.M relies on high-quality dataset to train a reference model, which impractical in fine-tuning tasks. Therefore, we train the reference model using subset of the fine-tuning dataset to simulate the effect of SLM as closely as possible.
TC: This is fine-grained token-level data selection method for LLM supervised fine- tuning. dedicated filtering uninformative tokens while preserving task cusk-specific infor- mative oncs- effectivvy addressing token-level noise even in high-quality samples Pang et al. (2025). our experiment, we adopt TC's Self-Evolving Cleaning strategy, with implementation tailored to the epoch-wise training process: Specifically, in each training epoch, we treat model trained in the previous epoch as the reference model and the model to be updated in the current epoch the base model. For every token in the training dataset, we calculate its quality score using the loss disparity between this base model and the reference model (a core scoring logic of Self- Cleaning).
For a fair experiment, we use the same validation dataset to select the best model for all baselines.
B.3 HYPERPARAMETERS SETTING
We carefully adjust the hyperparameter to make sure our fine-tuning process is correct and effective, and we finally select common setting shown in Table 3. Besides, set torch, torch_ dtype bfloat16 6 and adopt default generation setting (e.g.. temperature) in all of our experiments. We conduct experiment NVIDIA A100-SMX-80G, H20-96G and 4.20-48G.
20

---

Table Hyperparameters for different models and tasks. Mal denotes math (gsm8k).Ma2 denotes math( (authnaMaMatabeeo for MATH-500 evaluatio); Fi denotes finance; Co denotes code; Me denotes medicine. Lr denotes learning rate, Ep denotes epoch number of fine-tuning, and MNT denotes max new tokens for model evaluation.
Model
Task
Data scale
Lr
Lr_LoRA
Ep
Ep LoRA
MNT
1B
Mal
500
9e-6
3
512
IB
Ma2
500
9e-6
2048
IB
Fi
500
9e-6
3
1024
IB
Co
500
9e-6
3
512
IB
Me
500
le-5
1024
1,5B
Mal
600
9e-6
512
1,5B
Ma2
600
9--6
3
2048
1,5
600
9e-6
1024
1.5B
Co
600
9e-6
3
512
1.5B
Me
600
le-5
3
1024
3B
Mal
1000
8e-6
3
512
Ma2
1000
8e-6
2048
3B
Fi
1000
8e-6
3
1024
3B
1000
8c-6
512
3B
Mc
1000
3c-6
3
1024
7B&8B
Mal
3000
5c-/6
5e-5
512
7B&8B
Ma2
3000
5e-6
5e-5
3
2048
7B&8B
3000
5e-6
5e-5
3
1024
7B&8B
Co
3000
5e-6
5e-5
3
512
7B&8B
3000
2e-6
5e-5
3
1024
14B
Mal
3000
3e-6
3e-5
512
14B
Ma2
3000
3e-6
3e-5
2048
14B
3000
3e-6
3e-5
1024
14B
Co
3000
3e-6
3e-5
512
14B
Me
3000
3e-6
3e-5
1024
C
MORE EXPERIMENT RESULTS
C.I EXTENSION MAIN EXPERIMENT
Two experimental setups were implemented: (1) Models were fine-tuned NuminaMath- CoT and subsequently evaluated on MATH-500; (2) Models were fine-tuned and evaluated end-to-end on the FIQA dataset. For space considerations, additional experimental results from one economics dataset are included in the appendix, where all configurations are detailed in Table 3.
The result are shown Table 4. math task, XTF has average 4.6% higher accuracy than normal fine-tuning and 1.3% higher accuracy than the best baseline SLM. In finance task, XTF has average 4.8% higher accuracy than normal fine-tuning and 2.5% higher accuracy than the best baseline TC. The two tasks are more complex than the gsm8k and medicine tasks used in Section 4.2, and the final model accuracies are generally lower, especially for small-scal scale models. Therefore, the average advantage achieved by XTF is not as pronounced in Section 4.2. However, XTF still achieves the best performance in 16 out of 20 cases and the second-best performance in cases, which is sufficient to demonstrate the reliable advantage of the XTF method.
C.2 ABLATION STUDY ABOUT THRESHOLD
In the threshold ablation study, explore how the three thresholds influence the percentage of filtered tokens and the final training result. This ablation study is conducted using Deepseck- distilled- qwen-1.5BB on the GSM8K dataset.
21

---

Table 4: Extension Main Experiment. We show the accuracy of LLMs across different fine-tuning methods. Best results are marked in bold and the second best results are marked with underline.
MATH--ine-tuning models on NuminaMath- and Evaluate models on MATH-500
Model
[0]
LoRA
CA
Normal
x2
Ep
DF
DA
SLM
TC
XTF
Llama-3.2
IB
X
0.0
2.8
3.0
1.5
2.1
3.0
2.9
3.6
Llama-3.2
3B
X
0.8
4.8
4.9
5.2
4.9
8.2
8.4
8.6
Llama-3.1
8B
4.9
19.8
19.8
23.6
19.2
22.8
22.3
23.4
1.amm-3.1
8B
4.9
20.7
21.1
19.5
18.8
24.7
24.2
28.7
Mistral
7B
X
26.6
26.9
26.4
25.8
27.1
30.8
Mistral
7B
12.3
26.4
26.8
29.1
27.3
33.4
33.2
32.5
1.5B
X
33,4
50.6
50.8
52.3
49.2
51.2
50.5
52.6
Deepseck-distee
distilled-qwen
7B
44.5
65.3
65.9
55.7
61.4
69.2
69.5
70.3
7B
44.5
49.6
49.8
51.2
52.8
55.3
53.2
57.4
Deepseck-distilled
14B
52.8
72.1
72.1
63.4
68.9
75.3
74.6
77.2
Average
21.0
33.9
34.1
32.8
33.0
37.0
36.6
38.3
Fine-tuning and evaluate models on fiqa
Model
[0]
LoRA CA
Normal
x2 Ep
DF
DA
SLM
TC
XTF
Llama-3.2
IB
x
0.0
0.4
0.4
0.6
0.3
0.5
0.5
0.5
Llama-3.2
3B
X
0.5
1.1
1.0
2.2
1.6
3.2
3.4
3.5
1lama-3.1
8B
X
2.5
6.8
6.9
5.1
8.2
7.9
7.6
8.4
Llama-3.1
8B
2.5
5.2
5.4
4.3
4.8
8.9
8.6
8.1
Mistral
7B
X
13.3
13.3
12.4
14.1
15.1
14.8
18.5
Mistral
7B
8.7
12.8
12.9
13.6
12.9
14.2
14.9
17.2
Deepseek-distilled--ee
1.5B
X
11.9
18.4
18.7
15.7
19.2
21.5
20.8
25.9
Deepseck-distie
7B
18.4
28.5
28.9
28.3
30.9
31.6
31.9
36.1
gopsseek-disilleedaee
7B
18.4
22.6
22.7
20.8
23.9
22.8
22.1
29.3
Deepseek ceepseck-distilled-ee
14B
26.3
35.8
36.2
31.5
34.1
40.7
41.2
45.6
Average
9.8
14.5
14.6
13.5
15
16.6
16.8
19.3
502
Nun 500
1920
1950
Num
Num 2000
2000
Nur 3020
3020
34
82
1
025 009
6.55
1905 Threshald
1.50
0,57
Threshold
Threshold
(a) Filtering
(b) KN Filtering
(c) Filtering
Figure 6: Filtered token percentage under different threshold disturbance. In the ablation study of RI and TR, the value on the horizontal axis represents the original threshold without any perturbation. In the ablation study of KN, the horizontal axis corresponds different PCP values.
Filtered Tokens. First, we investigate whether adjusting the XTF thresholds produces noticeable effect on the total number of filtered tokens. Specifically, for RI and TR, first apply normalization to the scores and then modify the thresholds obtained by XTF to compute the number of filtered tokens. For KN, we use different PCP values to perform token filtering. To evaluate the stability of the filtering effect, compute the proportion of filtered tokens relative to all label tokens under four different training-set sizes.
The results are shown in Figure 6. Directly adjusting the threshold value produces significant impact on the number filtered tokens. For RI, slight increase the threshold causes a large number of tokens (>70%) be filtered. For KN, each 1% change in PCP corresponds to approximately 2% of all tokens, which exceeds 15% of the number of tokens filtered under the 0.95 PCP threshold. For TR, both increasing and decreasing the threshold lead to substantial changes. It is noteworthy that the trends of threshold variation are highly similar across different training set sizes, which indicates that the threshold-selection scheme based score distributions is resistant to dataset scale.
22

---

Table The ablation study on the rationality of the thresholds conducted as follows: +3%, -3%, and -0% correspond to adjustments in the number of tokens filtered under the respective attributes, based on the scores decide whether a token should be filtered not. The top five results are marked in the table
RI(J)
-3%
-3%
-3%
-3%
-3%
-3%
-3%
-3%
-3%
KN(()
-3%
-3%
-3%
-0%
-0%
-0%
+3%
+3%
+3%
TR(L)
-3%
-0%
+3%
-3%
-0%
+3%
-3%
-0%
+3%
ACC (%)
51.2
48.3
53.2
50.7
51.7
47.8
53.7
47.3
45.3
Order
5
RI(J)
-0%
-0%
-0%
-0%
-0%
-0%
0%
0%
0%
KN(L)
-3%
-3%
-3%
-0%
-0%
-0%
+3%
+3%
+3%
TR(()
-3%
-0%
+3%
-3%
-0%
+3%
-3%
-0%
+3%
ACC (%)
44.3
50.5
47.8
47.3
56.2
54.2
49.8
48.0
52.8
Order
3
RI(L)
+3%
+3%
+3%
+3%
+3%
+3%
+3%
+3%
+3%
KN(L)
-3%
-3%
-3%
-0%
-0%
-0%
+3%
+3%
+3%
TR(L)
-3%
-0%
+3%
-3%
-0%
+3%
-3%
-0%
+3%
ACC (%)
48.8
47.8
47.3
48.3
50.3
50.3
48.8
47.3
55.7
Order
Final Training Result. Since directly adjusting the threshold produces a large impact the number of filtered tokens, we adopt a more fine-grained filtering scheme, namely, perturbing the percentage of filtered tokens in each sentence directly according to the ranking of the scores. We varied the number of filtered tokens along different attribute dimensions by +3% We systematically combined all possible threshold settings, resulting a total of 27 cases.
As shown in Table 5, our thresholds, which are computed based on statistical principles, consistently lead to the best fine-tuning performance across all cases. While we observe that certain alternative threshold settings could also yield competitive results, these settings do not conform to consistent patterns, such as linear or normal distributions. Therefore, we believe that remains challenging to derive reliable and generalizable threshold computation method through post-hoc adjustment of our statistically derived thresholds.
D COMPUTATIONAL OVERHEAD
Table 6: Memory Usage (peak/normal) for different models. Only the scoring process is considered.
Model
Metric
Math
Code
Medicine
Llama IB
Mb (peak/normal)
5362.2 4746.3
20149.0
4746.3
7617.2/4746,
Llama 3B
Mb (peak/normal)
13093.7
12288.7
32465.6
12288.7
16035.3
12288.7
Llama 8B
Mb (peak/normal)
31740.7 30665.0
60487.2
30665.0
36525.37
30665.0
Deepspeek 1,5B
Mb (peak/normal)
7509.17 6811.8
18715.8
6811.8
7617.2
/ 6811.8
Deepspeek 7B
Mb (peak/normal)
30439.1/29223.
22660.8/9925.3
34626.5
29225.3
Deepspeck 14B
Mb (peak/normal)
38941.7/56375.2
92438.5/56375.
68705.6 56375.2
We evaluate the GPU memory usage and time consumption of XTF across 6 models from two mainstream LLM (Large Language Model) series: Llama and Deepseek. The experimental setting of this experiment is aligned with that of the main experiment in Table 1. Additionally, we provide basic information about the selected datasets.
A perturbation range of 3% is appropriate for our setting because the number of filtered tokens in sentence is relatively small. If perturbation of or 2% is used, the filtered tokens in many sentences remain completely unchanged. A value of 3% is therefore reasonable choice hased the experimental setting and granularity.
23

---

Table 7: Time Usage for different models. Only the scoring process considered.
Model
Metric
Math
Code
Medicine
LJama IB
Time (s)
134.62
150.42
253.68
Llama 3B
Time (s)
324.95
355.31
608.85
Llama 8B
Time (s)
1167.23
1541.27
2637.92
Deepspeek I.5B
Time (s)
191.44
191.32
377.20
Deepspeek 7B
Time (s)
1179.11
1445.44
2276.91
Deepspeek 14B
Time (s)
3910.83
4132.56
4235.85
The GPU memory consumption of XTF is at the inference-level, and remains close to the memory required for model loading (normal state), without causing significant increase peak memory usage. Specifically, the maximum sequence length of the Code dataset reaches 2806, which is much longer than that of the Math (529) and Medicine (1231) datasets; thus, the Code dataset exhibits relatively higher memory peak. However, this peak is still acceptable-far below the memory usage of fine-tuning, which is generally four times that of the model loading memory usage.
Time consumption is mainly determined by the average sequence length of the dataset. The Medicine dataset has an average sequence length of 401, which is longer than that of the Math (181) and Code (236) datasets, resulting in the highest time cost. Notably, although increasing the model size leads to higher computational costs, the theoretical overhead XTF remains significantly lower than that of the most competitive token-level baseline, i.e., SLM (Token-Level Supervised Language Model)- the latter requires training an additional reference model. Details of this experiment are provided in the appendix of the paper.
Cost Analysis. In data processing techniques that involve LLMs, the time and computational costs mainly arise from model inference and training, and the cost of training far exceeds that of inference. Therefore, the cost levels can be divided into two major categories: (1) training-level cost: the token-level baselines, including SLM and TC, require training a reference model to assist with filtering, and they need perform one inference on the original model and one inference on the reference model to obtain token-level loss values before computing scores. (2) inference-level cost: two types of sample-level methods fall into this category. The DF baseline needs to compute average perplexity for each sample, and DA requires multiple I.I. inferences to perform data augmentation. Although our XTF method is a token-level method, it completes token filtering with only two inferences. The first inference produces the output logits and attention, from which the RI score, KN score, and the domain vector for TR are obtained. The second inference computes the TR score for each token. Arguably, existing token-level methods that rely multiple rounds of training on a reference model cannot compete with XTF in terms of cost, and the cost of sample-level methods is also not substantially lower than that of XTF.
D.1 DISTRIBUTION FIGURES
Here we present more detailed distribution plots of the three scores. As shown in Figure 7, although the score distributions differ across various models or datasets, they still generally align with the patterns discussed in Section 3. Specifically, the reasoning importance scores are often concentrated certain fixed values, with only a few extreme values observed. The knowledge novelty score distribution is relatively uniform, making it difficult to partition. The task relevance score distribution exhibits distinct clustering characteristics, thus enabling effective partitioning via clustering methods.
E
LLM USAGE
An OpenAl LLM (GPT-40) was utilized as an assistant for writing and formatting, specifically to refine and suggest edits to figure and table captions, including improvements in grammar, phrasing, clarity, and layout (e.g., column alignment, caption length, and placement). The model's 's role was strictly limited to surface-level text and visual edits, without contributing to research ideation, experimental design, data analysis, or technical content. All outputs were thoroughly reviewed and revised by the authors, who retain full responsibility for the final text and visuals.
24

---

Medicine
(a) SKI Deepseek7B
(b) SKN Deepseek7B
(c) STr Deepseek7B
Math
Code
Medicine
Math
Code
Medid ne
Math
Code
Medicine
(d) STU on LlamalB
(e) SKN on Ilamal
(f) Stn on Ilama
Medicine
Math
(g) Sni on Llama3B
(h) SKN on Llama3B
(i) STR on Llama3B
Math
Code
Medicine
Math
Code
Medicino
Math
Code
Medi cinc
(j) SkI Llama8B
(k) SKN on Llama&B
()) STR on Llama8B
Math
Code
Medicine
Math
Code
Medicine
Math
Code
Medicine
(m) SeI on Mistral7B
(n) SKN on Mistral7B
(0) STR on Mistral7B
Figure 7: Distribution of three scores across different LLMs.
F EXAMPLES FOR TOKEN-LEVEL NOISE
Here we show some specific results of token-level noise filtering. As shown in Figure 8, Figure 9, Figure 10, Figure and Figure 12. the part marked in yellow is the noisy tokens filtered out according to the corresponding scores.
We would like to point out that the filtering decisions made by XTF may, in some cases, be more reliable than human intuition. Take GSM8K as example: tokens related to mathematical operations (e.g., "+", "=") or even specific computed results are not always necessary for effective fine-tuning. From the perspective of Knowledge Novelty (KN)-a dimension easier to interpret Figure that the filtering prediction probability of the third "+" in the expression "2P + +++4= 3P + +4= = 220" exceeds 95%, and the token "216" in "220 4 216" is also filtered out. This outcome deviates from typical human intuition, indicating that learning reasoning logic is more critical for the model than memorizing symbols or specific results.
show in
25

---

Input I: Trevor is a comic book il lustrraorr. In the past three months, he has drawn 220 pages of the comic. The comic he i illusrrates releases issue once a month. The first and second months' issues were the same number of pages, but the third month' s issue was four pages longer than e eit of the others. How many pages was the first issue of the comic? " www.
Low RI: Let P be the number of pages of the first issue of the comic. The second issue is the same number of pages, so the first and second issues are P + P = 2P pages. The third issue is pages longer, so it is P + 4 pages. All three issues are 2P
220 pages long. Thus. 3P 220 = 216 pages. Therefore, the first ssse of the comic was P 216 / <<216/3-72>>7 pages.
3P
Low KN: Let be the number of pages of the first issue of the comic. The second issue is the same number of pages, so the first and second issues
are P + P 2P pages. The third issu is pages longer. so it is pages. All three issues are 2P + P + 3P + 220 pages long. Thus, 3P 220 - 4 216 pages. Therefore, the first issue of the comic was P = 216 / <<216/3=72>>72 pages.
Low TR:Let P be the number of pages of the first issue of the comi c The second issue is the same number of pages, so the first and second issues are P + P = 2P pages, The third issue is pages longer, so it is
pages. All three issues
2P + + 3P + 4 220 pages long. Thus, 3P 220 -4 = 216 pages, Therefore, the first issue of the comic was P 216 3 <<216/3=72)>72 pages.
Input II: Don buys recyclable bottles in small town. Shop A normal sel him 150 bottles, shop sells him 180 bott les and Shop sells him the rest. How many bottles does Don buy from Shop C if he is capable of buying only 550 bottles?
Low RI: Between Shop A and Shop B he buys 150 bottles 180 bottles <<150
180 330>>330 bottles. Thus he can buy 550 bottles 330 bottles les <<550
330 220>>220 bottles from shop C.
Low KN:: Between Shop and Shop he buys 150 bottles 180 bottles
<<150 + 180 330>>330 bottles. Thus he can buy 550 bottles 330 bottles
<<50 330 = 220>>220 bottles from shop C.
Low TR: Between Shop A and Shop he buys 150 bottles 180 bottles
<<150 + 180 330>>330 bottles. Thus he can buy 550 bottles 330 bottles
<<550 330 220>>220 bottles les from shop
Figure 8: Low score tokens in gsm8k dataset.
We do not claim that XTF 's scoring mechanism is absolutely flawless. To mitigate the risk of mistakenly removing important tokens, we intentionally adopt conservative thresholds for all three scoring dimensions. This strategy reduces the likelihood of filtering out valuable tokens, enabling XTF maintain consistently strong performance.
26

---

Input I: Conside the terms of an ar th sequence: S-V (1) (3). y+2, 4y, Widtts. Solve for Sy$. \n.
Low RI: For an ar arithmetic sequence, the difference between consecut i ve terms set up the fol lowing equations based the
must be equal. Therefore, sequence diven:\n\\/ (y 2) Weeet (-llfrc (1) [3) Wright)
Low KN: For an arithmetic ic sequence, the difference between consecut ive terms must be equal. Therefore, sequence gi \n\\[ [y 2) --Veft (3) \right)
set up the following equations based the
Low FR:For an ar arithmetic sequence, the difference between consecut ive terms must be equal. Therefore, we can set the following equations based the sequence give \n\\[ (y 2) \lleft (-\lfrac (1) (3) \right)
Input II: Find all solutions to the equation ( [[\liipplasstylee\arr[[]]]]] www.
\lfrac [x] [3]] -2S. \n.
Low RI: Start by isolat ing the cube root: \n$$ [3sqrt[3] [3 \lrrac [x] [3)) -2
SS'un both sides to iminnae the cube root: \n$$ \ fIrrac [x] (3) (-2)
$$\ 3
{\frac [x] [3] = =- SS'n)nSol vv for Sx$ \n$$ 3 + 8 \\frac [x] (3)
$$'n$$ 11 (lraac [x] (3] $S\n$$ 33 $Sin\nThus, the solution to the equat
is: (n$ Wbooxed [x 33] $$
Low KN: Start by isolating the cube root: \n$S [|sqrt[3]] [3 \ifrac [x] [3]]
22 (n both sides iminatte the cube root: \n$S 3 \lrrac [x] [3] (-2) 3
$$'n$$ 3
Afraa [x] [3] -8 S$ininSolvee lve for Sx$ |n$$ 3 + \lrac [x] (3)
$$\n$$ 11
\\frac [x] [3] $S'n$$$ 33 $S)n\nThus the so iolution to the equat ion
AnSA \lboxed 33] $$
Low TR:
Start by isolating the cube root: \n$$ Asqrt [3] [3 Afra [x] [3)]
-2 $$ nnCube both sides el iminate the cube root: AnSS: Afrra [x] [3) (-2) "
$$\n$$ 3
Nfrac [x] [3] -8 $S\n\nSolve for Sx$ \n$$ 3 + Afrac [x] (3)
SS\ 11 frac [x] [3] $S'n$$ x 33 $81n)nThus, the solution to the equat ion
is: In$$ [bxxed 33) $$
Figure 9: Low score tokens in NuminaMath-Con CoT dataset.
27

---

Input: Write a Python program to count the number of vowels Is in given string.
Low RI: python def count lowels(stringg
This function takes a str ing as input and counts the number of vowe
in it.
Parameters:
str ing (str): The input str ing
Returns:
int: The number of vowe ls in the string
vowe Is "aeiouAEIQU" # Def ine a string of all vowe (both lowercase and upper
count 0 # Initial ize counter variable
for char in string: # Iterate over each character in the str ing if char vowe count + 1 # Increment the counter by 1 return count Return the final count of vowe
Check if the character is a vowel
Low KN: python
def count vowels (str ing)
This function takes string as input and counts the number of vowe in it.
Parameters
str ing (str): The input str ing
Returns:
int: The number vowels in the str ing
vowe Is "aeiouUEOOU" # Def ine string of all vowe Is (both lowercase and oppercasel
count = 0 # Initialize counter variable
for char in str ing: # Iterate over each character in the str ing if char in vowels: count += 1 # Increment the counter by 1 return count # Return the final count of vowe Is Low TR: python def count vowels
Check if the character is vowe
This function takes a str ing as input and counts the number of vowe Is in it.
Parameters:
str ing (str): The input string Returns:
int The number of vowels in the str string
vowe Is "aeiooλEIOUU # Def ine string of all vowels (both lowercase and uppercase)
count = 0 # Initial ize counter variable for char in str string: if char in vowels s Check if the character is a vowe count
terrate over each character in the str ing
# Increment the counter by 1
return count # Return the final count of vowels
Figure 10: Low score tokens Codeexxercise-Python--222 dataset.
28

---

Input: Answer the question based on the following context: Dur ing augmentation and Mitrofanoff procedures, conduits are usual ly imp Implanted into the poster ior bladder wall. Anatomical consi derations may recessitate compare the relat drainage efficiency in patients with poster ior and anter ior conduits, we studied their rates of bladder stone format ion and inary tract infection (UTI). A retrospective chart review identifies exstrophy patients who underwent augmentation and Mitrofaaofff between 1991 and 2003. Patients with 3 years or greater fol low up inc lude. Fifty-four patients fit this criterion. with conduit impl anted anter irr ly (33) or posteriorly (21). We compared rates of bladder stone format on and UTI. Stomal revisions and the status of the bladder higher in the
an anter ior conduit.
neck also noted. Stone formation and UTI rates anter ior conduits, although only UTI showed a statistical ly significan difference. Patient demographics were simi lar between the two groups, inc mncluding inding age and sex. The rates of stomal compl iaaiions and the bladder neck status were also simi lar.
Question: Mitrofanoff cont inett catheter izalee conduits: top down bottom up?
Low RI: Patients with anter ior conduits had increased risk of UTI and bladder stone format ion compared to those with posterior conduits. although this was not signi ficant in the case of b bladder stone rate. This may indicate sub-opt ima I aldder drainage and should be addressed wi with careful preoperative counse I ing and close up
Low KN: Patients with anterion ior conduïts had an increased risk of UTI and b bladerr stone formation ion compared to those with posterior conduits, although this was not signi cantt in the case of b bladder stone rate. This may cate sub-opt imal bladder drainage and should be addressed wi with careful preoperative counse ing and close low
Low TR: Patients with anterior conduits had an increased risk UTI and bl bladder stone formation in compared to those with posterior conduits, although this not signi iusiffcann cicatt in the case of b bladder stone rate, This may indi sub-optime imal bladder drainage and should be addressed with careful preoperative counse ing and close follow-up
Figure 11: Low score tokens in PubMedQA dataset.
29

---

summary of them with the rorrespond ig amounts to be claimed. Personal ly prefer print out a paper copy and file that away with that quarter s and year's other tax documents. do my own taxes and find paper copies handy as I can go through each invoice/ cetccc///reippp rece and make sure have entered its information by ticking it. find that when hand ing a large number of documents that paper copies are more easy to handle than electronic ones. In the end you Input Based on your financial expert ise, provide your response or viewpoi nt on the given financial question or topic. \nDo need to keep paper records for my bus iness?.
output: Scanned or electronic copies invoices should be sufficient as long as they are accurate and you can del iver them dur ing an audit. Also. if you have an accountant prepare your taxes you would either need to provide them copy of the invoices
will need to use system that you feel comfortab wi and are able to use effectively.
*****
Low RI: Scanned or e electronic copies of invoices should be suff icient as long as they are accurate and you can del iver them dur ing an aud Also, if you have accountant prepare your taxes you would either need to provide them copy of the invoices or a summary them w th the cor corresponding amounts to claimed. Personall ly I prefer to print out a paper copy and file that away with that quarter's ss and year's other tax documents. do my taxes and find paper copies handy can go through each rnviicerrcceilo information by ticking it. find that when hand ing a large number of documents that paper cop es are more easy to handle than e electronic ones. In the end you will need to use system that you feel comfortable with and are able to effectively.
and make sure have entered its
Low KN: Scanned or electronic copies of invoices should be sufficient as long as they are accurate and you can del iver them dur ing an aud Also, if you have accountant prepare your taxes you would either need to provide them copy of the invoices or a summary them th the corresponding amounts to claimed. Personal lly I prefer to print out a paper copy and file that away with that quarter's s and year" other tax documents. I do my own taxes and find paper copies handy can go through each inwoice/receiio information by ticking it. findd that when hand ing a large number of documents that paper cop is are more easy to handle than lectronic ones. In the end you iffectiielle
and make sure have entered its
will need to use system that you feel comfor with and are able to
Low TR: Scanned or electronic copies of invoices should be sufficient as long as they are accurate and you can del iver them dur ing an audit, Also, if you have an accountant prepare your taxes you would either need to provide them copy of the invoices or a summary of them th the cor corponsonding amounts to claimed, Personalll ly I prefer to print out a paper copy and file that away with that quarter's iss and year's 's other tax documents. do my taxes and find paper copies go through each invoice/ information by ticking it. I find that when hand ing a large number of documents that paper cap rspies are more easy to handle than e electronic ones. In the end you
handy
and make sure have entered its
will II need to use system that you feel comfortabe taboltaable le wi th and are able to use effective
Figure 12: Low score tokens in fiqa dataset.
30


Reviews:
[{"review_id": "mryhBTPWXr", "reviewer": "Reviewer_GsNo", "summary": "This paper proposes a method called XTF (Explainable Token-level Noise Filtering) to improve fine-tuning for large language models (LLMs). The authors argue that most fine-tuning datasets are designed at the sentence level, while LLMs are optimized at the token level. This mismatch introduces token-level noise that can hurt model performance. The paper brings out three interpretable attributes: reasoning importance, knowledge novelty, and task relevance. Before fine-tuning the model, it will mask the one with the lowest score. This method is tested on three downstream tasks and it shows gaining accuracy compared to normal fine-tuning.", "questions": "Threshold analysis is good but could do more in the future: It is good to see that the authors already include the threshold ablation in Appendix. A little bit of suggestion on this will be analyzing interactions effects between the three thresholds to see whether there is a trade-off or something, also it will be better to make some more discussion on this visible in the main paper.\n\nHow sensitive is the final performance when calculating three attributes in a specific way? For example, how would the different interaction among them affect the result?\n\nCould the paper provide more specific examples on the process of filtering tokens?\n\nHow stable or consistent are these improvements the paper mention?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**Novel perspective:** The paper focuses on token-level dataset optimization, a topic that has not been well studied in LLM fine-tuning.\n\n**Explainable approach:** The design and introduction of three attributes provides a clear and interpretable way to analyze which tokens are useful.\n\n**Good experiment results:** The authors test on diverse tasks (math, medicine, code) and multiple model sizes, showing stable improvements.\n\n**Good theoretical support:** Appendix A provides formal justification linking token filtering to gradient alignment, which adds credibility.", "weaknesses": "**Could do more on stable analysis:** Although results show improvements on accuracy, we don’t know whether the result is chosen based on the best result the experiment gets and whether it is reproducible. Thus, it could be better to include some variance, confidence intervals, or significance tests. It is hard to see how stable the improvements are.\n\n**Could do more on examples:** Although the paper repeatedly emphasizes “explainable” token-level filtering, there is almost no qualitative examples or visualizations showing which tokens were removed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CW2Em1RB99", "reviewer": "Reviewer_Ucn3", "summary": "This paper introduces XTF, a novel learning framework that enhances Large Language Model (LLM) fine-tuning by filtering token-level training data. XTF identifies and removes noisy tokens based on three proposed metrics: reasoning importance, knowledge novelty, and task relevance. Tokens scoring below a threshold on any metric are considered noise, and their loss contribution is blocked during backpropagation. Extensive experiments on mathematical, coding, and medical tasks across seven different LLMs demonstrate the strong performance of the proposed framework.", "questions": "1. The threshold selection process requires deeper analysis. A sensitivity analysis exploring different threshold values is needed to justify the current choices.\n\n2. To firmly establish XTF's superiority, evaluations should be expanded to include more datasets and a broader comparison with baselines.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-  Unlike traditional token-level filtering methods that rely solely on loss values, Authors propose three distinct metrics: reasoning importance, knowledge novelty, and task relevance, to comprehensively assess each token's contribution. These metrics are designed to capture different aspects of the relationship between the model, the training data, and the target task.\n\n- Theoretical analysis are provided to demonstrates the effectiveness and efficiency of the XTF framework.\n\n- Extensive experiments across three downstream domains (math, code, and medicine) and seven LLMs confirm the strong performance of our approach.", "weaknesses": "- The threshold selection process for the three metrics appears to be largely empirical. For instance, the use of a quantile method in Equation (5) and a fixed threshold of 0.05 in Equation (6). The generalizability of these specific threshold needs further investigation.\n\n- The paper posits that a lower PCP indicates a token containing novel knowledge. However, a low score can also reflect an incorrect or nonsensical token. This ambiguity creates a paradox that is not addressed: how can the framework reliably distinguish between \"novel knowledge\" and \"noise\"?\n\n- The empirical evaluation, while demonstrating positive results, is conducted on a limited number of datasets (three). This contrasts with related work in the field (e.g., [R1, R2]), which often employs more extensive benchmarks (e.g., seven datasets) to validate generalizability. \n\n- Typos. \"Section ??\" on Line 101, should be corrected.\n\n[R1] RHO-1: Not All Tokens Are What You Need\n\n[R2] Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N80ucs4HEW", "reviewer": "Reviewer_XoFh", "summary": "This paper investigates the token-level noise problem in fine-tuning datasets, motivated by recent findings that noisy tokens can significantly degrade model performance. To address this issue, the paper proposes XTF, a token-level noise filtering framework that identifies and removes noisy tokens based on three key attributes: reasoning importance, knowledge novelty, and task relevance. These attributes are quantified using the attention score, PCP score, and relevance score, respectively. Extensive experiments across multiple base models and benchmarks demonstrate that XTF consistently outperforms all baseline methods.", "questions": "1. The citation format throughout the paper is inconsistent with standard academic conventions. Please revise all references to follow the correct style \n\n2. A reference is missing in Line 101.\n\n3. Could incorporating additional attributes further improve the accuracy of noisy token identification?\n\n4. The effectiveness of SLM largely depends on the quality of its reference model. For SLM (also known as RHO), how many samples were used to train the reference model?\n\n5. Could you provide a detailed comparison of GPU hours between the proposed method and the baselines to clearly illustrate the computational overhead?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The writing is clear and easy to follow.\n\n- The paper lies in introducing a noisy token filtering framework that jointly leverages several scores (including attention, PCP, relevance scores) to accurately identify and remove noisy tokens, which is novel and interesting.\n\n- The claims are well-supported by empirical experiments across multiple models (LLaMA-3, Mistral, Deepseek-distilled-qwen), and popular evaluation benchmarks.", "weaknesses": "-  The authors fail to provide a detailed comparison of computational overhead with existing baselines, such as SLM. Including such analysis would help clarify the efficiency of the proposed method.\n\n-  The benchmarks used in the paper are relatively weak and do not convincingly demonstrate the method’s effectiveness. For instance, in Figure 4, the baselines—namely, the original base model (CA) and the standard fine-tuning version (Normal)—are too naive. The reviewer suggests incorporating more competitive baselines for comparison. Recent work such as [1] provides a strong theoretical foundation for noisy token cleaning and should be considered as an additional baseline.\n\n- The paper would benefit from evaluation on more diverse and challenging datasets to further validate its effectiveness—for example, BBH, ARC, and MMLU, which are widely recognized benchmarks for reasoning and knowledge assessment.\n\n\n[1] Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning, ICML 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v5i0FvHiMl", "reviewer": "Reviewer_9uW5", "summary": "This paper addresses the mismatch between sentence-level labeling and token-level optimization in LLM fine-tuning. Most fine-tuning datasets provide labels at the sentence or document level, causing models to learn all tokens with equal importance. As a result, noisy or irrelevant tokens within a sentence can distort the optimization direction.\nTo resolve this issue, the authors propose XTF (eXplainable Token-level Filtering). XTF decomposes each token’s contribution into three interpretable attributes: Reasoning Importance (RI), Knowledge Novelty (KN), and Task Relevance (TR). Based on statistically derived thresholds, noisy tokens are identified and masked during training, ensuring that the model learns from meaningful tokens only. \n\nExperiments across three downstream tasks and multiple LLM architectures demonstrate that XTF consistently outperforms both standard fine-tuning and existing data filtering methods. The authors also provide theoretical evidence showing that filtering improves gradient alignment and accelerates convergence. While token-level computation introduces additional overhead, the paper discusses potential efficiency improvements through model distillation.\n\nOverall, this study presents a theoretically grounded and empirically validated framework for token-level data refinement, offering a practical and interpretable solution to one of the fundamental limitations in LLM fine-tuning.", "questions": "Q1. The paper uses distinct methods (Quantile, fixed threshold, Multi-Otsu) for thresholding the three scores based on observed distributions. Could a more unified approach, perhaps learning optimal thresholds or using a combined score, be feasible or beneficial?\n\nQ2. Appendix C.2 shows scoring time. How does the total time (scoring + XTF fine-tuning) compare to achieving similar performance with baseline methods (e.g., normal fine-tuning for more epochs, or sample-level filtering + normal fine-tuning)?\n\nQ3. Could you provide further justification or analysis on how the adaptive constant (r₁) for RI and the heuristic threshold (e.g., PCP ≥ 0.95) for KN were determined? In particular, it would be helpful if you could include additional discussion or experimental evidence showing how variations in these parameters influence model performance.\n\nQ4. In the current study, the filtering ratio is adjusted in ±3% intervals to examine the effect of threshold changes, but this interval may appear somewhat arbitrary. Could you provide additional experimental results using finer intervals (e.g., ±1%) or a wider range of values to analyze the performance patterns and identify the potential optimal threshold range more precisely?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "S1. The three metrics are designed in a complementary manner, which is a notable strength of the paper. Knowledge Novelty (KN) increases as the model’s probability of correctly predicting the next token (PCP) decreases, but on its own, a low PCP could simply indicate meaningless noise. The authors address this by jointly considering Reasoning Importance (RI) and Task Relevance (TR), filtering out tokens with low RI or TR even if KN is high. Only when all three metrics are high is a token regarded as meaningful new knowledge. This complementary design effectively overcomes the limitations of each individual metric and demonstrates a distinctive approach to fine-grained token-level filtering.\n \nS2. The paper takes a logical and rigorous approach by providing mathematical proofs that explain why the proposed filtering mechanism is theoretically effective. This theoretical grounding enhances the credibility of the method and clearly supports the intuition behind its practical usefulness.", "weaknesses": "W1. The core techniques are borrowed from existing methods without significant innovation: attention scores for importance, which is a standard interpretability technique, prediction probability, which is a straightforward application, and semantic distance, which is basic embedding similarity. The contribution is primarily engineering existing components rather than introducing novel algorithms or architectures.\n\nW2. While motivated by score distributions (Fig 3), the specific thresholding methods involve some heuristics (e.g., the exact quantile parameter for RI, the fixed 95% PCP threshold for KN, choice of cluster for TR via Multi-Otsu). Sensitivity analysis (Appendix C.1) shows the chosen thresholds work best, but optimality across all scenarios isn't guaranteed.\n\nW3. Ablations are missing. The paper needs: 1) threshold sensitivity, where Table 4 tests ±3% but not other ranges or methods, 2) attribute weighting, where Instead of union, can we weight attributes?, 3) layer selection, where does using different attention layers change results?, and 4) dataset size, where how does XTF perform with more/less training data?\n\nW4. Calculating the three scores for all tokens in a dataset requires inference passes with the base LLM, adding a non-trivial computational cost before fine-tuning can begin. The paper quantifies this in Appendix C.2, but it could be a barrier for very large datasets or models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)