Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning
Authors: Ali Taheri
Alireza Taban
Qizhou Wang
Shanshan Ye
Abdolreza Mirzaei
Tongliang Liu
Bo Han
Venue: Accepted by TMLR ()

Abstract: Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs), notably enhancing their capacity to acquire domain-specific knowledge while preserving or potentially augmenting their general-purpose capabilities. However, the efficacy of SFT hinges on data quality as well as data volume, otherwise it may result in limited performance gains or even degradation relative to the associated baselines. To mitigate such reliance, we suggest categorizing tokens within each corpus into two parts---positive and negative tokens---based on whether they are useful to improve model performance. Positive tokens can be trained in common ways, whereas negative tokens, which may lack essential semantics or be misleading, should be explicitly forgotten. Overall, the token categorization facilitates the model to learn less informative messages, and the forgetting guides the model on what information to learn more precisely. We conduct experiments across diverse and well-established benchmarks using various model architectures, demonstrating that this forgetting mechanism enhances model performance.

Published in Transactions on Machine Learning Research (03/2026)
Forgetting: A New Mechanism Towards Better Large Lan- guage Model Fine-tuning
Ali Taheri" Mar Planck Institute for Software Systems
Alireza Taban*
Isfahan University of Technology
Qizhou Wang Hong Kong Baptist University
Shanshan Yel University of Technology Sydney
Abdolreza Mirzaei Simon Proser University
Tongliang Liu The University of Sydney
Bo Han
Hong Kong Baptist University
Reviewed on OpenReview: optee:ee/penrreveeu. net/ forum? id= 336smEoUoX
Abstract
Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs), notably enhancing their capacity to acquire domain-specific knowledge while pre- serving or potentially augmenting their general-purpos capabilities. However, the efficacy of SFT hinges on data quality as well as data volume, otherwise it may result in limited performance gains or even degradation relative to the associated baselines. To mitigate such reliance, suggest categorizing tokens within each corpus into two parts posst-vosiive and negative tokens based on whether they are useful to improve model performance. Positive tokens can be trained in common ways, whereas negative tokens, which may lack essential semanties or be misleading, should be explicitly forgotten. Overall, the token categoriza- tion facilitates the model to learn less informative messages, and the forgetting guides the model on what information to learn more precisely. We conduct experiments across diverse and well-established benchmarks using various model architectures, demonstrating that this forgetting mechanism enhances model performance.
1 Introduction
In recent years, we have witnessed emerging advancements in large language models (LLMs) (Brown et al.
2020 Achiam et al. 2023). powered by transformer-basell architectures (Vaswani et. al. 2017) with bil- lions of parameters and extensive pre-training on trillions of tokens (Zhao et al. 2023 These models
have evolved rapidly with continuous improvements in architectural design, training strategics, and scaling
"Equal contribution. i "Correpponding author.

---

Published in Transactions on Machine Learning Research (03/2026)
techniques (Hoffmann et al
2022j. They exhibit exceptiooal performance across wide range of complex
linguistic tasks, including reasoning, solving mathematics (Shao et al. 2024). summarization Nallapati
et al. 2016), language understanding, code generation (Chen et al. 2021 Jiang et al., 2026), question answering [Rajpurkar et al. 2016). etc.
Although powerful, L.MSs still require SFT to enhance their performance in specialized tasks (Chung et al. 2024 Aggarwal et al. 2024 Strangmann et al.) 2024 Lialin et al. 2023). SFT typically involves adapting the current LLM using conditional maximum likelihood principles
fine-tuning data comprising prompt- response pairs. However, its success heavily relies on the quality and volume of the data: Low quality can
mislead the model learning (Dodge et al. 2021 Luccioni & Viviano, 2021 Welbl et al. 2021 Longpre et al., 2024), introducing biases or inaccuracies that degrade performance, and small-scale datasets will hinder the model's ability to generalize well (Ghosh et al.
2024). On the other hand, collecting the ideal data needed for SFT can be challenging in practice. Generally speaking, task-specific data are often scarce, particularly in niche or emerging domains (Ghosh et al., 2024, Ma et al., 2024). making it difficult to collect a sufficiently diverse dataset. Additionally, ensuring data quality is non-trivial lask, as it involves curating examples that are both representative and free from noise or errors. Even for humans, identifying whether the data meet high-quality standards can be difficult due to the subtlcties of language and context. Consoquently, the lack of high-quality, task-sp data becomes a bottleneck for SFT, limiting the potential of LLMs to excel in specialized applications.
How can we mitigate the impacts of data on fine-tuning? Data filtering (Albalak et al. 2024) offers a promising solution. Specifically, it involves selecting a subset of data from the whole set that expected to be more beneficial for the targeted LLM than the original. With proper selection rules, such as gradient behaviors (Albalak et al. 2023), margins, loss, and influence (Bejan et al., 2023). filtering can refine data quality effectively. However, this comes at the cost of reducing the scale of the dataset, raising open questions about the trade-off between quality and scales and its impact on the generalization of the resulting model, Existing literature has attempted to mitigate this issue by exploring data rephrasing (Eldan & Russinovich, 2023 jin el al. 2024), while this approach heavily depends on manual efforts and/or expensive generators that are task-specific.
In this paper, we explore new mechanism towards better LLM fine-tuning, referred to forgetting.
Following previous wisdom (Yuan et al. 2024 Eldan & Russinovich 2023 Wang et al. 2025b Koh & Liang, 2017j, we begin by performing data filtering at the token level, categorizing tokens as either positive or negative based on their influence to enhancing performance. Note that token-level filtering helps preserve the data scale as much as possible, thus adopting as a default choice. Then, for positive tokens, conditional maximum likelihood is applied as usual, since our selection rules ensure that their learning will benefit the current model. Furthermore, for negative tokens, rather than simply discarding them, we propose applying forgetting (also referred to unlearning (Li et al. 2025 De Cao et al., 2021 Jang et al. 2023 Maini et al. 2024 Yao et al.) 2024b; Wang et al., 2025bb) to reduce the likelihood of their generation. Compared to positive ones, negative tokens are more likely to carry uninformative or even misleading knowledge. Explicitly forgetting these tokens not only prevents the model from generating them but also helps avoid overfitting to the current corpus. Moreover, we maintain the same data scale as in conventional fine-tuning, while taking some data (tokens more accurately) as negative samples during training to facilitate model generalization.
Although straight/orward to implement, we demonstrate the importance of forgetting in SFT for improved generalization through our extensive experiments. Specifically, we build our training corpus across 5 repre- sentative reasoning, knowledge and conversational datasets, and evaluate our forgetting mechanism alongside bascline methods on 5 diverse benchmark datasets, incorporating various LLMs as base models. For exam- ple, as shown in Table g in Section 5 using LLLMAA.2211 as the base model, our approach achieved a 2.51% improvement over that without forgetting and a 4.49% improvement over the fine-tuned model on full tokens. Similarly, with LLaMA3.2-3B, we obtained a 3.4% improvement over that without forgetting and 5.28% over fine-tuned model on full tokens. Additionally, with LLaMA3.1-8B, our approach resulted in 4.21% improvement over the no forgetting approach, and a 8.25% improvement over the fine-tuned model on full tokens. To validate scalability to larger model sizes, we conducted experiments with LLaMA- 2-13B in Appendix B.1 confirming the forgetting mechanism's 's generalization capability across different
2

---

Published in Transactions on Machine Learning Research (03/2026)
scales. Furthermore, demonstrate effectiveness across other model architectures (Qwen2.5-3B and CPT-Neo-2.7B) and diverse evaluation tasks in Appendix B.3
Connection with broader literature. The mechanism of forgetting is closely connected to preference optimization (PO) (Rafailov et al. 2023). Recalling that, many representative PO methods, such as direct preference optimization (DPO) (Rafailov et al.) 2023) and proximal policy optimization (PPO) (Schulman iet al, 2017), can broadly be reviewed as combining the objectives of learning and forgetting. They aim increase the likelihood of generating preferred corpora while reducing that of the dispreferred one. However, these methods are derived from the original PO objectives, which are inherently tied to problem setups and rely on manual labeling or reward models for preference annotation. In contrast, we focus on the SFT problems, where the forgetting mechanism acts as an enhancement strategy rather than a indispensable component of the problem formulation. Our method is inspired by PO but more focuses on the mechanism of forgetting as an integral component within learning. This approach helps mitigate the negative effects of low-quality data meanwhile enhancing generalization. In the long term, we aim bridge the methodological gap between SFT and PO, striving for a more unified and flexible framework for adapting LLMs.
2
Related work
2.1 Data selection for SFT
SFT is well-known fine-tuning technique that maximizes the likelihood of generating target tokens under the assumption that all tokens are informative. However, data quality has emerged critical bottleneck for this approach (Luo et al. 2024), with errors arising from various sources including human annotators, tool annolators, LLM hallucinations, and data processing inconsistencies (Luo et al. 2024).
LIMA Zhouu et al. 2023a hypothesized that LLMs primarily learn the style of dataset responses, rather than updating their pre-trained knowledge toward specialized tasks, by showing that fine-tuning on a 10k carefully curated dataset, they can obtain better performance than fine-tuning on a larger dataset.
To address quality challenges, rescarchers have investigated the advantages of data quality over quantity, proposing selection algorithms based on quality and diversity metrics to filter misleading samples and improve instruction- following capabilities (Chen et al. 2023 Maharana et al., 2023 Lu et al.) 2024 Wu et al. 2023) Xia et al.) 2024). While effective at improving performance, these approaches suffer from a fundamental limitation: they operate at the sample level, discarding entire examples and thus reducing the overall data scale available for training. This creates an inevitable trade-off between quality and quantity that remains unresolved.
Several data quality metrics have been introduced, such gradient matching (Zhou et al. 2023b). human feedback (Köpf et al. 2023) and influence function scores (Xia et al. 2024). Moreover, Dai et al. 2025) demonstrated that naturally higher influence scores for certain tasks can introduce bias in data selection, and proposed normalizing influence scores across different tasks before iteratively selecting samples for under- represented skills. Tn (Luo et al. 2024), authors propose two-stage noise-robust. framework that performs noise detection using multiple expert systems and then relabels the downstream task data by finding similar examples from the clean set to provide context. In another approach, researchers showed that selecting training samples aligned with the model's existing knowledge can improve performance by generating multi- ple instruction-responss pairs and choosing those with the highest probability according to the target model Zhang et al. 2025
Recent studies have explored various high-quality data selection algorithms for LLM fine-tuning, yet they predominantly overlook a crucial insight: even in noisy samples, some tokens still contain valuable informa- tion. By discarding entire samples, these methods inadvertently remove useful training signals. Furthermore, these approaches fail to utilize the rejected data as learning signal.
3

---

Published in Transactions on Machine Learning Research (03/2026)
2.2 LLM unlearning and PO
Several approaches have been proposed to remove specific information from LLM without complete retraining them from scratch, including data replacement and relabeling strategies (Eldan & Russinovichi 2023 jin al, 2024h, and knowledge editing techniques by predicting targeted parameter updates to change specific facts while preserving other knowledge (De Cao et al. 2021 Gradient ascent (GA) based methods are usually
used for their simplicity, which maximize the negative log- ogglikelihood of specific token sequences( Jequences(Jangg et al.
2023 Maini et al. 2024 Yao et. al.
2024b; Tian et al.
2024 Cha et al.) 2024
Yu et al., 2023). However,
some of them lead to degradation in LLM's outputs globally and damage the overall integrity of LLMs when
removing targeted knowlodge (Yu et al., 2023; Wang et al. 2025a Zhang et al.
2024 Lizzo & Hock 2025
called excessive unlearning, which some regularization techniques such as minimizing the KL-Div between the output distributions of the pre-trained and fine-tuned models (Yao et al.
2024 is proposed to maintain performance retain dataset. This introduce additional computational overhead and hyperparameter scnsitivity. Rescarchers in (Wang et al. 2025b) introduced WGA, which applics confidence-based weights to mitigale the excessive unlearning on a controlled forgetting manner.
In the PO field, DPO has emerged as an alternative to PPO-based aligument methods. However, PPO has been successful for its sample efficiency compared to earlier policy gradient methods, it still suffers from explicitly modeling reward model, and complex hyperparameter luning (Schulman et al. 2017). To address these challenges and making it more robust and less computationally expensive, DPO formulates the aligmment objective into a maximum likelihood formulation a preference-paires data, trying to make preferred responses more likely and dispreferred responses less likely. There are extensive studies to address
the limitations of DPO (Fthayarajh et al., 2024 Gheshlaghi Azar et al. 2024 Xu et al. 2024 Hong et al., 2024 Meng et al. 2024 Zeng et al. 2024), a new approach for preference- based unlearning was proposed by MManni et al.] 2024), which defines the forget set as the dispreferred responses, and the preferred response contains the refusal responses like 'I do not know the answer'. Inspired by this research, (Zhang et al. 2024) proposed new variant of DPO, called negative preference optimization (NPO) that uses only negative responses, disregarding the positive ones. In the (Wang et al., 2025b) further proposed Token-level NPO (TNPO) and Weighted TNPO (WTNPO), applying unlearning at the individual token level for more precise control over knowledge removal, yet these methods were developed specifically for targeted forgetting rather than as a complement to lcarning during SFT.
3 Preliminary
In this section, we present the foundational background essential to our work. We start by introducing SFT for autoregressive language modeling, followed by discussing the data quality issues within SFT.
3.1 SFT
Autoregressive language modeling, known sequential prediction of outputs conditioned on previous con- text, plays dominant rolc in contemporary LLMs. After pro-training, SFT is typically adopted to further improve LLMs for specific tasks by optimizing on task-specific instruction-response pairs. Specifically, rep- resenting a training corpus as = {X,,, Y)) including N sequence sample pairs, each pair containing X; as an input prompt and Y; as a completion response. Each prompt X, denoted as X; = { [ 1==1 with m2 indicating the sequence length of the i-th prompt. Similarly, each i-th completion response with sequence length of n; is denoted Y; [ In an autoregressive manner, the model learns to estimate the probability distribution P(y..j X,, Vi.gj 0) for each token Vi, in the response, conditioned the entire prompt X, and all preceding generated tokens in the response Vi.j [ V1. ...........jj... vi.j-1,, where d denotes the model parameters.
The standard cross-entropy objective is typically adopted for SFT, following the formulation of
(())
1
Σ
-log P(un,) Wt 0),
(1)
(I.J)Z
4

---

Published in Transactions on Machine Learning Research (03/2026)
where the index set is defined as:
I: E( (1.2..... NY. E .1.2,....,,
(2)
and the per-token loss function is defined as:
Einal 0) == - llog P(yi.l Xix Misii 0).
(3)
3.2 Data Quality of SFT
LLMs acquire diverse capabilities and knowledge representations through pretraining on extensive corpora. However, for utilizing them in specialized tasks, techniques such SFT play a remarkable role enhancing their performance by fine-tuning the LLM the training corpus without any selection discarding on the dataset's components (Pareja et al. 2025 Albalak et al. 2024).
However, collecting high-quality data, representing the required specific knowledge, is crucial to prevent inac- curacies and effectively align the LLM (Albalak et al. 2024). High-quality data collection can be challenging in practice due to several factors. Generally, task-specific data are often scarce, particularly in emerging domains. In addition, datasets are collected from various resources, often leading to inconsistent linguistic styles and quality, and errors due the use of annotator tools, human manual annotating (Luo et al. 2024) Therefore, each of them contribute noisy and mislcading tokens into the dataset thus jeopardizing the optimization process, leading to poor generalization.
To mitigate the impacts of low-quality and misleading data/tokens, existing methods proposed various data selection methods to maintain beneficial and high-quality data for fine-tuning (Albalak et al., 2024). More specifically, existing methods address data filtering al the data level; however, token-level filtering seems to preserve dataset scale and fine-grained information much more.
Although progress has been made in previous studies, they discard the low-quality data during fine-tuning, which significantly reduces the original dataset scale and potentially limits the model generalization. This remains an open question: how to leverage the full training dataset at its original scale while improving model performance? Specifically, is it possible to not only learn from high-quality samples but also utilize misleading data/tokens to make improvements in model generalization without overfitting to noise, while maintaining the comprehensive scope of the original dataset?
4 Method
SFT a well-established approach for aligning extensively knowledge-augnenteed pretrained LLMs with specialized tasks. As discussed in Section 3.2, practical datasets make challenging for SFT achieve high performance, their collection process leads to noisy dataset that jeopardizes the optimization process through misleading gradients. While many studies have attempted to address this issue by selecting high- quality subsets from SFT training data, these approaches sacrifice dataset scale instead of taking advantage from noisy tokens. This remained an open challenge to mitigate the effect of misleading tokens in the dataset, while preserving its scale. In this study, we propose a new approach for better LLM supervised fine- tuning, based on forgetting mechanism. Unlike traditional data selection approaches that treat all tokens uniformly and discard low-quality data, our method explicitly distinguishes between informative (positive) and uninformative or misleading (negative) tokens at a granular level. This token level approach preserves training data scale, while utilizing the tokens' training signals more effectively.
Specifically, actively forgetting negative tokens, rather than merely ignoring them, can significantly improve model performance by aligning better with larget data, freeing model capacity from undesired patterns, and preventing overfitting to noisy patterns. This insight particularly valuable when working with practical datascts that inevitably include noisy tokens that should be forgotten to preserve the model's general capa- bilities. The overall pipeline is outlined in Algorithm I In the following parts, we introduce the components of our pipeline, including the data preprocessing and training objective function.
5

---

Published in Transactions on Machine Learning Research (03/2026)
Algorithm Forgetting
Require: Base model 8, dataset D, proportion ps Emins tmax
Ensure: Finc-tuned model 8*
1: // Stage 1: Reference Model Fine-tuning
2: 0 < fine-tune 0 on sampled subset Dref C D
3: Stage 2: Token Quality Assessment
4: + All token indices (i:j) in Drrain
for (i,5) I do
Inf(yr.)) Q(vi.)) (nf(1))) end for
Quality score
// Stage 3: Token Selection
10: Sort tokens by Q(yi.) to partition into positive and negative subsets
11: + {(i,j) E II: Q(vr. 01,5i0,01 2 Fs(1 - p)}
b Positive tokens
12: I
D Negative tokens
13: // Stage Training with Forgetting
14: for slep 0 to tolal sleps do
stop X(step) + (tmax - tmin) total_ steps
15:
16:
Lp t Mean weighted loss over positive tokens in P
17:
LN + Mean weighted loss over negative tokens in N
18:
((0) + Lp - A(step) LN
19:
Update using optimizer step on L())
20: end for
21: return
4.1 Token quality assessment
To quantify token quality, we leverage the concept of influence functions (Koh & Liang 2017) between the base and reference models. Given base model with parameters and reference model with parameters 0' (introduced in Section 5.1.2), we define the eross-model influence for token yo, as follows.
1n = {(virlar,. 0') E(yi, ( 0).
(4)
The intuition is that tokens that become more predictable after initial training (resulting in loss reduction) represent patterns that the model has successfully learned and are likely to be informative.
The token quality score formulation is as follows:
Qun 0,0') Inf (3 0, 0').
(5)
A positive quality score indicates that the token became more predictable on the reference model (lower loss in 0' than in f), indicating that represents a generalizable pattern. In contrast, a negative score suggests that the token might represent noise or mislcading information.
4.2 Token selection
As a preprocessing step, we partition the tokens into positive and negative sets based on the quality scores. We first compute quality scores for all tokens in the training corpus, then sort them in descending order form the set S. Given proportion hyperparameter p (0, 1), partition the tokens follows:
P ((i,)) 1 0,00 Fs(1 p)} =
(6)
(7)

---

Published in Transactions on Machine Learning Research (03/2026)
where Fs(1-p) denotes the (1- (1-p)-th percentile threshold in S. The top p proportion of tokens considered as positive tokens form the P set, while the remaining tokens form the negative set W. In practice, we found that setting in the range of 0.7 to 0.8 achieves best results in our experiments. Furthermore, our experiments reveal that partitioning tokens by a zero threshold score (i.c. Q > as positive tokens) negatively affects performance. This challenges the int that tokens with higher confidence improvement are informative and beneficial, while the others are harmful, introducing an open challenge for proposing more robust methods to identify high-quality tokens.
4.3 Training objective
While standard SFT algorithms maximize the likelihood over all tokens uniformly (potentially reinforcing noisy patterns that mislead optimization) and data selection methods discard the distinguished noisy data before training, our approach maintains the benefits of full-scale training while addressing quality concerns, which enables improvements in the model performance, by minimizing the likelihood of generating the noisy tokens and freeing model capacity from misleading noisy patterns. As mentioned in the Section 2.2] unlearning techniques proven to be effective to mitigate the influence of undesirable data while preserving the model utility. In our context, rather than forgetting some specified knowledgo(c.g. copyrighted content), we forget misleading lokens through GA, effectively utilizing both positive and negalive lokens. This approach enhances the model generalization while maintaining the original data scale with no information loss. We propose training objective for selective learning and forgetting follows.
((0) = E(eyer Mis IOJER (yngltnn)) M(step) EGiee Yis LUAEE - Elnggliia 0)
(8)
E(ieer Mig M(aEe
Euseer l(a.)
where the first term represents the average weighted loss over positive tokens, and the second term represents the average weighted loss over negative tokens. We use X(step) (Imax - -min)) total steps as an adaptive coefficient that scales linearly with training progress, ensuring an effective balancing of positive and negative gradients through the optimization process. Please refer to Appendix B.5 for more experiments the X function selection.
In this training objective, optimization initially shares goals with generalization, but their objectives later diverge. The forgetting mechanism acts as a regularization technique that pulls optimization back for generalization when their goals conflict. By using the adaptive balancing coefficient, this enables to better capture the underlying preferred data distributtin rather than overfitting to the noise or merely following the pattern of low-scale high-quality data.
However, work differs from NPO (Zhang et al. 2024 and TNPO WWag et al. 2025b) in problem setting and mechanism design. While NPO (Zhang et al. 2024 and TNPO (Wang et al. 2025b) address unlearning removing predetermined unwanted knowledge (such as private data, copyrighted content) from trained models, our method focuses on SFT, where forgetting serves as a regularization term rather than the primary objective. We use token-level influence scores to automatically identify low-quality tokens within the training corpus to respect both the datasot quantity and quality. Then, apply forgetting to free model capacity from misleading noisy patterns, simultaneously learning positive tokens and forgetting negative ones. In contrast, NPO (Zhang et al. 2024) and TNPO (Wang al.) 2025b) operate predefined forget sets where unlearning itself is the goal, not regularization mechanism for improving generalization during task adaptation.
5 Experiments
5.1
Experimental setups
5.1.1
Datasets
Training data. We constructed our training corpus by randomly sampling from five datasets, Flan_v.
Chung et al. 2024h, Dolly (Conover et al. 2023), Open Assistant. 1 (Köpf et. al. 2023), Stanford Alpaca
7

---

Published in Transactions on Machine Learning Research (03/2026)
Table Dataset distribution comparison
50k Sample
10k Sample
Dataset
Samples
Percentage
Samples
Percentage
Dolly
2,617
5.23%
503
5.03%
Flan_ v2
17,803
35.61%
3,593
35.93%
Open Assistant
5,960
11.92%
1,135
11.35%
Stanford Alpaca
9,276
18.55%
1,834
18.34%
WizardLM
14,344
28.69%
2,935
29.35%
Taori et al. 2023 and WizardLM (Xu et al. 2023
Please refer to Appendix A for more datasets details.
The datasct distribution presented detail in Table 1 This corpus provides a comprehensive coverage of
domains and response styles, thereby enhancing the model's generalization capabilities (Wang al. 2023
Evaluation benchmarks. For the evaluation part, we have performed comprehensive evaluations five diverse benchmark datasets. They are TruthfulQA (Lin et al.
2022 to evaluate the ability of LLM in dataset
providing truthful and accurate information, BoolQ (Clark et al. and evaluates LLM's ability in making precise boolean judgernents, LogiQA (Liu et al. 2020) focused on logical reasoning, TydiQA (Clark et al. 2020) to evaluate the LLM on multilingual question-answering and ASDiv (Miao al. 2020) to evaluate the LLM on math word problems. The benchmarks' attributes are presented in Table
2019
binray question-
The evaluation processed on all benchmark samples, by using the Im-eval-harenes,
repository.
5.1.2 Models
Base models. In this paper, we choose 3 open-source LLMs including -LaMMA-3.2-1B. LLaMA-3.2-3B and LLaMA-3.1-8B
Grattafiori et al.] 2024 in diverse complexity as our base models for fine-tuning.
Reference models. The reference models obtained by fine-tuning the base models on subset Dref DD with Dref O Drrain = 0 where Dirain is the training corpus and DD is combination of training datasets. The fine-tuned LLM will be used for calculating the influence scores. We also investigate the robustness of our approach when the reference dataset contains duplicate samples (see Appendix B.2
Baselines. In this study, our baselines include the base model, the supervised fine-tuned version of the base model the whole training dataset with full tokens, and the fine-tuned version of the base model the preprocessed training dataset including only the top k% clean tokens.
5.1.3 Training configurations
For the reported results in Table 3 we employed model-specific hyperparameter pairs (Imin, (max) as follows: (10-5, 5,-5,00 0.25) for LLaMA-3.2-1 and (10-4,0.25, 0.25) for both 1.AMA-----33 and LLaMA-3. for our adap- tive balancing coefficient A(step). These values were determined through ablation studies optimizing for performance across our benchmark tasks. For fine-tuning the LLMs, we used LoRA (Hu et al. 2022 for its memory efficiency and stability during training. We set rank-size of 64, the scaling factor of 16 and dropout 0.1 for LoRA. We used the AdamW optimizer (Loshchilov & Hutter 2019), with the overall batch size equal to and the fine-tuning process is performed for 1 epoch with a learning rate 10-4 and a linear learning rale scheduler with 0.03 warm-up ratio. Moreover, we conducted our experiments on 4 NVIDIA L40S-48GB GPUs with Intel Xeon 6338 CPUs, running on Ubuntu 20.04.6 LTS. The systems utilize Transformers version 4.51.3 and CUDA version 12.5. Training time for 1B, 3B and 8B models approximately takes 3, and 5 hours, respectively.
http:://gihuub.ca//lleeteee

---

Published in Transactions on Machine Learning Research (03/2026)
Table 2: Evaluation datasets attributes
Dataset
Focus Area
Data Size
Question Length
TruthfulQA
Truthfulness
817
Medium
BoolQ
Boolean QA
15,942
Short
LogiQA
Logical reasoning
8,678
Medium
TydiQA
Multilingual QA
204k
Varied
ASDiv
Math Word Problem Solving
2,305
Varied
Table 3: Performance comparison of different methods across five different benchmarks using LLaMA-3.2 IB, LLaMA-3.2-3B and LLaMA-3.1-8B variants as our base models. We evaluate four approaches: Base (unmodified), Full Tokens (standard SFT), Ignoring, and our proposed Forgetting. The results show accuracy (%) for TruthfulQA, BoolQ, LogiQA, and ASDiv, and onc-shot F1 score for TydiQA Bold values demonstrrte best performance on each benchmark. Results show mean values with standard deviations from independent. training runs. Our proposed Forgetting method achieves significant. improvements across different benchmarks and model scales.
Method
TruthfulQA
BoolQ
LogiQA
TydiQA
ASDiV
AVG
Base model:
LLaMA-3.2-IB
Base
37.83:10
63.80:10
22.17400
14.36:0
0410
27.63:10
Full Tokens
38.7410.39
59.8411.94
24.6010.25
28.10-0.46
0.55.10.48
30.3710.39
Ignoring (seq-level)
39.56:10.57
61.17:00.66
24.03:10.25
27.90-0.34
1.46:10.15
30.88:10.28
Forgetting (seq-level)
38.93++0.08
63.13+0.46
24.80+0.12
28.75=0.23
2.50++.04
31.62++0.10
Ignoring [ooken-level]
13.40:10.13
60.2111.66
24.34110.31
33.87=0.64
0.91
32.35=10:46
Forgetting (loken-level)
44.83:10.45
65.3
25.60:10:48
36.2111.77
2.28
34.86:10.22
Base model:
LLaMA-3.2-3B
Base
39.45++0
73.04+0
22.17+0
21.12+0
31.24+0
37.40+0
Full Tokens
42.95+0044
72.54+0.59
25.51=0.21
44.04=0.27
49.46=0.14
46.90=:016
Ignoring (seq-level)
0..5
72.93::0.28
24.36:1047
44.82=1.03
49.11
46.36=0.41
Forgetting (seq-level) 40.95+0.30
77.80:1.38
25.2710.23
47.52=0.45
49.8310.93
48.27110.6
Ignoring (token-level)
47.23+0.86
75.40+0.37
25.12+0.31
47.63=0.42
48.51+0.74
48.78
Forgetting (token-level)
50.32:t0.96
76.66+0.07
27.09+0.377
56.36:10.06
50.47+0.3
52.181:012
Base model:
LLaMA-3.1-8B
Base
45.08110
82.15110
26.511:0
46.67.10
12.9310
42.67-00
Full Tokens
44.51+00.48
81.4410.47
25.68-10.14
52.03=0.18
51.46:10.22
51.0244.11
Tgnoring (seg-level)
47.05+00.21
85.17+0.45
24.64+0.18
52.31-0.08
51.62+0022
52.16+0.24
Forgetting (seq-level) 47.83+0.09
85.56=10.18
24.85+0.37
57.56-0.33
57.76+00118
54.71+0.10
Ignoring (token-level)
52.38-10.22
82.7610.07
25.5310.11
56.66-0.06
57.95-10.35
55.06110.16
Forgetting (token-level)
58.39-10.65
83.14+0.15
31.1510.88
66.2110..23
57.48+0.12
59.27:10.35
5.2 Empirical Results
We conducted comprchensive experiments to evaluate our forgetting approach against all basclines. Re- markably, our method outperformed all baselines in average performance. The forgelting met achieved superior results with in the range of 70% to s0%, while the ignoring has its best-case performance with p in the range of 50% to 60% across all benchmarks. We demonstrate the results of our experiments utilizing three different variants of LLaMA in Table comparing the method in their best-case performance, specifi- cally, setting p 0.7 for our forgetting approach and p 0.5 for the ignoring approach. Notably, compared

---

Published in Transactions on Machine Learning Research (03/2026)
to the standard SFT our method has achieved an average performance improvement of 4.49% on the 1B model, 5.28% on the 3B model and 8.25% on the 8B model. Furthermore, compared to ignoring baseline, our method has achieved performance improvement of 2.51% on the 1B model, 3.4% on the 3B model and 4.21% on the 8B model. Please see Appendix B.6 for a detailed analysis of computational overhead.
Additional experiments with LLAMA-2-13B (Tovrron et al., 2023) confirms these forgetting mechanism's generalization capability in larger scales, with detailed results provided in Appendix [..1 To further validate the generalizability of our forgetting mechanism across different model architectures and benchmarks, we conducted additional experiments Qwen2.5-3B (Qwen et al. 2025) and GPT-Neo-2.7BB (Black et al, (Zhou et al. 2023c), ARC-Challenge (Clark et al. 2018), LAMBADA (Paperno et al.) (016)) specially using OpenAl preprocessing (Radford et al. 2019) from the EleutherA P repository, and Arithmetic (Brown et al., 2020). The results, presented in Appendix [3.] demonstrate the superiority of our forgetting mechanism. Notably, our forgetting method achieved 5.33% improvement over the ignoring bascline on Qwen2.5- 3-3B and a 3.56% improvement on GPT- Neo-2.7B, confirming that the benefits of our approach extend beyond the LLaMA family and linguistics task.
2021) across four diverse benchmarks, Instruction-Followee
Token-level vs. sequence-level granularity. A key design choice in our approach is operating at the token level rather than the sequence level. This granular approach is motivated by the observation that individual sequences often contain a mixture of both informative and misleading tokens. Sequence-level selection would classify entire sequences as either positive or negative, potentially discarding valuable tokens within otherwise noisy sequences, or conversely, retaining harmful tokens within generally useful sequences. Token-level selection allows us to preserve beneficial information while selectively forgetting problematic content, maximizing the utility of our training data. The Table 3 shows comparison of the different approaches.
TableBishow that token-level approaches consistently outperform their sequence-level counterparts across all model sizes. For example, with LLaMA-3.2-3B, token-level forgetting achieves 52.18% average performance compared to 48.27% for sequence-level lorgetting. This superiority stems from token-level selection's 's ability to preserve useful information even in partially noisy sequences, while sequence-level selection discards entire soquences that may contain valuable tokens alongside problematic ones.
5.3 Ablation study
Impact of p. Our empirical evidence indicates that the forgetting approach demonstrates superior gener- alization capability when p has higher value, partitioning a larger subset of tokens as positive tokens and treating all remaining tokens as negative tokens (forget rate of 1 - p). However, forgetting only a subset of the remaining tokens and discarding the others leads to suboptimal performance, indicating the effectiveness of forgetting all the 1 1-p p tokens as negative tokens. Figure b illustrates the average performance for different forget rates. Morcover, the choice of the hyperparameter p, directly affects the noise distribution in positive and negative sets. Higher value of p can introduce noisy tokens to the positive set,, while lower value of p can add informative tokens to the negative set. Figure [[a shows the comparison between different values of for the forgetting and ignoring approaches. The average performance of the forgetting method has significantly decreased for the lowest value p = 0.4, due to the higher proportion of informative tokens in the negative set.
Impact of A(step). As explained in Section | effectively balancing the training and forgetting gradients is crucial for optimization stability. As related studies typically use a constant coefficient in the range (0.1) to reduce the learning rate of forgetting gradients. However, through empirical investigation, we observed that training iterations progress, the learning rate roduction leads to the vanishing of the forgetting gradients. Thus, we used an adaptive function X(slep), as a coefficient on forgetting loss term of our dual objective function, not only to balance the learning and forgetting gradients, but also to efficiently preserve the ef- feets of forgetting gradients during fine-tuning. According to the dual objective function formula, ignoring approach is equivalent to forgetting with a balancing coefficient of zero. In a comparison of balancing co- efficient strategies, we evaluated three approaches: static approaches with constant values zero (ignoring)
10

---

Published in Transactions on Machine Learning Research (03/2026)
and 0.0001 (optimal value for static strategy), and dynamic approach using the linear function X(step) with biin = 0.0001 and lmaz = 0.25. The corresponding average improvements are 48. 18.78,, 49.59%, and 52.18%, respectively. These results demonstrate that adaptive adjustment via linear function significantly outperforms static coefficient assignment, highlighting the critical role of selecting an appropriate balancing coefficient strategy. By incorporating ((step); the forgetting learning rale decreases more gradually with a shallower slope. We investigated the impact of the adaptive parameter (step) through series of experi- ments.
Hyperparameter sensitivity analysis. To evaluate the robustness of our approach to hyperparameter choices, we conducted extensive experiments varying the key parameters mii and max while keeping p 0.7 fixed. As shown in Figure a), our method demonstrates impressive robustness to values across a wide range. For practical selection of P, users can use the ratio of tokens with positive influence scores as an initial estimate- our experiments, this ratio was 0.67, leading us to select = 0.7 as optimal. Compre- hensive results across different combinations of Eiin and max values using L.LMAA3.2-3B are presented Appendix [.4]
Impact of forgetting. As demonstrated in previous sections, the forgetting mechanism significantly im- proves the performance of fine-tuning with respect to that without forgetting and standard SFT. Specifically, when comparing the forgetting and ignoring approaches with the same selection ratio (p= 0.7), the forgetting method achieves an accuracy of 52. 58,,8,, putperromingg the ignoring approach (48.39%). This performance gap indicates that the negative tokens set has a high noise ratio, reinforcing the impact of forgetting mis- leading tokens, leading to higher performance.
Performance
Performance vs Forget rate
54
51.31
52.04
Averaqe performance
Average performance
52
51,32
50
50,61
48:78
40h71
47.02
47.97
50
19.21
48.10
30.90
Forgetting
Ignortng
Forgetting
1.3
0.4
0.5
0.6
0.7
0.8
0.5
1.0
44
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Hyperparameter P
Forget rate
(a.
(b)
Figure 1: Performance analysis: (a) Average performance of forgetting versus ignoring methods across different values. (b) Average performance of the forgetting method with different forget rates.
6 Limitations
Despile our method's improvements, some limitations remain. The approach sensitive dataset size and noise ratio, leading to performance degradation for smaller negative token sets. However, it worth noting that noise existence is common in real-world practical datasets. Additionally, computational budget restricted our experiments to models up to 13B parameters with limited-scale training data. The performance remains uncertain how well the mechanism would perform on larger-scale base models and datasets.
7 Conclusion
This paper aims to reduce the reliance of LLM fine-tuning on data quality, an important and on-going topic that has boen receiving increasing attentions these days. Unlike previous works that primarily focus on improving data selection, we suggest that exploring new learning paradigms equally crucial. Specifically, propose novel fine-tuning mechanism named forgetting, which explicitly enables the model to forget
11

---

Published in Transactions on Machine Learning Research (03/2026)
misleading message carried by those filtered-out tokens. It mitigates the negative impact of noisy mis- leading data while preserving the dataset scale, helping to improve generalization and overall performance. In the future, we will explore more formal and rigorous ways to defiiing and enhancing data quality, as well as extend the forgetting mechanism to other related areas within LLMs, such as pro-training, preference optimization, and inference.
References
Josh Achiam; Steven Adler; Sandhini Agarwal, Lama Ahmad, Nlee Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXit:2303.0877 08774, 2023.
Divyanshu Aggarwal, Ashutosh Sathe, Ishaan Watts, and Sunayana Sitaram. MAPLE: Multilingual eval- uation of parameter efficient finetuning of large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguisties: ACL 2024, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: pp. htt.///..2.........e./e/ www.iiinnng-cc..s... acl.88.
10. 10.18653/v1 /t//////22222/////
Alon Albalak, Colin A Raffel, and William Yang Wang. Improving few-shot generalization by explor- ing and exploiting auxiliary data. In A. Oh, T. Naumann, A. Globerson, K. Sacnko, M. Hardt, and
S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 24754-24780, Cur- ran Associates, Inc., 2023. URL https: httpa/:///ppoeosespeeeeee neurips. pttpg//weerrrpppp.ppp.pepe.ee..e http://p.....pa.eeeeeeeeee//////
pdf
Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Mucnnighoff, Bairu Hou, Liangming Pan, Haewon Jcong, et al. A survey on data selection for language models. Xiv preprint ar 2rv:222002. 16827, 2024.
Irina Bejan, Artem Sokolov, and Katja Filippova. Make every example count: On the stability and utility of self-influence for learning from noisy NLP datasets. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.); Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 10107 10121, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023.emnlp- main.625. 6255. URL https::/accannthooogy org/ 2023. emnle- mmain. 625/
Sid Black, Gao Leo, Phil Wang, Connor Leahy, and Stella Biderman. Gpt-neo: Large scale autoregressive language modeling with mesh-lensorflow. Zenodo, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Vore Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Lan- guage models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (ods.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Cur- ran Associates, Inc., 2020. URL btttps:/prroceeeee neurips. wwtpseee.rppppppppeeeeeeeeeeeeee fflles//paeer/222//fiee 245T0660110040771111888442222280 Paper. pdf
Sungmin Cha, Sungjun Cho, Dasol Hwang, and Moontac Lec. Towards robust and cost-efficient knowledge unlearning for large language models. In Adaptive Foundation Models: Evolving AT for Personalized and Efficient Learning, 2024. URL http::://pperrrviee.neete///// http://re.....eetoooooo//ttaaait
Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yifan Yanggong, and Junbo Zhao. Maybe only 0.5% data nceded: preliminary exploration of low training data. instruction tuning. arXiv preprint ar Xiv:22300..0 092466, 2023.
Mark Chen, Jerry Tworck, Hoewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. or Xiu preprint ar arXio:2107.03374, 2021.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkum Yu, Slav Petrov, Ed H. Chi, Jell Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason language models. Journal of Machine Learning Research, 25(70):1-53, 2024. URL http://jmlr org/ wwwpg//apers/aee (atpeerr/vv2//3--800 html
Wei. Scaling instruciion-fnetume
Christopher Clark, Kenton Lee, Ming-Way Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2924-2936, Minneapolis, Minnesota, June 2019. Association for Computational Linguisties. doi: 10. 10.18653/v1/N19 1300. URL
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. Tydi qa: benchmark for informmalion--seeiing question answering in lypologically diverse languages. Transactions of the Association for Computational Linguistics, 8:454 470, 07 2020. ISSN 2307-387X. doi: 10.1162 a 00317. URL https://dol. 1et...111tcccccccc hatp:/ttcl___a1111
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafford. Think you have solved question answering? try arc, the ai2 reasoning challenge. orXiv preprint 1803v 1833055003 2018.
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tumed 1mm, 2023. URL hhttps://wwr htttp://www.databrikkee databricks wwt.tss..ooooooe/eeeeee http/////gg/2///////2/ sotableeinstruction-tuee
Qirun Dai, Dylan Zhang, Jiaqi W. Ma, and Hao Peng. Improving influence-based instruction tun- ing data selection for balanced learning of diverse capabilities. In Christos Christodoullpoollo,, Tan- moy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Compu- tational Linguistics: EMNLP 2025, pp. 7079-7102, Suzhou, China, November 2025. Association for URL
Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2083 http://////22...innnesseeee
https://accantholla org org/ findinggn-eniinn e.nlp.372
Nicola. De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Maric- Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/200 cmnlp-main.522 522. URL lattps://cclatthol.gg. org1221.. maii..22///.................
Jesse Dodge, Maarten Sap, Ana Marasovié, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (cds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1286 1305, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18633vvl http:////222/./// www.2p-1mmmppmmmi...... URL
org/2021. emnlp caalpp-mmii.ee. 98/
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning for 1lms. 2023.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Model aligmment as prospect theoretic optimization. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (ods.), Proceedings of the 41st International Con- ference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 12634-12651. PMLR, 21 27 Jul 2024. URL http::ttps/ppeet htttps://proceedingg.. mlr. prwpss/235/ ethayarajh24aaa.... html wttp::awwwr.ree.eeeeeeeeee
13

---

Published in Transactions on Machine Learning Research (03/2026)
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human prefer- ences. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li (eds.), Proceedings of The 27th Interna- tional Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learn- PMLR, 02 04 May 2024. URL httpps://proceedings... mlr. pte....rees//28//
ing Research, pp.
dheshlaghi-azane html]
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. A closer look at the limitations of instruction tuning. Tn Proceedings of the 41st International Conference on Machine Learning, ICML'24 JMLR.org, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The lama 3 herd of models. arXiv preprint Xrxi:24077 2024.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, DDL Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optinal large language models. arXiv preprint arXiv:2203.15533
10, 2022.
Jiwoo Hong, Noah Lee, and James Thorne ORPO: Monolithic preference optimization without reference model. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Melhods in Natural Language Processing, pp. 11170-11111 Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v /v 220044..mmmmmmmm.6666666 URL https:
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https:// http://www.......orrooooooo//aa.
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Min- joon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pp. 14380-14408, Toronto, Canada, URL https:
July 2023. Association for Computational Linguistics. doi: /200533311/223.cl--bb //aclanthology.o www.lllogyyrgg/222020 wrg/2023.........oo wa:0022..ccc1110 acl-long.80l. 805/
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. ACM Trans. Softw. Eng. Methodol. 35(2), January 2026. ISSN 1049-331X. doi:
10.1145/3747588. URL https://doi..rg///
1145/3747588
Zhuoran jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. Rwku: Benchmarking real-world knowledge unlearning for large language models. In A. Globerson, L.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomozak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 98213-98263. Curran Associates, Inc., 2024. URL https:
neurips cc.pppprrrpprrr...p.ppppp wtw.ss/paper///2////// doi: 01.55222/0099017-000 Dataeaeeetaaaaatrtoeeeeereeaaaar and Benchmarks _are___rcckppi pdf bIT7Brcccco066582400222cddeeeer2
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1885-1894, PMLR, 06-11 Aug 2017. URL
https://proceeding mlr. http:///.nn......pees////////aaa press//700kkhi7a..ht
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdul- lah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Ar- nav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. Openassis- tant conversations democratizing large language model alignment. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (ods.), Advances in Neural Information Processing Systems, volume 36, pp. 47049-7667777 177669-47661..1 Curran Associates, Inc., 2023. URL https: attp:UUULLLtpp::/roooooeiss wwwpoediiiinnnnneuiieeeeee neurips ccciips.cc//ppppppp http://ccppppppapaaaaeeea files/ anw_Beechmarrs.preeee .df
14

---

Published in Transactions on Machine Learning Research (03/2026)
Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, and Anmin Fu. Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects. TEEE Transactions on Neural Networks and Learning Systems, 36(8):13709 13729, 2025. doi: 1AL.00/TNNNNSS.222550000000
Vladislav Lialin, Vijeta Deshpande, and Anna Rumshisky. Scaling down to scale up: A guide to parameter- efficient fine-tuning. Xiv preprint or urXiv:2303.156, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human false- hoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214 3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. acl-long.222 229. URL https://aclanthooogy/ .rg. 2022. 2all-oong2220 229/
Jian Liu; Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge datasct for machine reading comprchension with logical reasoning. In Christian Bessiere (cd.), Proceedings of the Twenly-Ning Ninth International Joinl Conference on Artificial Intellignnce, IJCAI-20, pp. 3622 3628. International Joint Conferences on Artificial Intelligence Organization, 7 2020. doi: 10.2 10.24963 10.24963/ijcaa.2200 /ij/a 501. URL https:://oi..rg 1tarr//10.2000 10. 24963/1jcaai 2020/501 Main track.
Tyler Lizzo and Larry Heck. UNLEARN efficient removal of knowledge in large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Lin- guistics: NAACL 2025, pp. 1272-7283, Albuquerque, New Mexico, April 2025. Association for Compu- tational Linguisties. ISBN 111-8-89176-19555 doi: 10.18653/v1 (LL6653/v1222255 finddiigfnnnn----i. -aaiggs-naal.4555. URL https:
//aclantholoog.orggg
findingnnnnnannnn naacl. .40/
Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, and Daphne Ippolito. A pretrainer's guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Compulational Linguislics: Human Language Technologies (Volume Long Pupers), pp. 3245 3276, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/ 11/2024.naacl-oogg
URL http::tt::// https:://cclaaataa http://aaaaatooooorrrr.rr./
maacl- 179/
lyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference Learning Representations, 2019. URL https:// htt.://oooooomorm//.//o.
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. #instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https:///pperrevee..c
forum?id--szeehybbU
Alexandra Luccioni and Joseph Viviano. What's in the box? analysis of undesirable content in the Common Crawl corpus. In Chongqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.). Proceedings of the 59th Annuul Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 182-189, Online, August 2021. Association for Computational Linguistics. doi: 1/653/v1120211 acl-short.24 URI https:// aclanthology org/2021.
wwwnthooogy..rg2221...ooo ac0021.allssh0 seelsshort.22 24/
Junyu Luo, Xiao Luo, Kaize Ding, Jingyang Yuan, Zhiping Xiao, and Ming Zhang. Robustft: Robust supervised fine-tuning for large language models under noisy response. arXiv preprint or Xix:2412. 14922. 2024.
Runyuan Ma, Wei Li, and Fukai Shang. Investigating public fine-tuning datasets: A complex review of current practices from a construction perspective. arXiv preprint ar arXiv:2407.08/71 2024.
Adyasha Maharana, Pratock Yadav, and Mohit Bansal. D2 pruning: Message passing for balancing diversity and difficulty in data pruning. arXiv preprint ar 200:203000.79,3, 2023.
15

---

Published in Transactions on Machine Learning Research (03/2026)
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and Zico Kolter. Tofu: A task of fictitious unlearning for llms. Xiv preprint or Niv:2401.06122 2024.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a relerence-Iree reward. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomezak, and C. Zhang (eds.), Advances in Newral Information Processing Systems, volume 37, pp. 124198 124235. Curran Associates, Inc., 2024. doi: 10.52202 00700000000000077900000 URL https://proceeddggss neurips wwtpppsrccpppp.pppppppp wttpc/cpeerepeeepppeeeeeeeeeeeee fttrsf/lee/ppp///////// pdf
Shen-yun Miao, Chao-Chun Liang, and Keh- Yih Su. A diverse corpus for evaluating and developing English math word problem solvers. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (ods.), Proceedings of the 58th Annual Meeting of the Associalion for Compulational Långuistics, pp. 975 984, Online, July 2020. Association for Computational Linguistics. doi: 10. (0.18653/v1, 11853/11/20200ac///// /2200000ccllmmin290.0 URL
attps://aclanthoology www.aattthhooooyooo/222222/ wttp/o00..//...o.. www.acclmmn... 92/
Ramesh Nallapati, Bowen Zhou, Ciccro dos Santos, Çağlar Gulgehre, and Bing Xiang. Abstractive text summarization using sequence-lo-seqqeeee RNNs and beyond. In Stefan Riezler and Yoav Goldberg (eds.), Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp. 280 -9002 Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v//K16 1116--028.
URL htttaa///acclanaaooog..oogg////
Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaclla Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Annwal Meeting of the Association for Computational Lánguistics (Volume 1: Long Papers), pp. 1525 1534, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1 /P11111111111 1/11/1111111111. URL
Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang, Krishnateja Killamsetty, Shivchander Sudalairaj, Wen- long Zhao, Seungwook Han, Abhishek Bhandwaldar, Guangxuan Xu, Kai Xu, Ligong Han, Luke In- glis, and Akash Srivastava. Unveiling the secret recipe: A guide for supervised fine-tuning small L.Mss. Tn The Thirleenth International Conference on Learning Representations, 2025. URL https: //openreview weee:/forummdddeEEnnni
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https: htapss///axxvv...
abs/ 2412. 15115
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and 1lya Sutskever. Lan- guage models are unsupervised multitask learners. OpenAI, 2019. URL httpp::/ openai. haster--onngage-ndeee//aaagag 2024-11-15.
saciigeemmmell__eeeeeeee are_ aelaaareeea__a umsupervised_multtine mulliitsttt meta pdf Accessed:
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Newral Information Processing Sys- tems, volume 36, pp. 53728 53741. Curran Associates, Inc., 2023. URL URL/http:://proceedngg. neuriips.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang- SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2302, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653/v1/D16- 1264. URL https: htt.aaatthooooooooo/1/------20
16

---

Published in Transactions on Machine Learning Research (03/2026)
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimiza- tion algorithms. Xrviv preprint Xin: 11:::10070000 1707.06347, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint urXiv:2402.0833 2024.
Tobias Strangmann, Lennart Purucker, Jörg KH Franke, Ivo Rapant, Fabio Ferreira, and Frank Hutter. Transfer learning for finetuning large language models. arXiv preprint arXiv:2411.01195 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloinng llama model. https://github... com/
hatpo/lllasstaffrrdrrrroo alpaca 2023.
Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. To forget or not? towards practical knowledge unlearning for large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 1524 1537, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 111653/v1120244 findings- findings-emmll.... emnlp.82. URL https://
aclanthology wra.lloggyrrg//22/ httg//224.findingess findings-emnlp... 82/
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. Xixiv preprint arXiv:2307.09288 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, F. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances Newral Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL /ttp::/ttt/// neurips wwwprrissccppppppeee wttppccc_paeataeeeeeee fateer_fileepppprr// pddf
Qizhou Wang, Bo Han, Puning Yang, Jianing Zhu, Tongliang Liu, and Masashi Sugiyama. Towards effec- tive evaluations and comparison for Ilm unlearning methods. In International Conference on Learning Representations, 2025a.
Qizhou Wang, Jin Peng Zhou, Zhanke Zhou, Saebyeol Shin, Bo Han, and Kilian Q Weinberger. Rethinking Ihm unlearning objectives: A gradient perspective and go beyond. In International Conference on Learning Representations, 2025b.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Tz Beltagy, and Hannanch Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources. In A. Oh, T. Naunann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neurol Information Processing Systems, volume 36, pp. 74764-747887 Curran Associates, Inc., 2023. URL
neur wwwrrip..ccppaaaaep. files// and www.ceernss..pppp. pdf
Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hen- dricks, Kirsty Anderson, Pushmoct Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxify- ing language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Compulational Länguislies: FMNLP 2021, pp. 2447-2469, Punta Cana, Dominican Republic, November 2021, Association for Computational Linguistics. doi: findings emnlp. 210/
10.18653/v1 /02/55////2221111// w1t.021..ndingnnmmmm/ppppmmpppp findings-emmllp2110. URL httt::://aclaathoooog..oro
Shengguang Wu, Keming Lu, Benfeng Xu, Junyang Lin, Qi Su, and Chang Zhou Self-evolved diverse data sampling for efficient instruction tuning, arXiv preprint ar 2xX1v:2311.08182 2023.
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: selecting influential data for targeted instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, ICML/24. oM,. 2024.
17

---

Published in Transactions on Machine Learning Research (03/2026)
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprinl arXiv:230,.1224, 2023.
Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: pushing the boundaries of Ilm performance in machine translation. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. Jog..org, 2024.
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. In Lun-Wet Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Länguistics (Volume 1: Long Papers), pp. 8403 8419, Bangkok, Thailand, August 2024a. Association for Computational Lin- guistics. doi: 10.18653/v 100/3/3/12222.///// a2024.acl-oong. 457. URL https://aclaantoooogy wwwp:/ggy..rg//222..../ mong.llo...... 457/
Yuanshun Yao, Xiaojun Xu, and YangLiu. Large language model unlearning. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in New- ral Information Processing Systems, volume 37, pp. 105425-105475. Curran Associates, Inc., 2024b. doi: 10. 10.52200.0079017-3344 URL https://proceeddigg
cwpppprrrrppppp....pppppp wwtppapprfilleeppppppppeeeeeeeep f1les//pappr222///ill
bes/aacfbbccc4aacca0eeeeeteeeeed Paper Conferenceeeeeeeece
Charles Yu, Sullam Jeoung, Anish Kasi, Penglei Yu, and Heng Ji. Unlearning bias in language models by partitioning gradients. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Compulational Linguislies: ACL 2023, pp. 6032 6048, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2088 hing://i////...ddinnnngg--a/s ac1.375. URL https: //aclanthology.o org/2023. f
Xiaojian Yuan, Tianyu Pang, Chao Du, Kejiang Chen, Weiming Zhang, and Min Lin. A closer look at machine unlearning for large language models. arXiv preprint (00:2410.08109 2024.
Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Token-level direct preference optimization. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 58348 58365. PMLR, 21-27 Jul 2024. URL
mlr press/v235/zone
html
Dylan Zhang, Qirun Dai, and Hao Peng. The best instruction-tuningg data are those that fit. In The Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openrevier. net/forum?idee htt./oommmmdd-4/F3etaaa)
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint 0556668 2024.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Be- ichon Zhang, Junjic Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303. 20X2223303.8822,, 1(2):1 124, 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, IIIII YU, Susan Zhang, Gargi Ghosh, Mike Lewis, Tuke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 55006 55021. Cur- ran Associates, Inc., 2023a. URL https: http:::s///roceednngge neurips neurips.cc/paprr http:///oooooooo.o/po//ooe//e//a /ites//appr///22///111 46652114879244477cc1117777774333 Patp://eereoaaaaaaaaa---rrreeee Conf .df
Daquan Zhou, Kai Wang, Jianyang Gu, Xiangyu Peng, Dongze Lian, Yifan Zhang, Yang You, and Jiashi Feng. Dataset quantization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17205-17216 October 2023b.
18

---

Published in Transactions on Machine Learning Research (03/2026)
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-folloiing evaluation for large language models. arXiv preprint or Xii:2331.0079) 2023c.
A Dataset Details
Table provides comprchensive information about the datascts used create training corpus, including Cheir quality assessment, size, total length of samples, and source.
Table 4: Datasets attributes
Dataset
Data Quality
Size
Length
Resource
Dolly
High
15.01k
Varied
Human-annotated
Flan v2
High
100k
Varied
Human-annotated
Open Assistant
Moderate
33.92k
Varied
aamaan-nnnotatdd
Stanford Alpaca
High
52k
Varied
L.M-generated
WizardLM
High
100k
Longer
LLM-generated
B Additional experimental results
B.1 LLaMA-2-13B results
To further validate the robustness and scalability of our forgetting mechanism, we conducted additional experiments using LLaMA-2-13B as the base model. These results provide additional evidence that our ap- proach consistently improves performance across different model architectures and scales, extending beyond the LLaMA-3.x series reported in the main paper.
The results Table 5 demonstrate that our forgetting method maintains its effectiveness with larger models, achieving a 6.16% improvement over standard SFT and a 4.16% improvement over the ignoring baseline. This consistency across model scales (from 1B 13B parameters) reinforces the gencralizability of our approach and suggests that the forgetting mechanism provides fundamental benefits for supervised fine- tuning regardless of model size architecture.
Table 5: Performance comparison different methods across five benchmarks using LLaMA-2-13B as the base model. Results show accuracy (%) for TruthfulQA, BoolQ, LogiQA, and ASDiv, and one-shot F1 score for TydiQA. Bold values demonstrate best performance on each benchmark. Our proposed Forgetting method achieves significant improvements across different benchmarks, with average improvement of 6. 6.16% over standard SFT and 4.16% over the ignoring approach.
Method
Dataset
TruthfulQA
BoolQ
LogiQA
TydiQA
ASDiV
AVG
Base model:
LLaMA-2-131
Base
36.73
80.67
26.05
34.27
0.35
35.61
Full Tokens (standard SFT)
42.65
82.24
27.44
36.77
8.76
39.57
Ignoring
43.01
84.50
27.29
38.39
15.34
41.71
Forgetting (Ours)
52.82
84.13
27.95
48.71
17.80
46.28
B.2 Impact Reference Dataset Duplicates
We conducted additional experiments to investigate the robustness of our approach when the reference dataset contains duplicate samples. However our pipeline's preprocessing step removes duplicate samples
19

---

Published in Transactions on Machine Learning Research (03/2026)
from the both training and references datasets, this analysis is important for understanding how data quality in the reference model training affects the overall forgetting mechanism performance.
Table shows results using LLaMA-3.2-3B when the reference dataset includes duplicate samples. Interest- ingly, our forgetting method remains effective even under these suboptimal reference conditions, achieving a 4.93% improvement over standard SFT and a 2.05% improvement over the ignoring baseline. This demon- strates the robustness of our influence-based token quality assessment even when the reference model is trained imperfect data, suggesting that approach can handle practical scenarios where perfect data curation is not feasible.
Table 6: Performance comparison with duplicate samples in reference dataset using LLaMA-3.2-3B as base model. Results show mean values with standard deviations from 3 independent training runs. Our forgetting method maintains effectiveness with imperfect reference data quality.
Method
Dataset
TruthfulQA
BoolQ
LogiQA
TydiQA
ASDIV
AVG
Base model:
LLaMA-3.2-3B (Reference with Duplicates)
Base
39.45:10
73.0410
22.17-00
21.12410
31.244:
37.40:10
Full Tokens (standard SFT)
42.95++0.47
0..54+0.59
20.51
44.04=0.27
49.46
46.90=t0.16
Ignoring
09.91
75.60:10886
24.9911.355
48.6110.20
49.81110.01
49.78:0.22
Forgetting (Ours)
51.09=0..54
07.00=0.009
26.57=00.08
54.88=00229
49.60:t0.14
51.83+0.11
B.3
Evaluation on Diverse Model Architectures
To demonstrate the broad applicability of our forgetting mechanism, we extended our evaluation additional model architectures beyond the LLaMA family. Specifically, conducted experiments Qwen2.5-3B and GPT-Neo-2.7B evaluating performance across four diverse benchmarks including Eestruction-Follwwee (IFEval)) ARC- ChC-Challenge, LAMBADA, and Arithmetic. The characteristics of these evaluation benchmarks are detailed in Table 7
For these experiments, we maintained our L.IMAA3.2-3B experimental setup and hyperparameters, as de- scribed in Section 5.1.3] The results presented in Table show that our forgetting method consistently outperforms both standard SFT (full tokens) and the ignoring bascline across both model architcctures. On Qwen2.5-3B, our method achieves an average performance of 59.01%, representing 16.49% improvement over standard SFT and a 5.33% improvement over the ignoring approach. Similarly, on GPT-Neo-2.7B, our forgetting mechanism attains 28.15% average performance, demonstrating a 4.37% improvement over standard SFT and a 3.56% improvement over ignoring. These results confirm that the effectiveness of our forgetting mechanism generalizes well across diverse model architectures and evaluation tasks, validating its broad applicability for improving SFT of large language models.
Table 7: Charactcristics of diverse evaluation benchmarks
Dataset
Focus Area
Data Size
Question Length
1FEval
Instruction Following
541
Varied
ARC- CRC-Challenge
Scientific Reasoning
1,172
Medium
LAMBADA
Language Modeling
5,153
Short
Arithmetic
Mathematical Computation
Varied
Short
B.4 Hyperparameter sensitivity analysis
Table presents comprchensive results across different combinations of fmin and fmax values using LLaMA-
3.2-3B. The results demonstrate remarkable stability, with performance variations remaining small across different hyperparameter settings (standard deviation 0.5% across configuration)). This robustness ensures
20

---

Published in Transactions on Machine Learning Research (03/2026)
Table Performance comparison across diverse model architectures and benchmarks. Results show accuracy (%) for all benchmarks. Bold values indicate best performance. Our forgetting method demonstrates consis- tent improvements across different model families (Qwen and GPT-Neo),, validating its broad applicability beyond the LLaMA architecture family.
Model
Method
Fval
ARC-Challenge
LAMBADA
Arithmetic
AVG
Base model: Qwen2.5-3B
Base
23.13
44.70
66.72
13.16
36.93
Qwen2.5-3B
Full Tokens
19.59
43.66
68.64
38.19
42.52
Ignoring
19.22
45.63
68.82
81.03
53.68
Forgetting (Ours)
33.49
45.39
69.76
87.40
59.01
Base model: GPT-Neo-2.7B
Base
1.90
27.48
61.74
0.43
22.89
GPT-Neo-2.7B
Full Tokens
1.49
29.72
63.05
0.84
23.78
Ignoring
0.19
30.55
66.15
1.46
24.59
Forgetting (Ours)
11.66
31.63
67.29
2.03
28.15
that our method maintains superiority over baselines without requiring extensive hyperparameter tuning, The stability is partly attributed to the inherent robustness of large language models and their extensive pre- trained knowledge, which provides strong foundation that is resilient to moderate changes in fine-tuning parameters.
Table 9: Hyperparameter sensitivity analysis for fmin and fmax using LLaMA-3.2-3B with fixed p 0.7. Results demonstrate robustness across different parameter combinations.
min
fmax
TruthfulQA
BoolQ
LogiQA
TydiQA
ASDiV
AVG
0.00001
0.45
52.75
74.38
25.89
54.27
48.10
51.08
0.00001
0.35
51.55
75.11
26.15
56.74
48.42
51.59
0.00001
0.25
50.93
76.58
25.99
56.13
50.26
51.98
0.00001
0.15
50.17
75.45
26.19
54.37
50.67
51.37
0.0001
0.45
50.90
77.56
25.83
54.33
48,90
51.50
0.0001
0.35
51.20
75.67
26.65
57.21
48.78
51.90
0.0001
0.25
50.32
76.64
27.09
56.36
50.47
52.18
0.0001
0.15
50.09
74.79
25.27
55.21
51.82
51.44
0.001
0.15
49.05
76.03
26.36
54.85
51.49
51.56
0.001
0.25
48.96
76.50
28.68
56.35
49.66
52.03
0.001
0.35
51.25
74.41
26.51
56.58
50.05
51.76
0.001
0.45
50.69
74.50
25.98
56.97
48.46
51.32
0.01
0.15
50.46
75.24
26.12
54.17
50.95
51.39
0.01
0.25
51.02
76.28
27.75
55.48
49.93
32.09
0.01
0.35
52.78
74.44
25.58
55.92
48.30
51.40
0.01
0.45
50.09
74.87
27.60
54.69
48.68
51.19
21

---

Published in Transactions on Machine Learning Research (03/2026)
B.5 X(step) Constant À
In this section, compare our adaplive function X(slep) against using constant value for A. To ensure fair comparison, we conducted extensive experiments on LLaMa-3.2-3B. evaluating wide range of constant values. Table [0 presents the results across different constant settings, demonstrating that oven the best- performing constant value outperformed by our adaptive {(step) approach.
Table 10: ((ste) selection experiments on L.AMA-3.2-3B with fixed p 0.7.
A
TruthfulQA
BoolQ
LogiQA
TydiQA
ASDiV
AVG
constant value
47.85
76.53
25.19
50.41
49.27
49.85
(emax
Lmin)
step
(linear)
50.32
76.64
27.09
56.36
50.47
52.18
total steps
B.6 Computational Cost
As shown in the Table I1 we conducted a detailed analysis of computational cost. Given the consistent and significant performance improvements across different scales, this computational overhead is reasonable and justified. The preprocessing phase (warmup + influence computation) is one-time cost that can be ignored if multiple training runs are conducted.
Table 11: Training time comparison of different strategies for 3B and 8B models.
Component
3B Model
8B Model
Preproccssing Phase
Warmup Training
36 min
52 min
Influence Score Computation
47 min
73 min
Training Phase
Full Token
205 min
347 min
Ignoring
166 min
282 min
Forgetting
187 min
311 min
Total Time
Full Token (training only)
205 min
347 min
Ignoring (warmup + score + training)
249 min
407 min
Forgetting (warmup + score + training)
270 min
436 min
Overhead
Ignoring Full Token
+21.46%
+17.29%
Forgetting Full Token
+31.71%
+25.65%
Forgetting Ignoring
+8.43%
+7.12%
22


Reviews:
[{"review_id": "grDsOz5isA", "reviewer": "Reviewer_NBXC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a \"forgetting\" mechanism for LLM SFT, classifying tokens as positive (informative) or negative (misleading) via influence functions from a reference model. Positive tokens use standard MLE; negative ones apply gradient ascent with adaptive balancing. This preserves data scale, mitigates noise, and improves generalization, shown via gains (2.51–8.25%) on benchmarks with LLaMA, Qwen, and GPT-Neo models.\n\n### Key Strengths  \n- Token-level forgetting preserves scale while addressing noise, outperforming sample-level filtering.  \n- Strong experiments across models, tasks, and ablations; links SFT to PO for unified adaptation.  \n\n### Key Weaknesses  \n None.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Performance claims backed by averaged results on 5+ benchmarks, ablations on ρ/λ/granularity, and appendices for other models/tasks. Methods clear with equations/algorithm; \"knowledge boundaries\" inferred from gains, not directly probed.", "audience_yn": "Yes", "audience_explanation": "Researchers working on LLM unlearning would be interested in these findings.", "requested_changes": "- **Clarify examples (strengthening):** Add positive/negative token examples with scores/context.  \n- **Compare unlearning baselines (critical):** Benchmark vs. NPO/TNPO in SFT.  \n- **Analyze boundaries (strengthening):** Include output probes pre/post-forgetting.  \n- **Scale tests (strengthening):** Discuss/add larger models/datasets.  \n- **Fix issues (critical):** Correct citations/typos; define w_{i,j}.  \n- **Costs (strengthening):** Table for overhead vs. SFT.  \n- **Clarify 2.1 (critical):** Define \"noisy samples\" (e.g., annotation errors); add: \"Hence, we adopt token-level filtering to retain useful signals in noisy samples.\"  \n- **Alternative metrics (strengthening):** Ablate token quality with gradients/loss/confidence.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3UssJkQkVT", "reviewer": "Reviewer_sGcN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a token-level “forgetting” mechanism in SFT. It splits tokens into positive vs negative based on the loss difference between a base and reference model. It trains the model with a dual objective: standard likelihood training on positive tokens and gradient-ascent-style “forgetting” (rather than discarding) on negative tokens. The method yields consistent average improvements over standard SFT and “ignoring” baselines across multiple model sizes and architectures.\n\nstrengths\n- The paper proposes a token-level “forgetting”, and compares token-level vs sequence-level and shows token-level doing better\n- The paper proposes to “forget” low-quality tokens instead of just discarding them\n- Experiments show consistent performance gains across different model scales on diverse benchmarks\n\nweaknesses\n- The \"forgetting\" mechanism is essentially gradient ascent on negative samples, similar to DPO/NPO negative component.\n- The method requires training a reference model first.\n- Missing qualitative analysis of what tokens are being classified as negative.\n- Missing comparison with recent advances.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper provides emprical evidence that adding the forgetting term improves average benchmark scores compared to standard SFT and Ignoring mechanisms. \n\nHowever, the concepts of \"negative tokens lack essential semantics or are misleading\", is not directly supported. It is measured by the cross-model loss change. The metric introduces several possible factors, such as reference model fine-tuning may introduce distribution shift or catastrophic forgetting on rare tokens. The paper would benefit from qualitative and quantitative analyses of what kinds of tokens are classified negative.\n\nThe claims about “shapes a knowledge boundary” and improves “diversity” are also not directly supported.", "audience_yn": "Yes", "audience_explanation": "The data quality and selection is a practically important problem in SFT, which is highly relevant to ML community.", "requested_changes": "- Add comparison with stronger recent advances for noisy/robust SFT or data selection under matched compute and data, such as D^2pruning, LESS.\n- Report the cost of the method such as reference training + token scoring + final training (time/GPUs), and discuss scalability for larger models/datasets.\n- Provide qualitative and quantitative analyses of what kinds of tokens are classified negative, such as token type, position, dataset sources, etc., and discuss whether the “negative” tokens are really correspond to actual annotation noise, factual errors, or low-quality generations.\n- Discuss how reference model quality affects results.\n- Weaken or remove the claim about “shapes a knowledge boundary” and improves “diversity”, unless providing more evidences.", "broader_impact_concerns": "The reference model could be biased without carefully safety/regression testing.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Q3GW1za261", "reviewer": "Reviewer_egFx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Forgetting, a novel LLM fine-tuning method that focuses on lower-quality data. Unlike prior work that performs sequence-level filtering, the paper introduces a token-level filtering approach that classifies each token as positive or negative. Moreover, building on an intuition similar to that of preference optimization methods, the negative tokens are explicitly forgotten (i.e., unlearned).\n\nEmpirical evaluations on a range of models with sizes up to 13B parameters demonstrate a significant improvement in model performance with the proposed method. The ablation study further shows that both token filtering and unlearning are effective.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The proposed method is simple, intuitive, and effective, and it is backed by sound empirical evaluation. The experiments cover models of different sizes and multiple datasets.\n\nFor the two fundamental contributions—token-level filtering and unlearning—the ablation study demonstrates that both components are effective, and that combining them achieves the best performance. These results support the theoretical claims made in the earlier sections, namely that finer-grained filtering is beneficial and that negative signals should not be ignored.", "audience_yn": "Yes", "audience_explanation": "SFT is an important problem for the community and would have quite large audience.", "requested_changes": "From my perspective, no major changes are required. I only have a few minor recommendations:\n- Providing more details on how the reference model is trained would further strengthen the claims. For example, what proportion of the dataset is used to train the reference model, and does this incur significant additional computational overhead?\n- It would be helpful to include more theoretical or intuitive explanations for the choice of the adaptive coefficient. In particular, this could better support the claim that the strength of unlearning should be gradually increased over time.\n- Additionally, although it is not directly related to the paper’s main claims, it would be interesting to analyze which tokens tend to be classified as positive or negative, and whether the filtering exhibits any bias toward particular types of tokens or token distributions.", "broader_impact_concerns": "Not any", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LQ93ErYkey", "reviewer": "Reviewer_ggfb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a token quality assessment within the SFT framework in order to better utilize the tokens' training signals. To do so, the method relies on the cross-model influence metric to come up with a token quality score. The tokens in the corpus are then sorted according to this quality score and positive/negative tokens are determined by slicing this ranking according to percentiles. Negative tokens are then forgotten during the training process using gradient ascent.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The proposed method is sound, the equations are meaningful and appear to be right. The empirical results support the applicability of the method.", "audience_yn": "Yes", "audience_explanation": "Yes, this work is of interest to folks in the community working on post training optimization such as SFT.", "requested_changes": "No changes requested - good paper.", "broader_impact_concerns": "No such concerns from my side.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)