UNDIAL : Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models (2025)

Abstract: Mitigating the retention of sensitive or private information in large language models is essential for enhancing privacy and safety. Existing unlearning methods, like Gradient Ascent and Negative Preference Optimization, directly tune models to remove unwanted information. However, these methods often become unstable because they fine-tune by maximizing loss, which is the opposite of traditional loss minimization in learning. This reversal creates instability, especially on larger datasets, as the model struggles to balance unlearning with maintaining language capacity, leading to over-unlearning. In this paper, we introduce UnDIAL (Unlearning via Self-Distillation on Adjusted Logits), a novel and robust unlearning method. Our approach leverages self-distillation to adjust logits and selectively reduce the influence of targeted tokens. This technique ensures smooth convergence and avoids catastrophic forgetting, even in challenging unlearning tasks with large datasets and sequential unlearning requests. Extensive experiments show that UnDIAL is the first direct tuning method to achieve both robustness in unlearning and scalability, while maintaining stable training dynamics and resilience to hyperparameter tuning.

## introduction the increasing widespread use of large language models (llms) @xcite @xcite in userfacing applications raises significant privacy concerns. trained on vast, unmoderated web data, these models risk unintentionally exposing personally identifiable information (pii), such as names and addresses @xcite . furthermore, llms are vulnerable to malicious exploitation, i.e. adversarial attacks @xcite @xcite , allowing confidential data to be extracted and heightening concerns about data security with @xcite . in addition to these privacy risks, data protection regulations such as the eu's general data protection regulation @xcite and the california consumer privacy act (ccpa, 2018) enforce the "right to be forgotten," enabling individuals to request the removal of their personal data from online platforms. this creates an urgent need for techniques that allow llms to effectively "unlearn" and prevent the disclosure of specific information-a process known as llm unlearning. recent advances in llm unlearning fall into two main categories. the first category involves using an auxiliary model to explicitly memorize sensitive information, which is later removed from the original model using techniques such as contrastive decoding @xcite @xcite or parameter merging @xcite . however, this approach introduces infrastructure overhead and poses a significant risk if the auxiliary model is exposed, as it contains exactly the data meant to be forgotten. another line of research focuses on directly tuning the base llm model to unlearn sensitive information, using techniques such as gradient ascent (ga) @xcite and negative preference optimization (npo) @xcite . these approaches are gaining more attention as they align more closely with the growing emphasis on ai safety @xcite . despite these advances, the recent unlearning benchmark muse @xcite highlights a major drawback in current methods: applying unlearning to larger corpora leads to a decline in general language usefulness. this limits its usage in real-world settings, as an effective unlearning method must scale reliably with increasing data sizes, and accommodate continual updates-all figure 1 : an illustration of the self-distillation process in the proposed undial method: the original logits generated by the model are adjusted by subtracting the one-hot distribution of the target token. the student model is then fine-tuned to approximate this modified logit distribution. since the adjustments rely solely on the original model's outputs, this is a self-distillation process to de-emphasize the token to be forgotten. in this work, we introduce a novel direct-tuning method, undial, which enables unlearning via self-distillation with adjusted logits. as shown in figure 1 , we generate a target distribution by reducing the logit of the token to be unlearned. this target distribution is fixed during self-distillation, ensuring a stable optimization process. unlike ga and npo, which suffer from significant model capacity degradation as datasets scale and training extends, undial demonstrates strong robustness to data scaling, hyperparameter tuning, and sequential unlearning, offering the first robust unlearning method for direct tuning llms. our main contributions are as follows. 1) we identify the robustness issues in current unlearning methods and propose a new, more robust method based on self-distillation. 2) we demonstrate the effectiveness and robustness of undial across various hyperparameter settings, forget set sizes and a number of unlearning requests. 3) we also explore a variant of undial that focuses solely on specific set of tokens like named entities or nouns, which can further improve its overall performance. ## methodology motivation with an example. as highlighted by @xcite , direct tuning methods face the challenge of instability. methods like ga and npo, designed to directly unlearn from the original model, often lead to the so-called over-unlearning issue, where the algorithm continues to unlearn after the corresponding knowledge is forgotten. this often leads to the model to collapse with the model capacity dropping to zero. while npo partially mitigates this by adding a regularization term to slow the rate of divergence, it fails to prevent longterm collapse in practice @xcite . to demonstrate this critical issue, we apply ga and npo methods on the muse dataset and illustrate the training dynamics in figure 2 . as shown in the results, both ga and npo exhibit model collapsing, although npo diverges more slowly than ga as also shown in @xcite ; @xcite ,where both ga and npo lead to over-unlearning and thus to a substantial decline in model usefulness and performance. moreover, npo is also very sensitive to different hyperparameter setups and thus difficult to tune. in the early stages of training, the distance on the forget set remains approximately constant (see figure 2 left), showing that the model is not unlearning as expected. after this initial plateau stage, the model briefly begins to unlearn but quickly collapses. this instability reflects how sensitive npo is to hyperparameter tuning and the need to stop training at exactly the right moment. even slight overshooting can lead to severe performance degradation, i.e. over-unlearning. in contrast, undial consistently shows robust performance throughout training, converging to a stable distribution. this stability allows for flexible stopping points without any degradation risk. un-dial also achieves a substantially lower forget-set distance in far fewer steps, making it not just robust, but highly efficient. ## case study two: muse benchmark we now evaluate our method on the muse dataset @xcite . muse is the most recent and comprehensive unlearning benchmark with the data obtained from bbc news passages. the original work separates the data into two sets: forget and retain. the goal is to unlearn a set of bbc news passages while retaining knowledge on other news passages. for evaluation, question answering is conducted with respect to the news coming from the two sets. the questions related to the forget set will test knowledge memorization, which we want to keep low. in contrast, the questions targeting the retain set will test whether the unlearning procedure impacts unrelated topics, referred to as utility preservation. following the setup of @xcite , we use llama-2 7b @xcite as the base model and lora @xcite with rank 8 to fit the fine-tuning onto one nvidia a100. ## conclusion we introduced undial, a novel unlearning method based on self-distillation, which effectively balances reducing memorization with preserving language generation and understanding capabilities. our approach represents a significant advancement in direct-tuning unlearning methods, offering improved robustness from multiple angles. extensive experiments on the extraction data and muse benchmarks demonstrated state-of-the-art unlearning performance. additionally, we show that undial is highly resilient across varying hyperparameters, different forget set sizes, and sequential unlearning requests. with its ability to prevent model collapse and scale efficiently, un-dial presents a promising next step for real-world applications requiring unlearning from llms.