N euro R eset : LLM Unlearning via Dual Phase Mixed Methodology (2025)

Abstract: This paper presents the method for the unlearning of sensitive information from large language models as applied in the SemEval 2025 Task 4 challenge. The unlearning pipeline consists of two phases. In phase I, the model is instructed to forget specific datasets, and in phase II, the model is stabilized using a retention dataset. Unlearning with these methods secured a final score of 0.420 with the 2nd honorary mention in the 7B parameter challenge and a score of 0.36 in the 13th position for the 1B parameter challenge. The paper presents a background study, a brief literature review, and a gap analysis, as well as the methodology employed in our work titled NeuroReset. The training methodology and evaluation metrics are also presented, and the trade-offs between unlearning efficiency and model performance are discussed. The contributions of the paper are systematic unlearning, a comparative analysis of unlearning methods, and an empirical analysis of model performance post-unlearning.

## introduction large language models (llms) demonstrate excellent natural language understanding and language generation capabilities. they are trained on vast amounts of data. curating the data to remove private or sensitive information is a laborintensive task and is often overlooked. as a result, once an llm is trained, it may contain sensitive or erroneous information that ideally should be deleted for ethical, privacy, or regulatory reasons @xcite . unlearning in machine learning is an emerging area of interest, focusing on the selective removal of unwanted information while preserving the overall model integrity and generalization ability @xcite . the unlearning must guarantee that the information erased is not recoverable by model inversion attacks but, at the same time, allow the model to function on unrelated tasks @xcite . the traditional unlearning methods, such as knowledge distillation and fine-tuning, have issues such as uncertain outputs post-unlearning and catastrophic forgetting, whereby the said models lose relevant knowledge alongside the ones they do not want to lose @xcite . this work describes a scalable unlearning paradigm using fine-tuned llms. llm unlearning with multitask evaluations are also proposed in recent times @xcite . we propose a two-phase unlearning framework comprising target-specific gradient-based forgetting and post-unlearning stabilization. hence, it leads to effective removal of specified data while substantially reducing catastrophic forgetting of general knowledge. the system was evaluated on multi-tasking scenarios measuring unlearning efficacy and overall model performance post-unlearning. primary contributions of this work are threefold: ## conclusion this paper presents the neuroreset framework employing the dual phase unlearning method for llm unlearning. sensitive information unlearning is a very challenging task, and one of the key parameters is the mia score. our proposed approach achieved the mia score of 0.876 for 7b parameter and 0.841 for 1b parameters placing us in the top 15 teams in the semeval challenge 2025. the approach presents some limitations, such as an inefficient retain phase and lack of support for multidomain and multilingual aspects. the presented work can be extended further to mitigate these challenges by further fine-tuning and experimentation. the long-term goal also includes assessing the effects on downstream applications.