Soft Prompting for Unlearning in Large Language Models (2025)

Abstract: The widespread popularity of Large Language Models (LLMs), partly due to their emerging in-context learning ability, has highlighted the importance of ethical and safety considerations for deployment. Motivated by corresponding data protection guidelines, we investigate machine unlearning for LLMs. In contrast to the growing literature on fine-tuning methods to achieve unlearning, we focus on a comparatively lightweight alternative called soft prompting to realize unlearning in LLMs. With losses designed to enforce forgetting as well as utility preservation, our framework Soft Prompting for Unlearning (SPUL) learns prompt tokens that are prepended to a query to induce unlearning of specific training examples at inference time without updating LLM parameters. We conduct a rigorous evaluation of the proposed method, and results indicate that SPUL can significantly improve the trade-off between utility and forgetting for text classification and question-answering. We further validate our method with LLMs of varying parameter sizes to highlight its flexibility and provide detailed insights into the choice of hyperparameters and the influence of the size of unlearning data.

## introduction with evolving transformer models @xcite and the availability of massive text corpus, language models are progressing rapidly. the pretrain and fine-tune pipeline has garnered wide popularity, especially since the release of llms such as gpt @xcite and llama @xcite . however, several ethical and security concerns have been raised due to the presence of private, sensitive, or harmful information in the training data. for example, llms can regurgitate personal information @xcite , or mimic harmful and/or hateful behavior as a consequence of such content being prevalent in the data @xcite . the non-consented and unwarranted use of copyrighted content for llm training has also raised significant concerns @xcite . current policies governing the use and distribution of such models do not encompass all ethical avenues; nonetheless, certain regulations such as california consumer privacy act (ccpa) and gdpr's right to be forgotten (rtbf) serve as guidelines for organizations to ensure that their operations do not infringe upon user privacy. specifically, these regulations stipulate that businesses and data collectors provide and exercise an optout mechanism essentially allowing individuals to request the deletion of their data on reasonable grounds. in machine learning literature, these regulations have been conceptualized as machine unlearning @xcite , which aims to eliminate the influence of unwanted data points on a model's behavior as if they had never been observed during training. naturally, machine unlearning should be integrated into the llm pipeline to address previously outlined issues resulting from the presence of sensitive or harmful data in pre-training. however, unlearning in llms faces unique challenges due to the inaccessibility of model and pre-training data, and the sheer size of the pre-trained llms making re-training practically infeasible. much of the research in this direction therefore focuses on the fine-tuning approach which involves training all or a subset of llm parameters to enforce unlearning @xcite @xcite @xcite . in contrast, we present a resource-efficient approach to unlearning in llms via soft prompting. soft prompting optimizes a set of task-specific token embeddings that learn contextual information from a corresponding dataset and instruct a frozen llm during inference @xcite . we leverage this ability to modulate llm outputs using learnable prompts and formulate soft prompting for unlearning (spul), a resource-efficient mechanism to achieve llm unlearning in text classification and multiple-choice question answering (mcqa) tasks. spul optimizes soft prompts so that they learn to encode underlying information in the data relevant for unlearning. when prepended to the input tokens of an llm during inference, the soft prompts guide the llm towards a generic response. we implement a multi-objective loss aligned with specific unlearning goals to facilitate soft prompt learning. spul unlearns undesirable outcomes without updating large-scale llm parameters and can fully capitalize on the language understanding capability offered by the pre-trained llms. consequently, the same pre-trained llm can be utilized for different unlearning tasks and datasets during inference. we formulate llm unlearning as the forgetting of a training data subset composed of benchmark nlp datasets for sentiment classification or mcqa @xcite @xcite . the unlearning subset could contain data corresponding to entities or sensitive and potentially harmful information. to evaluate whether spul can achieve unlearning, we focus on quantifying the predictive performance on the unlearning subset as well as a general retain subset and consider the trade-off between them. in this setting, spul can effectively induce forgetting during inference while preserving the pre-trained utility with performance comparable to or significantly better than multiple baselines that implement fine-tuning. we conduct experiments to analyze the influence of spul hyperparameters including the contribution of loss components and the size of the soft prompts. we further validate spul on multiple pre-trained llms with different parameter sizes and using varied sizes of unlearning sets. ## conclusion we investigate unlearning in llms to remove the influence of unwanted/harmful training examples during text classification and mcqa. we present a soft prompting strategy to unlearn subsets of training data while keeping pre-trained llm parameters frozen to maintain the model's generalizability. the proposed spul framework optimizes a small number of prompt tokens using a multi-objective loss function defined on disjoint training data subsets representing the forget data subjected to removal and the retain data that aims to preserve model utility. experimental evaluation on sentiment classification and mcqa datasets demonstrates the efficiency of spul over fine-tuningbased baselines for trade-offs between forget quality, retain utility, and generalizability. we also empirically show that spul can adapt to multiple llms and is robust to large unlearning requests.