S em E val-2025 Task 4: Unlearning sensitive content from Large Language Models (2025)

Abstract: We introduce SemEval-2025 Task 4: unlearn- ing sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1) unlearn long form synthetic creative documents spanning different genres; (2) un- learn short form synthetic biographies contain- ing personally identifiable information (PII), in- cluding fake names, phone number, SSN, email and home addresses, and (3) unlearn real docu- ments sampled from the target model’s training dataset. We received over 100 submissions from over 30 institutions and we summarize the key techniques and lessons in this paper.

## introduction large language models (llms) have achieved enormous success recently due to their ability to understand and solve various non-trivial tasks in natural language. however, they have been shown to memorize their training data @xcite which, among other concerns, increases risk of the model regurgitating creative or private content. often, such issues are discovered post model training during testing or red teaming. furthermore, stakeholders may sometimes request to remove their data after model training to protect copyright, or exercise their right to be forgotten (general data protection regulation). in these instances, retraining models after discarding such data is one option but doing so after each such removal request is prohibitively expensive. machine unlearning is a promising line of research for removing sensitive information from models' parametric memory. while unlearning has been studied for sometime in classification problems, it is still a relatively underdeveloped area of study in llm research since the latter operate in a potentially unbounded output label space. current algorithms often fall short of effectively and efficiently unlearning sensitive information from llms, without impacting model utility. further, there is a need for benchmarks which can provide thorough evaluations of new unlearning algorithms in removing different categories of sensitive information. to address these needs and to spur further research on this topic, we developed a new challenge (and an associated benchmark) for llm unlearning as part of the semeval 2025 competition. this document provides a summary of our challenge foot_0 along with the benchmark, results and key takeaways. ## related work given the nascent stage of unlearning research in llms, few prior works exist which address the task of robustly evaluating the success of unlearning. @xcite presented a new challenge task in which the goal was to to unlearn information contained in select images within the task of image based age prediction. while successful, the specific task addressed in this challenge was narrow, focusing only on image based age prediction -a classification problem with 10 classes with limited applicability in the unbounded text generation task of large language models. @xcite released a new evaluation framework named tofu which partially addressed this task of evaluating llm unlearning algorithms. their framework was evaluated on question answering task applied on biographies of synthetically created fake authors. they train target models on this synthetic data and evaluate the ability of unlearning algorithms to forget a portion of this synthetic dataset. while being a promising first step, this work has a few key limitations: unlearning the targeted information required for the qa task is unlikely to cause loss of any other substantial information, specially linguistic attributes such as grammar. further, this work leverages gpt4 to generate the synthetic content, which may have downstream implications owing to gpt4's proprietary license. more recently, @xcite released a benchmark named muse which evaluated model unlearning using real data set for containing news documents and harry potter book chapters. this benchmark released detailed evaluation metrics to robustly evaluate the unlearning algorithms. however since it only leverages real data set the benchmark does not provide a clean test bed to evaluate model performance. specifically, the information contained in the unlearn documents may also appear in other disjoint training documents, limiting the effectiveness of unlearning. while the tofu benchmark mentioned before avoids this by only using synthetic documents, the data set coverage is rather limited (it only containts biographic information). the benchmark developed in our challenge addresses both these shortcomings together and presents a single holistic testbed to evaluate model unlearning in llms. ## challenge description to robustly evaluate unlearning algorithms on their effectiveness in removing different categories of information from llm, we developed 2 a new unlearning benchmark (on english language), covering three distinct sub-tasks spanning (1) creative content, (2) personally identifiable information (pii) of synthetic individuals and (3) real biographies of individuals sampled from wikipedia. please refer to @xcite for detailed information on the dataset creation process. within each sub-task, we further test the models for regurgitation (where model is prompted to complete partial documents) and knowledge (via generated question-answer pairs), leading to 12 different sub-tasks for the challenge. to score highly in the challenge, participants are expected to do well in all sub-tasks. in comparison, existing unlearning benchmarks such as tofu @xcite and muse @xcite cover only a portion of the subtasks we test for. for each subtask, we released retain (r) (i.e. model should retain these documents in memory) and forget (f ) datasets (i.e. model should forget information from these documents) in size) which were fine-tuned to memorize documents from all three tasks. participants were encouraged to explore various unlearning algorithms which enable them to remove the sensitive information present in f without affecting model knowledge on the r. our initial data release was further split in 80:20 ratio as train and validation subsets for optional hyperparameter tuning. participants were expected to submit working python scripts containing their unlearning code for the evaluation phase, which were executed on privately held subsets of retain and forget sets from each sub-task. we provide further details on our dataset creation for the three tasks below, followed by details on the evaluation phase. ## benchmarked algorithms we benchmarked our dataset on several state of the art unlearning algorithms described below. gradient ascent: this is a straightforward unlearning algorithm where we reverse the direction of model update by flipping the sign in gradient descent, in order to steer the model away from the sensitive model outputs in the forget set. while easy to implement, this approach has a significant drawback since the gradient ascent training objective is unbounded, which can lead to model divergence with nonsensical outputs for all inputs. the loss term in this algorithm reverses sign of the standard cross entropy training loss (l ce ) and is applied only on the forget set f : gradient difference @xcite : in this approach, we augment the gradient ascent objective applied on forget set, by adding a gradient descent objective on the retain set. by jointly optimizing on both sets, we steer the model away from regurgitating the sensitive information from the retain set, while ensuring it does not lose performance in the retain set. despite being a promising alternative to gradient ascent, this quality of model performance on non-sensitive dataset depends on the size of the retain set used in model training, and can lead to poor generalization on new examples. the loss term jointly increases the likelihood of generating responses in the retain set r while reducing the likelihood of generating f , as shown below. negative preference optimization @xcite : this baseline uses a modified version of the direct preference optimization objective, adapted to remove the sensitive information from forget set. ## participant systems we received over 100 submissions from 24 teams with nearly 70 individuals spanning over 30 institutions across the world. we list key ideas explored by participants in most teams used variations of gradient difference (gd), kl regularization or negative preference optimization (npo) with specific hyperparameters coupled with clever optimizations leading to faster training within the fixed compute time. other teams explored new and innovative solutions for unlearning by leveraging novel loss objectives, selective layer/parameter training, etc. the best performing team, ails-ntua, leveraged a parameter-efficient unlearning method based on gd with lora adapters added to transformer projection layers. they carefully sampled chunks of forget set mixed with a large (resampled) retain set. the second place team, zjuk-lab merged two different models unlearned with distinct hyperparameters, to balance under/overunlearning in the two models. the third place team, ynu used alternating gd with randomly sampled forget labels. team mr. snuffleupagus applied targeted unlearning using rmu on 3 layers selected using the validation set. ishumei-chinchunmei explored a new inverted loss function for the forget set, which avoids the gradient explosion commonly found in ga. ## conclusion and future improvements this paper summarizes semeval-2025 task 4 on unlearning sensitive content from llms. our task presents a significant challenge, as most baselines struggle to maintain model utility while unlearning unwanted information. we received several innovative solutions which made strong contributions towards solving this task. we hope our challenge and the associated benchmark inspire further research into efficient methods for unlearning sensitive content from llms. we note several avenues for future exploration: 1. evaluation metrics. outside llms, unlearning literature typically uses some form of statistical hypothesis testing between the model posteriors from the unlearned and the retrained (i.e. trained without the sensitive information) model candidates. however, this is not always feasible for llms since the model would have to be trained from ground up, including pretraining which is a computationally expensive undertaking. 2. larger model checkpoints: we limited our challenge to 7 and 1 billion parameter models due to limited compute availability with most participants. in future work we may expand on this challenge by inviting a subset of teams to onboard to specialized compute platforms to motivate further research on unlearning larger models. 3. unlearning other attributes: unlearning of sensitive information or a class of model capabilities (such as coding in a specific language).