Rethinking Evaluation Methods for Machine Unlearning (2024)

Abstract: Machine *unlearning* refers to methods for deleting information about specific training instances from a trained machine learning model. This enables models to delete user information and comply with privacy regulations. While retraining the model from scratch on the training set excluding the instances to be “*forgotten*” would result in a desired unlearned model, owing to the size of datasets and models, it is infeasible. Hence, unlearning algorithms have been developed, where the goal is to obtain an unlearned model that behaves as closely as possible to the retrained model. Consequently, evaluating an unlearning method involves - (i) randomly selecting a *forget* set (i.e., the training instances to be unlearned), (ii) obtaining an unlearned and a retrained model, and (iii) comparing the performance of the unlearned and the retrained model on the test and forget set. However, when the forget set is randomly selected, the unlearned model is almost often similar to the original (i.e., prior to unlearning) model. Hence, it is unclear if the model did really unlearn or simply copied the weights from the original model. For a more robust evaluation, we instead propose to consider training instances with significant influence on the trained model. When such influential instances are considered in the forget set, we observe that the unlearned model deviates significantly from the retrained model. Such deviations are also observed when the size of the forget set is increased. Lastly, choice of dataset for evaluation could also lead to misleading interpretation of results.

## introduction datasets that are used to train natural language processing models are often very large and may contain sensitive information, which raises concerns regarding user privacy @xcite . to address these concerns, the "right to be forgotten" has been introduced and realized through laws such as the general data protection regulation and the california consumer privacy act. this right provides users with the ability to request the removal of their personal data from a machine learning pipeline. while personal information can simply be deleted from a database, this is much more complicated for the training data in machine learning, as the final trained model still encapsulates the user's information, posing a potential risk for data leakage @xcite . to fully comply with the law, not only the data point but also its influence on the trained model needs to be removed. this is realized through machine unlearning techniques, which remove the influence of specific points in efficient ways without compromising model performance. note that a trivial solution to the problem would be to retrain the model from scratch. however, this would be infeasible for larger models, which involve long training times. nevertheless, the goal of the unlearning approaches is to achieve an unlearned model that behaves as closely as possible to a retrained model. while machine unlearning research has gained traction in recent years, the focus has been mainly on computer vision tasks @xcite @xcite . for models involving textual data, only a few unlearning approaches @xcite exist. typically, an unlearning method is evaluated on its ability to replicate the performance of the retrained model on the test set and the forget set (i.e., the set of points whose influence is to be removed). however, the forget set is usually small and randomly sampled from the training set. we argue that a small, randomly selected forget set lacks the necessary complexity to effectively evaluate the unlearning process and is not representative of the true performance of the unlearning method. this is illustrated in ## experiments in this section, we delve into the experimental setup and results of our analysis on machine unlearning evaluation. ## discussion in this section, we discuss the most important results from our experiments as well as their implications and our recommendations. ## conclusion in this paper, we introduce novel techniques in order to improve the robustness of machine unlearning evaluation for natural language processing. our approach of using influential forget data created via influence functions provides a challenging unlearning scenario for state-of-the-art machine unlearning concerning the forget accuracies. moreover, we show that increasing the size of the forget dataset also enables a more robust evaluation. our results further demonstrate that for evaluating these unlearning methods, one needs to carefully consider the evaluation dataset. otherwise, it might lead to incorrect interpretation of results. our results advocate for development of unlearning evaluation scenarios resembling real-world challenges.