Remember What You Want to Forget: Algorithms for Machine Unlearning

1 Introduction . Many organizations and companies employ user data to train machine learning models for a wide array of applications , ranging from movie recommendations to health care . While some of these organizations allow users to withdraw their consent from their data being used ( at which point the organization will delete the user ’ s data ) , less savory businesses might covertly retain user data . Given the potential for misuse , legislators worldwide have wisely introduced laws that mandate user data deletion upon request . These include the European Union ’ s General Data Protection Regulation ( GDPR ) , the California Consumer Privacy Act ( CCPA ) , and Canada ’ s proposed Consumer Privacy Protection Act ( CPPA ) . There is some natural ambiguity present in these guidelines . Is it sufficient to simply delete the user ’ s data , or must one also take action on machine learning systems that used this data for training ? Indeed , by now , privacy researchers are well-aware that user data may be extracted from trained machine learning models ( e.g. , Shokri et al . [ 2017 ] , Carlini et al . [ 2019 ] ) . In a potentially landmark decision , the Federal Trade Commission recently ordered a company to delete not only data from users who deleted their accounts , but also models and algorithms derived from this data [ Federal Trade Commission , 2021 ] . This suggests that organizations have an obligation to retrain any machine learning models after excluding users whose data has been deleted . ⇤Authors in alphabetical order . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . However , naïvely retraining models after every deletion request would be prohibitively expensive : training modern machine learning models may take weeks , and use resources of value in the millions . One could instead imagine more careful methods , which attempt to excise the required datapoints from the model : crucially , without incurring the cost of retraining from scratch . This notion is called machine unlearning . The goal would be to obtain a model which is identical to the alternative model that would be obtained when trained on the dataset after removing the points that need to be forgotten . This requirement is rather strong : Ginart et al . [ 2019 ] proposed a relaxed notion of deletion , in which the model must only be close to the alternative , where closeness is defined in a way reminiscent of differential privacy [ Dwork et al. , 2006a , b ] ( our variant of this notion is described in Definition 2 ) . This relaxation has inspired the design of several efficient algorithms for data deletion from machine learning models [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] . As mentioned before , one naïve strategy involves retraining the model from scratch , sans the deleted datapoints . When the training dataset is large , this approach is undesirable for several reasons . First , it is computationally very expensive . Even iteration over the training data can be too costly , let alone training a new model on it . Second , preserving the entire training dataset consumes a significant amount of storage.2 Another straightforward approach involves model checkpointing , in which the learner preemptively stores backup models in which certain points have been excluded . While this strategy makes it easy to quickly return an appropriate backup model upon receiving a deletion request , the downside is that one typically has to store a number of additional models which scales with the training data size , which may be prohibitively large . As we can see from these examples , computational and storage complexity are two vital metrics when designing a machine unlearning algorithm . Finally , while there has recently been a wealth of results in machine unlearning , all of it has focused on the core problem of empirical risk minimization , where the goal is to minimize the training loss . However , to fulfil the promise of machine learning , we desire algorithms that can generalize to previously unseen test data . Motivated simultaneously by all of these concerns , our goal is to address the following question : How do we design resource-efficient machine unlearning algorithms which generalize ? Our contributions . We initiate a new line of inquiry in machine unlearning : • We investigate generalization properties of unlearning algorithms , in particular asking : how many samples can we unlearn while still ensuring good performance on unseen test data ? In comparison , prior work focused on the empirical training loss only . • We consider machine unlearning simultaneously under storage constraints as well as the previously studied computation constraints . Unlike prior work , our algorithms do not require the training data to be available to the unlearning algorithm when deleting samples . • A clean approach for unlearning is to ignore which particular samples are being unlearnt and directly apply known algorithms and guarantees from differential privacy ( DP ) . We show a strict separation between DP and machine unlearning . In particular , algorithms based on DP can delete at most e⇥ ( n/ p d ) samples while still retaining test loss performance , where d denotes the dimension of the problem . On the other hand , we provide efficient unlearning algorithms that take into account the particular samples to be unlearnt and show that we can delete up to eO ( n/d1/4 ) samples , thus giving a quadratic improvement in terms of dependence of d over DP . Our results apply to both strongly convex and convex loss functions . 1.1 Related work . Cao and Yang [ 2015 ] introduced the term “ machine unlearning , ” and gave efficient deterministic algorithms for exact unlearning in certain settings . This definition requires an algorithm to have identical outputs on a dataset after deleting a point , and if that point was never inserted . However , their 2Orthogonal to storage constraints , an additional issue is that government regulations may restrict the learner from storing raw user data for extended periods of time due to privacy concerns . However , we focus on storage as it captures undesirability of a wider range of unsatisfactory solutions . algorithms are restricted to very structured problems only . Bourtoule et al . [ 2021 ] provide unlearning algorithms using a sharding-based strategy , though in a weaker unlearning model ( requiring only that it be possible that the output may have arisen ) , and without error guarantees . Ginart et al . [ 2019 ] introduced the probabilistic notion of unlearning , inspired by differential privacy [ Dwork et al. , 2006b , a ] . Their definition requires the output distribution of the unlearning algorithm to be similar to the output distribution obtained by running the learning algorithm on the dataset without the deleted points . Several recent works [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] provide theoretical error guarantees for various problem settings under this probabilistic notion of unlearning . While our unlearning setup is closely related to that of Ginart et al . [ 2019 ] and in the related works , there are two major differences . First , the prior work focuses on empirical risk minimization [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] . In their setup , the goal of the unlearning algorithm is to find approximate minimizers of the empirical loss on the remaining training dataset after deleting samples . In comparison , our focus in this paper is on the test loss , and we wish to understand how many samples can be deleted from a learnt model while still ensuring that the updated model performs well on unseen examples ( i.e. , the generalization error ) . As we discuss in Section 3.1 , the goal of minimizing the training loss is qualitatively different from that of minimizing the test loss . Second , the prior work focuses exclusively on the computational cost of unlearning , without concern for associated storage requirements . This has led to approaches which involve memory-intensive checkpointing data structures , which enables fast processing of deletion requests , but consumes potentially impractical amounts of storage . In contrast , we are additionally concerned with memory usage , which highlights the drawbacks of such approaches . Unlike prior work , our algorithms do not require the training data to be available to the unlearning algorithm when deleting samples , and only rely on some cheap-to-store data statistics . The most closely related work to ours is the certified data removal framework of Guo et al . [ 2020 ] which provides efficient data deletion algorithms for generalized linear models ( linear and logistic regression ) . While our deletion algorithm is similar to the Newton update removal mechanism considered in their work , there are some important technical differences . First , their unlearning setup requires access to the entire training dataset for deleting samples ; we do not require this . Second , they provide theoretical guarantees in terms of the norm of the empirical gradient being small after data removal . In comparison , our guarantees are for the test loss . Third , their unlearning definition requires the learning algorithm to be randomized , and this leads to worse performance guarantees due to added noise . In comparison , we do not need to randomize the learning algorithm . Finally , our guarantees hold for arbitrary convex loss functions and are thus broader in scope . Several other models of unlearning have been considered . Garg et al . [ 2020 ] give an alternative perspective on machine unlearning , grounded in cryptography . Other works in this space focus on exploring privacy risks [ Chen et al. , 2020 ] and verification [ Sommer et al. , 2020 ] in machine unlearning settings . For specific learning models like SVMs , exact unlearning has been considered under the framework of decremental learning [ Cauwenberghs and Poggio , 2001 , Tveit et al. , 2003 , Karasuyama and Takeuchi , 2010 , Romero et al. , 2007 ] . However , the primary motivation in these works is to use the framework of decremental learning to estimate the leave-one-out error in order to provide generalization guarantees for the learnt model . Finally , there has also been recent empirical and theoretical work in developing definitions and algorithms for machine unlearning with deep neural networks for application domains in computer vision [ Du et al. , 2019 , Golatkar et al. , 2020a , b , Nguyen et al. , 2020 ] . 2 Preliminaries . Let D be a distribution over an instance space Z and W ✓ Rd be the parameter space of a hypothesis class . Let f : W ⇥ Z ! R be a loss function . The goal is to minimize the test loss population risk ( test loss ) , given by F ( w ) : = Ez⇠D [ f ( w , z ) ] , ( 1 ) where f ( w , z ) is the loss of the hypothesis corresponding to w 2 W on the instance z 2 Z . Let F ⇤ = minw2W F ( w ) be the value of this minimum and w⇤ be a corresponding minimizer . Since the distribution D is often unknown , we are restricted to rely on samples to find a small test loss model . Given S = ( z1 , z2 , . . . , zn ) , a set of n samples drawn independently from D , standard learning algorithms minimize the empirical loss given by bFn ( w ) : = 1 n nX i=1 f ( w , zi ) . ( 2 )

Summary: The paper considers the problem of machine unlearning: after a model is trained on a dataset, there is a request to delete a point from the dataset. The goal is to design an *efficient* method to update the trained model such that is nearly indistinguishable from what we would have obtained had we trained on the dataset without the point. The authors use differential privacy as the unlearning criteria and look at unlearning in convex learning problems. The give a method which can *unlearn more points* that DP-training with small runtime and storage complexity.