Remember What You Want to Forget: Algorithms for Machine Unlearning

1 Introduction . Many organizations and companies employ user data to train machine learning models for a wide array of applications , ranging from movie recommendations to health care . While some of these organizations allow users to withdraw their consent from their data being used ( at which point the organization will delete the user ’ s data ) , less savory businesses might covertly retain user data . Given the potential for misuse , legislators worldwide have wisely introduced laws that mandate user data deletion upon request . These include the European Union ’ s General Data Protection Regulation ( GDPR ) , the California Consumer Privacy Act ( CCPA ) , and Canada ’ s proposed Consumer Privacy Protection Act ( CPPA ) . There is some natural ambiguity present in these guidelines . Is it sufficient to simply delete the user ’ s data , or must one also take action on machine learning systems that used this data for training ? Indeed , by now , privacy researchers are well-aware that user data may be extracted from trained machine learning models ( e.g. , Shokri et al . [ 2017 ] , Carlini et al . [ 2019 ] ) . In a potentially landmark decision , the Federal Trade Commission recently ordered a company to delete not only data from users who deleted their accounts , but also models and algorithms derived from this data [ Federal Trade Commission , 2021 ] . This suggests that organizations have an obligation to retrain any machine learning models after excluding users whose data has been deleted . ⇤Authors in alphabetical order . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . However , naïvely retraining models after every deletion request would be prohibitively expensive : training modern machine learning models may take weeks , and use resources of value in the millions . One could instead imagine more careful methods , which attempt to excise the required datapoints from the model : crucially , without incurring the cost of retraining from scratch . This notion is called machine unlearning . The goal would be to obtain a model which is identical to the alternative model that would be obtained when trained on the dataset after removing the points that need to be forgotten . This requirement is rather strong : Ginart et al . [ 2019 ] proposed a relaxed notion of deletion , in which the model must only be close to the alternative , where closeness is defined in a way reminiscent of differential privacy [ Dwork et al. , 2006a , b ] ( our variant of this notion is described in Definition 2 ) . This relaxation has inspired the design of several efficient algorithms for data deletion from machine learning models [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] . As mentioned before , one naïve strategy involves retraining the model from scratch , sans the deleted datapoints . When the training dataset is large , this approach is undesirable for several reasons . First , it is computationally very expensive . Even iteration over the training data can be too costly , let alone training a new model on it . Second , preserving the entire training dataset consumes a significant amount of storage.2 Another straightforward approach involves model checkpointing , in which the learner preemptively stores backup models in which certain points have been excluded . While this strategy makes it easy to quickly return an appropriate backup model upon receiving a deletion request , the downside is that one typically has to store a number of additional models which scales with the training data size , which may be prohibitively large . As we can see from these examples , computational and storage complexity are two vital metrics when designing a machine unlearning algorithm . Finally , while there has recently been a wealth of results in machine unlearning , all of it has focused on the core problem of empirical risk minimization , where the goal is to minimize the training loss . However , to fulfil the promise of machine learning , we desire algorithms that can generalize to previously unseen test data . Motivated simultaneously by all of these concerns , our goal is to address the following question : How do we design resource-efficient machine unlearning algorithms which generalize ? Our contributions . We initiate a new line of inquiry in machine unlearning : • We investigate generalization properties of unlearning algorithms , in particular asking : how many samples can we unlearn while still ensuring good performance on unseen test data ? In comparison , prior work focused on the empirical training loss only . • We consider machine unlearning simultaneously under storage constraints as well as the previously studied computation constraints . Unlike prior work , our algorithms do not require the training data to be available to the unlearning algorithm when deleting samples . • A clean approach for unlearning is to ignore which particular samples are being unlearnt and directly apply known algorithms and guarantees from differential privacy ( DP ) . We show a strict separation between DP and machine unlearning . In particular , algorithms based on DP can delete at most e⇥ ( n/ p d ) samples while still retaining test loss performance , where d denotes the dimension of the problem . On the other hand , we provide efficient unlearning algorithms that take into account the particular samples to be unlearnt and show that we can delete up to eO ( n/d1/4 ) samples , thus giving a quadratic improvement in terms of dependence of d over DP . Our results apply to both strongly convex and convex loss functions . 1.1 Related work . Cao and Yang [ 2015 ] introduced the term “ machine unlearning , ” and gave efficient deterministic algorithms for exact unlearning in certain settings . This definition requires an algorithm to have identical outputs on a dataset after deleting a point , and if that point was never inserted . However , their 2Orthogonal to storage constraints , an additional issue is that government regulations may restrict the learner from storing raw user data for extended periods of time due to privacy concerns . However , we focus on storage as it captures undesirability of a wider range of unsatisfactory solutions . algorithms are restricted to very structured problems only . Bourtoule et al . [ 2021 ] provide unlearning algorithms using a sharding-based strategy , though in a weaker unlearning model ( requiring only that it be possible that the output may have arisen ) , and without error guarantees . Ginart et al . [ 2019 ] introduced the probabilistic notion of unlearning , inspired by differential privacy [ Dwork et al. , 2006b , a ] . Their definition requires the output distribution of the unlearning algorithm to be similar to the output distribution obtained by running the learning algorithm on the dataset without the deleted points . Several recent works [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] provide theoretical error guarantees for various problem settings under this probabilistic notion of unlearning . While our unlearning setup is closely related to that of Ginart et al . [ 2019 ] and in the related works , there are two major differences . First , the prior work focuses on empirical risk minimization [ Guo et al. , 2020 , Izzo et al. , 2021 , Neel et al. , 2021 , Ullah et al. , 2021 ] . In their setup , the goal of the unlearning algorithm is to find approximate minimizers of the empirical loss on the remaining training dataset after deleting samples . In comparison , our focus in this paper is on the test loss , and we wish to understand how many samples can be deleted from a learnt model while still ensuring that the updated model performs well on unseen examples ( i.e. , the generalization error ) . As we discuss in Section 3.1 , the goal of minimizing the training loss is qualitatively different from that of minimizing the test loss . Second , the prior work focuses exclusively on the computational cost of unlearning , without concern for associated storage requirements . This has led to approaches which involve memory-intensive checkpointing data structures , which enables fast processing of deletion requests , but consumes potentially impractical amounts of storage . In contrast , we are additionally concerned with memory usage , which highlights the drawbacks of such approaches . Unlike prior work , our algorithms do not require the training data to be available to the unlearning algorithm when deleting samples , and only rely on some cheap-to-store data statistics . The most closely related work to ours is the certified data removal framework of Guo et al . [ 2020 ] which provides efficient data deletion algorithms for generalized linear models ( linear and logistic regression ) . While our deletion algorithm is similar to the Newton update removal mechanism considered in their work , there are some important technical differences . First , their unlearning setup requires access to the entire training dataset for deleting samples ; we do not require this . Second , they provide theoretical guarantees in terms of the norm of the empirical gradient being small after data removal . In comparison , our guarantees are for the test loss . Third , their unlearning definition requires the learning algorithm to be randomized , and this leads to worse performance guarantees due to added noise . In comparison , we do not need to randomize the learning algorithm . Finally , our guarantees hold for arbitrary convex loss functions and are thus broader in scope . Several other models of unlearning have been considered . Garg et al . [ 2020 ] give an alternative perspective on machine unlearning , grounded in cryptography . Other works in this space focus on exploring privacy risks [ Chen et al. , 2020 ] and verification [ Sommer et al. , 2020 ] in machine unlearning settings . For specific learning models like SVMs , exact unlearning has been considered under the framework of decremental learning [ Cauwenberghs and Poggio , 2001 , Tveit et al. , 2003 , Karasuyama and Takeuchi , 2010 , Romero et al. , 2007 ] . However , the primary motivation in these works is to use the framework of decremental learning to estimate the leave-one-out error in order to provide generalization guarantees for the learnt model . Finally , there has also been recent empirical and theoretical work in developing definitions and algorithms for machine unlearning with deep neural networks for application domains in computer vision [ Du et al. , 2019 , Golatkar et al. , 2020a , b , Nguyen et al. , 2020 ] . 2 Preliminaries . Let D be a distribution over an instance space Z and W ✓ Rd be the parameter space of a hypothesis class . Let f : W ⇥ Z ! R be a loss function . The goal is to minimize the test loss population risk ( test loss ) , given by F ( w ) : = Ez⇠D [ f ( w , z ) ] , ( 1 ) where f ( w , z ) is the loss of the hypothesis corresponding to w 2 W on the instance z 2 Z . Let F ⇤ = minw2W F ( w ) be the value of this minimum and w⇤ be a corresponding minimizer . Since the distribution D is often unknown , we are restricted to rely on samples to find a small test loss model . Given S = ( z1 , z2 , . . . , zn ) , a set of n samples drawn independently from D , standard learning algorithms minimize the empirical loss given by bFn ( w ) : = 1 n nX i=1 f ( w , zi ) . ( 2 )

Summary: Machine unlearning is new and hot topic. This paper makes several interesting contributions. The idea that the unlearning operation can only access a summary of the entire dataset is a great conceptual innovation since many deployment environments are memory-constrained compared to the training environment. The main contributions of this paper lean theoretical. I did not closely check all the details in the math, but it seems correct at a high-level. I consider the theoretical contributions interesting and strong.