Privacy-preserving Learning via Deep Net Pruning

Neural network pruning has demonstrated its success in significantly improving the computational e ciency of deep models while only introducing a small reduction on final accuracy . In this paper , we explore an extra bonus of neural network pruning in terms of enhancing privacy . Specifically , we show a novel connection between magnitude-based pruning and adding di↵erentially private noise to intermediate layers under the overparameterized regime . To the best of our knowledge , this is the first work that bridges pruning with the theory of di↵erential privacy . The paper also presents experimental results by running the model inversion attack on two benchmark datasets , which supports the theoretical finding . 1 Introduction . Data privacy has become one of the top concerns in the application of deep neural networks , since there has been an increasing demand to train deep models on private data sets . For example , hospitals are now training their automated diagnosis systems on private patients ’ data Litjens et al . ( 2016 ) ; Lakhani & Sundaram ( 2017 ) ; De Fauw et al . ( 2018 ) ; and advertisement providers are collecting users ’ online trajectories to optimize their learning-based recommendation algorithm Covington et al . ( 2016 ) ; Ying et al . ( 2018 ) . These private data , however , are usually subject to the regulations such as California Consumer Privacy Act ( CCPA ) Legislature ( 2018 ) , Health Insurance Portability and Accountability Act ( HIPAA ) Act ( 1996 ) , and General Data Protection Regulation ( GDPR ) of European Union . Di↵erential privacy ( DP ) Dwork et al . ( 2006b ) ; Dwork ( 2009 ) ; Dwork & Roth ( 2014 ) has emerged , during the past few years , as a strong standard to provide theoretical privacy guarantees for algorithms on aggregate databases . The core idea of achieving di↵erential privacy is to add controlled noise to the output of a deterministic function , such that the output can not be used to infer much about any single individual in the database . Recent years have seen an increasing number of applications that adapt di↵erential privacy mechanisms to address privacy concerns in deep learning Shokri & Shmatikov ( 2015 ) ; Abadi et al . ( 2016 ) ; Phan et al . ( 2016 ) ; McMahan et al . ( 2018 ) . Neural network pruning ( or pruning in short ) , a seemingly orthogonal field to privacy , has also been the subject of a great amount of work in recent years . Pruning aims to reduce the number of model parameters , such that the compressed model can be applied even under the memory constraints of the edge-devices . Various pruning techniques have succeeded in significantly compressing models with little or no loss of accuracy Han et al . ( 2015 ; 2016a ) ; Li et al . ( 2016 ) ; Ding et al . ( 2018 ) ; Evci et al . ( 2019 ) ; Tanaka et al . ( 2020 ) . However , the majority of existing literature only demonstrate the benefits of pruning in terms of energy saving and inference speedup , while in this work , we investigate another interesting bonus of pruning – preserving data privacy . Our investigation is mainly inspired by the observation that neural network pruning makes the inversion from hidden-layers harder , as the percentage of remained weight decreases ( see Figure 1 ) . Motivated by this empirical observation , we build under the over-paramterized regime of deep learning theory , and show an interesting connection between neural network pruning and adding di↵erentially private noise to intermediate layers . We believe this con- nection may have important practical implications since the pruned model only incurs small accuracy loss , and we leave that as future work . We list our contributions as follow : • We explore the benefits of pruned neural networks in terms of preserving data privacy . To the best of our knowledge , this is the first step towards drawing a theoretical connection between neural network pruning and di↵erential privacy . • To build the connection between pruning and adding di↵erentially private noise to intermediate layers , we generalize the famous anti-concentration inequality due to Carbery and Wright Carbery & Wright ( 2001 ) . This generalization might find more applications in the theoretical analysis of neural network pruning in the future . • We provide empirical results in support of our theoretical finding . Specifically , we demonstrate on two benchmark datasets that pruned neural networks are more secure in the sense that running the model inversion attack becomes harder . Roadmap . The rest of this paper is organized as follow . Section 2 covers existing literature in di↵erent privacy , neural network pruning , and over-parameterized deep learning theory . Section 3 provides theoretical preliminaries and Section 4 presents our main theoretical result . Section 5 shows empirical results on MNIST and CIFAR-10 benchmarks that are in support of our theoretical findings . We conclude this work in Section 6 . 2 Related work . Neural network pruning Traditional deep neural network models are computationally expensive and memory intensive , which hinders their deployment in applications with limited memory resources or strict latency requirements . Many progress has been made to perform model compression in deep networks , including low-rank factorization Sainath et al . ( 2013 ) ; Lebedev et al . ( 2015 ) , network pruning LeCun et al . ( 1990 ) ; Srinivas & Babu ( 2015 ) ; Han et al . ( 2016b ) ; Li et al . ( 2017 ) , and knowledge distillation Hinton et al . ( 2015 ) ; Chen et al . ( 2017 ) . Among them , neural network pruning has been widely adopted because it is able to reduce model sizes by up to one order of magnitude without significant accuracy loss . The idea of network pruning dates back to the Optimal Brain Damage in 1990s LeCun et al . ( 1990 ) . Recently , it has been shown that removing the weights with low magnitude can also achieve a highly compressed model Han et al . ( 2016b ) , which is referred to as ‘ magnitude-based pruning ’ . Di↵erential privacy The concept of ✏-di↵erential privacy was originally introduced by Dwork , McSherry , Nissim and Smith Dwork et al . ( 2006b ) . Later , it was generalized to a relaxation of ( ✏ , ) -di↵erential privacy Dwork et al . ( 2006a ) ; Dwork ( 2009 ) ; Dwork & Roth ( 2014 ) . Di↵erential privacy has been successfully applied to many problems . For more detailed surveys of the applications of di↵erential privacy , we refer the readers to Dwork ( 2008 ; 2011 ) . Applying di↵erential privacy techniques in deep learning is an interesting but non-trivial task . Previous research have customized di↵erential privacy for di↵erent learning tasks and settings Shokri & Shmatikov ( 2015 ) ; Abadi et al . ( 2016 ) ; Phan et al . ( 2016 ) . Although there are existing studies about applying di↵erential privacy in neural network , but there is little exploration on presenting di↵erential privacy using prune network yet . To the best of our knowledge , this paper is the first work that shows a connection between di↵erential privacy and pruned neural network . Over-parameterized deep learning theory Recently , there is super long line of work focusing showing the convergence of deep neural network training under overparameterization regime Li & Liang ( 2018 ) ; Du et al . ( 2019 ) ; Allen-Zhu et al . ( 2019b ; c ; a ) ; Arora et al . ( 2019a ; b ) ; Song & Yang ( 2019 ) ; Oymak & Soltanolkotabi ( 2020 ) ; Brand et al . ( 2020 ) . The theory suggested as long as the neural network is su ciently wide , i.e. , m poly ( n , d , 1/ , L ) then running ( stochastic ) gradient descent algorithm is able to find the global minimum , where n is the number of input data points , d is the dimension of data , is the minimum ` 2 distances between all pairs and L is the number of layers in neural network . However , unlike the above classical deep learning convergence theory , this work explored the over-parameterized theory in a very di↵erent perspective , e.g . privacy . Our result is not an optimization result which indicating neural network can learn a set of input data points in certain sense , however our result is suggesting neural network can be private in the di↵erential privacy sense . 3 Backgrounds . Notations . For a positive integer n , we use [ n ] to denote set { 1 , 2 , · · · , n } . For vector x 2 Rn , we use kxk1 to denote Pn i=1 |xi| , kxk2 to denote ( Pn i=1 x 2 i ) 1/2 , kxk1 to denote maxi2 [ n ] |xi| . We use N ( µ , 2 ) to denote random Gaussian distribution . For a matrix A , we use kAk to denote its spectral norm . This section presents some backgrounds before theoretically establishing the equivalence between magnitude-based pruning and adding di↵erentially private noise in Section 4 . Section 3.1 revisits the notion of ( ✏dp , dp ) -di↵erential privacy . Section 3.2 describes the magnitude pruning algorithm . 3.1 Differential privacy . The classical definition of di↵erential privacy is shown as follow : Definition 3.1 ( ( ✏dp , dp ) -di↵erential privacy Dwork et al . ( 2006a ) ) . For a randomized function h ( x ) , we say h ( x ) is ( ✏dp , dp ) -di↵erential privacy if for all S ✓ Range ( h ) and for all x , y with kx yk1 1 we have Pr h [ h ( x ) 2 S ] exp ( ✏dp ) · Pr h [ h ( y ) 2 S ] + dp . Definition 3.1 says that , if there are two otherwise identical records x and y , one with privacy-sensitive information in it , and one without it , and we normalize them such that kx yk1 1 . Di↵erential Privacy ensures that the probability that a statistical query will produce a given result is nearly the same whether it ’ s conducted on the first or second record . Parameters ( ✏dp , dp ) are called the privacy budget , and smaller ✏dp and dp provide a better di↵erential privacy protection . One can think of a setting where both parameters are 0 , then the chance of telling whether a query result is from x or from y is no better than a random guessing . A standard strategy to achieve di↵erential privacy is by adding noise to the the original data x or the function output h ( x ) . In order to analyze it , we need the following definition : Definition 3.2 ( Global Sensitivity Dwork et al . ( 2006b ) ) . Let f : Rn ! Rd , define GSp ( f ) , the ` p global sensitivity of f , for all x , y with kx yk1 1 as GSp ( f ) = sup x , y2Rn kf ( x ) f ( y ) kp . The global sensitivity of a function measures how ‘ sensitive ’ the function is to slight changes in input . The noise needed for di↵erential privacy guarantee is then calibrated using some well-known mechanisms , e.g. , Laplace or Gaussian Dwork & Roth ( 2014 ) , and the amount of noise ( the standard deviation of the noise distribution ) is proportional to the sensitivity , but inversely proportional to the privacy budget ✏dp . That is to say , for a given function with fixed global sensitivity , a larger amount of noise is required to guarantee a better di↵erential privacy ( one with a smaller budget ✏dp ) .

Summary: Overview: This paper aims to establish a theoretical connection between differential privacy and magnitude based pruning. The authors show theoretically that outputs of pruned single layer neural networks have some similarity to outputs of the same network with differential privacy added. The paper then empirically demonstrates that model inversion attacks are harder against magnitude pruned neural networks.