Differentially Private SGD with Sparse Gradients

1 INTRODUCTION . The success of machine learning , and deep neural networks in particular , combined with ubiquitous edge computation and digital record keeping , has led to a surge in privacy sensitive learning applications . Internet-scale data promises to accelerate the development of data-driven statistical approaches , but the need for privacy constrains the amalgamation of such datasets . Private data are in fact isolated , constraining our ability to build models that learn from a large number of instances . On the other hand , the information contained in locally stored data can also be exposed through releasing the model trained on a local dataset ( Fredrikson et al. , 2015 ; Shokri et al. , 2017 ) , or even reconstructed when gradients generated during training are shared ( Zhu et al. , 2019 ; Geiping et al. , 2020 ; Zhu & Blaschko , 2021 ) . To address these issues , many applications of machine learning are expected to be privacypreserving . While differential privacy ( DP ) provides a rigorously defined and measurable privacy guarantee for database operations ( Dwork & Roth , 2014 ) , it also contains intriguing properties , such as robustness to post-processing and composability , which enables conveniently computing an overall privacy guarantee for several DP components . Differential privacy1 defines privacy with respect to the difficulty of distinguishing the outputs . For a pair of neighboring databases X , X ′ ∈ X , i.e . X can be obtained from X ′ by adding or removing an element . Definition 1 . A randomized mechanism M : X → R is ( ε , δ ) -differentially private , if for any subset of outputs S ⊆ R it holds that : Pr [ M ( X ) ∈ S ] ≤ eε Pr [ M ( X ′ ) ∈ S ] + δ . A common paradigm for a randomized mechanismM in deep learning is perturbed gradient descent : M ( X ) : = f ( X ) +N ( 0 , S2fσ2I ) , ( 1 ) 1In this work we only consider approximate differential privacy which includes the δ term . where f : X → R computes an aggregated gradient given a database X or X ′ . The isotropic Gaussian distributed noise ξDP ∼ N ( 0 , S2fσ2I ) is calibrated to f ’ s sensitivity S2f , which is the maximal ` 2 distance ‖f ( X ) − f ( X ′ ) ‖ , i.e . the maximal ` 2 norm of gradient among all individual examples . X , X ′ could be batches of training data , for instance , in our experiments they are batches of image-label pairs . The factor σ is a noise multiplier controlling the strength of the privacy guarantee : higher σ leads to lower privacy loss . Differentially private stochastic gradient descent ( DP-SGD ) upper bounds the certainty of connecting data with arbitrary subset of gradient space using the privacy budget variables ( ε , δ ) . Bassily et al . ( 2014 ) show that in a convex setting , DP-SGD achieves excess risk of Õ ( √ d/nε ) for a modelw ∈ Rd that minimizes the empirical risk ∑n i=1 ` ( w , xi ) , where x1 , x2 , ... , xn are drawn from X . While we show that in non-convex general setting , the mean square error ( MSE ) of perturbed gradient g̃ = g + ξDP is between Ω ( d ) and Ω ( d2 ) by assuming the gradients follow a Gaussian distribution : Theorem 1 . Assuming that the gradient is drawn from N ( ∇w , Σ ) , centered at the true gradient ∇w and with respect to the covariance matrix Σ whose trace goes linearly up with dimension d. The MSE of perturbed gradient g̃ = g + ξDP can be lower bounded by : MSE ≥ Tr [ Σ ] ( 1 + dσ2 ) . ( 2 ) from which we conclude that the lower bound on MSE is between linear and quadratic in d in practice . For the proof and conclusion , refer to Appendix A . In terms of deep learning , as d is a large number for modern network architectures , this can lead to a significant increase in error . The work of Abadi et al . ( 2016 ) proposed to clip the gradient of each individual example in ` 2 norm to a preset bound C , i.e . ḡ = g ·min ( 1 , C‖g‖2 ) . They then apply this clipping bound to compute the variance of Gaussian distributed noise . The Gaussian noise mechanism can be expressed as : M ( D ) : = f ( D ) +N ( 0 , C2 · σ2Id ) . ( 3 ) DP-SGD with gradient clipping has been empirically verified to be effective as it constraints the amount of injected noise by setting a small clipping bound . However , clipping removes the magnitude information of the gradient and therefore results in gradient estimation bias . Setting a small clipping bound with a deeper network will not result in better performance . The expected MSE of the perturbed gradient is only constrained to O ( d ) , the impact of perturbation is non-negligible in practice . The biggest network where this strategy is successfully applied so far is a CNN with Tanh proposed by Papernot et al . ( 2021 ) , which reaches ∼ 66 % accuracy on CIFAR10 in a low privacy regime and is regarded as the state-of-the-art ( SOTA ) end-to-end network with DP-SGD . To address this curse of dimensionality , most recent works concentrate on gradient dimension reduction . 1.1 RELATED WORKS . Abadi et al . ( 2016 ) propose to pretrain a network on an auxiliary dataset and then transfer the feature extraction , so that only a linear classifier will be replaced and trained on the private data . Tramer & Boneh ( 2021 ) adopt ScatterNet ( Oyallon et al. , 2019 ) to extract handcrafted features and train a relatively shallow network based on the features . Both work decrease d by excluding the majority of parameters during DP learning , which also constrains the learning ability of network . Inspired by the empirical observation that the optimization trajectory is contained in lowerdimensional subspace ( Vogels et al. , 2019 ; Gooneratne et al. , 2020 ; Li et al. , 2020 ) , a line of work intend to reduce d by exploiting the low-rank property of the gradient while considering privacy . Several recent works ( Zhou et al. , 2021 ; Yu et al. , 2021a ; Kairouz et al. , 2021 ) project the gradient into a subspace which is identified by auxiliary data or released historical gradients . In practice , they use the power method to search for the subspace . The computational cost of running the power method and projecting gradients as well as the memory footprint of storing the projection matrix limits the application of such method to large models . Zhang et al . ( 2021 ) target an NLP task where networks are heavily over-parameterized and gradients are extremely sparse , and propose to adopt DP selection to privately select top-k significant gradients for optimization . Also targeting an NLP task , Yu et al . ( 2021b ) propose a low-rank reparameterization of weights via released historical gradients . In addition to the aforementioned works , McMahan et al . ( 2017 ) ; Yang et al . ( 2019 ) and others study how to incorporate differential privacy in collaborative training , e.g . federated learning , in the interest of protecting the privacy of participants . Federated learning also suffers from large d , as it is usually deployed on edge devices and local models are periodically synchronized , so communication cost becomes expensive both in time and power usage ( Pathak et al. , 2012 ) . Therefore , gradient dimension reduction can have large benefits in federated learning involving power-restricted edge devices . A line of work studies how to tackle this issue by utilizing the low-rank property ( Shokri & Shmatikov , 2015 ; Yang et al. , 2019 ; Liu et al. , 2020 ) . 1.2 OUR CONTRIBUTION . In this work , we demonstrate an axis-aligned gradient dimension reduction method . Our work is orthogonal to previous works , we do not extract any characteristic information of the gradient or model , or approximate the gradient from a subspace . Instead , we randomly zero-out a fraction of the gradient during training and force the gradient to have a sparse representation . We provide a theoretical study on this strategy and reveal that random freeze exhibits a trade-off between signal loss and perturbation moderation in DP-SGD . We remark that our theory and approach do not necessarily rely on the low-rank assumption . To the best of our knowledge , we are the first to study this trade-off in DP-SGD and provide an effective approach . We use the benchmark CIFAR10 ( Krizhevsky , 2012 ) which is to date standard in benchmarking DP learning and show that well implemented random freeze exhibits various advantages and can be widely applied . More specifically , we maintain accuracy when we adapt projected DP-SGD with random freeze , while we reduce the computational cost and memory footprint induced by the power method and projection . Applying it to various frameworks , we achieve a high representation sparsity of gradient without a loss in performance . Federated learning can take advantage of the resulting sparse representation to reduce communication costs . We further note that the random freeze strategy improves the accuracy of large networks , which we demonstrate with the SOTA End-to-end CNN proposed by Papernot et al . ( 2021 ) . 2 ANALYSIS OF RANDOM FREEZE . In this section we theoretically prove that to a certain freeze rate r , DP-SGD with random freeze will converge if the approach without random freeze can converge . Then we investigate the tradeoff between signal loss and perturbation moderation induced by random freeze . Furthermore , we empirically demonstrate benefits of applying random freeze from the perspective of the gradient distribution . 2.1 CONVERGENCE RATE OF DP-SGD WITH RANDOM FREEZE . Let L be the objective function L ( w ) : = Ex∼X [ ` ( w , x ) ] , m ∈ { 0 , 1 } d the freeze mask and r the freeze rate , we randomly draw rp indices and set these positions in the mask to 0 and others to 1 so that ∑ m = ( 1 − r ) d. We assume an oracle telling us the true gradient ∇w and individual gradient gt , i = ∇w + ξt ( xi ) , where ξt is independent gradient deviation with zero mean . Let pt be the distribution of ξt . For random freeze the sparse gradient is g′t , i = ∇w′t + ξ′t ( xi ) , where ∇w′t : = m ∇wt , ξ′t ( xi ) : = m ξt ( xi ) . Denote for DP-SGD the averaged clipped gradient of B samples ḡt : = 1B ∑ i gt , i ·min ( 1 , C ‖gt , i‖ ) , and for DP-SGD with random freeze ĝt : = 1 B ∑ i g ′ t , i · min ( 1 , C‖g′t , i‖ ) . We have for DP-SGD with random freeze the following inequality : Theorem 2 . Assume G-Lipschitz smoothness of∇w such that ||∇wt+1−∇wt|| ≤ G||wt+1−wt|| . Consider an algorithm with clipping bound C , learning rate γ and choose a symmetric probability density distribution p̃ ( · ) satisfying p̃t ( ξt ) = p̃t ( −ξt ) , ∀ξt ∈ Rd . Then ∃κ ≥ 1− r such that : 1 T T∑ t=1 Pξt∼p̃t ( ‖ξt‖ < C 4 ) h ( ∇wt ) ‖∇wt‖ ≤ 1 κ ( ∆L γT +γ∆C + ( 1−r ) γ∆DP − 1 T T∑ t=1 Em [ bt ] ) , ( 4 ) where we define bt : = ∫ 〈∇w′t , g′t , i · min ( 1 , C‖g′t , i‖ ) 〉 ( p ′ t ( ξ ′ t ) − p̃′t ( ξ′t ) ) dξ′t , p′t , p̃′t are corresponding projected distribution , and define ∆L : = E [ L1 − minw L ( w ) ] , ∆C : = GC 2 2 , ∆DP : = C2σ2dG 2B2 , h ( ∇wt ) : = min ( ‖∇wt‖ , 3C4 ) . The proof can be found in Appendix B . When no freeze r = 0 , i.e . m is a vector filled with value 1 . Note that κ = 1 for r = 0 , we can obtain the inequality 4 in the following form : 1 T T∑ t=1 Pξt∼p̃t ( ‖ξt‖ < C 4 ) h ( ∇wt ) ‖∇wt‖ ≤ ∆L γT + γ ( ∆C + ∆DP ) − 1 T T∑ t=1 bt , ( 5 ) which describes DP-SGD without random freeze . Chen et al . ( 2020b ) argue that by tweaking p̃t it is possible to boundPξt∼p̃t away from zero which means the l.h.s . is proportional to ‖∇wt‖ or ‖∇wt‖2 at each iteration , while letting the convergence bias term−bt tend to be small as pt is approximately symmetric . They then prove that by setting a certain learning rate γ , the r.h.s . diminishes to zero , so the network under DP-SGD can converge . Adapted from that , we see at the r.h.s . of inequality 4 for random freeze , −Em [ bt ] also tends to be small as p′t is expected to be approximately symmetric if pt is . Additionally , 1/κ will not be large as along as r is not extremely close to 1 . So with the same learning rate and γ adapted for DP-SGD without random freeze , the r.h.s . of inequality 4 will also tend to zero , which proves the convergence of DP-SGD with random freeze .

Summary: This paper considers the problem of differentially privately learning deep neural networks. In order to improve the accuracy and reduce the communication cost, this paper proposes to randomly freeze a progressively increasing subset of parameters, which results in sparse gradient updates. Empirical results also show that the new algorithm can largely reduce the communication cost, while maintaining the performance. Furthermore, the extra computation cost is negligible.