Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification

1 INTRODUCTION . Many fundamental machine learning tasks involve solving empirical risk minimization ( ERM ) : given a loss function ` , find a model w ∈ Rp that minimizes the empirical risk L̂n ( w ) = 1 n ∑n i=1 ` ( w , zi ) , where z1 , . . . , zn are i.i.d . examples drawn from a distribution P . In many applications , the training data may contain highly sensitive information about some individuals . When the models are given by deep neural networks , their rich representation can potentially reveal fine details of the private data . Differential privacy ( DP ) ( Dwork et al. , 2006 ) has by now become the standard approach to provide principled and rigorous privacy guarantees in machine learning . Roughly speaking , DP is a stability notion that requires that no individual example has a significant influence on the trained model . One of the most commonly used algorithm for solving private ERM is the differentially-private stochastic gradient descent ( DP-SGD ) ( Abadi et al. , 2016 ; Bassily et al. , 2014 ; Song et al. , 2013 ) –a private variant of SGD that perturbs each gradient update with random noise vector drawn from an isotropic Gaussian distribution N ( 0 , σ2Ip ) , with appropriately chosen variance σ2 . Due to the gradient perturbation drawn from an isotropic Gaussian distribution , the error rate of DP-SGD has a dependence on the ambient dimension p—the number of parameters in the model . In the case of convex loss ` , Bassily et al . ( 2014 ) show that DP-SGD achieves the optimal empirical excess risk of Õ ( √ p/ ( n ) ) . For non-convex loss ` , which is more common in neural network training , minimizing L̂n ( w ) is in general intractable . However , many ( non-private ) gradient-based optimization methods are shown to be effective in practice and can provably find approximate stationary points with vanishing gradient norm ‖∇L̂n ( w ) ‖2 ( see e.g . Nesterov ( 2014 ) ; Ghadimi and Lan ( 2013 ) ) . Moreover , for a wide family of loss functions L̂n under the Polyak-Łojasiewicz condition ( Polyak , 1963 ) , the minimization of gradient norm implies achieving global optimum . With privacy constraint , Wang and Xu ( 2019 ) recently showed that DP-SGD minimize the empirical gradient norm down to Õ ( p1/4/ √ n ) when the loss function ` is smooth . Furthermore , exsiting lower bounds results on private ERM ( Bassily et al. , 2014 ) show that such dependence on p is inevitable in the worst case . However , many modern machine learning tasks now involve training extremely large models , with the number of parameters substantially larger than the number of training samples . For these large models , the error dependence on p can be a barrier to practical private ERM . In this paper , we aim to overcome such dependence on the ambient dimension p by leveraging the structure of the gradient space in the training of neural networks . We take inspiration from the empirical observation from Li et al . ( 2020 ) ; Gur-Ari et al . ( 2018 ) ; Papyan ( 2019 ) that even though the ambient dimension of the gradients is large , the set of sample gradients at most iterations along the optimization trajectory is often contained in a much lower-dimensional subspace . While this observation has been made mostly for non-private SGD algorithm , we also provide our empirical evaluation of this structure ( in terms of eigenvalues of the gradient second moments matrix ) in Figure 1 . Based on this observation , we provide a modular private ERM optimization framework with two components . At each iteration t , the algorithm performs the following two steps : 1 ) Gradient dimension reduction . Let gt be the mini-batch gradient at iteration t. In general , this subroutines solves the following problem : given any k < p , find a linear projection V̂k ( t ) ∈ Rp×k such that the reconstruction error ‖gt − V̂k ( t ) V̂k ( t ) ᵀgt‖ is small . To implement this subroutine , we follow a long line of work that studies private data analysis with access to an auxiliary public dataset Sh drawn from the same distributionP , for which we don ’ t need to provide formal privacy guarantee ( Bassily et al. , 2019b ; 2020 ; Feldman et al. , 2018 ; Avent et al. , 2017 ; Papernot et al. , 2017 ) . In our case , we compute V̂k ( t ) which is given by the top-k eigenspace of the gradients evaluated on Sh . Alternatively , this subroutine can potentially be implemented through private subspace identification on the private dataset . However , to our best knowledge , all existing methods have reconstruction error scaling with √ p ( Dwork et al. , 2014 ) , which will be propagated to the optimization error . 2 ) Projected DP-SGD ( PDP-SGD ) . Given the projection V̂k ( t ) , we perturb gradient in the projected subspace : g̃t = V̂k ( t ) V̂k ( t ) ᵀ ( gt+bt ) , where bt is a p-dimensional Gaussian vector . The projection mapping provides a large reduction of the noise and enables higher accuracy for PDP-SGD . Our results . We provide both theoretical analyses and empirical evaluations of PDP-SGD : Uniform convergence for projections . A key step in our theoretical analysis is to bound the reconstruction error on the gradients from projection of V̂k ( t ) . This reduces to bounding the deviation ‖V̂k ( t ) V̂k ( t ) ᵀ − VkVk ( t ) ᵀ‖2 , where Vk ( t ) denotes the top-k eigenspace of the population second moment matrix E [ ∇ ` ( wt , z ) ∇ ` ( wt , z ) ᵀ ] . To handle the adaptivity of the sequence of iterates , we provide a uniform deviation bound for all w ∈ W , where the set W contains all of the iterates . By leveraging generic chaining techniques , we provide a deviation bound that scales linearly with a complexity measure—the γ2 function due to Talagrand ( 2014 ) —of the set W . We provide lowcomplexity examples of W that are supported by empirical observations and show that their γ2 function only scales logarithmically with p. Convergence for convex and non-convex optimization . Building on the reconstruction error bound , we provide convergence and sample complexity results for our method PDP-SGD in two types of loss functions , including 1 ) smooth and non-convex , 2 ) Lipschitz convex . Under suitable assumptions on the gradient space , our rates only scales logarithmically on p. Empirical evaluation . We provide an empirical evaluation of PDP-SGD on two real datasets . In our experiments , we construct the “ public ” datasets by taking very small random sub-samples of these two datasets ( 100 samples ) . While these two public datasets are not sufficient for training an accurate predictor , we demonstrate that they provide useful gradient subspace projection and substantial accuracy improvement over DP-SGD . Related work . Beyond the aforementioned work , there has been recent work on private ERM that also leverages the low-dimensional structure of the problem . Jain and Thakurta ( 2014 ) ; Song et al . ( 2020 ) show dimension independent excess empirical risk bounds for convex generalized linear problems , when the input data matrix is low-rank . Kairouz et al . ( 2020 ) study unconstrained convex empirical risk minimization and provide a noisy AdaGrad method that achieves dimensionfree excess risk bound , provided that the gradients along the optimization trajectory lie in a lowdimensional subspace . In comparison , our work studies both convex and non-convex problems and our analysis applies to more general low-dimensional structures that can be characterized by small γ2 functions ( Talagrand , 2014 ; Gunasekar et al. , 2015 ) ( e.g. , low-rank gradients and fast decay in the magnitude of the gradient coordinates ) . Recently , Tramer and Boneh ( 2021 ) show that private learning with features learned on public data from a similar domain can significantly improve the utility . Zhang et al . ( 2021 ) leverage the sparsity of the gradients in deep nets to improve the dependence on dimension in the error rate . We also note a recent work ( Yu et al. , 2021 ) that proposes an algorithm similar to PDP-SGD . However , in addition to perturbing the projected gradient in the top eigenspaces in the public data , their algorithm also adds noise to the residual gradient . Their error rate scales with dimension p in general due to the noise added to the full space . To achieve a dimension independent error bound , their analyses require fresh public samples drawn from the same distribution at each step , which consequently requires a large public data set with size scaling linearly with T . In comparison , our analysis does not require fresh public samples at each iteration , and our experiments demonstrate that a small public data set of size no more than 150 suffices.1 2 PRELIMINARIES . Given a private dataset S = { z1 , ... , zn } drawn i.i.d . from the underlying distribution P , we want to solve the following empirical risk minimization ( ERM ) problem subject to differential privacy:2 minw L̂n ( w ) = 1 n ∑n i=1 ` ( w , zi ) . where the parameter w ∈ Rp . We optimize this objective with an iterative algorithm . At each step t , we write wt as the algorithm ’ s iterate and use gt to denote the mini-batch gradient , and ∇L̂n ( wt ) = 1n ∑n i=1∇ ` ( wt , zi ) to denote the empirical gradient . In addition to the private dataset , the algorithm can also freely access to a small public dataset Sh = { z̃1 , . . . , z̃m } drawn from the same distribution P , without any privacy constraint . Notation . We write Mt ∈ Rp×p to denote the second moment matrix of gradients evaluated on public dataset Sh , i.e. , Mt = 1m ∑m i=1∇ ` ( wt , z̃i ) ∇ ` ( wt , z̃i ) ᵀ and write Σt ∈ Rp×p to denote the population second moment matrix , i.e. , Σt = Ez∼P [ ∇ ` ( wt , z ) ∇ ` ( wt , z ) ᵀ ] . We use V ( t ) ∈ Rp×p as the full eigenspace of Σt . We use V̂k ( t ) ∈ Rp×k as the top-k eigenspace ofMt and Vk ( t ) ∈ Rp×k as the top-k eigenspace of Σt . To present our result in the subsequent sections , we introduce the eigen-gap notation αt , i.e. , let λ1 ( Σt ) ≥ ... ≥ λp ( Σt ) be the eigenvalue of Σt , we use αt to denote the eigen-gap between λk ( Σt ) and λk+1 ( Σt ) , i.e. , λk ( Σt ) −λk+1 ( Σt ) ≥ αt . We also define W ∈ Rp as the set that contains all the possible iterates wt ∈ W for t ∈ [ T ] . Throughout , for any matrix A and vector v , ‖A‖2 denotes spectral norm and ‖v‖2 denotes ` 2 norm . Definition 1 ( Differential Privacy ( Dwork et al. , 2006 ) ) A randomized algorithm R is ( , δ ) differentially private if for any pair of datasetsD , D′ differ in exactly one data point and for all event 1Note that the requirement of a large public data set may remove the need of using the private data in the first place , since training with the large public data set may already provide an accurate model . 2In this paper , we focus on minimizing the empirical risk . However , by relying on the generalization guarantee of ( , δ ) -differential privacy , one can also derive a population risk bound that matches the empirical risk bound up to a term of order O ( + δ ) ( Dwork et al. , 2015 ; Bassily et al. , 2016 ; Jung et al. , 2020 ) . Y ⊆ Range ( R ) in the output range of R , we have P { R ( D ) ∈ Y } ≤ exp ( ) P { R ( D′ ) ∈ Y } + δ , where the probability is taken over the randomness ofR . To establish the privacy guarantee of our algorithm , we will combine three standard tools in differential privacy , including 1 ) the Gaussian mechanism ( Dwork et al. , 2006 ) that releases an aggregate statistic ( e.g. , the empirical average gradient ) by Gaussian perturbation , 2 ) privacy amplification via subsampling ( Kasiviswanathan et al. , 2008 ) that reduces the privacy parameters and δ by running the private computation on a random subsample , and 3 ) advanced composition theorem ( Dwork et al. , 2010 ) that tracks the cumulative privacy loss over the course of the algorithm . We analyze our method under two asumptions on the gradients of ` . Assumption 1 For any w ∈ Rp and example z , ‖∇ ` ( w , z ) ‖2 ≤ G. Assumption 2 For any example z , the gradient ∇ ` ( w , z ) is ρ-Lipschitz with respect to a suitable pseudo-metric d : Rp × Rp 7→ R , i.e. , ‖∇ ` ( w , z ) −∇ ` ( w′ , z ) ‖2 ≤ ρd ( w , w′ ) , ∀w , w′ ∈ Rp . Note that Assumption 1 implies that L̂n ( w ) is G-Lipschitz and Assumption 2 implies that L̂n ( w ) is ρ-smooth when d is the ` 2-distance . We will discuss additional assumptions regarding the structure of the stochastic gradients and the error rate for different type of functions in Section 3 .

Summary: The paper considers the problem of solving differentially private empirical risk minimization. To reduce the dependence on dimensionality $p$, they propose Projected DP-SGD (PDP-SGD) that projects the noisy gradients to a low-dimensional subspace computed from a free public dataset at each iteration. They prove that PDP-SGD is differentially private and has only a logarithmic dependence on $p$.