On Dynamic Noise Influence in Differential Private Learning

1 INTRODUCTION . In the era of big data , privacy protection in machine learning systems is becoming a crucial topic as increasing personal data involved in training models ( Dwork et al. , 2020 ) and the presence of malicious attackers ( Shokri et al. , 2017 ; Fredrikson et al. , 2015 ) . In response to the growing demand , differential-private ( DP ) machine learning ( Dwork et al. , 2006 ) provides a computational framework for privacy protection and has been widely studied in various settings , including both convex and non-convex optimization ( Wang et al. , 2017 ; 2019 ; Jain et al. , 2019 ) . One widely used procedure for privacy-preserving learning is the ( Differentially ) Private Gradient Descent ( PGD ) ( Bassily et al. , 2014 ; Abadi et al. , 2016 ) . A typical gradient descent procedure updates its model by the gradients of losses evaluated on the training data . When the data is sensitive , the gradients should be privatized to prevent excess privacy leakage . The PGD privatizes a gradient by adding controlled noise . As such , the models from PGD is expected to have a lower utility as compared to those from unprotected algorithms . In the cases where strict privacy control is exercised , or equivalently , a tight privacy budget , accumulating effects from highly-noised gradients may lead to unacceptable model performance . It is thus critical to design effective privatization procedures for PGD to maintain a great balance between utility and privacy . Recent years witnessed a promising privatization direction that studies how to dynamically adjust the privacy-protecting noise during the learning process , i.e. , dynamic privacy schedules , to boost utility under a specific privacy budget . One example is ( Lee & Kifer , 2018 ) , which reduced the noise magnitude when the loss does not decrease , due to the observation that the gradients become very small when approaching convergence , and a static noise scale will overwhelm these gradients . Another example is ( Yu et al. , 2019 ) , which periodically decreased the magnitude following a predefined strategy , e.g. , exponential decaying or step decaying . Both approaches confirmed the empirically advantages of decreasing noise magnitudes . Intuitively , the dynamic mechanism may coordinate with certain properties of the learning task , e.g. , training data and loss surface . Yet there is no theoretical analysis available and two important questions remain unanswered : 1 ) What is the form of utility-preferred noise schedules ? 2 ) When and to what extent such schedules improve utility ? To answer these questions , in this paper we develop a principled approach to construct dynamic schedules and quantify their utility bounds in different learning algorithms . Our contributions are summarized as follows . 1 ) For the class of loss functions satisfying the Polyak-Lojasiewicz condition ( Polyak , 1963 ) , we show that a dynamic schedule improving the utility upper bound is shaped by the influence of per-iteration noise on the final loss . As the influence is tightly connected to the loss curvature , the advantage of using dynamic schedule depends on the loss function consequently . 2 ) Beyond gradient descent , our results show the gradient methods with momentum implicitly introduce a dynamic schedule and result in an improved utility bound . 3 ) We empirically validate our results on convex and non-convex ( no need to satisfy the PL condition ) loss functions . Our results suggest that the preferred dynamic schedule admits the exponentially decaying form , and works better when learning with high-curvature loss functions . Moreover , dynamic schedules give more utility under stricter privacy conditions ( e.g. , smaller sample size and less privacy budget ) . 2 RELATED WORK . Differentially Private Learning . Differential privacy ( DP ) characterizes the chance of an algorithm output ( e.g. , a learned model ) to leak private information in its training data when the output distribution is known . Since outputs of many learning algorithms have undetermined distributions , the probability of their privacy leakages is hard to measure . A common approach to tackle this issue is to inject randomness with known probability distribution to privatize the learning procedures . Classical methods include output perturbation ( Chaudhuri et al. , 2011 ) , objective perturbation ( Chaudhuri et al. , 2011 ) and gradient perturbation ( Abadi et al. , 2016 ; Bassily et al. , 2014 ; Wu et al. , 2017 ) . Among these approaches , the Private Gradient Descent ( PGD ) has attracted extensive attention in recent years because it can be flexibly integrated with variants of gradient-based iteration methods , e.g. , stochastic gradient descent , momentum methods ( Qian , 1999 ) , and Adam ( Kingma & Ba , 2014 ) , for both convex and non-convex problems . Dynamic Policies for Privacy Protection . Wang et al . ( 2017 ) studied the empirical risk minimization using dynamic variation reduction of perturbed gradients . They showed that the utility upper bound can be achieved by gradient methods under uniform noise parameters . Instead of enhancing the gradients , Yu et al . ( 2019 ) ; Lee & Kifer ( 2018 ) showed the benefits of using a dynamic schedule of privacy parameters or equivalently noise scales . In addition , adaptive sensitivity control ( Pichapati et al. , 2019 ; Thakkar et al. , 2019 ) and dynamic batch sizes ( Feldman et al. , 2020 ) are also demonstrated to improves the convergence . Utility Upper Bounds . A utility upper bound is a critical metric for privacy schedules that characterizes the maximum utility that a schedule can deliver in theory . Wang et al . ( 2017 ) is the first to prove the utility bound under the PL condition . In this paper , we improve the upper bound by a more accurate estimation of the dynamic influence of step noise . Remarkably , by introducing a dynamic schedule , we further boost the sample-efficiency of the upper bound . With a similar intuition , Feldman et al . ( 2020 ) proposed to gradually increase the batch size , which reduces the dependence on sample size accordingly . Recently , Zhou et al . proved the utility bound by using the momentum of gradients ( Polyak , 1964 ; Kingma & Ba , 2014 ) . Table 1 summarizes the upper bounds of methods studied in this paper ( in the last block of rows ) and results from state-of-the-art algorithms based on private gradients . Our work shows that considering the dynamic influence can lead to a tighter bound . 3 PRIVATE GRADIENT DESCENT . Notations . We consider a learning task by empirical risk minimization ( ERM ) f ( θ ) = 1 N ∑N n=1 f ( θ ; xn ) on a private dataset { xn } Nn=1 and θ ∈ RD . The gradient methods are defined as θt+1 = θt − ηt∇t , where ∇t = ∇f ( θt ) = 1N ∑ n∇f ( θt ; xn ) denotes the non-private gradient at iteration t , ηt is the step learning rate . ∇ ( n ) t = ∇f ( θt ; xn ) denotes the gradient on a sample xn . Ic denotes the indicator function that returns 1 if the condition c holds , otherwise 0 . Assumptions . ( 1 ) In this paper , we assume f ( θ ) is continuous and differentiable . Many commonly used loss functions satisfy this assumption , e.g. , the logistic function . ( 2 ) For a learning task , only finite amount of privacy cost is allowed where the maximum cost is called privacy budget and denoted as R. ( 3 ) Generally , we assume that loss functions f ( θ ; x ) ( sample-wise loss ) are G-Lipschitz continuous and f ( θ ) ( the empirical loss ) is M -smooth . Definition 3.1 ( G-Lipschitz continuity ) . A function f ( · ) is G-Lipschitz continuous if , for G > 0 and all x , y in the domain of f ( · ) , f ( · ) satisfies ‖f ( y ) − f ( x ) ‖ ≤ G‖y − x‖2. . Definition 3.2 ( m-strongly convexity ) . A function f ( · ) is m-strongly convex if f ( y ) ≥ f ( x ) + ∇f ( x ) T ( y − x ) + m2 ‖y − x‖ 2 , for some m > 0 and all x , y in the domain of f ( · ) . Definition 3.3 ( M -smoothness ) . A function is M -smooth w.r.t . l2 norm if f ( y ) ≤ f ( x ) + ∇f ( x ) T ( y − x ) + M2 ‖y − x‖ 2 , for some constant M > 0 and all x , y in the domain of f ( · ) . For a private algorithmM ( d ) which maps a dataset d to some output , the privacy cost is measured by the bound of the output difference on the adjacent datasets . Adjacent datasets are defined to be datasets that only differ in one sample . In this paper , we use the zero-Concentrated Differential Privacy ( zCDP , see Definition 3.4 ) as the privacy measurement , because it provides the simplicity and possibility of adaptively composing privacy costs at each iteration . Various privacy metrics are discussed or reviewed in ( Desfontaines & Pejó , 2019 ) . A notable example is Moment Accoutant ( MA ) ( Abadi et al. , 2016 ) , which adopts similar principle for composing privacy costs while is less tight for a smaller privacy budget . We note that alternative metrics can be adapted to our study without major impacts to the analysis . Definition 3.4 ( ρ-zCDP ( Bun & Steinke , 2016 ) ) . Let ρ > 0 . A randomized algorithmM : Dn → R satisfies ρ-zCDP if , for all adjacent datasets d , d′ ∈ Dn , Dα ( M ( d ) ‖M ( d′ ) ) ≤ ρα , ∀α ∈ ( 1 , ∞ ) where Dα ( ·‖· ) denotes the Rényi divergence ( Rényi , 1961 ) of order α. zCDP provides a linear composition of privacy costs of sub-route algorithms . When the input vector is privatized by injecting Gaussian noise of N ( 0 , σ2t I ) for the t-th iteration , the composed privacy cost is proportional to ∑ t ρt where the step cost is ρt = 1 σ2t . For simplicity , we absorb the constant coefficient into the ( residual ) privacy budget R. The formal theorems for the privacy cost computation of composition and Gaussian noising is included in Lemmas B.1 and B.2 . Generally , we define the Private Gradient Descent ( PGD ) method as iterations for t = 1 . . . T : θt+1 = θt − ηtφt = θt − ηt ( ∇t + σtGνt/N ) , ( 1 ) where φt = gt is the gradient privatized from ∇t as shown in Algorithm 1 , G/N is the bound of sensitivity of the gathered gradient excluding one sample gradient , and νt ∼ N ( 0 , I ) is a vector element-wisely subject to Gaussian distribution . We use σt to denote the noise scale at step t and use σ to collectively represents the schedule ( σ1 , . . . , σT ) if not confusing . When the Lipschitz constant is unknown , we can control the upper bound by scaling the gradient if it is over some constant . The scaling operation is often called clipping in literatures since it clips the gradient norm at a threshold . After the gradient is noised , we apply a modification , φ ( · ) , to enhance its utility . In this paper , we consider two types of φ ( · ) : φ ( mt , gt ) = gt ( GD ) , φ ( mt , gt ) = [ β ( 1− βt−1 ) mt + ( 1− β ) gt ] / ( 1− βt ) ( Momentum ) We now show that the PGD using Algorithm 1 guarantees a privacy cost less than R : Algorithm 1 Privatizing Gradients Input : Raw gradients [ ∇ ( 1 ) t , . . . , ∇ ( n ) t ] ( n = N by default ) , vt , residual privacy budget Rt assuming the full budget is R and R1 = R. 1 : ρt ← 1/σ2t , ∇t ← 1n ∑n i=1∇ ( i ) t . Budget request 2 : if ρt < Rt then 3 : Rt+1 ← Rt − ρt 4 : gt ← ∇t +Gσtνt/N , νt ∼ N ( 0 , I ) . Privacy noise 5 : mt+1 ← φ ( mt , gt ) or g1 if t = 1 6 : return ηtmt+1 , Rt+1 . Utility projection 7 : else 8 : Terminate Theorem 3.1 . Suppose f ( θ ; x ) is G-Lipschitz continuous and the PGD algorithm with privatized gradients defined by Algorithm 1 , stops at step T . The PGD algorithm outputs θT and satisfies ρ-zCDP where ρ ≤ 12R . Note that Theorem 3.1 allows σt to be different throughout iterations . Next we present a principled approach for deriving dynamic schedules optimized for the final loss f ( θT ) .

Summary: The paper studies private gradient descent when the noise added to each of the iteration is dynamically scheduled. Prior to this work, the work of Zhou et al. tries to achieve the same for DP-SGD and they analyze their algorithm for many variants of adaptive gradient descent based method. The difference with Zhou et al. is that they do not gradient norm and the generalization property of DP. As a result, the authors claim that Zhou et al. achieves suboptimal utility guarantee.