Differentially Private Learning with Adaptive Clipping

1 Introduction . Deep learning has become ubiquitous , with applications as diverse as image processing , natural language translation , and music generation [ 27 , 13 , 28 , 7 ] . Deep models are able to perform well in part due to their ability to utilize vast amounts of data for training . However , recent work has shown that it is possible to extract information about individual training examples using only the parameters of a trained model [ 12 , 30 , 26 , 8 , 19 ] . When the training data potentially contains privacy-sensitive user information , it becomes imperative to use learning techniques that limit such memorization . Differential privacy ( DP ) [ 10 , 11 ] is widely considered a gold standard for bounding and quantifying the privacy leakage of sensitive data when performing learning tasks . Intuitively , DP prevents an adversary from confidently making any conclusions about whether some user ’ s data was used in training a model , even given access to arbitrary side information . The formal definition of DP depends on the notion of neighboring datasets : we will refer to a pair of datasets D , D′ ∈ D as neighbors if D′ can be obtained from D by adding or removing one element . Definition 1.1 ( Differential Privacy ) . A ( randomized ) algorithm M : D → R with input domain D and output rangeR is ( ε , δ ) -differentially private if for all pairs of neighboring datasets D , D′ ∈ D , and every measurable S ⊆ R , we have Pr ( M ( D ) ∈ S ) ≤ eε · Pr ( M ( D′ ) ∈ S ) + δ , where probabilities are with respect to the coin flips of M . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Following McMahan et al . [ 18 ] , we define two common settings of privacy corresponding to two different definitions of neighboring datasets . In example-level DP , datasets are considered neighbors when they differ by the addition or removal of a single example [ 9 , 4 , 2 , 22 , 31 , 23 , 15 ] . In user-level DP , neighboring datasets differ by the addition or removal of all of the data of one user [ 18 ] . Userlevel DP is the stronger form , and is preferred when one user may contribute many training examples to the learning task , as privacy is protected even if the same privacy-sensitive information occurs in all the examples from one user . In this paper , we will describe the technique and perform experiments in terms of the stronger user-level form , but we note that example-level DP can be achieved by simply giving each user a single example . To achieve user-level DP , we employ the Federated Averaging algorithm [ 17 ] , introduced as a decentralized approach to model training in which the training data is left distributed on user devices , and each training round aggregates updates that are computed locally . On each round , a sample of devices are selected for training , and then each selected device performs potentially many steps of local SGD over minibatches of its own data , sending back the model delta as its update . Bounding the influence of any user in Federated Averaging is both necessary for privacy and often desirable for stability . One popular way to do this to cap the L2 norm of its model update by projecting larger updates back to the ball of norm C. Since such clipping also effectively bounds the L2 sensitivity of the aggregate with respect to the addition or removal of any user ’ s data , adding Gaussian noise to the aggregate is sufficient to obtain a central differential privacy guarantee for the update [ 9 , 4 , 2 ] . Standard composition techniques can then be used to extend the per-update guarantee to the final model [ 20 ] . Setting an appropriate value for the clipping threshold C is crucial for the utility of the private training mechanism . Setting it too low can result in high bias since we discard information contained in the magnitude of the gradient . However setting it too high entails the addition of more noise , because the amount of Gaussian noise necessary for a given level of privacy must be proportional to the norm bound ( the L2 sensitivity ) , and this will eventually destroy model utility . The clipping bias-variance trade-off was observed empirically by McMahan et al . [ 18 ] , and was theoretically analyzed and shown to be an inherent property of differentially private learning by Amin et al . [ 3 ] . Learning large models using the Federated Averaging/SGD algorithm [ 17 , 18 ] can take thousands of rounds of interaction between the central server and the clients . The norms of the updates can vary as the rounds progress . Prior work [ 18 ] has shown that decreasing the value of the clipping threshold after training a language model for some initial number of rounds can improve model accuracy . However , the behavior of the norms can be difficult to predict without prior knowledge about the system , and if it is difficult to choose a fixed clipping norm for a given learning task , it is even more difficult to choose a parameterized clipping norm schedule . While there has been substantial work on DP techniques for learning , almost every technique has hyperparameters which need to be set appropriately for obtaining good utility . Besides the clipping norm , learning techniques have other hyperparameters which might interact with privacy hyperparameters . For example , the server learning rate in DP SGD might need to be set to a high value if the clipping threshold is very low , and vice-versa . Such tuning for large networks can have an exorbitant cost in computation and efficiency , which can be a bottleneck for real-world systems that involve communicating with millions of samples for training a single network . Tuning may also incur an additional cost for privacy , which needs to be accounted for when providing a privacy guarantee for the released model with tuned hyperparameters ( though in some cases , hyperparameters can be tuned using the same model and algorithm on a sufficiently-similar public proxy dataset ) . Related Work The heuristic of clipping to the median update norm was suggested by Abadi et al . [ 1 ] , but no method for doing so privately was proposed , nor was the heuristic empirically tested . One possibility would be to estimate the median unclipped update norm at each round privately by adding noise calibrated to the smooth sensitivity as defined by Nissim et al . [ 21 ] . However this approach has several drawbacks compared to the algorithm we will present . It would require the clients to send their unclipped updates to the server at each round,1 which is incompatible with the foundational 1Alternatively , clients could first report their update norms , after which the server could reply with the clipping norm for the round , and finally the clients would send the clipped updates . However , this would still give the server strictly more information than our approach , and would require an extra round of communication , both of which are undesirable . principle of federated learning to transmit only focused , minimal updates , and precluding the use of secure aggregation [ 5 ] and certain forms of compression [ 16 , 6 ] . Also , by incorporating information across multiple rounds , our method is able to track the underlying quantile closely , with less jitter associated with sampling at each round , and using only a negligible fraction of the privacy budget . Contributions In this paper , we describe a method for adaptively and privately tuning the clipping threshold to track a given quantile of the update norm distribution during training . The method uses a negligible amount of privacy budget , and is compatible with other FL technologies such as compression and secure aggregation [ 5 , 6 ] . We perform a careful empirical comparison of our adaptive clipping method to a highly optimized fixed-clip baseline on a suite of realistic and publicly available FL tasks to demonstrate that high-utility and high-privacy models—sometimes exceeding any fixed clipping norm in utility—can be trained using our method without the need to tune any clipping hyperparameter . 2 Private adaptive quantile clipping . In this section , we will describe the adaptive strategy that can be used for adjusting the clipping threshold so that it comes to approximate the value at a specified quantile . Let X ∈ R be a random variable , let γ ∈ [ 0 , 1 ] be a quantile to be matched . For any C , define ` γ ( C ; X ) = { ( 1− γ ) ( C −X ) if X ≤ C , γ ( X − C ) otherwise , so ` ′γ ( C ; X ) = { ( 1− γ ) if X ≤ C , −γ otherwise . Hence , E [ ` ′γ ( C ; X ) ] = ( 1 − γ ) Pr [ X ≤ C ] − γ Pr [ X > C ] = Pr [ X ≤ C ] − γ . For C∗ such that E [ ` ′γ ( C∗ ; X ) ] = 0 , we have Pr ( X ≤ C∗ ) = γ . Thus , C∗ is the γth quantile of X . Because the loss is convex and has gradients bounded by 1 , we can get an online estimate of C that converges to the γth quantile of X using online gradient descent ( see , e.g. , Shalev-Shwartz [ 25 ] ) . See Figure 1 for a plot of the loss for a random variable that takes six values with equal probability . Suppose at some round we have m samples of X , with values ( x1 , . . . , xm ) . The average derivative of the loss for that round is ¯̀′ γ ( C ; X ) = 1 m m∑ i=1 { ( 1− γ ) if xi ≤ C , −γ otherwise = 1 m ( ( 1− γ ) ∑ i∈ [ m ] Ixi≤C − γ ∑ i∈ [ m ] Ixi > C ) = b̄− γ , where b̄ , 1m ∑ i∈ [ m ] Ixi≤C is the empirical fraction of samples with value at most C. For a given learning rate ηC , we can perform the update : C ← C − ηC ( b̄− γ ) . Geometric updates . Since b̄ and γ take values in [ 0 , 1 ] , the linear update rule above changes C by a maximum of ηC at each step . This can be slow if C is on the wrong order of magnitude . At the other extreme , if the optimal value of C is orders of magnitude smaller than ηC , the update can be very coarse , and may overshoot to become negative . To remedy such issues , we propose the following geometric update rule : C ← C · exp ( −ηC ( b̄− γ ) ) . This update rule converges quickly to the true quantile even if the initial estimate is off by orders of magnitude . It also has the attractive property that the variance of the estimate around the true quantile at convergence is proportional to the value at that quantile . In our experiments , we use the geometric update rule with ηC = 0.2 .

Summary: This works studies the problem of training neural networks with user-level differential privacy using federated learning. To bound user contribution, the authors propose to clip the gradient norms to the private estimate of the median of gradient norms, as opposed to some fixed clipping threshold. This method reduces the need to tune the clip threshold and achieves competitive performance across a range of tasks.