D IFFERENTIALLY P RIVATE SGD WITH S PARSE G RADIENTS
Anonymous authors
Paper under double-blind review
A BSTRACT
To protect sensitive training data, differentially private stochastic gradient descent (DP-SGD) has been adopted in deep learning to provide rigorously deﬁned privacy.
However, DP-SGD requires the injection of an amount of noise that scales with the number of gradient dimensions, resulting in large performance drops compared to non-private training.
In this work, we propose random freeze which randomly freezes a progressively increasing subset of parameters and results in sparse gradient updates while maintaining or increasing accuracy.
We theoretically prove the convergence of random freeze and ﬁnd that random freeze exhibits a signal loss and perturbation moderation trade-off in DP-SGD.
Applying random freeze across various DP-SGD frameworks, we maintain accuracy within the same number of iterations while achieving up to 70% representation sparsity, which demonstrates that the trade-off exists in a variety of DP-SGD methods.
We further note that random freeze signiﬁcantly improves accuracy, in particular for large networks.
Additionally, axis-aligned sparsity induced by random freeze leads to various advantages for projected DP-SGD or federated learning in terms of computational cost, memory footprint and communication overhead.
I NTRODUCTION
The success of machine learning, and deep neural networks in particular, combined with ubiquitous edge computation and digital record keeping, has led to a surge in privacy sensitive learning applications.
Internet-scale data promises to accelerate the development of data-driven statistical approaches, but the need for privacy constrains the amalgamation of such datasets.
Private data are in fact isolated, constraining our ability to build models that learn from a large number of instances.
On the other hand, the information contained in locally stored data can also be exposed through releasing the model trained on a local dataset (Fredrikson et al., 2015; Shokri et al., 2017), or even reconstructed when gradients generated during training are shared (Zhu et al., 2019;
Geiping et al., 2020; Zhu & Blaschko, 2021).
To address these issues, many applications of machine learning are expected to be privacypreserving.
While differential privacy (DP) provides a rigorously deﬁned and measurable privacy guarantee for database operations (Dwork & Roth, 2014), it also contains intriguing properties, such as robustness to post-processing and composability, which enables conveniently computing an overall privacy guarantee for several DP components.
Differential privacy 1 deﬁnes privacy with respect to the difﬁculty of distinguishing the outputs.
For a pair of neighboring databases X, X (cid:48) ∈ X , i.e.
X can be obtained from X (cid:48) by adding or removing an element.
Deﬁnition 1. A randomized mechanism M : X → R is ( ε, δ ) -differentially private, if for any subset of outputs S ⊆ R it holds that:
[Equation]
A common paradigm for a randomized mechanism M in deep learning is perturbed gradient descent:
[Equation]
where f : X → R computes an aggregated gradient given a database X or X (cid:48) .
The isotropic Gaussian distributed noise ξ DP ∼ N (0 , S 2 f σ 2 I ) is calibrated to f ’s sensitivity S 2 f , which is the maximal (cid:96) 2 distance (cid:107)
f ( X ) − f
( X (cid:48) )
(cid:107) , i.e. the maximal (cid:96) 2 norm of gradient among all individual examples.
X, X (cid:48) could be batches of training data, for instance, in our experiments they are batches of image-label pairs.
The factor σ is a noise multiplier controlling the strength of the privacy guarantee: higher σ leads to lower privacy loss.
Differentially private stochastic gradient descent (DP-SGD) upper bounds the certainty of connecting data with arbitrary subset of gradient space using the privacy budget variables ( ε, δ ) .
Bassily et al.
(2014) show that in a convex setting, DP-SGD achieves excess risk of ˜ O ( √ d/nε ) for a model w ∈ R d that minimizes the empirical risk (cid:80) ni
=1 (cid:96) ( w, x i ) , where x 1 , x 2 , ..., x n are drawn from
X .
While we show that in non-convex general setting, the mean square error (MSE) of perturbed gradient ˜ g = g + ξ DP is between Ω( d ) and Ω( d 2 ) by assuming the gradients follow a Gaussian distribution:
Theorem 1. Assuming that the gradient is drawn from N ( ∇ w, Σ) , centered at the true gradient ∇ w and with respect to the covariance matrix Σ whose trace goes linearly up with dimension d .
The MSE of perturbed gradient ˜ g
= g + ξ DP can be lower bounded by:
[Equation]
from which we conclude that the lower bound on MSE is between linear and quadratic in d in practice.
For the proof and conclusion, refer to Appendix A. In terms of deep learning, as d is a large number for modern network architectures, this can lead to a signiﬁcant increase in error.
The work of Abadi et al.
(2016) proposed to clip the gradient of each individual example in (cid:96) 2 norm to a preset bound C,
i.e. ¯ g = g · min(1 , C (cid:107) g
(cid:107) 2 ) .
They then apply this clipping bound to compute the variance of Gaussian distributed noise.
The Gaussian noise mechanism can be expressed as:
[Equation]
DP-SGD with gradient clipping has been empirically veriﬁed to be effective as it constraints the amount of injected noise by setting a small clipping bound.
However, clipping removes the magnitude information of the gradient and therefore results in gradient estimation bias.
Setting a small clipping bound with a deeper network will not result in better performance.
The expected MSE of the perturbed gradient is only constrained to O ( d ) , the impact of perturbation is non-negligible in practice.
The biggest network where this strategy is successfully applied so far is a CNN with Tanh proposed by Papernot et al.
(2021), which reaches ∼ 66% accuracy on CIFAR10 in a low privacy regime and is regarded as the state-of-the-art (SOTA) end-to-end network with DP-SGD.
To address this curse of dimensionality, most recent works concentrate on gradient dimension reduction.
1. R ELATED WORKS
Abadi et al.
(2016) propose to pretrain a network on an auxiliary dataset and then transfer the feature extraction, so that only a linear classiﬁer will be replaced and trained on the private data.
Tramer & Boneh (2021) adopt ScatterNet (Oyallon et al., 2019) to extract handcrafted features and train a relatively shallow network based on the features.
Both work decrease d by excluding the majority of parameters during DP learning, which also constrains the learning ability of network.
Inspired by the empirical observation that the optimization trajectory is contained in lowerdimensional subspace (Vogels et al., 2019; Gooneratne et al., 2020; Li et al., 2020), a line of work intend to reduce d by exploiting the low-rank property of the gradient while considering privacy. Several recent works (Zhou et al., 2021; Yu et al., 2021a; Kairouz et al., 2021) project the gradient into a subspace which is identiﬁed by auxiliary data or released historical gradients.
In practice, they use the power method to search for the subspace. The computational cost of running the power method and projecting gradients as well as the memory footprint of storing the projection matrix limits the application of such method to large models.
Zhang et al.
(2021) target an NLP task where networks are heavily over-parameterized and gradients are extremely sparse, and propose to adopt DP selection to privately select top- k signiﬁcant gradients for optimization. Also targeting an NLP task, Yu et al.
(2021b) propose a low-rank reparameterization of weights via released historical gradients.
In addition to the aforementioned works, McMahan et al.
; Yang et al.
and others study how to incorporate differential privacy in collaborative training, e.g. federated learning, in the interest of protecting the privacy of participants.
Federated learning also suffers from large d , as it is usually deployed on edge devices and local models are periodically synchronized, so communication cost becomes expensive both in time and power usage (Pathak et al., 2012).
Therefore, gradient dimension reduction can have large beneﬁts in federated learning involving power-restricted edge devices.
A line of work studies how to tackle this issue by utilizing the low-rank property (Shokri &
Shmatikov, 2015; Yang et al., 2019; Liu et al., 2020).
1. O UR CONTRIBUTION
In this work, we demonstrate an axis-aligned gradient dimension reduction method.
Our work is orthogonal to previous works, we do not extract any characteristic information of the gradient or model, or approximate the gradient from a subspace.
Instead, we randomly zero-out a fraction of the gradient during training and force the gradient to have a sparse representation.
We provide a theoretical study on this strategy and reveal that random freeze exhibits a trade-off between signal loss and perturbation moderation in DP-SGD. We remark that our theory and approach do not necessarily rely on the low-rank assumption. To the best of our knowledge, we are the ﬁrst to study this trade-off in DP-SGD and provide an effective approach. We use the benchmark CIFAR10 (Krizhevsky, 2012) which is to date standard in benchmarking
DP learning and show that well implemented random freeze exhibits various advantages and can be widely applied.
More speciﬁcally, we maintain accuracy when we adapt projected DP-SGD with random freeze, while we reduce the computational cost and memory footprint induced by the power method and projection.
Applying it to various frameworks, we achieve a high representation sparsity of gradient without a loss in performance.
Federated learning can take advantage of the resulting sparse representation to reduce communication costs.
We further note that the random freeze strategy improves the accuracy of large networks, which we demonstrate with the SOTA End-to-end CNN proposed by Papernot et al.  A NALYSIS OF R ANDOM F REEZE In this section we theoretically prove that to a certain freeze rate r , DP-SGD with random freeze will converge if the approach without random freeze can converge.
Then we investigate the tradeoff between signal loss and perturbation moderation induced by random freeze. Furthermore, we empirically demonstrate beneﬁts of applying random freeze from the perspective of the gradient distribution.
2. C ONVERGENCE RATE OF DP-SGD WITH RANDOM FREEZE
Let L be the objective function L ( w ) := E x ∼X
[ (cid:96) ( w, x )] , m ∈ { 0 , 1 } d the freeze mask and r the freeze rate, we randomly draw rp indices and set these positions in the mask to [-
-]0 and others to 1 so that (cid:80) m = (1 − r ) d . We assume an oracle telling us the true gradient ∇ w and individual gradient g t,i = ∇ w + ξ t ( x i ) , where ξ t is independent gradient deviation with zero mean. Let p t be the distribution of ξ t .
For random freeze the sparse gradient is g (cid:48) t,i = ∇ w (cid:48) t + ξ (cid:48) t ( x i ) , where ∇ w (cid:48) t : = m (cid:12) ∇ w t , ξ (cid:48) t ( x i ) : = m (cid:12) ξ t ( x i ) .
Denote for DP-SGD the averaged clipped gradient of
{+B samples ¯ g t := 1 B (cid:80) i g t,i · min(1 , C (cid:107) g t,i (cid:107) ) , and for DP-SGD with random freeze ˆ g t := 1 B (cid:80) i g (cid:48) t,i · min(1 , C (cid:107) g (cid:
+}
) t,i (cid:107) ) . We have for DP-SGD with random freeze the following inequality: Theorem 2. Assume G-Lipschitz smoothness of ∇ w such that ||∇ w t +1 − ∇ w t || ≤ G || w t +1 − w t
|| .
Consider an algorithm with clipping bound C , learning rate γ and choose a symmetric probability density distribution ˜ p ( · ) satisfying ˜ p t ( ξ t ) =
˜ p t ( − ξ t ) , ∀ ξ t ∈ R d .
Then ∃ κ ≥ 1 − r such that:
[Equation]
{+where we deﬁne b t := (cid:82) (cid:104)∇ w (cid:48) t , g (cid:
+}
) t,i · min(1 , C (cid:107) g (cid:48) t,i (cid:107) )
(cid:105) ( p (cid:48) t ( ξ (cid:48) t ) − ˜ p (cid:48) t ( ξ (cid:48) t )) dξ (cid:48) t , p (cid:48) t , ˜ p (cid:48) t are corresponding projected distribution, and deﬁne ∆ L :=
E [ L 1 − min w L ( w )] , ∆ C :=
GC 2 2 , ∆ DP := C 2 σ 2 dG 2 B 2 , h ( ∇ w t ) :
= min( (cid:107)∇ w t (cid:107) , 3 C 4 ) .
The proof can be found in Appendix B.
When no freeze r = 0 , i.e. m is a vector ﬁlled with value 1 .
Note that κ = 1 for r = 0 , we can obtain the inequality 4 in the following form:
[Equation]
which describes DP-SGD without random freeze
Chen et al.
(2020b) argue that by tweaking ˜
p t it is possible to bound P ξ t ∼ ˜ p t away from zero which means the l.h.s.
is proportional to (cid:107)∇ w t (cid:107) or (cid:107)∇ w t (cid:107)at each iteration, while letting the convergence bias term − b t tend to be small as p t is approximately symmetric.
They then prove that by setting a certain learning rate γ , the r.h.s.
diminishes to zero, so the network under DP-SGD can converge.
Adapted from that, we see at the r.h.s.
of inequality 4 for random freeze, − E m [ b t ] also tends to be small as p (cid:48) t is expected to be approximately symmetric if p t is.
Additionally, 1 /κ will not be large as along as r is not extremely close to 1 .
So with the same learning rate and γ adapted for DP-SGD without random freeze, the r.h.s.
of inequality 4 will also tend to zero, which proves the convergence of DP-SGD with random freeze.
[Figure]
2. T RADE - OFF BETWEEN SIGNAL LOSS AND PERTURBATION MODERATION
From Theorem 2, we see that by applying random freeze the injected noise term ∆ DP is reduced by 1 − rκ < 1 while other terms could be increased by 1 /κ .
From an operational view, random freeze moderates perturbation while removing some signal.
Therefore, the convergence rate of random freeze is a trade-off between signal loss and perturbation moderation.
Consider that ∆ DP is proportional to the number of parameters d , which is large for neural networks.
It might be worthwhile to remove some signal while moderating perturbation.
To investigate this question, we consider the MSE of gradient estimation of DP-SGD with and without random freeze at iteration t :
[Equation]
[Equation]
Two statistics of interest are: First, the difference of signal loss ∆ S :=
E [ (cid:107)∇ w t − ¯ g (cid:107) 2 ] − E
[ (cid:107)∇ w t − ˆ g (cid:107) 2 ] .
DP-SGD induces signal noise due to clipping bias while random freeze has additional loss due to freezing.
Second, the difference of MSE ∆ MSE :
= MSE DP − MSE RF which implies the trade-off between signal loss and perturbation moderation.
We train the End-to-end CNN with 80 epochs on a privacy budget ( ε = 7 .
δ = 10 − 5 ) and measure these two statistics with respect to empirical distribution at the end of three training stages (see Figure 1).
The result shows that although random freeze has more signal loss as the freeze rate increasing, due to a large amount of injected noise, MSE of gradient estimation with random freeze is clearly lower.
The result reﬂects that injected noise is dominant during optimization.
Combining this with Theorem 2, it shows the possibility that we achieve better convergence by removing some signal while mitigating noise.
For example, in addition to taking advantage of noise mitigation, we can also raise the noise to ξ + DP so that E [ (cid:107) m (cid:12) ξ + DP (cid:107) ] matches E [ (cid:107) ξ DP (cid:107) ] .
This allows us to run more iterations T on the same privacy budget, which also implies better convergence by Theorem 2.
In section 4 we will demonstrate that when d is large, which implies large ∆ DP , we can improve the performance by applying random freeze.
2. A GRADIENT DISTRIBUTION VIEW OF RANDOM FREEZE
We also observe the potential beneﬁt of random freeze from the perspective of the gradient distribution.
Compared to no freeze, random freeze leads to less clipping bias and gradient distortion, as shown in Figure 2.
We adopt the same clipping bound for random freeze and no freeze.
As fewer dimensions contribute to the norm computation, random freeze reduces the clipping probability and therefore alleviates clipping bias (Zhang et al., 2020).
We also note that the norm of sparse gradients are not equally scaled down, weak gradients can spontaneously become larger during training, which mitigates the distortion of gradients due to perturbation, while the perturbation of random freeze is already moderated compared to no freeze.
T ECHNICAL
[Figure] ETAILS Algorithm 1 outlines our approach. Since the coordinates that have been selected to freeze do not depend on the dataset, their indices can be exposed or transferred in clear text and do not lead to additional privacy loss to the dataset.
We apply random freeze to optimization with SGD with or without momentum.
In case of non-zero momentum, the velocity is updated with the sparse gradient as normal.
That means, parameters that have been frozen can still be updated as long as their velocity has not decayed to zero.
Next, we discuss the properties and implementation details of random freeze, compare different variants and provide insight into the strategy, 3. G RADUAL COOLING We ﬁnd that if we initiate the training with constant freeze rate r , the network converges slowly and performs poorly when the privacy budget has been fully consumed. The reason is that in the early stages of training, the network is far from its optimal position, it is better to let all parameters stay
Input: Initialized parameters: w 0 ; Loss : (cid:96) ; Iterations ; Epochs ; Freeze rate:
; Cooling time: e ; Clipping bound: C ; Momentum: µ Learning rate for e 0...E
1 do r ( = r · min( e e − 1 ,
Randomly generate a freeze mask m ∈ { 0 , 1 } d subject to (cid:80) m = d · (1 − r ( e )) ; for = 0...T-1 do For each x i in minibatch of size B , compute g t ( x i ) = ∇ (cid:96) ( w t , x i ) ;
Partially zero out each gradient g t i
= m g x i ) ; Clip each individual gradient ˆ g t ( i = g t
) · min C (cid:107) g x Add noise ˜ g t = 1 B
i ˆ g t x i
+ m
N 0 , C 2 σ
I d )) ;
Update v t +1 = µ · v t + g t , w t +1 = w t − γv t +1 ; end end active.
So we present gradual cooling, which is inspired by the gradual warm-up of the learning rate adopted for non-privacy-preserving training (Goyal et al., 2017).
Gradual cooling linearly ramps up the freeze rate from 0 to r ∗ within a predeﬁned cooling time e ∗ and stays at r ∗ for the remaining training epochs, i.e. r = r ∗ · min( e e ∗ − 1 , 1) .
3. P ER - ITERATION RANDOMIZATION VS . P ER - EPOCH RANDOMIZATION
Although conducting random freeze leads to negligible additional computational cost and memory footprint, we ﬁnd that it is sufﬁcient to generate one freeze mask per epoch, while re-randomizing the freeze mask at each iteration slightly decreases the performance.
As we have shown in Section 2.2, noise dominates throughout training.
Therefore, by per-iteration randomization, parameters have been perturbed at this round could be frozen during subsequent iterations and stay biased.
By contrast, per-epoch randomization lets the selected parameters update for one epoch, allowing the noise of multiple iterations to be averaged out.
Another strength of per-epoch randomization is that for one epoch there are certainly (1 − r ) d parameters updated, which is favorable in collaborative learning schemes as communication cost is a signiﬁcant issue, and data are not transmitted every iteration.
While for per-iteration randomization, the cumulative number of updated parameters depends on the freeze rate r and iterations between two communication rounds, resulting in higher communication overheads than per-epoch randomization. We present non-privately ranked freeze, i.e. ranking after excluding noise, for comparison.
3. R ANDOM FREEZE VS . R ANKED FREEZE
Based on the observation of the low-rank property of gradients (Vogels et al., 2019; Gooneratne et al., 2020; Li et al., 2020), freezing the parameters with respect to the mean of past perturbed gradients instead of a random draw might be helpful:
First, taking the mean of past perturbed gradients will average out zero-mean noise.
Second, the magnitude of the gradients is a diagonal approximation to the principal components and may be indicative of a useful working subspace.
We therefore consider ranking the dimensions of the gradient by their magnitude and freezing the smallest ones.
However, empirical results do not match these intuitions.
Comparing Table 3 with Table 4, ranked freeze performs similarly to random freeze.
We further ﬁnd that ranked freeze is itself inherently random as the ranking is dominated by Gaussian noise.
To demonstrate this, we run random freeze and ranked freeze 2 on End-to-end CNN for 100 epochs, then statistically analyse the distribution of how many times a parameter is frozen.
The result shows the equivalency between these two strategies, which implies even averaging the perturbed gradients over a full epoch cannot sufﬁciently mitigate the noise added in gradient perturbation (see Figure 3).
This result also reﬂects that even in a low privacy regime, noise has a signiﬁcant impact throughout training. [Figure] 5 ) , refer to Appendix F.
3. I NVERSELY PROPORTIONAL SCALING RULE FOR ADJUSTING THE CLIPPING BOUND AND MOMENTUM
Previous works commonly adopt ﬁrst order momentum in the optimization, because momentum can alleviate oscillation and accelerate gradient descent (Sutskever et al., 2013).
As a result, it is believed to reduce the number of iterations of training and therefore achieve less privacy loss.
However, for privacy-preserving training, momentum will also exaggerate the additive i.i.d.
Gaussian noise by incorporating current and all historical noise.
For instance, using the Pytorch (Paszke et al., 2019) implementation of SGD, the velocity update can be written as: v t +1 = µ · v t + g t +1 , where v, µ, g denote perturbed velocity, momentum and perturbed gradients, respectively.
Using the expression of one step noise in Equation 3 and denoting by ˆ v t the velocity after separating the noise, we have v t +1 − ˆ v t +
=
(1+ µ + µ
+ ... + µ t ) ·N (0 , C 2 · σ 2 I d ) .
After many iterations, the scalar approximates a geometric series, i.e. v t +1 − ˆ v t +1 ≈ 1 1 − µ · N (0 , C 2 · σ 2 I d ) .
Pulling the clipping bound C out and forming the noise as C 1 − µ · N (0 , σ 2 I d ) , we present an inversely proportional scaling rule for adjusting C and 1 − µ , i.e. with other hyperparameters ﬁxed, networks trained with same value of the ratio C 1 − µ perform similarly (see Figure 4).
Our conjecture is that the inversely proportional scaling rule ensures the same amount of injected noise.
It is worth noticing that the inversely proportional scaling rule is general and amenable to no freeze and random freeze.
Tuning hyperparameters in the context of privacy-perserving training has been observed to be brittle.
This rule helps reduce the tuning workload.
E XPERIMENTAL R ESULTS
Privacy budget According to Deﬁnition 1, we have that each iteration of training is ( ε, δ ) - differentially private with respect to a batch of training data, while shufﬂing and partitioning the dataset into batches implies that each iteration is ( O ( qε ) , qδ ) -differentially private with respect to the full dataset according to the privacy ampliﬁcation theorem (Balle et al., 2018;
Wang et al., 2019), where q = B/N , B is the batchsize and N is the size of dataset.
To track the cumulative privacy loss over multiple training epochs, we adopt R´enyi differential privacy (Mironov, 2017), which is more operationally convenient and quantitatively accurate.
Advantages of sparsity Projected DP-SGD induces a signiﬁcant additional computation cost by running the power method and projecting gradients into and out of subspace.
For the power method, the basic operation is WW (cid:124) V ,
W ∈ R
d × s denotes sample gradients, V ∈ R d × b denotes eigenvectors, the computational cost is O ( dbs ) .
Similarly, for projection V (cid:124) X ; X ∈ R d × n denotes original gradients, the computational cost is O ( dbn ) .
Applying random freeze, a random selection of rows of X are deleted, while corresponding rows of V, W can be removed as no information of gradient exits in that subspace. We note that b, s might also be able to be reduced. Overall, the computational cost is between O (1 − r ) and O ((1 − r ) 3 ) .
Another issue of projected DP-SGD is the memory footprint of V . Saving sparse V by random freeze can be achieved by storing non-zero values and indices of zeros. The cost of indexing is logarithmic of the number of parameters, consider that log 2 10 9 < 32 , we can decrease the memory footprint by removing a single 32 bit ﬂoat gradient. Communication overhead can similarly be reduced.
We note that random freeze uses the same mask during one training epoch, which could contain multiple groups of eigenvectors and communication rounds. Therefore, the cost of indexing is negligible: communication overhead and memory footprint are ˜ O (1 − r ) . Further, we deﬁne the total density as the total amount of non-zero gradients by random freeze over the total amount of gradients by the original dense representation to reﬂect these advantages of sparsity.
Our experiments are implemented in the Pytorch framework.
To compute the gradients of an individual example in a minibatch, which is required for gradient clipping, we use the BackPACK package (Dangel et al., 2020).
The privacy loss of multiple iterations has been tracked with Opacus.
We use the benchmark CIFAR10, which is to date standard in benchmarking DP learning.
Our code is available for download from http://anonymized.for.review/ .
To validate the reliability of random freeze, we conduct experiments on several SOTA networks cross different frameworks, including End-to-end CNN (Papernot et al., 2021); Handcrafted CNN (Tramer & Boneh, 2021) which incorporates ScatterNet (Oyallon et al., 2019) as feature extractor; Gradient
Embedding Perturbation (GEP) proposed by Yu et al.
(2021a) which injects noise in projected gradient; DP-Transfer Learning which adopts a pretrained network and replaces the linear classiﬁer layer, in particular we use SIMCLR v2 (Chen et al., 2020a)
pretrained on unlabeled ImageNet (Deng et al., 2009), which has been benchmarked by Tramer & Boneh (2021).
For a fair comparison, we run every experiment 5 times then compute the average of best accuracy and the standard error.
We do not tune the hyperparameters when adapting the SOTA works with random freeze, instead we adopt the optimal hyperparameters for DP-SGD without random freeze provided in the respective works.
Random freeze is applied as follows: if the optimal number of epochs from the previous work is set as e ∗ , we linearly ramp up the freeze rate r from r = 0 at epoch e = 0 to r = r ∗ at epoch e = e ∗ , i.e. r = r ∗ · e e ∗ − 1 .
These straightforward experiments allow us to demonstrate that random freeze is a safe add-on in a variety of methods.
We summarize the performance results in Table 1, and the corresponding total density by random freeze in Table 2.
We document the hyperparameters in Appendix D.
We ﬁnd that when the network is large, for instance End-to-end CNN has the most parameters among all frameworks, random freeze is able to improve the accuracy.
To demonstrate this, we further tune the End-to-end CNN.
We still adopt the best hyperparameters from the previous work, then ﬁrst adjust the clipping bound and momentum from ( C = 0 . 1 , µ = 0 . 9) to ( C = 1 , µ = 0) [Table] [Table]
[Table]
with random freeze
Similar to the previous experiment we also linearly ramp up the rate to but then we extend the training for 20 additional epochs at freeze rate , this help us to improve the performance further.
The result is summarized in Table 3.
Note that, after extending the training epochs, the noise multiplier σ has been increased accordingly to ensure the same privacy budget will be consumed.
As a result, more epochs without random freeze will decrease the performance.
Moreover, we achieve lower total density with higher freeze rate.
For hyperparameters, total density and accuracy as a function of privacy loss, refer to Appendix E.
D ISCUSSION AND C ONCLUSIONS
In this work we propose random freeze which is an axis-aligned random gradient dimension reduction method.
We provide a fundamental theoretical study on random freeze and investigate the trade-off between signal loss and perturbation moderation.
Although simple to implement, random freeze can be safely (without further tuning) applied across different frameworks and architectures.
Even in conjunction with other gradient space reduction methods performance is maintained.
With the sparse representation of the gradient update, random freeze can reduce the computational cost and memory footprint of projected DP-SGD.
Also, it is able to reduce the total amount of transferred data in federated learning or other collaborative learning frameworks.
Moreover, we signiﬁcantly improve the performance of SOTA End-to-end CNN using the random freeze strategy and inversely proportional scaling rule that we have proposed.
We note that the computational cost of the random freeze procedure is negligible.
Therefore we believe that random freeze can be incorporated as a general method in DP-SGD.
and expect random freeze to exhibit strong improvements as DP-SGD is adapted to larger and deeper networks in the future.
We also hope this work can shed light on gradient dimension reduction in DP-SGD and motivate further research in this direction.
A M EAN SQUARE ERROR OF GRADIENT IN DP-SGD
Theorem 1.
Assuming that the gradient is drawn from N ( ∇ w, Σ) centered at the true gradient ∇ w and with respect to the covariance matrix Σ whose trace goes linearly up with dimension d . The MSE of perturbed gradient ˜ g = g + ξ DP can be lower bounded by: [Equation]
Proof.
Assuming an oracle can tell us the true gradient ∇ w given a network with parameters w ∈ R d while the individual gradient g can be seen as drawn from the Gaussian distribution centered at ∇ w with respect to the covariance matrix Σ
i e g ∼ N ( ∇ w, Σ) . Denote ξ g as gradient deviation, i e. ξ g = g − ∇ w the perturbed gradient ˜ g in DP-SGD can be expressed as:
[Equation]
and the mean square error of perturbed gradient is:
[Equation]
Since ξ DP and ξ g are independent and plugging the deﬁnition of injected noise.
i.e. ξ DP ∼
N (0 , max( g (cid:124) g ) σ 2 I d ) , into Equation 9, we have:
[Equation]
where equality can be achieved when the network converges to stationary point, i.e. ∇ w → 0 .
Applying a spectral decomposition to Σ , we obtain Q (cid:124) Λ Q = Σ , where Q is an orthonormal basis and Λ is diagonal matrix of eigenvalues of Σ .
Denote by z a random vector drawn i.i.d.
from a normal distribution, we can further derive:
[Equation]
As random variable with respect to the same distribution leads to the same expectation, plugging
Equation 14 into Equation 13, we have:
[Equation]
Since z (cid:124) Λ z
= λ 1 z 21 + λ
z 22 + · · · + λ d z 2 p ; z i ∼ N (0 , 1) , we have:
[Equation]
So far either the exact PDF of a Gamma distribution summation can only be represented with (another) series of gamma distributions (Moschopoulos, 1985; Mathai, 1982), or its exact CDF has a series representation (Moschopoulos & Canada, 1984).
For the convenience of further derivation of the expected maximum, we use the Welch-Satterthwaite approximation which provides a certain
Gamma distribution:
[Equation]
Denote M n = max( x 1 , x 2 , ..., x n ) , where x i is drawn from Gamma (cid:16)
Tr 2 { Λ } 2Tr { Λ 2 } , Tr 2 { Λ } 2Tr { Λ 2 } (cid:17) .
While from the Cauchy-Schwarz inequality, we have: [Equation]
[Equation]
Denote k as Tr 2 { Λ } 2Tr { Λ 2 } , Figure 5 illustrates the PDF of Gamma distribution with parameters Gamma ( k, k ) [Equation] As k becomes large
[Equation] the distribution becomes more concentrated [Equation] and we have the following inequality:
[Equation]
Substituting inequality 24 back into Equation 17, we have: [Equation]
The proof is completed.
We note that Tr[Σ] is the trace of a d -dimensional covariance matrix.
In the extreme case that Σ = λI , this will scale linearly in d , while in the opposite extreme of a rank deﬁcient covariance matrix, the trace is constant in d .
Thus, in the former case
[Equation]
26) while in the latter we have
[Equation]
which leads to our conclusion that the lower bound on MSE is between linear and quadratic in d in practice.
B T HE CONVERGENCE RATE OF DP-SGD WITH RANDOM FREEZE
Lemma 1.
u, v ∈ R d are too arbitrary vectors, m is a { 0 , 1 } d random mask independent from u, v , and subject to (cid:80) m
= (1 − r ) d , where r denotes the freeze rate.
We have the following expectation:
[Equation]
Proof.
[Equation]
The proof is completed.
Corollary 1.1.
Following the notation of lemma 1, we have the expectation below:
[Equation]
Proof.
[Equation]
The proof is completed.
Lemma 2.
u ∈ R d is an arbitrary vector, m is a { 0 , 1 } d random mask independent from u and subject to (cid:80) m = (1 − r ) d , where r denotes the freeze rate. We have the following inequality:
[Equation]
Proof.
[Equation]
Equality is taken when r = 0 .
The proof is completed.
Theorem 2.
Assume G-Lipschitz smoothness of ∇ w such that ||∇ w t +1 − ∇ w t || ≤ G || w t +1 − w t
|| .
Consider an algorithm with clipping bound C , learning rate γ and choose a symmetric probability density distribution ˜ p ( · ) satisfying ˜ p t ( ξ t ) =
˜ p t ( − ξ t ) , ∀ ξ t ∈ R d .
Then ∃ κ ≥ 1 − r such that:
[Equation]
Proof.
Denote the injected noise at each iteration ξ DP := 1 B z , where z is drawn from
N (0 , C 2 σ 2 I d ) , follow from the smoothness assumption, we have:
[Equation]
[Equation]
Taking expectations on both sides and rearranging, we have:
[Equation]
Now focusing on the l
[Equation]
h
[Equation]
s., we have:
[Equation]
where we have deﬁned b t :
= (cid:82) (cid:104)∇ w (cid:48) t , g (cid:48)
[Equation]
[Equation]
) (cid:105) ( p (cid:48) t ( ξ (cid:48) t ) − ˜ p
[Equation]
t ( ξ (cid:48) t )) dξ (cid:48) t .
Equation 52 is achieved because ξ t and m are independent.
We note that for any given m , ξ (cid:48) t is still independent deviation with zero mean and ˜ p (cid:48) t ( ξ (cid:48) t ) =
˜ p (cid:48) t ( − ξ (cid:48) t ) since projection of symmetric distribution to subspace is symmetric.
According to Theorem 2 from Chen et al.
(2020b), for a given m we have the following inequality:
[Equation]
Back to Inequality 47 and considering the overall T steps, we have:
[Equation]
where we have deﬁned ∆ L :=
E [ L 1 − min w L ( w )] , ∆ C :=
GC 2 2 , ∆ DP := C 2 σ 2 dG 2 B 2 .
Plugging
Inequality 54 into Equation 53 and combining with Inequality 56:
[Equation]
where we have deﬁned h ( ∇ w (cid:48) t )
=
min( (cid:107)∇ w
[Equation]
t (cid:107) , 3 C 4 ) .
Now we look at the l.h.s.
and consider each step with respect to the following cases:
[Equation]
case 2 : (cid:107)∇ w t (cid:107) > 34 C , consider two events
A 1 : (cid:107)∇ w (cid:48) t
[Equation]
≤ 34 C ;
A 2 : (cid:107)∇ w
[Equation]
t (cid:107) > 34 C then we have:
[Equation]
from which we conclude that:
[Equation]
The inequality above implies that ∃ κ ≥ 1 − r such that:
[Equation]
[Equation] We note that κ = 1 if r = 0 . Plugging equation 68 into inequality 57 we obtain: [Equation]
The proof is completed.
It is worthwhile to note that the l.h.s. can also be written as the general form below and the inequality still holds: [Equation]
Algorithm 2 describes ranked freeze, the performance is recorded in Table 4.
Note that for aggregated gradient estimation we also inject noise to gradient estimation g e of frozen coordinates. If not, the coordinates get frozen at the ﬁrst iteration will receive aggregated gradient estimation as 0 and never get updated for all the remaining iterations, which will degrade the network.
Adding noise to frozen coordinates can give these coordinates a chance to be ranked in higher positions, while in turn the coordinates in updating but with low magnitude of true gradient may get frozen in the next iteration.
Input: Initialized parameters: w 0 ; Loss function: (cid:96) ; Iterations per epoch: T ; Epochs E ; Freeze rate: r ∗ ; Cooling time: e ∗ ; Clipping bound: C ; Momentum: µ ; Learning rate γ .
for e = 0...E-1 do g e = { 0 } d ; r ( e ) =
r ∗ · min( e e ∗ − 1 , 1) ; if e is 0 then m = { 1 } d else
Sort the indices [1,...,d] with respect to corresponding aggregated gradient of the last epoch g e − 1 in ascending order then set the ﬁrst d · r ( e ) positions in mask m to 0 and the rest to 1; endfor t = 0...T-1 do
For each x i in minibatch of size B , compute g t ( x i ) = ∇ (cid:96) ( w t , x i ) ;
Partially zero out each gradient g t ( x i ) = m (cid:12) g t ( x i ) ;
Clip each individual gradient ¯ g t ( x i ) =
g t ( x i ) · min(1 ,
C (cid:107) g t ( x i ) (cid:107) 2 ) ;
Add noise ˜ g t = 1 B ( (cid:80) i ¯ g t ( x i ) + m (cid:12) N (0 , C 2 σ 2 I d )) ;
Update v t +1 = µ · v t + g t , w t +1 = w t − γv t +1 , g e = g e + ¯ g t + N (0 , C 2 σ 2 I d )) ; end end
[Table]
trained hyperparameters recorded in table D H YPERPARAMETERS FOR T ABLE The δ term is 10 − 5 for all settings.
There exist two ways of determining the injected noise scale per iteration: 1. Set the privacy budget and noise multiplier then stop the training when the privacy budget is fully consumed; 2. Set the privacy budget and iterations of training then compute the noise multiplier so that the privacy budget will be fully consumed at the last iteration. In this work we refer to the second method when talking about hyperparameters.
Hyperparameters are listed separately for each framework, see Table 5, 7, 6, 8.
There is a minor difference for hyperparameters adopted from Tramer & Boneh (2021).
For instance, they set batchsize=1024 for End-to-end CNN, which leads to non-exact batch partitions.
We set batchsize=1000 to avoid that and achieve equally good or slightly better baseline performance.
Additonally, their hyperparameters of End-to-end CNN are tuned for ε = 3 and then applied to ε = 7 . 53 , we ﬁnd out that based on their hyperparameters for batchsize, clipping bound and momentum, setting the number of epochs to 80 can achieve the best accuracy.
[Table]
[Table]
[Table]
[Table] [Table] E I MPROVING THE ACCURACY OF E ND - TO - END CNN Table 9 includes the hyperparameters and total density of ajusted End-to-end CNN with random freeze.
Figure 6 shows the accuracy as a function of privacy loss [Table] F I NVERSELY PROPORTIONAL SCALING RULE We include here an additional experiments of inversely proportional scaling rule on low privacy budget (see Figure 7).
[Figure] rule can achieve good performance, otherwise the network performance is degraded. Combining with Figure 4, we see this rule is relatively general and does not dependent on privacy level. G A NUANCE OF RANDOM FREEZE
Gradient clipping is usually the ﬁrst step we need to do after backpropagation in DP-SGD. However, random freeze ﬁrst zeros out part of the gradient then clips with respect to the respective norm. We note that the strategy of ﬁrst clipping then freezing appears in a recent work (Zhang et al., 2021) as a baseline for comparison, but shows no improvement in performance. We name this strategy posterior random freeze for convenience.
For DP-SGD with posterior random freeze, denote the gradient
[Equation]
) , assume G-Lipschitz smoothness, we have:
[Equation]
Similarly for DP-SGP with random freeze [Equation] we have: [Equation]
We see both methods moderate the perturbation, while E [ [Equation] ] but it is complicated to compare E [ (cid:104)∇ w t , ¯ ¯ g t (cid:105) ] and E [ (cid:104)∇ w t , ˆ g t (cid:105) ] , we leave this for future work.
From an operational view, denote ¯¯ g t,i = m (cid:12) g t,i · min(1 , C (cid:107) g t,i (cid:107) ) and ˆ g t,i = m (cid:12) g t,i · min(1 , C (cid:107) m (cid:12) g t,i (cid:107) ) , we see that (cid:104) ¯¯
g t,i , ˆ g t,i (cid:105) / (cid:107) ¯¯ g t,i (cid:107)(cid:107) ˆ g t,i (cid:107) = 1 and (cid:107) ¯¯ g t,i (cid:107) ≤ (cid:107) ˆ g t,i (cid:107) ≤ (cid:107) g t,i (cid:107) , i.e. random freeze and posterior random freeze contain the same direction information of individual gradient, but random freeze retains more distance information. Therefore, we suppose random freeze to perform better than posterior random freeze.