Losing Less: A Loss for Differentially Private Deep Learning

Differentially Private Stochastic Gradient Descent , DP-SGD , is the canonical approach to training deep neural networks with guarantees of Differential Privacy ( DP ) . However , the modifications DP-SGD introduces to vanilla gradient descent negatively impact the accuracy of deep neural networks . In this paper , we are the first to observe that some of this performance can be recovered when training with a loss tailored to DP-SGD ; we challenge cross-entropy as the de facto loss for deep learning with DP . Specifically , we introduce a loss combining three terms : the summed squared error , the focal loss , and a regularization penalty . The first term encourages learning with faster convergence . The second term emphasizes hard-to-learn examples in the later stages of training . Both are beneficial because the privacy cost of learning increases with every step of DP-SGD . The third term helps control the sensitivity of learning , decreasing the bias introduced by gradient clipping in DP-SGD . Using our loss function , we achieve new stateof-the-art tradeoffs between privacy and accuracy on MNIST , FashionMNIST , and CIFAR10 . Most importantly , we improve the accuracy of DP-SGD on CIFAR10 by 4 % for a DP guarantee of ε = 3 . 1 INTRODUCTION . Releasing machine learning ( ML ) models may risk the privacy of training data as ML models unintentionally leak information about the training data ; this is due to limitations of learning , including overfitting ( Song & Shmatikov , 2019 ) and data memorisation ( Fredrikson et al. , 2015 ; Carlini et al. , 2019 ) . The framework of Differential Privacy ( DP ) by Dwork et al . ( 2014 ) is the gold standard for formalising privacy guarantees . When training ML models , a DP training algorithm provides guarantees bounding the leakage of private data that may occur during training . This requires that one bound the influence of individual training examples on the output of the algorithm ( e.g . parameters or gradients of the model ) . In addition to these strong theoretical guarantees , DP models are empirically resistant to various attacks – membership inference attacks ( Shokri et al. , 2017 ; Rahman et al. , 2018 ) , training data extraction ( Carlini et al. , 2019 ) and data poisoning attack ( Ma et al. , 2019 ) . Differentially Private Stochastic Gradient Descent , DP-SGD , trains a ML model under the framework of DP by ( 1 ) clipping per-example gradients to a fixed norm to bound their sensitivity and ( 2 ) adding noise to these clipped gradients before they are applied to update the model parameters ( Abadi et al. , 2016 ) . When done separately and at the level of a minibatch , clipping ( Menon et al. , 2019 ; Zhang et al. , 2019 ) or noising ( Foret et al. , 2021 ) effectively regularize learning because they respectively control the dynamics of iterates and smoothen the loss landscape . However , gradient clipping and noising are detrimental to model performance in DP-SGD because they are applied in combination and at the granularity of individual training examples . In addition to this , the accuracy of DP-SGD is negatively affected by the limited number of iterations that can be computed : each iteration of DP-SGD increases the privacy budget expended by learning . Recent works have explored bounded activation functions ( Papernot et al. , 2021 ) , publicly available feature extractors and data ( Tramèr & Boneh , 2021 ) , randomised smoothing ( Wang et al. , 2021 ) and gradient embedding ( Yu et al. , 2021 ) as a means to improve the tradeoffs between privacy and accuracy of DP-SGD . However , training deep neural networks with strong DP guarantees in DPSGD still comes at a significant cost in accuracy for datasets like CIFAR10 . In this paper , we show that one key aspect of formulating the optimization problem solved in learning is left out of studies seeking to improve deep learning with DP : the loss function . All existing implementations of deep learning with DP we are aware of optimise cross-entropy ( which performs well in the non-privacy-preserving setting with SGD ) . However , we first observe that optimising cross-entropy with DP-SGD leads to exploding model weights , layer pre-activations and logit values . This is true even if the activation functions themselves are bounded ( Papernot et al. , 2021 ) . This phenomenon makes it difficult to control the sensitivity of the learning algorithm at a minimal impact to its correctness . Indeed , clipping and noising large gradients in DP-SGD introduces an information loss and additional biases . More precisely , the direction of a minibatch of clipped per-example gradients in DP-SGD is not necessarily aligned with the direction of the original gradients in SGD . Furthermore , we also note that the slow training convergence of cross-entropy ( Allen-Zhu et al. , 2019 ) negatively impacts the performance of DP-SGD for which a limited number of iterations are required to achieve strong privacy guarantees . In this paper , to tackle these two limitations of the cross-entropy loss , we propose to tailor the loss function to the specifics of DP-SGD . We design a novel loss function that takes into account sensitivity , training convergence and the order in which examples are learned with the overarching goal of improving the tradeoffs between privacy and accuracy of DP-SGD . We achieve these objectives with a novel loss function that combines the Sum Squared Error ( SSE ) between the model logits and ground-truth labels , the focal loss ( Lin et al. , 2017 ) , and a regularisation penalty on pre-activations . The first two penalties impose a curriculum structure in the training to improve privacy-accuracy tradeoffs of DP-SGD by exploiting the generalisation and fast convergence speed ( Wang et al. , 2019 ; Guo et al. , 2018 ) of curriculum learning ( Bengio et al. , 2009 ) . We start training with SSE which has faster convergence ( Allen-Zhu et al. , 2019 ) and smaller gradients than cross-entropy . Then , we gradually shift toward emphasising hard-to-learn examples using the focal loss : this down-weights gradients assigned to examples that are already learned correctly to focus more on hard-to-learn and misclassified examples . Finally , our regularisation penalty prevents the weights from exploding to further reduce the magnitude of per-example gradients prior to clipping . Implementation-wise , our new loss function can be easily integrated within existing implementations of deep learning with DP ; it requires a single line change to edit the loss function being optimized . In summary , our main contributions are as follows : • We are the first to investigate the loss function in the context of deep learning with DP . We analyse the effect of the loss function on the sensitivity of DP models by providing extensive analysis of the norm of activations , weights and gradients in Section 4 . • We propose a novel loss function tailored to deep learning with DP . In Section 3 and Section 4 , we analytically and experimentally connect the superior performance of our proposed loss function to the gradient norm and direction , smoothness of the loss surface , and convergence of DP-SGD . Our proposed loss function better controls the sensitivity of the learning algorithm by better preconditioning gradients to being clipped and noised . • We show in Section 4 that our proposed loss function is compatible with other DP-SGD improvement strategies , and more importantly allows us to establish new state-of-the-art tradeoffs between privacy and accuracy on several key benchmarks for deep learning with DP , including CIFAR10 . Finally , we perform an ablation study to analyse the effect of each component of our proposed loss function on DP learning . 2 DIFFERENTIAL PRIVACY , DP-SGD AND STATE-OF-THE-ART APPROACHES . Trained machine learning models memorise and leak information about their training data ( Shokri et al. , 2017 ; Rahman et al. , 2018 ; Song & Shmatikov , 2019 ; Fredrikson et al. , 2015 ) . Differential privacy ( Dwork et al. , 2014 ) is the established gold standard to reason about the privacy guarantees of learning algorithms . An algorithm is differentially private if its outputs are statistically indistinguishable on neighbouring datasets . More formally , a randomised learning algorithm A is ( ε , δ ) differentially private if the following holds for any two neighbouring datasets D and D′ that differ only in one record and all S ∈ Range ( A ) : Pr [ A ( D ) ∈ S ] ≤ eεPr [ A ( D′ ) ∈ S ] + δ . ( 1 ) The privacy budget ε upper bounds the privacy leakage in the worst possible case : the smaller the ε , the tighter the upper bounds ( or stronger privacy guarantee ) . Note that the factor δ is very small ( generally it is chosen to be in the order of the inverse of the dataset size ) . In order to train ML models with DP , we can add randomness to the learning algorithm in three ways : output perturbation ( Wu et al. , 2017 ; Zhang et al. , 2017 ) , objective perturbation ( Chaudhuri et al. , 2011 ; Iyengar et al. , 2019 ) or gradient perturbation ( Bassily et al. , 2014 ; Abadi et al. , 2016 ) . Among these approaches , differentially private stochastic gradient , DP-SGD of Abadi et al . ( 2016 ) , has established itself as the de facto strategy because of its versatility . DP-SGD perturbs gradients computed at each step of stochastic gradient descent using the Gaussian mechanism to achieve competitive tradeoffs between privacy and accuracy ( Yu et al. , 2020 ; Tramèr & Boneh , 2021 ) . Next , we describe the process of training a classifier using DP-SGD on a private dataset . Let X = { xi|i = 1 , ... , N } be the training set containing N private examples xi , and Y = { yi|i = 1 , ... , N } be the ground-truth label set where each yi is a D-dimension one-hot encoded vector . We consider a D-class classifier containing M layers parameterised with weights W = { Wm|m = 1 , ... , M } . In each DP-SGD iteration , similarly to the non-private SGD one , the classifier receives each training example xi from a minibatch containing L data points that are sampled randomly from X . The activation of each intermediate layerm of size dm , ami = f ( h m i ) ∈ R1×dm , is computed by applying a non-linear function , f ( · ) , on the pre-activation hmi = Wma m−1 i . The pre-activation is a linear combination of the weight of that layer , Wm , and the activation of the previous layer , am−1i . The last layer outputs the logit values associated with each class without any activation function as aMi = hMi = W MhM−1i ∈ R1×D . The cross-entropy loss , LCE , for each training example xi is computed as LCE = − D∑ d=1 y ( i , d ) log ( p ( i , d ) ) , where p ( i , d ) = ea M ( i , d ) ∑D d′=1 e aM ( i , d′ ) . ( 2 ) The probability that xi belongs to d-th class , p ( i , d ) ∈ [ 0 , 1 ] , is computed by applying a Softmax function on the logit value of its corresponding class aM ( i , d ) . The DP-SGD optimiser computes per-example gradient of LCE with respect toW , as opposed to per minibatch gradient in the SGD optimiser , to bound the influence of each individual training example on the output of the learning algorithm ( i.e . weight gradients ) . The gradients , Gm , for each training example xi are computed from the last layer and back-propagated until the first layer as : GMi = ( pi − yi ) TaM−1i , G M−1 i = ( ( pi − yi ) WM ) T aM−2i , GM−2i = f ′ ( ( pi − yi ) WMWM−1 ) TaM−3i . ( 3 ) One can continue this gradient computation until the first layer . All the per-example gradients are concatenated as Gi = [ G1i ; ... ; G M i ] , where ; denotes the concatenation operation . As there is no a priori bound on Gi , the l2 norm , ‖ · ‖2 , of each Gi are artificially clipped by C to bound the influence of each xi on the final gradients , G : G = 1 L ( L∑ i=1 Gi +N ( 0 , σ2C2I ) ) where Gi ← Gi ·min ( 1 , C ‖Gi‖2 ) . ( 4 ) Finally , the classifier weights are updated through the average of the clipped and noisy ( scaled by the noise multiplier σ ) per-example gradients of a minibatch . The accuracy of classifiers trained with DP-SGD is lower than that of classifiers trained by nonprivate SGD . Recently , researchers investigated the choice of f ( · ) and data representation for improving the accuracy of the classifier trained by DP-SGD . Papernot et al . ( 2021 ) demonstrated that exploiting a bounded family of activation functions instead of the more commonly employed unbounded ReLU activation for the choice of f ( · ) can decrease the bias introduced by DP-SGD and improve the tradeoffs between privacy and accuracy of DP-SGD . Their approach though does not fully bridge the gap between non-private and private learning for datasets like CIFAR10 . Tramèr & Boneh ( 2021 ) proposed to train the classifier on a representation of data outputted by a public ScatterNet feature extractor as opposed to using the raw pixels xi . This however requires access to public data in addition to the private dataset .

Summary: The paper proposes a tailored loss for DP-SGD, which includes summed squared error, the focal loss, and a regularization penalty. The summed squared error is for fast convergence at initial stage. The focal loss is used for identify hard samples. The regularization penalty is used for reducing the gradient/weight norm and avoid explosion. For each component of the tailored loss, the paper has empirical/theoretical evidence to argue the necessity. It is a good try to improve the performance of DP-SGD from amending the loss.