Losing Less: A Loss for Differentially Private Deep Learning

Differentially Private Stochastic Gradient Descent , DP-SGD , is the canonical approach to training deep neural networks with guarantees of Differential Privacy ( DP ) . However , the modifications DP-SGD introduces to vanilla gradient descent negatively impact the accuracy of deep neural networks . In this paper , we are the first to observe that some of this performance can be recovered when training with a loss tailored to DP-SGD ; we challenge cross-entropy as the de facto loss for deep learning with DP . Specifically , we introduce a loss combining three terms : the summed squared error , the focal loss , and a regularization penalty . The first term encourages learning with faster convergence . The second term emphasizes hard-to-learn examples in the later stages of training . Both are beneficial because the privacy cost of learning increases with every step of DP-SGD . The third term helps control the sensitivity of learning , decreasing the bias introduced by gradient clipping in DP-SGD . Using our loss function , we achieve new stateof-the-art tradeoffs between privacy and accuracy on MNIST , FashionMNIST , and CIFAR10 . Most importantly , we improve the accuracy of DP-SGD on CIFAR10 by 4 % for a DP guarantee of ε = 3 . 1 INTRODUCTION . Releasing machine learning ( ML ) models may risk the privacy of training data as ML models unintentionally leak information about the training data ; this is due to limitations of learning , including overfitting ( Song & Shmatikov , 2019 ) and data memorisation ( Fredrikson et al. , 2015 ; Carlini et al. , 2019 ) . The framework of Differential Privacy ( DP ) by Dwork et al . ( 2014 ) is the gold standard for formalising privacy guarantees . When training ML models , a DP training algorithm provides guarantees bounding the leakage of private data that may occur during training . This requires that one bound the influence of individual training examples on the output of the algorithm ( e.g . parameters or gradients of the model ) . In addition to these strong theoretical guarantees , DP models are empirically resistant to various attacks – membership inference attacks ( Shokri et al. , 2017 ; Rahman et al. , 2018 ) , training data extraction ( Carlini et al. , 2019 ) and data poisoning attack ( Ma et al. , 2019 ) . Differentially Private Stochastic Gradient Descent , DP-SGD , trains a ML model under the framework of DP by ( 1 ) clipping per-example gradients to a fixed norm to bound their sensitivity and ( 2 ) adding noise to these clipped gradients before they are applied to update the model parameters ( Abadi et al. , 2016 ) . When done separately and at the level of a minibatch , clipping ( Menon et al. , 2019 ; Zhang et al. , 2019 ) or noising ( Foret et al. , 2021 ) effectively regularize learning because they respectively control the dynamics of iterates and smoothen the loss landscape . However , gradient clipping and noising are detrimental to model performance in DP-SGD because they are applied in combination and at the granularity of individual training examples . In addition to this , the accuracy of DP-SGD is negatively affected by the limited number of iterations that can be computed : each iteration of DP-SGD increases the privacy budget expended by learning . Recent works have explored bounded activation functions ( Papernot et al. , 2021 ) , publicly available feature extractors and data ( Tramèr & Boneh , 2021 ) , randomised smoothing ( Wang et al. , 2021 ) and gradient embedding ( Yu et al. , 2021 ) as a means to improve the tradeoffs between privacy and accuracy of DP-SGD . However , training deep neural networks with strong DP guarantees in DPSGD still comes at a significant cost in accuracy for datasets like CIFAR10 . In this paper , we show that one key aspect of formulating the optimization problem solved in learning is left out of studies seeking to improve deep learning with DP : the loss function . All existing implementations of deep learning with DP we are aware of optimise cross-entropy ( which performs well in the non-privacy-preserving setting with SGD ) . However , we first observe that optimising cross-entropy with DP-SGD leads to exploding model weights , layer pre-activations and logit values . This is true even if the activation functions themselves are bounded ( Papernot et al. , 2021 ) . This phenomenon makes it difficult to control the sensitivity of the learning algorithm at a minimal impact to its correctness . Indeed , clipping and noising large gradients in DP-SGD introduces an information loss and additional biases . More precisely , the direction of a minibatch of clipped per-example gradients in DP-SGD is not necessarily aligned with the direction of the original gradients in SGD . Furthermore , we also note that the slow training convergence of cross-entropy ( Allen-Zhu et al. , 2019 ) negatively impacts the performance of DP-SGD for which a limited number of iterations are required to achieve strong privacy guarantees . In this paper , to tackle these two limitations of the cross-entropy loss , we propose to tailor the loss function to the specifics of DP-SGD . We design a novel loss function that takes into account sensitivity , training convergence and the order in which examples are learned with the overarching goal of improving the tradeoffs between privacy and accuracy of DP-SGD . We achieve these objectives with a novel loss function that combines the Sum Squared Error ( SSE ) between the model logits and ground-truth labels , the focal loss ( Lin et al. , 2017 ) , and a regularisation penalty on pre-activations . The first two penalties impose a curriculum structure in the training to improve privacy-accuracy tradeoffs of DP-SGD by exploiting the generalisation and fast convergence speed ( Wang et al. , 2019 ; Guo et al. , 2018 ) of curriculum learning ( Bengio et al. , 2009 ) . We start training with SSE which has faster convergence ( Allen-Zhu et al. , 2019 ) and smaller gradients than cross-entropy . Then , we gradually shift toward emphasising hard-to-learn examples using the focal loss : this down-weights gradients assigned to examples that are already learned correctly to focus more on hard-to-learn and misclassified examples . Finally , our regularisation penalty prevents the weights from exploding to further reduce the magnitude of per-example gradients prior to clipping . Implementation-wise , our new loss function can be easily integrated within existing implementations of deep learning with DP ; it requires a single line change to edit the loss function being optimized . In summary , our main contributions are as follows : • We are the first to investigate the loss function in the context of deep learning with DP . We analyse the effect of the loss function on the sensitivity of DP models by providing extensive analysis of the norm of activations , weights and gradients in Section 4 . • We propose a novel loss function tailored to deep learning with DP . In Section 3 and Section 4 , we analytically and experimentally connect the superior performance of our proposed loss function to the gradient norm and direction , smoothness of the loss surface , and convergence of DP-SGD . Our proposed loss function better controls the sensitivity of the learning algorithm by better preconditioning gradients to being clipped and noised . • We show in Section 4 that our proposed loss function is compatible with other DP-SGD improvement strategies , and more importantly allows us to establish new state-of-the-art tradeoffs between privacy and accuracy on several key benchmarks for deep learning with DP , including CIFAR10 . Finally , we perform an ablation study to analyse the effect of each component of our proposed loss function on DP learning . 2 DIFFERENTIAL PRIVACY , DP-SGD AND STATE-OF-THE-ART APPROACHES . Trained machine learning models memorise and leak information about their training data ( Shokri et al. , 2017 ; Rahman et al. , 2018 ; Song & Shmatikov , 2019 ; Fredrikson et al. , 2015 ) . Differential privacy ( Dwork et al. , 2014 ) is the established gold standard to reason about the privacy guarantees of learning algorithms . An algorithm is differentially private if its outputs are statistically indistinguishable on neighbouring datasets . More formally , a randomised learning algorithm A is ( ε , δ ) differentially private if the following holds for any two neighbouring datasets D and D′ that differ only in one record and all S ∈ Range ( A ) : Pr [ A ( D ) ∈ S ] ≤ eεPr [ A ( D′ ) ∈ S ] + δ . ( 1 ) The privacy budget ε upper bounds the privacy leakage in the worst possible case : the smaller the ε , the tighter the upper bounds ( or stronger privacy guarantee ) . Note that the factor δ is very small ( generally it is chosen to be in the order of the inverse of the dataset size ) . In order to train ML models with DP , we can add randomness to the learning algorithm in three ways : output perturbation ( Wu et al. , 2017 ; Zhang et al. , 2017 ) , objective perturbation ( Chaudhuri et al. , 2011 ; Iyengar et al. , 2019 ) or gradient perturbation ( Bassily et al. , 2014 ; Abadi et al. , 2016 ) . Among these approaches , differentially private stochastic gradient , DP-SGD of Abadi et al . ( 2016 ) , has established itself as the de facto strategy because of its versatility . DP-SGD perturbs gradients computed at each step of stochastic gradient descent using the Gaussian mechanism to achieve competitive tradeoffs between privacy and accuracy ( Yu et al. , 2020 ; Tramèr & Boneh , 2021 ) . Next , we describe the process of training a classifier using DP-SGD on a private dataset . Let X = { xi|i = 1 , ... , N } be the training set containing N private examples xi , and Y = { yi|i = 1 , ... , N } be the ground-truth label set where each yi is a D-dimension one-hot encoded vector . We consider a D-class classifier containing M layers parameterised with weights W = { Wm|m = 1 , ... , M } . In each DP-SGD iteration , similarly to the non-private SGD one , the classifier receives each training example xi from a minibatch containing L data points that are sampled randomly from X . The activation of each intermediate layerm of size dm , ami = f ( h m i ) ∈ R1×dm , is computed by applying a non-linear function , f ( · ) , on the pre-activation hmi = Wma m−1 i . The pre-activation is a linear combination of the weight of that layer , Wm , and the activation of the previous layer , am−1i . The last layer outputs the logit values associated with each class without any activation function as aMi = hMi = W MhM−1i ∈ R1×D . The cross-entropy loss , LCE , for each training example xi is computed as LCE = − D∑ d=1 y ( i , d ) log ( p ( i , d ) ) , where p ( i , d ) = ea M ( i , d ) ∑D d′=1 e aM ( i , d′ ) . ( 2 ) The probability that xi belongs to d-th class , p ( i , d ) ∈ [ 0 , 1 ] , is computed by applying a Softmax function on the logit value of its corresponding class aM ( i , d ) . The DP-SGD optimiser computes per-example gradient of LCE with respect toW , as opposed to per minibatch gradient in the SGD optimiser , to bound the influence of each individual training example on the output of the learning algorithm ( i.e . weight gradients ) . The gradients , Gm , for each training example xi are computed from the last layer and back-propagated until the first layer as : GMi = ( pi − yi ) TaM−1i , G M−1 i = ( ( pi − yi ) WM ) T aM−2i , GM−2i = f ′ ( ( pi − yi ) WMWM−1 ) TaM−3i . ( 3 ) One can continue this gradient computation until the first layer . All the per-example gradients are concatenated as Gi = [ G1i ; ... ; G M i ] , where ; denotes the concatenation operation . As there is no a priori bound on Gi , the l2 norm , ‖ · ‖2 , of each Gi are artificially clipped by C to bound the influence of each xi on the final gradients , G : G = 1 L ( L∑ i=1 Gi +N ( 0 , σ2C2I ) ) where Gi ← Gi ·min ( 1 , C ‖Gi‖2 ) . ( 4 ) Finally , the classifier weights are updated through the average of the clipped and noisy ( scaled by the noise multiplier σ ) per-example gradients of a minibatch . The accuracy of classifiers trained with DP-SGD is lower than that of classifiers trained by nonprivate SGD . Recently , researchers investigated the choice of f ( · ) and data representation for improving the accuracy of the classifier trained by DP-SGD . Papernot et al . ( 2021 ) demonstrated that exploiting a bounded family of activation functions instead of the more commonly employed unbounded ReLU activation for the choice of f ( · ) can decrease the bias introduced by DP-SGD and improve the tradeoffs between privacy and accuracy of DP-SGD . Their approach though does not fully bridge the gap between non-private and private learning for datasets like CIFAR10 . Tramèr & Boneh ( 2021 ) proposed to train the classifier on a representation of data outputted by a public ScatterNet feature extractor as opposed to using the raw pixels xi . This however requires access to public data in addition to the private dataset .

Summary: The paper proposes a new loss function to improve the performance of neural network models trained by DP-SGD. The new loss function is a weighted average of the sum of squared error, the focal loss, and a penalty on the squared norm of the pre-activation output of different layers. The new loss achieves state-of-the-art accuracy on the CIFAR-10, FashionMNIST, and MNIST datasets. It is also shown that the new loss can reduce the bias of gradient clipping and encourage learning on hard examples.