Two Birds, One Stone:  Achieving both Differential Privacy and Certified Robustness for Pre-trained Classifiers via Input Perturbation

1 INTRODUCTION . Deep learning with pre-trained classifiers are increasingly powerful for solving difficult machinelearning tasks in the real-world , including image classification ( Krizhevsky et al. , 2012 ; Simon et al. , 2016 ) and natural language processing ( Vaswani et al. , 2017 ; Devlin et al. , 2018 ) . However , deep learning models with pre-trained classifiers are subject to adversarial examples attacks ( Szegedy et al. , 2013 ; Bruna et al. , 2013 ; Goodfellow et al. , 2014 ) which apply small perturbations on inputs to cause the model to misclassify . To solve this challenge , certified robustness ( Li et al. , 2018 ) for pre-trained classifiers is proposed by ( Salman et al. , 2020 ) as a provable defense approach , where a denoiser , e.g. , autoencoder ( AE ) ( Hinton & Zemel , 1994 ) , is trained on the data perturbed with Gaussian noise and aims to reconstruct denoised data . Once denoiser finishes training , it takes the Gaussian perturbed data for denoising reconstruction and feeds the denoised data to the pretrained classifier . Randomized smoothing is then applied and provides certified robustness without retraining the large pre-trained model . On the other hand , the training of denoiser requires training data that can include sensitive information , e.g. , clinical records , financial records , user profiles , etc . Several works have shown that attackers can infer private information from training data through trained models ( Fredrikson et al. , 2015 ; Wang et al. , 2015 ; Shokri et al. , 2017 ) . A popular and powerful technique to address this issue in deep learning is differential privacy ( DP ) ( Dwork et al. , 2006 ; Dwork , 2011 ; Dwork et al. , 2014 ) , which has been adopted in a large volume of works to provide a rigorous protection from leaking private information contained in training data . Gaussian Mechanism ( GM ) ( Dwork et al. , 2014 ) is a basic technique to achieve DP by injecting isotropic Gaussian perturbations to a computation output . Depending on where to inject the perturbations , existing works on machine learning with DP can be mainly categorized into : input perturbation ( Fukuchi et al. , 2017 ; Kang et al. , 2020a ; b ) , output perturbation ( Zhang et al. , 2017 ) , gradient perturbation ( Song et al. , 2013 ; Bassily et al. , 2014 ; Shokri & Shmatikov , 2015 ; Abadi et al. , 2016 ; 2017 ; Wang et al. , 2017 ; Lee & Kifer , 2018 ; Yu et al. , 2019 ) , objective perturbation ( Kifer et al. , 2012 ; Zhang et al. , 2012 ; Phan et al. , 2016 ; 2017 ; Iyengar et al. , 2019 ) , and noisy labelling ( Papernot et al. , 2016 ; 2018 ) . Therefore , a straightforward way to prevent these two critical risks , i.e. , adversarial examples and privacy leakage , is independently applying certified robustness and DP to models with pre-trained classifiers . A few works ( Phan et al. , 2019 ; 2020 ) follow this idea and simultaneously achieve both DP and certified robustness . However , we observe that the randomized smoothing for certified robustness requires that the input data is perturbed with Gaussian noise during training . This Gaussian perturbation brings randomization to the training process and could have been utilized to provide a certain level of DP guarantee . Independently applying certified robustness and DP fails to exploit this connection and incurs unnecessary additional randomization during training , which leads to the degradation of model utility . Directly analyzing the DP guarantee through input perturbation methods ( Fukuchi et al. , 2017 ; Kang et al. , 2020a ; b ) is nontrivial in deep learning , because these methods pose strict constraints on loss function , which are not satisfied by deep learning models . An alternative way is to transform the input perturbation into gradient perturbation and then analyze the DP guarantee it provides . However , existing work ( Kang et al. , 2020a ) gives strong assumption and simply regards the transformed gradient perturbation as isotropic Gaussian perturbation , and fails to recognize that in most deep learning models , this transformed gradient perturbation follows multivariate Gaussian distribution . In this paper , we propose a novel framework TransDenoiser ( Figure 1 ) to simultaneously achieve certified robustness and DP for models with pre-trained classifiers . TransDenoiser has a similar architecture as ( Salman et al. , 2020 ) by adding a denoiser before pre-trained classifier . Compared with ( Salman et al. , 2020 ) , TransDenoiser can provide similar level of certified robustness without retraining the pre-trained model , as well as guarantee DP for the training data . Compared with existing works that achieve both certified robustness and DP including SecureSGD ( Phan et al. , 2019 ) and StoBatch ( Phan et al. , 2020 ) , TransDenoiser 1 ) provides a tigher guarantee of DP by utilizing all the randomization during training including input and gradient perturbations , and 2 ) achieves more effective certified robustness by leveraging randomized smoothing on the input instead of noisy layers in the model . Contributions . Our key contributions are : 1 . We propose a novel framework TransDenoiser that trains a denoiser through both input and gradient perturbation for achieving DP and certified robustness simultaneously on deep learning models with pre-trained classifiers . The input perturbation for achieving certified robustness is utilized to achieve partial DP and additional gradient perturbation is used as necessary for the overall DP , ensuring an enhanced privacy and utility performance . 2 . We present an analytical tool that leverages Taylor expansion to transform input perturbation into gradient perturbation so that it can be quantified and composed with the explicit gradient perturbation for the DP guarantee . We propose a Multivariate Gaussian Mechanism ( MGM ) to analyze DP of the multivariate Gaussian perturbation and prove that MGM is a generalization of Heterogeneous Gaussian Mechanism ( Phan et al. , 2019 ) . 3 . Observing that the transformed gradient perturbation itself can not satisfy the DP guarantee requirement in some scenario , we add additional perturbation following isotropic Gaussian distribution to the gradient , and propose Mixed Multivariate Gaussian Analysis ( MMGA ) to analyze the DP guarantee provided by transformed gradient perturbation and additional gradient perturbation . We also prove that MMGA can work with moments accountant ( Abadi et al. , 2016 ) to provide a tight bound on the privacy cost . 4 . We conduct extensive experiments on several benchmark datasets which demonstrate that TransDenoiser can 1 ) provide a significantly tighter bound on privacy cost with same utility performance , and 2 ) achieve similar level of certified robustness as other state-of-the-art works . 2 TRANSDENOISER . In this section , we will present our proposed framework TransDenoiser , which can be applied before any pre-trained classifier to guarantee certified robustness and DP without retraining the pre-trained model . A denoiser is trained to guarantee certified robustness of the final model via randomized smoothing or input perturbation on the input , where the training process introduces the input perturbation for better accuracy of the randomized smoothed model . We transform the input perturbation into equivalent gradient perturbation so that it can be quantified and composed with the explicit gradient perturbation for the DP guarantee . 2.1 DENOISER AND CERTIFIED ROBUSTNESS . As can be seen in Figure1 , TransDenoiser is a denoising AE trained on input data with Gaussian perturbation . Similar to ( Salman et al. , 2020 ) , this input Gaussian perturbation is used as randomized smoothing for certified robustness . Naively applying randomized smoothing on a pre-trained classifier without the denoiser gives very loose certification bounds because the pre-trained classifier is not trained to be robust to Gaussian perturbations of their input . The denoiser serves to “ remove ” this Gaussian perturbation and effectively reconstruct the input data like a pre-processing step before feeding input data into the pre-trained classifier while maintaining the benefit of certified robustness . The detailed proof of randomized smoothing can be found in ( Cohen et al. , 2019 ) , and we provide a brief proof in Appendix B . Different from ( Salman et al. , 2020 ) , given input data x ( i ) and perturbed data z ( i ) = x ( i ) + b ( i ) with b ( i ) ⇠ N ( 0 , 2I ) , the objective function we use to optimize the denoiser contains the standard reconstruction MSE : l ( z ( i ) , ✓ ) = kg ( z ( i ) ) x ( i ) k 2 2 , ( 1 ) where l is the loss function , g denotes the denoiser , and ✓ is the parameter of the denoiser . The stability objective of ( Salman et al. , 2020 ) is not included in our objective function , because it requires both x ( i ) and g ( z ( i ) ) to pass the pre-trained classifier h , which both incurs additional privacy cost and additional computation overhead when we calculate the transformation matrix . In this work , we assume l ( z ( i ) , ✓ ) is C-Lipschitz continuous , which is a mild and common assumption in existing works ( Bassily et al. , 2019 ; Feldman et al. , 2020 ) . 2.2 PERTURBATION TRANSFORMATION AND MULTIVARIATE GAUSSIAN MECHANISM . In this section , we will introduce perturbation transformation and Multivariate Gaussian Mechanism ( MGM ) to analyze the DP guarantees of the input perturbation . Perturbation transformation . As we introduced in 2.1 , input Gaussian perturbation is utilized to achieve certified robustness for TransDenoiser . Although theoretically this perturbation is only required at the testing phase , almost all existing approaches in practice demand the randomization during training to improve the performance . Our strategy is to transform input perturbation into gradient perturbation during training and analyze the DP guarantee that input perturbation can offer . The crucial step of this transformation is the Taylor expansion of MSE loss l ( z ( i ) , ✓ ) at the data point x ( i ) , which is formulated as follows , l ( z ( i ) , ✓ ) = l ( x ( i ) , ✓ ) + ( z ( i ) x ( i ) ) | rx ( i ) l ( x ( i ) , ✓ ) + o ( z ( i ) x ( i ) ) ( 2 ) Since the only mild constraint on l ( z ( i ) , ✓ ) is C-Lipschitz continuous , it is possible for the higher order terms o ( z ( i ) x ( i ) ) to be negative . We denote the examples with non-negative higher order terms as “ non-negative cases ” , and the others as “ negative cases ” . In the rest of this section , we use the superscript “ non ” and “ neg ” to denote samples of “ non-negative cases ” and “ negative cases ” , respectively . For “ non-negative cases ” , we have the following lemma : Lemma 1 . Given perturbed example znon ( i ) = x non ( i ) + b ( i ) with b ( k ) ( i ) ⇠ N ( 0 , 2 ) , and MSE loss l ( znon ( i ) , ✓ ) is C-Lipschitz continuous . The gradient r✓l ( z non ( i ) , ✓ ) can be reformulated as the gradient with respect to the original sample with a gradient perturbation : r✓l ( z non ( i ) , ✓ ) r✓l ( x non ( i ) , ✓ ) + p ( i ) , ( 3 ) where p ( i ) is the transformed perturbation with p ( i ) ⇠ N ( 0 , ⌃ ( i ) ) , ⌃ ( i ) = M ( i ) 2 , M ( i ) = A ( i ) A | ( i ) and A ( i ) = J✓rxnon ( i ) l ( x non ( i ) , ✓ ) . The detailed proof of Lemma 1 can be found in Appendix E. With Lemma 1 , we find that the righthand side is the lower bound of left-hand side , which means DP guarantee provided by transformed gradient perturbation p ( i ) is the lower bound of that provided by input perturbation b ( i ) . Multivariate Gaussian Mechanism . Since the transformed gradient perturbation p ( i ) in equation ( 3 ) follows a multivariate Gaussian distribution with correlated elements , which is in contrast to the independent and isotropic Gaussian noise used in standard Gaussian mechanism for DP , we introduce Multivariate Gaussian Mechanism ( MGM ) to analyze DP of this multivariate Gaussian perturbation . Theorem 1 . Multivariate Gaussian Mechanism . Let G : Rv ! Rw be an arbitrary w-dimensional function , and G = maxD , D0 kG ( D ) G ( D0 ) k2 . A Multivariate Gaussian Mechanism M with the covariance ⌃ 2 Rw⇥w adds noise to each of the w elements of the output . The mechanism M is ( ✏ , ) -DP , with ✏ 2 ( 0 , 1 ] , Smin ( M ) 1 2 p 2 ln ( 1.25/ ) G/✏ . where Smin ( M ) is the minimum singular value of M and ⌃ , M 2 . The proof of this theorem is in Appendix D. With Theorem 1 , multivariate Gaussian perturbation can be leveraged to preserve ( ✏ , ) -DP , and we have the following Corollary : Corollary 1 . Given a multivariate Gaussian perturbation p ⇠ N ( 0 , ⌃ ) , ⌃ = M 2 , the DP guarantee of the MGM is equivalent to that of a GM with its perturbation following a Gaussian distribution N ( 0 , Smin ( M ) 2 ) , where Smin ( M ) is the minimum singular value of M. Because the DP guarantee of MGM is equivalent to the GM by applying a transformed perturbation , the traditional DP analysis technique , e.g. , moments accountant Abadi et al . ( 2016 ) , can be leveraged to analyze the privacy cost in the training process of denoiser . We also note a special case of MGM , in which the covariance matrix of the multivariate Gaussian perturbation only contains the diagonal values , and the perturbation on each elements is independent from each other but they can have different scales . This mechanism is called Heterogeneous Gaussian Mechanism ( HGM ) Phan et al . ( 2019 ) . We re-define HGM in Appendix F and prove that MGM is a generalization of HGM .

Summary: In this paper, authors studied the problem of achieving both the overall differential privacy and certified robustness simultaneously for pre-trained models. They proposed a framework called TransDenoiser based on an existing framework (Salman et al, 2020) [1] by adding additional and transformed gradient perturbations for the overall DP. Authors analyzed DP guarantee provided by these perturbations and empirically evaluate their methods on MNIST and CIFAR-10, and shown that TransDenoiser is effective against FGSM and PGD attacks with guaranteed DP.