Two Birds, One Stone:  Achieving both Differential Privacy and Certified Robustness for Pre-trained Classifiers via Input Perturbation
Authors: Pengfei Tang
Wenjie Wang
Xiaolan Gu
Jian Lou
Li Xiong
Ming Li
Venue: ICLR 2022 Submitted (2022)
Keywords: Differential Privacy
Certified Robustness
Pre-trained Classifiers
Input Perturbation

Abstract: Recent studies have shown that pre-trained classifiers are increasingly powerful to improve the performance on different tasks, e.g, neural language processing, image classification. However, adversarial examples from attackers can trick pre-trained classifiers to misclassify. To solve this challenge, a reconstruction network is built before the public pre-trained classifiers to offer certified robustness and defend against adversarial examples through input perturbation. On the other hand, the reconstruction network requires training on the dataset, which incurs privacy leakage of training data through inference attacks. To prevent this leakage, differential privacy (DP) is applied to offer a provable privacy guarantee on training data through gradient perturbation. Most existing works employ certified robustness and DP independently and fail to exploit the fact that input perturbation designed to achieve certified robustness can achieve (partial) DP. In this paper, we propose perturbation transformation to show how the input perturbation designed for certified robustness can be transformed into gradient perturbation during training. We propose Multivariate Gaussian mechanism to analyze the privacy guarantee of this transformed gradient perturbation and precisely quantify the level of DP achieved by input perturbation. To satisfy the overall DP requirement, we add additional gradient perturbation during training and propose Mixed Multivariate Gaussian Analysis to analyze the privacy guarantee provided by the transformed gradient perturbation and additional gradient perturbation. Moreover, we prove that Mixed Multivariate Gaussian Analysis can work with moments accountant to provide a tight DP estimation. Extensive experiments on benchmark datasets show that our framework significantly outperforms state-of-the-art methods and achieves better accuracy and robustness under the same privacy guarantee.

Under review as a conference paper at ICLR 2022
TWO BIRDS, ONE STONE: ACHIEVING DIFFEREN- TIAL PRIVACY AND CERTIFIED ROBUSTNESS FOR PRE- TRAINED CLASSIFIERS VIA INPUT PERTURBATION
Anonymous authors
Paper under double-bling blind review
ABSTRACT
Recent studies have shown that pre-trained classifiers are increasingly powerful to improve the performance on different tasks, e.g, neural language processing, image classification. However, adversarial examples from attackers can trick pre- trained classifier to misclassify. To solve this challenge, a reconstruction network is built before the public pre- pre-trained classifiers to offer certified robustness and de- fend against adversarial examples through input perturbation. On the other hand, the reconstruction network requires training the dataset, which incurs privacy leakage of training data through inference attacks. To prevent this leakage, differ- ential privacy (DP) is applied to offer provable privacy guarantee on training data through gradient perturbation. Most existing works employ certified robustness and DP independently and fail exploit the fact that input perturbation designed to achieve certified robustness can achieve (partial) DP. In this paper, we propose perturbation transformation to show how the input perturbation designed for certi- fied robustness can be transformed into gradient perturbation during training. We propose Multivariate Gaussian mechanism to analyze the privacy guarantee of this transformed gradient perturbation and precisely quantify the level of DP achieved by input perturbation. To satisfy the overall DP requirement, we add additional gradient perturbation during training and propose Mixed Multivariate Gaussian Analysis to analyze the privacy guarantee provided by the transformed gradient perturbation and additional gradient perturbation. Moreover, we prove that Mixed Multivariate Gaussian Analysis can work with moments accountant to provide a tight DP estimation. Extensive experiments on benchmark datasets show that our framework significantly outperforms state-of-the-un methods and achieves better accuracy and robustness under the same privacy guarantee.
INTRODUCTION
Deep learning with pre-trained classifiers are increasingly powerful for solving difficult machine- learning tasks in the real-world, including image classification (Krizhevsky et al. 2012 Simon al.) 2016) and natural language processing (Vaswani et all, 2017 Devlin et al. 2018). However, deep learning models with pre-trained classifiers are subject adversarial examples attacks (Szegedy et al. 2013 Bruna et al. 2013. Goodfellow et al.) 2014) which apply small perturbations on inputs cause the model to misclassify. To solve this challenge, certified robustness (Li et al. 2018) for pre-trained classifiers is proposed by (Salman et al., 2020) as a provable defense approach, where a denoiser, e.g., autoencoder (AE) (Hinton & Zemel 1994,, trained on the data perturbed with Gaussian noise and aims to reconstruct denoised data. Once denoiser finishes training, it takes the Gaussian perturbed data for denoising reconstruction and feeds the denoised data to the pre- trained classifier. Randomized smoothing is then applied and provides certified robustness without retraining the large pre-trained model.
On the other hand, the training of denoiser requires training data that can include sensitive infor- mation, e.g., clinical records, financial records, user profiles, etc. Several works have shown that attackers can infer private information from training data through trained models (Fredrikson al. 2015 Wang et al. 2013) Shokri et al., 2017)p A popular and powerful technique to address this issue in deep learning is differential privacy (DP) (Dwork et al. 2006 Dwork 201
Dwork al. 2014

---

Under review as a conference paper at ICLR 2022
Adftiina kndiec pettrbation (uely trainin))
Tridkded labd
2.
s(2)
Iopet Sraininggadd tosting
DP Denoiser
Pretrained classißer
given a clean image x, input perturbation is added to generate per- turbed image z. is then reconstructed by denoiser to generate g(z). which is fed into the pre-trained classifier h for classification. The input perturbation x utilized to achieve certified robustness during test- ing. The denoiser trained under by leveraging input perturbation added on x and additional gradient perturbation during training.
Figure I: Framework of TransDenoiser;
which has been adopted in a large volume of works provide rigorous protection from leaking private information contained training data. Gaussian Mechanism (GM) (Dwork et al. (014)) is basic technique to achieve DP by injecting isotropic Gaussian perturbations
computation output.
Depending on where to inject the perturbations, existing works on machine learning with DP can be
mainly categorized into: input perturbation (Fukuchi al., 2017
Kang al. 2020a output per-
turbation (Zhag et al. (017)) gradient perturbation (Song ct al.
2013 Bassily et al. 2014 Shokri
Shmatikov [2013] Abadi et al. 2016 2017
Wang al. 2017
Lee & Kiter [018 Yu al. 2019)
objective perturbation (Kifer et al. 2012 Zhang et al. 2012 Phan et al. 2016 2017 [yengar et al.)
2019) and noisy labelling (Papernot
2016 2018).
Therefore, straightforward way to prevent these two critical risks, i.e.. adversarial examples and privacy leakage, independently applying certified robustness and DP to models with pre-trained classifiers. A few works (Phan et al. 2019, 2020) follow this idea and simultaneously achieve both DP and certified robustness. However, we observe that the randomized smoothing for certified ro- bustness requires that the input data is perturbed with Gaussian noise during training. This Gaussian perturbation brings randomization to the training process and could have been utilized to provide a certain level of DP guarantee. Independently applying certified robustness and DP fails to exploit this connection and incurs unnecessary additional randomization during training, which leads to the degradation of model utility.
Directly analyzing the DP guarantee through input perturbation methods (Fukuchi et al. 2017 Kang al. 2020a is nontrivial in deep learning, because these methods pose strict constraints loss function, which are not satisfied by deep learning models. An alternative way is to transform the input perturbation into gradient perturbation and then analyze the DP guarantee it provides. How- ever, existing work (Kang et al. 2020a) gives strong assumption and simply regards the transformed gradient perturbation as isotropic Gaussian perturbation, and fails recognize that in most deep learning models, this transformed gradient perturbation follows multivariate Gaussian distribution.
In this paper, we propose novel framework TransDenoiser (Figure D to simultaneously achieve certified robustness and DP for models with pre-trained classifiers. TransDenoiser has similar ar- chitecture as (Salman et al. 2020 by adding a denoiser before pre-trained classifier. Compared with (Salman et al. 2020). TransDenoise can provide similar level of certified robustness without retrain- ing the pre- pre-trained model, as well as guarantee DP for the training data. Compared with existing works that achieve both certified robustness and DP including SecureSGD (Phan et al. 2019) and StoBatch (Phan et al. 2020). TransDenoiser 1) provides a tigher guarantee of DP by utilizing all the randomization during training including input and gradient perturbations, and 2) achieves more ef- fective certified robustness by leveraging randomized s smoothing on the input instead of noisy layers in the model.
Contributions. Our key contributions are:
1. We propose novel framework TransDenoiser that trains denoiser through both input and gradient perturbation for achieving DP and certified robustness simultaneously on deep learning models with pre-trained classifiers. The input perturbation for achieving certified robustness utilized to achieve partial DP and additional gradient perturbation is used as necessary for the overall DP, ensuring an enhanced privacy and utility performance.
2

---

Under review as a conference paper at ICLR 2022
2. We present analytical tool that leverages Taylor expansion transform input perturba- tion into gradient perturbation so that it can be quantified and composed with the explicit gradient perturbation for the DP guarantee. We propose Multivariate Gaussian Mecha- nism (MGM) to analyze DP of the multivariate Gaussian perturbation and prove that MGM is generalization of Heterogeneous Gaussian Mechanism (Phan al. 20T9).
3. Observing that the transformed gradient perturbation itself cannot satisfy the DP guarantee requirement in some scenario, we add additional perturbation following isotropic Gaussian distribution to the gradient, and propose Mixed Multivariate Gaussian Analysis (MMGA) to analyze the DP guarantee provided by transformed gradient perturbation and additional gradient perturbation. We also prove that MMGA can work with moments accountant (Abadi et al., 2016) to provide tight bound on the privacy cost.
4. We conduct extensive experiments several benchmark datasets which demonstrate that TransDenoiser can 1) provide significantly tighter bound on privacy cost with same utility performance, and 2) achieve similar level of certified robustness as other state-of-the-art works.
2 TRANSDENOISER
In this section, we will present our proposed framework Trans Transsen,sser, which can be applied before any pre-trained classifier to guarantee certified robustness and DP without retraining the pre-trained model. A denoiser is trained to guarantee certified robustness of the final model via randomized smoothing or input perturbation on the input, where the training process introduces the input pertur- bation for better accuracy of the randomized smoothed model. We transform the input perturbation into equivalent gradient perturbation so that it can be quantified and composed with the explicit gradient perturbation for the DP guarantee.
2.1 DENOISER AND CERTIFIED ROBUSTNESS
As can be seen in Figure TransDenoiser is denoising AE trained on input data with Gaussian perturbation. Similar to (Salman et al., 2020). this input Gaussian perturbation is used as random- ized smoothing for certified robustness. Naively applying randomized smoothing classifier without the denoiser gives very loose certification bounds because the pre-trained classifier trained be robust Gaussian perturbations of their input. The denoiser serves to "remove" this Gaussian perturbation and effectively reconstruct the input data like a pre-processing step before feeding input data into the pre-trained classifier while maintaining the benefit of certified robustness. The detailed proof of randomized smoothing can be found in (Cohen et al. 2019) and we provide a brief proof in Appendix B
pre-trained
Different from (Salman et al. 2020), given input data X()) and perturbed data 2()) -x) + b with b()) ~ (0, the objective function we use to optimize the denoiser contains the standard reconstruction MSE:
(zz 0) (g(Z())) x(1112
where / is the loss function, g denotes the denoiser, and 0 is the parameter of the denoiser. The stability objective of (Salman et al. 2020 is not included in our objective function, because it requires both (()) and g(z to pass the pre-trained classifie h, which both incurs additional privacy cost and additional computation overhead when we calculate the transformation matrix. In this work, we assume ( (Z))) 0) is C-Lipschitz continuous, which a mild and common assumption in existing
works (Bassily al. 2019 Feldman et al. 2020
2.2 PERTURBATION TRANSFORMATION AND MULTIVARIATE GAUSSIAN MECHANISM
In this section, we will introduce perturbation transformation and Multivariate Gaussian Mechanism (MGM) to analyze the DP guarantees the input perturbation.
Perturbation transformation. As we introduced in 2.1 input Gaussian perturbation is utilized to achieve certified robustness for TransDenoiser. Although theoretically this perturbation only required at the testing phase, almost all existing approaches in practice demand the randomization
3

---

Under review as a conference paper at ICLR 2022
during training improve the performance. Our strategy is to transform input perturbation into gradient perturbation during training and analyze the DP guarantee that input perturbation can offer. The crucial step of this transformation is the Taylor expansion of MSE loss (Z))) 0) at the data point ((), which is formulated as follows,
((( 0) (x) 0) + (2(0)
x (x),,, 0) + o(Z(3) x())
(2)
Since the only mild constraint on (()1)) 0) is C-Lipschitz continuous, it is possible for the higher order terms o(z()) - x()) be negative. We denote the examples with non-negative higher order terms as "non-negative cases", and the others as "negative cases". In the rest of this section, we use the superscript "non" and "neg" to denote samples of "non-negative cases" and "negative cases", respectively. For "non-negative cases", we have the following lemma:
Lemma 1. Given perturbed example (1)) = x + -b() with b() ~ ~N(000²), and MSE loss (42))) 8) is C-Lipschitz continuous. The gradient Vollz () be reformulated as the gradient with respect to the original sample with a gradient perturbation:
Vollz() non 0) N Vol(x()) 0) + P(()
(3)
where P(a is the transformed perturbation with P( (O Eg)), E( Mojo2
M
( AT and (c) J&V (1)
(x 8).
The detailed proof of Lemmal 1 can be found in Appendix E With Lemma] D we find that the right-
hand side is the lower bound of left-hand side, which means DP guarantee provided by transformed gradient perturbation P()) is the lower bound of that provided by input perturbation b
Multivariate Gaussian Mechanism. Since the transformed gradient perturbation P(j) in equation (3) follows a multivariate Gaussian distribution with correlated elements, which is in contrast to the independent and isotropic Gaussian noise used in standard Gaussian mechanism for DP, we introduce Multivariate Gaussian Mechanism (MGM) to analyze DP of this multivariate Gaussian perturbation.
Theorem 1. Multivariate Gaussian Mechanism. Let 9 R" - th be an arbitrary u-dimensional function, and Ag = maxp,p' [IG GDD)) A Multivariate Gaussian Mechanism M with the covariance Σ E Rwxx adds noise to each of the w elements of the output. The mechanism M (e, ()DP, with
E (0, 1], Smin (M)²a 2 (/2m(1125/8)Ac/
where Smin(M) is the minimum singular value of M and Σ & Mo?.
The proof of this theorem in Appendix D With Theorem D multivariate Gaussian perturbation can be leveraged to preserve d)-D), and we have the following Corollary:
Corollary 1. Given a multivariate Gaussian perturbation
(0 E = Mo², the DP
guarantee of the MGM is equivalent to that of a GM with is perturbation following a Gaussian distribution Smia
Smin (M)a²), where Smin (M) is the minimum singular value of M.
Because the DP guarantee of MGM is equivalent to the GM by applying a transformed perturbation, the traditional DP analysis technique, e-g., moments accountant Abadi et al. (016)), can be leveraged to analyze the privacy cost in the training process of denoiser. We also note a special case of MGM, which the covariance matrix of the multivariate Gaussian perturbation only contains the diagonal values, and the perturbation on each elements is independent from each other but they can have different scales. This mechanism is called Heterogeneous Gaussian Mechanism (HGM) Phan et al. (019). We re-define HGM in Appendix F and prove that MGM is generalization of HGM.
2.3 TRANSDENOISER TRAINING ALGORITHM
As introduced in Section 2 the DP guarantee provided by transformed gradient perturbation de- pends on the transformation matrix and the scale of input noise. In some scenarios, this transformed gradient perturbation itself does not fully satisfy the DP requirement, because the scale of input noise
4

---

Under review as a conference paper at ICLR 2022
to achieve randomized smoothing of certified robustness is relatively small. To address this, we add additional gradient perturbation directly to the gradient in each iteration of the training process.
Algorithm shows the details of proposed TransDenoiser training algorithm achieve both certified robustness and DP. Each record is perturbed with input perturbation to achieve certified robustness (line 7). We utilize both input perturbation and gradient perturbation to achieve DP for "non-negative cases" (Line 12 18), and for the "negative cases", we directly employ gradient perturbation (Line 19 23). For non-negative cases, we transform input perturbation each iteration into gradient perturbation. We then add additional gradient perturbation with scale ō to Va, (Z,, 0c) each iteration given approach. mini-batch SGD, to train the denoiser, which is slightly different from our previous setting (Lemma D where only single sample fed into the model per iteration. We will show that our DP analysis works in both of these two settings.
flow and Eup- In Algorithm | we use the commonly used
Algorithm 1: Denoserr Training Algorithm
Input: pre-trained classifier h, total training epoch T, perturbation scale thresholds Elow and Eup, input perturbation scale a, learning rate 1y. training dataset Derain
t= 0:
load parameters of the pre-trained classifier h;
initialize parameters of the denoiser g:
while < '' do
get mini-bace batch data Xt from Derain; for each data X(e) in do
2(0) = (() + (0,
(() x()) = [(((0, 0) - (1(x(1)) 0) + (2(1) -x(1) (Vic ((x())))))
end
10
"Non-negative cases" z,on = {2 with o(2(3) x(6)) 2 0;
"Negative cases" z, = (z) with o(z(s) x(a) < 0;
if "Nonnnegative cases" then compute the gradient ((, =?"r):
13
14
calculate the input perturbation transformation matrix Mc; calculate Smin (M,);
15
Eup: Eup TSmin (M.
if VTSm (Mi)o < Elows
16.
else if VTSmi (M,)o < Eupri else,
0.
add additional perturbation the gradient Vol(f,, z5o)) = Vol(O,, 2("") +0000,
18
end else
19
compute the gradient Ve,((0,, z, 25));;
21
Eupi
add perturbation to the gradient Vol(O,, 2;;0)=
end 23
24
Vo, (fe, z,) = (ol 2 + Vol ";
25
update parameter for next iteration 0r+1 = 0: B 2,0)) + Vol(O., 20 "(";));
26 end
27
output Or and compute overall privacy cost through moments accountant.
Privacy Analysis. In order to compose the transformed gradient perturbation and the direct isotropic gradient perturbation in each iteration for DP analysis, introduce Mixed Multivariate Gaussian Analysis below.
Theorem Mixed Multivariate Gaussian Analysis. Let RR= R be an arbitrary w- dimensional function, and Åg maxp, d' [[G (D)))) Mixed Multivariate Gaussian Anal- ysis is the mix of a Multivariate Gaussian Mechanism M, with the covariance E E R and Gaussian Mechanism M2 with ō adding noise to each of the w elements of the output. This mixed mechanism is (e, 8)D,, with
E (0, O0 TS 72
2 Eup (0)
2211(1125/0))A
where ā is the isotropic gradient perturbation, T is the number of training steps, Smin (M()) is the minimum singular value of M.g and E(g) A M,ya2
5

---

Under review as a conference paper at ICLR 2022
Mixed Multivariate Gaussian Analysis (MMGA) can be leveraged to analyze privacy guarantee provided by transformed gradient perturbation and additional gradient perturbation. In the following of this section, will prove Theorem 2 and show that MMGA can work with moments accountant to provide a tighter DP estimation and preserve (c, 6)-DP for deep learning models with pre-trained classifiers.
Privacy Analysis for Vanilla SGD. In vanilla SGD, the algorithm randomly picks one sample at each iteration and feeds it into the model for optimization. Given the initial parameters On, iteration t, the parameters are updated as: 0z+1 = O, Ve, [[0,, ", where denotes the learning rate and z denotes one perturbed sample randomly picked at iteration In optimization process, the transformed perturbation P. is slightly different from the that in Equation 3
Given perturbed example z = xt + b, with b(() NNO,, o²), the number of training steps T. and C-Lipschitz continuous loss I. The gradient Vo, Vo, ,0,) at each step of vanilla SGD can be reformulated as:
Lemma 2.
Vo,1 llzoon 81) Vo,1(x?0n 01) +p
(4)
where the transformed perturbation Pt NO, min (M))²I), M. AOr A(a)
Jo Vx Xxpon((x(()) (i) 0z)-
The detailed proof of Lemma can be found in Appendix G arcorem@ear be proved by Lemma the definition of ō and Theorem Thus, we have the following corollary for "non-negatte" cases" :
Corollary 2. Given an inpul perturbation b, E RR with bya) ~N(0, the transformation matrix A, and additional gradient perturbation with scale o, the DP guarantee of the MMGA is equivalent to that of a GM with its perturbation following Gaussian distribution (0, e2
The above analysis for "non- negn-negttive cases" leverages perturbation transformation and MMGA to analyze DP. For "negative cases", because the perturbation transformation is no longer required and the gradient perturbation with Sup is directly added (ol =; the MMGA is equivalent to traditional GM with perturbation following (O, E.p). Therefore, can conclude that Corollary is applicable to both "non-negative cases" and "negative cases".
Privacy Analysis for Mini-batch SGD. The above claims for Vanilla SGD can be adapted to Mini- batch SGD by setting M, Σ A instead of M. A detailed proof can be found in Appendix
(0)4 A in Vanilla SGD. The
Tighter Composition via Moments accountant. While Corollary 2 is derived with simple compo- sitions, moments accountant can be applied with MMGA to provide tighter DP composition for Algorithm]]
B and the Theorem 3. There exist constants ej and C2 so that given sampling probability q N
number of training steps T, for any e < Giq², Algorithm 7 is (e, (e,df-diiferential private for any 5 > 0 if
qv Tlog(1/8)
Sup 2 02
(5)
The proof of Theorem can be found in Appendix
Discussion. We note that although the transformation matrix A() requires the clean example x()) the calculation of (g) does not incur privacy cost for the denoiser. This because the transforma- tion process and the calculation of A()) is only for analyzing the DP of the input perturbation, i.e., the clean example X()) does not actually contribute to the gradient Vol(z (). Another potential privacy concern is about the observations that "non-negative cases" and "negative cases" use dif- ferent scales of additional gradient perturbation. Even for "non-negative cases", different data will be applied with different σ. We claim that the different additional perturbation scales among "non- negative cases" and "negative cases" will not incur privacy violation, because we have proven that input perturbation also provides certain level of privacy guarantee, which is analyzed in way that we transform it into gradient perturbation. We first use perturbation transform matrix and MGM to calculate how much gradient perturbation scale can be transformed from input perturbation for

---

Under review as a conference paper at ICLR 2022
different data. Then add additional gradient perturbation onto transformed gradient perturbation to ensure that the overall gradient perturbation scale σ² + (M 2 Sup for each data. On the one hand, the calculation of σ is not visible users attackers. On the other hand, different ø can ensure the overall scale x² + Smmn (M₃) 2 a consistent lower hound for all training data.
EXPERIMENTS
In this section, will show experiments two benchmark datasets, MNIST and CIFAR- 10. These experiments are conducted to prove that 1) TransDenoiser can provide high level of certified robustness through randomized smoothing, 2) the input perturbation transformation can save considerable amount of DP budget and thus improve the model performance.
3.1 CONFIGURATIONS
Baseline and ablation studies. We employ SecureSGD (Phan et al. 2019 and StoBatch (Phan et al. 2020p two architectures in baseline approaches, and compare the certified robustness and DP performance on MNIST and CIFAR-10 datasets. We acquire two versions of SecureSGD through different training strategies: SecureSGD __st _sct is acquired by training an entire classifier from scratch, and SecureSGD.prr is acquired by training the denoiser and fixing the pre-trained classifier. For StoBatch, we only acquire the one training from scratch, because the denoiser with pre-trainied classifier not fit to its architecture. We also conduct two ablation studies with variants of Tenoseen: 1) Transsenoiseer, noddp which only contains input perturbation for certified ro- bustness, without the perturbation transformation and additional gradient perturbation for DP: 2) TransDenoiser which contains input perturbation for certified robustness and separate gradi- ent perturbation for DP, without utilizing input perturbation via perturbation transformation.
Models. Pre-trained classifiers are trained on public datasets, and we use convolutional and trans- posed convolutional layers to build the autoencoder based denoiser for both MNIST and CIFARIO datasets. The details of pre-trained classifiers and denoiser can be found in Appendix M
Adversarial examples. We use four different attack algorithms, i.e., FGSM, I-FGSM Kurakin al. 2016), Momentum Iterative Method (MIM) (Dong et al. 2017). and MadryEtAl (Madry al. 2017). to craft adversarial examples. These algorithms apply 12-norm attack on the pre-trained clas- sifter under white-box setting. Given threshold Latk of perturbation norm, adversarial example x' can be represented as x' x + d s.t. VA E R", ||bll2 S
Latk-
Certification. We employ the randomized smoothing of Cohen et al. (Cohen et al. 2019 in our work. function Radees or) is designed to return a certified radius 6 given the input and its perturbation scale. This indicates that the randomized smoothed model is certified robust around X()) within the radius K.
Evaluation metrics. We evaluate the performance in terms of certified accuracy (Cer- (Acc) on clean examples and conventional accuracy (ConvAce) on both clean and adversar- ial examples. CertAce = x issorrectt (1) CarrobustRRdduusxx,,
R), ConvAce =
isCorred(x(a) for adversarial example; isCorred(xaa) for clean example, where N denotes the test data size, X(E) denotes the adversarial example, isCorrect(x(n) denotes the function returning when the prediction X(3) is correct and 0 otherwise, isCorrect(x/e) works the same X(yy robust Radius (1); ()) 2 returns when the certified radius e is equal or larger than the thresh- old R and 0 otherwise.
N
Implementation details. The detailed implementation and code be found in Appendix M
3.2 EXPERIMENTAL RESULTS
We conduct our experiments on MNIST and CIFAR-10 10 to show that TransDenoiser can simultane- ously achieve both differential privacy and certified robustness via input and gradient perturbation.
For the following experiments, we will compare with 1) SecureSGD sct, SecureSGD prt and Sto- Batch to show that TransDenoiser achieves better performance than baselines certified robustness

---

Under review as a conference paper at ICLR 2022
ConAAc
CenAAe
CensAce
CenAAco
1000
0.
0.5
24
0.
05
0/5
18
05
02
00
36
1.2
radins
1
radius
2 radios
1
radios
(a) TransDenoiser and (h) TransDenoiser and ab- (c) Tennsse and (d) Tenosser and ab-
baselines on CIFARIO
lation cases on CIFARIO baselines on MNIST
lation cases on MNIST
Figure 2: Comparison among TransSenoiser. baselines and ablation cases for certified accuracy l2 radii on two datasets. The input perturbation scale 0.25, the overall gradient perturbation scale 2.0 (2 2.0 for TransDenoiser), and guarantee (1.0, 1e-5) 5)-DP for private models.
and DP, 2) compare with Transsenn nodp to show that TransDenoiser still achieves high level of certified robustness after adding gradient perturbation for DP, and 3) compare with TransDe- noiser_sepdp to show that input perturbation transformation effectively saves a considerable amount of DP budget and improves the utility performance.
Certified robustness. We demonstrate that TransDenoiser can achieve high level of certified ro- bustness on both MNIST and CIFARIO. We conduct experiments measure the CertAce on clean examples with different 22 radii of different methods on the two datasets, shown in FigurePaland Fig- ure 2c TransDenoiser significantly outperforms the state-of-the-ar SecureSGD prt, SecureSGD set and StoBatch algorithms on both datasets, thanks to the benefit of perturbation transformation and randomized smoothing.
Figure 2b and Figure 2d show the comparison with ablation cases. Besides TransDemoiser.nodp and TransDenoiser TransDenoiser_ing inn_.__eeriiii__111__1 0.1 denoting that the input perturbation scale for this TransDenoiser is 0.1 rather than 0.25. Comparing these ablation cases, TransDenoiser achieves similar CertAce as TransDe- noiser nodp, which means that TransDenoiser needs to add very little additional perturbation for ensuring DP thanks to the benefit of input perturbation that is being exploited. Compared with TransDenoiser _seddp that separate gradient perturbation for DP, TransDenoiser saves signifi- cant DP budget and thus requires less perturbation for DP. leading to significantly higher accuracy. Comparing with TransDenoiser inp_0. we find that TransDenoiser inp_0.1 can achieve highest CertAcc when l2 radius is small, but it drops quickly as radius increases. This is because 1) smaller input perturbation scale will bring less randomization to the model, and thus improves performance; 2) smaller input perturbation scale can only defend against less "powerful" adversarial attacks, and thus CertAce drops when attack radius increases. Comparing the two datasets, they show similar trend besides the fact that MNIST has higher accuracy for all methods in general due to its simplicity.
that we already introduced in See 3.I we add one more ablation case
Empirical defense. Certified robustness shows the theoretical defense against adversarial examples, also conduct experiments to show that TransDenoiser examples from different attacks. We only show the results against FGSM, I-FGSM attacks here, the more detailed experiments be found in Appendix M Figure 7a and Figure show the conv of T'ranDDenoise and baselines with respect to varying attack norm bound for different attack meth- ods on two datasets, respectively. Compared with these baselines, TransDenoiser achieves better empirical performance with any attack norm bound of all attacks. Figure 76 and Figure 7d show the comparison of TransDenoiser and ablation cases. Compared with TransDenoiser _nodp, Trans- Denoiser achieves similar ConvAce, which proves that perturbation for DP in TransDenoiser does not affect the ConvAcc too much. Compared with TransDenoiser sepdp. TransDenoiser effectively saves DP budget and achieves better empirical performance against adversarial examples. In all these figures, we add a curve named "Clean examples" to represent ConvAce that clean examples pass through Transeenoiser. As can be seen, ConvAcc for clean examples keep consistent attack norm bound increasing, and TransDenoisercan achieve similar ConAcc as "Clean examples" when attack norm bound is small.
empirically defend against adversarial
Observing the similar results between CertAce and ConvAcc, we see that the certified accuracy on clean examples provides good estimation for the empirical robustness of the model. If model achieves relatively high CertAce on clean examples, it can have a high probability to achieve high ConvAce on adversarial examples.
8

---

Under review as a conference paper at ICLR 2022
0.4
0.22 875 1.31 120 1.30 175 290
0,2 E35-035 180 12 130 1.5 210
0.23 133-3 1J6
1,0 230
0:2 5:75 1.39
1.31
PGS stock nomn hound
IPGSM snack nom bound
FGSM afRCc THE bound
[PDSM atracc norm hound
(a) TransDenoiser and baselines CIFARIO
(b) TransDenoiser and ablation cases CIFARIO
0.5
6.77
3 0.7
06
0.5
05
64
24-
044
03-
PGSM sttack nzem bound
IPGSM attack norm bound
16SM attack bound
IPGSM sttzck bound
(c) Denosser and baselines on MNIST
(d) Tenosser and ablation cases on MNIST
Figure 3: More comparison among TransDenoiser, baselines and ablation cases for conventional accuracy vs. ls radii two datasets. The input perturbation scale on CIFARIO = 0.1, MNIST 0.25, the overall gradient perturbation scale 2.0 (2 2.0 for TransDenoiser), and guarantee (1.0,1e-))-DP for private models.
PGSM julvacy trolget spiile
IPGSM jelvayy budge quilos
Mim- privacy budget quallon
Mady privacy Iradget spoilict
(a) Transeenoiser, baselines and ablation cases on CIFARIO
1.5
6.7
5.7
2077-
307
0.6
2.5
0.3
Sandl
5us
0.3
0.7
0.3
02-
02-
1752.82
IL250 50875
125255
grivacy qpilen
IFGSM privacy quills
privacy budgett
privacy spoilon
(b) TransDenoiser baselines and ablation cases on MNIST
Figure 4: Comparison among Denosee... baselines and ablation cases for conventional Accuracy vs. c on two datasets. The input perturbation scale on CIFARIO = 0. on MNIST = 0.25, attack norm bound 2.0, the overall gradient perturbation scale 2.0 (2 2.0 for Transsenosser), and 8 = le- for DP.
Differential privacy. We also evaluate the tradcoff between accuracy and privacy for different methods. As can be seen from Figure 4aland and Figure 4b given the same e, TransDenoiser can always achieve higher ConvAcc on different attacks similar to what we have observed so far. In addition, with increasing epsilon, all methods achive a higher accuracy expected. On MNIST dataset, beassDeniierrsppeee achieves similar result with Tenosee..sser.
4 CONCLUSIONS AND FUTURE WORK
In this paper, we have proposed Transsenoiser to achieve both DP and certified robustness via in- put perturbation. TransDenoiser stands as the first attempt achieve both for the vastly existing. yet under-studied, pre-trained model setting. We leverage input perturbation transformation to effi- ciently transform input perturbation into gradient perturbation. We propose MGM and MMGA to analyze DP of the transformed gradient perturbation and combine MMGA with moments accountant to provide a tight bound on DP guarantee. Therefore, TransDenoiser effectively saves a considerable DP budget and improves the utility performance compared to using gradient perturbation indepen- dently achieve DP. Our experiments on two benchmark datasets verify the performance advantage of TransDenoiser w.r.t. both DP and certified robustness compared to state-of-the-art methods. In future work, we plan to utilize more advanced DP analysis approach, e.g., analytical moments ac- countant (Balle & Wang 2018 to derive tighter bound on DP and further improve the privacy and utility tradeoff.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, Iyya Miconov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Proceedings of 2016 SIGSAC Conference Computer Communications Security. pp. 308-318. 2016.
Ahadi, Ulfar Frlingsson, Goodfellow, Brendan McMahan, Ilya Mironov, Nicolas Papemot, Kunal Talwar, Thang, the protection of private infornation in machine learning systems: Two recent approches. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF). pp. 1-6. IEEE, 2017.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. I Layer normalization. arXiv preprias orxiv 1607.0000065550 2016.
Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International Conference on Muchine Leurning. pp. 394 403.. PMLR, 2018.
Raef Bassily, Adam Smith, and Abhradoep Thakurta, Private empirical minimization: Efficient algorithms and tight error bounds, 2074 JEEE 55th Anmual Symposium Foundations of Computer Science, pp. 464-473, IEEE, 2014.
Raef Bassily, Vitaly Feldman, Kunal Talwar, and Abhradeep Thakurta. Private stochastic optimization with optimal rates. arXiv preprint arviv: 1908 09970, 2019.
Amos Beimel, Shiva Prasad Kasiviswanathan, and Kobbi Nissim, Bounds on the sample complexity for private learning and private datn release, Theory of Cryptogruphy Conference, pp. 437 Springer, 2010.
Bruna, Christian Szegedy, Ilya Sulskever, Ian Goodiellow, Wojeiech Zaremba, Fergus, and Dumitru Erhun. Intriguing properties neural networks. 2013.
Jeremy M Colen, Hlan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. crXiv:1902.029)) 2019.
arXiv preprint
Jacob Devlin, Ming-Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pre-tmining of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810..4500 2018.
Yinpeng Dong, Fangzhou Lian, Tianyu Pang, Xiaolin Hu, and Zhu. Discovering adversarial examples momentum, arXiv preprind 1100:1710.06081, 2017.
Cynthia Dwork. firm foundation for private data amalysis. Communications of the ACM, 54(1):86 95,86,95,,00 2011.
Cynthia Dwork, Frank McSherry. Kobbi Nissim, and Adam Smith. Calibrating noise sensitivity in private data analysis. In Theory of cryptegraphy conference, pp. 265. 284, Springer, 2006.
Cynthin Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and differential privacy. In IEEE of Computer Science. pp. 51 51-60. IEEE, 2010.
IEEE 31st Annual Symposium on Foundations
Cynthia Dwork Aaron Roth, et The algorithmic foundations of differential privacy. Foundations Trends in Theoretical Computer Science, 93-4):1114077, 2014.
Vituly Feldman, Tomer Koren, and Kunal Talwar. Privale stochustic convex optimization:: optimal in lineur time. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp- 439 449, 2020.
Matt Fredrikson, Somesh lha, and Thomas Ristcupart. Model inversion attacks that exploit confidence information and basic counterneasures. Proceedings of 22nd ACM SIGSAC Conference on Computer and Communicattons Security. pp. 1322 1333,
Kazuto Fakuchi, Quang Khai Tran, and Sakuma. Diffcrentially private empirical ri minimization inpur perturbation. In International Conference Discovery Science, pp. 82-90. Springer, 2017.
Manuel Gil. On Rényl divergence measures for continuous alphabet sources, PhD thesis, Citeseer, 2011.
Jan Goodfellow; Jonathon Shlens, and Christian Szegedly. Explaining and harnessing advensarial examples. arXiv preprimt arKiv: 1412,5572, 2014.
Kaiming He, Xiangyu Zhang, Shaoging Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Gooffrey E Hinton and Richard Zemel. Autoencoders, minimum description length, and helmholtz free energy. Advances in neural informa- rion processing systems, 6:3-10, 1994.
Roger Tyengar, Joseph Near. Dawn Song. Om Thakkar, Abhradeep Thakurta, and Lun Wang. Towards practical differentially private optinization. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 299-316. IEEE, 2019.
Yilin Kang, Yong Liu, Lizbong Ding. Xinwang Liu, Xinyi Tong, and Weiping Wang. Differentilly private erm based on data perturbation. arXiv preprint 2002 088788 2020a.
Yilin Kang. Vong Ben Nio, Xinyi Tong. Likun Zhang, Weiping Wang. Input perturhation: new paradigm hetween central local differential privacy. Xix preprint orXiv: 200220002700 2020h.
Daniel Kifer. Adam Smith, and Abhradeep Thakurta. Private convex empirical risk minimization and high-dimensional regression. In Confers once on Learning Theory, pp. 25 2012.
Alex Krizhevsky, llya Sutskever, and Geoffry E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in newral information processing systems, pp. 1097-1105, 2012.
Alexey Kurakin, Goedfellow, Samy Bengio, Adversarial machine learning scale. arXiv preprint Xix::6111.002230 2016.
10

---

Under review as a conference paper at ICLR 2022
Mathias Lecuyer. Vaggelis Atlidakis, Roxana Geumbasu, Duniel Hsu, Sumaa Jana. Ceriilied robustness to adversarial examples differential privacy. In 2019 IEEE Symposiun on Security and Privacy (SP). 656 672. IEEE, 2019.
Guang. He Lee, Yang Yuan, Shiyu Chang, and Tommi Jankkola. Tight certificates of adversurial robustness for randomly smoothed classifiers. Advances in Neural Information Processing Systems, pp. 4910-4921, 2019.
Jaewoo Lee and Daniel Kifer. Concentrated differentially private gradient descent with adaptive per-iteration privacy budget. In Proceedings the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 1656 1665, 2018.
Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Second order adversarial attack and certifiable robustmess. arXiv preprint arXiv: 031103 2018.
Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. In Advances in Nearal Information Processing Systems, pp. 9474, 2019.
Alcksander Madry, Alcksandar Makelov, Ludwig Schmidt, Dimitris Tsipnss, and Adrian Vladu, Towards deep leamning models resistant to adversarial altacks. arXiv preprint 1rXiv:1706.06000 2017.
Nicolas Pupernot, Martin Abodi, Ulfer Erlingsson, Ian Goodfellow, and Kunal Talwar, Semi-superrised knowledge transfer for deep learning from private training data, arXiv preprinz 161v::610005555 2016.
Nicolas Papemot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Ülfar Erlingsson. Scalable private learning with pate. arKiv preprint arXiv:1802 08968, 2018.
Hai Phan, My Thai, Han Hu, Ruoming Jin, Tong Sun, and Dejing Scalable differential privacy with certified rohustness in adversurial learning, In International Conference on Machine Learning, pp. 7683-7694, PMLR, 2020.
NhatHai Phan, Yue Wang. Xintao Wa, and Dejing Dou. Differential privacy preservation for deep auto-encoders: an application of human behavior prediction. Asai, volume 16, pp. 1309-1316, 2016.
NaitHaai Phan, Xintao Wa, Han Ha, and Dejing Dou. Aduptive laplace mechanism: Differential privacy preservation in deep learning, In 2017 JEFE International Conference Date Mining (ICDM), pp. 385-394, IEFE, 2017.
ShatHai Phan. Minh Yang Liu, Ruoming Jin, Dejing Deu, Xintao and My T'Tai. Heterogencous gaussian mechanism: Preserving differential privacy in deep learning with provable robustness, arXiv preprint arXin:1906.01466 2019.
Hadi Salman, Greg Yang. Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshieyn, and Sehastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. arXiv prepriet arviv (eee7906.000 1906.04584, 2019.
Hadi Salman. Mingie Sun. Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. arXiv preprint arXiv:2003 200:2003.01900, 2020.
Reza Shokri and Vitaly Shmatikov, Privacy-preservin deep leaminn. Proceeulings the 22nd ACM SIGSAC conference on computer and communications security. pp. 1310-1321, 2015.
Reza Shokri, Marco Stronati, Congzhong Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Sympozium on Security Privacy (SP) pp. 3-18, IEBE, 2017.
Marcel Simon, Rodner, and Joachim Denzler. Imagenet pre-trinned models with barch normalization. arXiv preprint arXiv:1612.01455: 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for largo-scale image recognition. arXiv preprint a409::1409.111 1556. 2014.
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. Stochastic gradient descent with differentially private updates, In 2013 IEEE Global Conference Signal and Information Processing, pp. 245-248. IEEE, 2013.
Christian Soegedy, Wojsiech Zaremba, Ilya Satskever. Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus, Intriguing properties neural networks, arXiv preprint 1312:1312.6199 2013.
Florian Tramer, Alesey Kurakin, Nicolas Papemot, Ian Goodfellow, Dan Boneh, and Parrick McDaniel, Fisemble adversarial training: Attacks and defenses. arXiv preprint (tfo::1750770000 2017.
Ashish Vaswani, Noam Shuseer, Niki Parmar, Jakob Uszkoreit. Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin, Attention is all you need. In Advances neunal information pmocesinn systems, pp. 5998 6008, 2017.
Di Wang. Minwei Ye, and Jinhui Xu. Differentially private empirical risk minimization revisited: Faster and more general. Advances Neunal Information Processing Systems, pp. 2732-2731, 2017.
Yue Wang. Cheng Si, and Xintao Wu. Regression model litting under differential privacy and model inversion attack. In LSAA pp. 1003-1009, 2015.
Lin Xiao and Tong Zhang. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization, 24(4): 2057-2075, 2014.
Lei Yu, Ling Liu, Calton Pa, Mehmet Emre Garsoy, and Stacey Truex Differentially private model publishing for deep learning, In 2079 JEEE Symposiuum on Security and Privacy (SP). pp. 332-349. IEEE, 2019.
Jiaqi Zhang, Kai Zheng, Wenlong Mon, and Liwei Wang. Efficient private emn for smooth objectives. arXiv preprint orXiv: 099477 2017.
Jun Zhang, Zhenjie Zhang, Xisokai Xiso, Yin Yang, and Marianne Winslett. Functional meclunism: regression analysis under differential privacy, Xrxiv preprint oixive 1208.0299 2012.
11


Reviews:
[{"review_id": "lRS68dVcE4Z", "reviewer": "Reviewer_wLgg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* The idea is well described.\n* The authors provided detailed analysis both theoretically and experimentally.\n\nWeaknesses / discussion questions:\n* The proposed TransDenoiser is lack of novelty. This paper mainly builds off the work by (Salman et al, 2020) [1], which proposed to train a denoiser on input perturbations and leveraged randomized smoothing to achieve certified robustness. The main difference is that in TransDenoiser, additional gradient perturbations are generated. However, the comparison with [1] is missing. The lack of comparison with the most relevant baseline reduces the confidence.\n\n* Throughout the paper, it is not made clear why achieving DP is important, and what is the difference between \"partial\" and \"overall\" DP. I think it would be good to include some brief background on DP in the related work section instead of in the appendix. A good chunk of your introduction could be moved to the related work section as well.\n\n* In terms of clarity, the overall writing could be greatly improved. There are several typos, confusing sentences and symbol choices (I listed several in minor issues). Specifically, it’s hard to follow your TransDenoiser Training Algorithm.\n\n* Does the proposed method work for L-infinity norm as well, both theoretically and experimentally?\n\n* The experimental results are not entirely convincing to me. For a thorough evaluation, it would be better to report robust accuracy against PGD (using MadryEtAl is very confucing...), CW and Auto Attack. My specific concerns are the following:\n  * All the methods should be given with a better name, current versions, like \"xxx_sct\", \"xxx_prt\", \"xxx_sepdp\" are not easy to follow.\n  * The captions of Figure 2 are mixed together and confusing.\n  * I’m not sure what's going on in Figure 3\n    * there is no figure reference in main text. I just suppose the corresponding explanations are under \"Empirical defense\", please correct me if I am wrong. \n    * only 'clean example' for TransDenoiser is provided, what about other methods? \n    * besides, the better robustness may be caused by the trade-off between natural accuracy and robustness, with lack of the 'clean example' given by other baselines, to me, it is not convincing to directly draw the conclusion that \"the certified accuracy on clean examples provides a good estimation for the empirical robustness of the model\". \n\nMinor:\n* I found authors make their statements misleading. For example, \n  * in page 2, the paper says \"compared with [1], TransDenoiser can .... without retraining the pre-trained models\", however [1] fixed the pre-trained models instead of retraining.\n  * in page 3, the paper says \"different from [1], ..., the objective function we use to optimize the denoiser contains the standard reconstruction MSE\", however [1] also used MSE. \n* Under \"Empirical defense\", the authors are mainly describe figures in the Appendix.\n* The proposed methods should be evaluated on larger datasets (e.g., CIFAR-100) and more \"popular\" models (e.g., ResNet-xx, WRN-xx) to demonstrate its effectiveness thoroughly.\n", "summary_of_the_paper": "In this paper, authors studied the problem of achieving both the overall differential privacy and certified robustness simultaneously for pre-trained models. They proposed a framework called TransDenoiser based on an existing framework (Salman et al, 2020) [1] by adding additional and transformed gradient perturbations for the overall DP. Authors analyzed DP guarantee provided by these perturbations and empirically evaluate their methods on MNIST and CIFAR-10, and shown that TransDenoiser is effective against FGSM and PGD attacks with guaranteed DP. ", "summary_of_the_review": "This paper mainly builds off the work by (Salman et al, 2020) [1]. Although DP analysis and tighter bound on DP guarantee are of some significance, the authors are suggested to 1) compare their proposed method with [1], 2) improve overall writing clarity, and 3) significant improvements over experiment settings.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Pxn02yTzuhF", "reviewer": "Reviewer_K9zu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "strengths: \n1). This paper considers transforming input perturbation into gradient perturbation, then the noise introduced by random smoothing can be quantified with the explicit gradient perturbation for the privacy guarantee. \n2). To analyze the privacy guarantee, Multivariate Gaussian mechanism is proposed by considering multivariate Gaussian perturbation\n3). The proposed method are conducted on various datasets and adversial attacks to show the effectiveness.\n\nweaknesses:\n1), Multivariate Gaussian Mechanism is not new, and many previous works are also investigated multivariate Gaussian differential privacy to achieve DP. For example, Chanyaswad et al in [1] proposed a MVG mechanism, which adds a matrix-valued noise drawn from a matrix-variate Gaussian distribution, and also introduce the directional noise in MVG that can further imporve the utility. Further, Yang et al in [2] proposed a Matrix Gaussian Mechanisms for matrix value with better utility. \n\n\n[1] Chanyaswad, Thee, Alex Dytso, H. Vincent Poor, and Prateek Mittal. \"Mvg mechanism: Differential privacy under matrix-valued query.\" In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 230-246. 2018.\n[2] Yang, Jungang, Liyao Xiang, Jiahao Yu, Xinbing Wang, Bin Guo, Zhetao Li, and Baochun Li. \"Matrix Gaussian Mechanisms for Differentially-Private Learning.\" IEEE Transactions on Mobile Computing (2021).\n", "summary_of_the_paper": "This paper focuses on providing both differential privacy and certified adversarial robustness to machine learning models. \nThe authors propose an algorithm called TransDenoiser to achieve such a goal. TransDenoiser consists a denoiser through both input and\ngradient perturbation for achieving DP and certified robustness, and following by a pre-trained classifier for classification. The privacy guarantee is carefully analyzed. Extensive experiments demonstrate the effectiveness of proposed method from model utility and adversarial robustness.", "summary_of_the_review": "This paper investigates how the random smoothing noise can be transformed into gradient perturbation, and then carefully compute the privacy loss, which seems an interesting method.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "89UXw6dcTWr", "reviewer": "Reviewer_t4Xc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nIt may be desirable to defend the same classifier against adversarial input perturbations and privacy attacks, so approaches which simultaneously guarantee certified robustness and differential privacy are interesting and useful.\n\nPrior work has noted that certified robustness techniques provide differential privacy, current work provides a quantification of this claim.\n\nExperiments show gain over prior methods.\n\n\nWeaknesses:\n\nThe proposed perturbation mechanism is a simple combination of input and gradient noise, the transformation process is only for analyzing the Differential Privacy guarantee and involves a simple Taylor estimate and weakly uses the properties of the loss function.\n\nThe multivariate gaussian mechanism is a simple generalization of known mechanism in prior work.\n\n\nQuestions:\n\nIt is not clear what is the proportion of negative and non-negative examples, or what effect this proportion has on differential privacy.\n\nHow are the perturbation scale thresholds xi_low and xi_up set in the experiments?\n\nCan gradient perturbation be useful for certified robustness?\n\nExperiments compare with other approaches that simultaneously try certified robustness and DP, but what is the gap from approaches that optimize for exactly one or the other?\n\n\nOther suggestions/comments:\n\nFormal statements are sometimes unclear or without sufficient detail. E.g. what is the domain of input x_(i).\n\nThe readability of the paper would benefit from a brief consolidated notation/terminology section, e.g. defining (epsilon,delta)-dp\n\nTheorem 1 simultaneously defines Multivariate Gaussian Mechanism and proves a property for it, would be clearer to separate out a complete formal definition.\n\nIn equation (2) is o the little-oh notation? It appears to not be the case in Algorithm 1? A clarification would be useful if there is abuse of notation, or if the little-oh choice is just coincidental.\n\nIn abstract and elsewhere the DP bounds with Moments accountant are said to be 'tight' but perhaps better to just say relatively tighter.", "summary_of_the_paper": "The paper studies differential privacy for pre-trained certifiers that offer certified robustness through input perturbation. The key insight is to analyze differential privacy afforded by the input perturbation by noise transformation (computing corresponding gradient noise), and augmenting with gradient perturbation when needed for differential privacy guarantees. This improves upon past work by reducing the differential privacy budget needed by showing that some differential privacy is obtained from the input perturbation. In addition to new analysis techniques for providing the privacy bounds, experiments show gains over prior methods, e.g. better differential privacy under adversarial attack.     ", "summary_of_the_review": "The paper considers an interesting and relevant question but the technical novelty and significance of the provided results does not seem to be sufficient. Algorithmic additions over prior work include using a combination of input and gradient noise and adding general multivariate Gaussian noise. The key analytic insight is to use Taylor approximation to estimate DP afforded by input noise. It is not clear how to set the hyperparameters of the proposed algorithm. Also formal presentation is somewhat lacking. Even though the approach beats some prior works empirically, I believe the work is marginally below acceptance threshold due to above issues.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5X8BBH8mOn", "reviewer": "Reviewer_s8jm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\nPrivacy and robustness to adversarial examples are two hot topics within the ML community, especially when considering large models for image or speech recognition. Therefore, I believe that the main focus of this paper, i.e., the integration of these two notions for pre-trained classifiers, is very relevant to the ICLR community. Also, the main point of the article is quite simple and easy to understand from a high-level perspective. Finally, the idea of trying to translate the input noise injection into gradient perturbation to simplify the privacy analysis seems interesting. \n\n**Weaknesses**\n\nMy main concern is with the technical quality of the article. In fact, I am not sure that the claims of the paper are technically correct, especially with respect to Lemmas 1 and 2. Although, from a general perspective, the concepts of input noise and gradient perturbation seem related, in my opinion, neither Lemma 1 nor Lemma 2 demonstrate a clear connection.  I provide some details below.\n\t\n1. Lemma 1 states that for a certain type of perturbed examples $z_{(i)}^{non}$ (defined in section 2.2), the gradient computed at $z_{(i)}^{non}$ can be lower bounded by the gradient computed at the initial point $x_{(i)}^{non}$ plus some noise that depends on the Jacobian matrix of the loss function at $x_{(i)}^{non}$. First, the statement itself appears to be very confusing to me because the authors compare two random vectors (with infinite support) without explaining the meaning of the term $\\geq$. Second, the analysis that the authors provide by stating that, according to Lemma 1, \"the DP guarantee provided by the perturbation of the transformed gradient is the lower bound of the one provided by the perturbation of the input\" lacks justification, especially since the lemma only holds for a specific family of perturbed inputs.  Finally, looking at the proof, I have some additional concerns, among which a) the reason for the jump from (9) to (10) is not clear to me, and b) the transition from (10) to the equality between the gradient of $z_{(i)}$ and the perturbed gradient of $x_{(i)}$ is also not clear to me. \n2. Lemma 2 provides a similar statement with an equality but was not provided with a formal proof. Instead, the authors present another lemma in the appendices (Lemma 3, also without a proof), which is very similar to Lemma 1 and claim that Lemma 2 can be derived from Lemma 3. This claim does not seem sufficiently conclusive to me.\n\nFinally, I have the impression that the paper claims too much its technical contribution on the analysis of multivariate Gaussian noise injection. In fact, as I understand it, this work can be considered as a special case of previous work studying matrix-valued Gaussian mechanisms [2]. I think the authors should compare with this previous work.\n\n**Additional comments and questions**\n\nIn experiments, I am not sure that the comparison of TransDenoiser with previous methods is fair in terms of privacy preservation. As I understand it in [3] and [4], the model is directly learned with differential privacy, thus protecting the dataset used to learn the model. However, in this paper, the authors only claim to preserve privacy on the fine-tuning dataset, thus leaving the dataset used in the pre-trained model unprotected. I have two concerns with this: a) the authors are comparing methods that do not preserve privacy on the same dataset, which makes the comparison unfair compared to previous methods, and b) I checked the pre-trained classifiers and it appears that they use the same datasets as the authors' trained auto-encoder (since these models are not trained with privacy, I think this represents a clear privacy breach).\n\nIn Thereom 3, the authors present a result on privacy preservation for Algorithm 1 based on a previous result in [5]. This result is only valid if we consider algorithms that use Poisson sampling to select the mini-batch at each round. However, Algorithm 1 does not seem to be using Poisson sampling since the size of the mini-batch is constant and equal to B (see line 25 of the algorithm).\n\nI think the presentation of the technical contribution could be improved. The appendix presents several statements re-demonstrating existing results in the literature on privacy and adversarial robustness (Theorem 5, 6 and Appendix B). From my point of view, these do not help the overall understanding of the contributions of the paper. I suggest presenting only the proofs of the original contributions in the appendix, and simply citing the existing papers for the earlier work when needed.\n\n\n[2] MVG Mechanism: Differential Privacy under Matrix-Valued Query Thee Chanyaswad, Alex Dytso, H. Vincent Poor, Prateek Mittal\n\n[3] Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness NhatHai Phan, Minh Vu, Yang Liu, Ruoming Jin, Dejing Dou, Xintao Wu, My T. Thai\n\n[4] Scalable Differential Privacy with Certified Robustness in Adversarial Learning NhatHai Phan, My T. Thai, Han Hu, Ruoming Jin, Tong Sun, Dejing Dou\n\n[5] Deep Learning with Differential Privacy Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang\n", "summary_of_the_paper": "This paper studies the problem of integrating differential privacy and robustness to adversarial examples for pre-trained machine learning models. Specifically, this work aims at designing methods that guarantee both privacy and robustness without having to re-train the model at hand. To achieve this goal, the authors build upon an existing technique in the adversarial example literature that involves placing a denoising auto-encoder in front of a pre-trained model before applying a noise injection scheme known as \"randomized smoothing\" [1]. While this technique is known to provide state-of-the-art \"certified accuracy\" against adversarial examples, its privacy guarantees remained to be studied. This work proposes to do just that by adapting the algorithm to guarantee differential privacy for the dataset used to train the auto-encoder. The authors claim three main contributions: \n\n1. Exploiting the intrinsic train-time input perturbation that existed in the previous implementation of the algorithm and composing it with an explicit gradient perturbation to satisfy differential privacy. The authors claim that their treatment of this input perturbation allows a finer analysis of the algorithm's privacy, which ultimately leads to better accuracy, for the same privacy guarantees.\n2. Introducing two new analytical tools, namely MGM and MMGA,  for analyzing the privacy guarantees of multivariate Gaussian noise injection. \n3. Conducting extensive experiments on several benchmark datasets to demonstrate that their algorithm, called « TransDenoiser »,  provides better privacy guarantees and achieves similar level of certified robustness compared to previous works. \n\n[1] Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, Sebastien Bubeck\n", "summary_of_the_review": "While I think this article studies an interesting problem, I do not think it presents its contributions convincingly enough. In particular, I have concerns about the technical quality and the novelty of the article that lead me to recommend its rejection.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper develops a technique to provide both privacy and robustness
at the same time using differential privacy.

Unfortunately the paper in its current form does not have meaningfully
interpretable security or privacy claims. The reviewers point at a number
of these flaws that the authors do not address to the satisfaction of
the reviewers, but there are a few others as well.
- What is actually private, at the end of this whole procedure? If the
  actual "pretrained classifier" is not made private, then what's the
  purpose of the entire privacy setup in this paper? Why does the denoiser
  need to be private if the classifier isn't?
- The proof of Lemma 1 appears incorrect. The proof in Appendix E says that
  Equation 10 is true, but this sweeps all of the remaining Taylor series
  terms under the rug and doesn't deal with them. How are they handled?
- In Figure 4(a), what does it even mean to have a "FGSM privacy budget
  epsilon"? Or a "MIM privacy budget epsilon"? A privacy budget is almost
  always something defined with respect to the *training data privacy*,
  how does this relate to the attack in this paper?
- How does this paper compare prior *canonical* defenses, both on the
  robustness and privacy side? In particular, comparisons to adversarial
  training on the robustness side, and some recent DPSGD result on the
  privacy side?
Decision: Reject