Differentially Private Optimizers Can Learn Adversarially Robust Models
Authors: Zhiqi Bu
Yuan Zhang
Venue: Accepted by TMLR ()

Abstract: Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities. One important yet worrying question is: Will training models under the differential privacy (DP) constraint have an unfavorable impact on their adversarial robustness? While previous works have postulated that privacy comes at the cost of worse robustness, we give the first theoretical analysis to show that DP models can indeed be robust and accurate, even sometimes more robust than their naturally-trained non-private counterparts. We observe three key factors that influence the privacy-robustness-accuracy tradeoff: (1) hyper-parameters for DP optimizers are critical; (2) pre-training on public data significantly mitigates the accuracy and robustness drop; (3) choice of DP optimizers makes a difference. With these factors set properly, we achieve 90\% natural accuracy, 72\% robust accuracy ($+9\%$ than the non-private model) under $l_2(0.5)$ attack, and 69\% robust accuracy ($+16\%$ than the non-private model) with pre-trained SimCLRv2 model under $l_\infty(4/255)$ attack on CIFAR10 with $\epsilon=2$. In fact, we show both theoretically and empirically that DP models are Pareto optimal on the accuracy-robustness tradeoff. Empirically, the robustness of DP models is consistently observed across various datasets and models. We believe our encouraging results are a significant step towards training models that are private as well as robust.

Differentially Private Optimizers Can Learn Adversarially Robust Models
Zhiqi Bu"
woody:218@gmaas com com
Yuan Zhang'
ewynannhhngg@@maii... (a com
Reviewed on OpenReview: https: // openreview. net/ forum? id- 08VgRNYh6n
Abstract
Machine learning models have shone in variety of domains and attracted increasing at- tention from both the security and the privacy communities. One important yet worrying question is: Will training models under the differential privacy (DP) constraint have unfavorable impact on their adversarial robustness? While previous works have postulated that privacy comes at the cost worse robustnoss, we give the first theoretical analysis to show that DP models can indeed be robust and accurate, even sometimes more robust than their naturally-trained non-private counterparts. We observe three key factors that influence the wrriaccyrobuuttues--ccnott
tradeoff: (1) hyper-parameters for DP optimizers are critical; (2) pre-training on public data significantly mitigates the accuracy and robust- ness drop; (3) choice of DP optimizers makes a difference. With these factors set properly, we achieve 90% natural accuracy, 72% robust accuracy (+9% than the non-private model) under (2.5.) atlack, and 69% robust accuracy (+16% than the non-private model) with pre-trained SimCLRv2 model under Loc (4/255) attack on CIFAR10 with e = In fact, we show both thcorctically and empirically that DP models are Pareto optimal on the accuracy- robustness tradeoff. Empirically, the robustness DP models is consistently observed across various datasets and models. We believe our encouraging results are significant. step to- wards training models that are private well robust.
1 Introduction
Machine learning models trained on large amount of data be vulnerable to privacy attacks and leak sensitive information. For example, Carlini et al. (2021) shows that attackers can extract text input from the training set via GPT2 (Radford et al., 2019), that contains private information such as address, phone number, name and so on; Zhu et al. (2019) shows that attackers can recover both the image input and the label from gradients ResNet (He et al., 2016) trained on CIFAR100 (Krizhovsky et al., 2009), SVHN (Neizer et al., 2011), and LFW (Huang et. al., 2008).
To protect against the privacy risk rigorously, the differential privacy (DP) is widely applied in various deep learning tasks (Abadi al., 2016; McMahan et al., 2017: Bu et al., 2020; Nori et al., 2021; Li et al., 2021), including but not limited to computer vision, natural language processing, recommendatinn ion system, federated learning and so on. At high level, the privacy is protected via DP optimizers such as DP-SGD and Adan,,,, while allowing the models to remain highly accurate. In other words, the privacy concerns have been largely alleviated by switching from regular optimizers to DP ones.
EEqual contribution.

---

An equally important concern from the security community is that, many models such as deep neural networks are known to be vulnerable against adversarial attacks. This robustness risk can be severe when the attackers can successfully fool models to make the wrong prediction, through modifying the input data by a negligible amount. An example from Engstrom el al. (2019) shows that a strong ResNet50 trained on ImageNet (Deng et al., 2009) with 76.13% accuracy can degrade to 0.00% accuracy, even if the input image is merely perturbed by 4/255 at cach pixel.
However, al the intersection of these two concerns, previous works have empirically observed an upselling privacy-robustuess tradeoff under some scenarios, implying the implausibility of achieving both robustness and privacy at the same time. In Song et al. (2019) and Mcjia et al. (2019), adversarially trained models are shown to be more vulnerable to privacy attacks, such as the membership inference attack, than naturally trained models. In Tursynbek et al. (2020) and Boenisch et al. (2021), DP trained models were more vulnerable to robustness attacks than naturally trained models on MNIST and CIFAR10. This leads to the following concern:
Does DP optimization necessarily lead to less adversarially robust models?
On the contrary, we show that DP models can be adversarially robust, sometimes even more robust than the naturally trained models. Indeed, we illustrate that DP models can be Pareto optimal, so that any model with higher accuracy than DP ones must have worse robustness. We observe that:
DP training itself does not worsen adversarial robustness in comparison to the natural training.
2. The robustness is largely affected by the DP optimization arper-parameters n), where different hyper- parameters are equally privacy-preseviigg but significantly different in accuracy and robustness.
3. DP optimization hyper-parameters (R, 1) achieving the best accuracy (which can be much less robust than the natural training) is different those achieving the best robustness.
4. The robustness is less affected by the privacy budget (e.g. Table 4 and Table 6). Even when we consider non-DP optimization (i.e, e = oo, no noise but with clipping), the models could be comparably or more robust than naturally trained models (see the blue boxplots in Figure 3).
In sharp contrast to the empirical nature of previous arts, we enhance our understanding about the adversarial robustness of DP models from a theoretical angle. Our analysis shows that DP classifiers without adversarial training may in fact be the most adversarially robust classifier. Motivated by our theoretical analysis, we claim that the hyperparameter tuning is vital to successfully learning a robust and private model, where the optimal choice is to use small clipping norms and large learning rates. This is interesting as such a hyperparameter choice is also observed to be the most effective in learning highly accurate models under DP (Li et al., 2021; Kurakin et al., 2022; De al., 2022). In fact, using small clipping norm is equivalent to normalizing all per-sample gradients, which allows a clear demonstration in Table 1 that the optimal hyperparamcter for natural accuracy is different to that for robust accuracy (only n is present because R is absorbed in the automatic DP-SGD (Bu et al., 2023b)).
Table 1: Robust and natural accuracy are achieved by different hyperparameter n on CIFAR10. Same setting as in Tramer & Boneh (2020), under (E =(2,le-5) and attacked by 20 steps of Loo (2/255) PGD.
learning rate v
2-8
2-7
2-6
2-5
2-1
2-3
2-
2-1
DP
natural accuracy
84.36
87.33
89.45
90.76
91.76
92.50
92.54
92.70
robust accuracy
75.45
78.92
81.03
80.96
78.87
72.77
58.29
26.97
learning rate v)
2-5
2-1
2-3
2-2
2-1
20
21
2
non-DP
natural accuracy
94.23
94.31
94.38
94.46
94.60
94.39
94.21
93.84
robust accuracy
79.32
79.57
74.18
66.20
55.87
46.00
41.75
45.21
2

---

Additionally, we advocate pretraining the model and selecting proper optimizers for DP training, which allow us to max out the performance on MNIST (LeCun et al., 1998), Fashion MNIST(Xiao et al., 2017), CIFAAIO(Kriahevsky et al., 2009), and CelebA et al., 2015).
Remark 1. Most of this work does not use adversarial training (except in Table 2 and Table 3) and should be distinguished from the certified robustness (Lecuyer et al., 2019), which does not guarantee DP in a per-sample sense rather than use the mathematical tools from DP in a per-pixel way.
2 Preliminaries
Notation. We use f X' X y to denote the model mapping from data space X to label space y. We denote the datapoints respectively. We denote the gradient of the i-th sample at step t ge yoi w, b), where w is the weights and b is the bias of model f.
(æ:) E Rdd and the labels {wi), following i.i.d. from the distributions and y
To start, we introduce the definition of DP, particularly the (s, 6)-DP (Dwork et al., 2014). Definition 1. A randomized algorithm M (e, 6)PD) if for any neighboring datasets S, S that differ by one arbitrary sample, and for any event E, holds that
P[M(S) E) < CMIM E +
(1)
In words, DP guarantees in the worst case that adding or removing one single datapoint (ax,, ys) does not affect the model much, as quantified by the small constants (c, Therefore DP limits the information possibly leaked from such datapoint.
In deep learning, DP is guaranteed by privatizing the gradient in two steps: (1) the per-sample gradient clipping' (specified by the clipping norm R, to bound the sensitivity of [z;))) (2) the random noising (specified by the noise multiplier ODP, to randomize the outcome so that each sample's contribution is indistinguishabl;; ODP can be determined by the privacy accountants Dwork et al. (2014); Abadi et al. (2016))- From an algorithmie viewpoint, DP training simply applies any optimizer on the private gradients instead of on the regular gradients.
NP training on regular gradient:
E,g y:)
(2)
DP training on private gradient:
E,Cr(e((e, ODP R (O
(3)
We are interested in the adversarial robustness of models trained by DP optimizers. To be sure, we consider the adversarially robust classification error and the natural classification error as
(f) = pl <7, s.t. f(x p) + v). == P((()))+)))
(4)
where p E Idd the adversarial perturbation, 7 is the attack magnitude, and Lo (and (2) attack considered. Notice that when y 0, the robust error in (4) reduces the natural crror.
3 Theoretical Analysis on Linear Classifiers
To theoretically understand the adversarial robustness of DP learning, we study the robustness of the DP and non-DP linear models on binary classification problem. We consider a mixed Gaussian distribution, where the positive class = +1 has a larger variance (i.e. it is more difficult to be classified correctly?) than the negative class -1:
{
N (0,, K2oLL N N-0,s, I
yf +1 yf
x
(5)
We use (R,) = de min(R//lldla. 1) as in Abadi per-sample gradient norm R. Note that in Table use the automatic clipping such that Cn(ge(az)) == ge/lgelll. The fact that larger variance indicates lower intra-class accuracy is proven by Xu et (2021, Theorem 1).
(2016) to denote gradient clipping, after which each
3

---

where y unif 1-1,+11, Os (8, ,0 IR
ann K 1.
0.50
Adversorta Natural
Adooneniia
0.25
Natural
0.00
-0255
-0.50
-0.7
-100
-125
1.50
-1.0
-0.5
00
05
Figure 1: Decision boundaries of linear classifiers for (5), K 4, 0.2, =1 1.
The sctting? in (5) is analyzable because of the data symmetry along the diagonal axis Er1 Era. as illustrated in Figure 1. We will show that this symmelry leads to an explicit decision hyperplane of
linear classifiers in Theorem 1, which further characterizes the strongest adversarial perturbation p arg (ggSpppp--- ~ < Pff( p) # y) explicitly.
In the following analysis, focus on the linear classifiers f(æ; w, sign(>) WyJz + b) with weights wy and bias b.
Remark 2. Within the family of linear classifiers, by the symmetry of data in (5), it can be rigorously shown by (6) that the optimal weights with respect to the natural and robust errors are always w1 = =... = wd: which matches our empirical calculation on DP models. That is, the weights do not distinguish between the robust and natural models. Consequently, the key to the adversarial robustness lies in the intercept b, which is analyzed in the subsequent. sections.
3.1 Optimal Robust and Natural Linear Classifiers
We start by reviewing the robust error of robust classifier and the explicit formula of its intercept. Thcorem (Extended from Thcorem 2 in Xu al. (2021)). For data distribution (x, v) Equation (5) and under the y allack magnilude, we define the optimal robust linear classifier
fr = arg min P(l < <,, s.t. f(x -p) #v = arg min R. (f). I linear
f is tneer
The optimal robust error is
Ry R, (f=) 1
1 + + (-KB(K,n) +
(BCl)n)++BBBBB(())) /B(K, + 9(K)).
where & is the cumulative distribution function of slandard normal, B(K 2 va(0-1) and g(K) =
2 K²-1 Furthermore, by the symmetry of the data distribution, we have
1, b, = arg min m w.b
(6)
k² +
4d 7)2 KV (K2 1)2 + do²g(K).
by
K2 d(0 ((0--))
(7)
TThis sctting also studied our interest.
al. (2021), which focuses on the robustucss--ain fairnesss tradeoff, thus different to

---

Theorem 1 gives the closed form of the optimal robust classifier fr. or equivalently its intercept by: and the optimal robust error. The special case of natural error can be easily recovered by setting y = 0:
1. 1. bo arg min (=;W(,
w.b
We know for sure from Theorem that there exists tradeoff between robustness and accuracy: it impos- sible for the natural classifier fo to be optimally robust or the robust classifier fy to be optimally accurate, since bo + by (c.f. Figure 2).
10
Ro.lfoo.
Ry
Fact 1. by in (7) is strictly decreasing in y, ranging from by to -X.
Roz
10-2
y-0
y=02
Proof of Fact 1. Proof in Xu et al. (2021) shows that dh. --K-dd 25 - < 0, thus by is strictly decreasing in 7. Therefore, the range of by is bul. Finally, we note that by < R
==0.4
bo.4 b
Ro(fo)
b
Figure 2: Intercepts and robust/ natural accuracy
K²+1(0 (0- - 7)), hence bxo = -00.
under (5), with K = 0, = 0.2, 0 1.
3.2 Adversarially Robust Errors of Private Linear classifiers
Now analyze the robust error of DP classifiers, which requires a different analysis from Theorem because (1) DP modifies the optimization instead of the objective function, unlike R, which modifies the natural error Ro; (2) consequently, deriving the DP parameters w and b is much more difficult: we need to solve arg min R. (R;;;,;; under the additional constraint (imposed by DP) that (3) equals 0 in expectation. While is possible to simplify the problem, for example, by analyzing the deterministic gradient flow Bu et al. (2023a) or by formulating the actual objective that DP-SGD optimizes Song et al. (2021), this is beyond the scope of this work.
We consider specific linear classifier f(; where only the intercept b is learned and privatized*. This is known as the bias term fine-tuning (Zaken et al., 2022; Bu et al., 2022b), popular approach in fine-tuning the DP or standard neural networks.
For this classifier and any b, the robust error (7 き 0) and the natural error (7 0) are:
(1) =P(]lpl>> S S.L.
s.l. f(x + p) + y) = max P( + p) +
W/
P(f ( ra) - 1y Plf(x - 7u) # +1 y +1)
_ P
1
Σ Wj (xz + +b>0 +
P
Σ
Wj (zj 7) - + -b<00==+++
2
2
1
Vd(0 - -7)
vd(0- -7)
K dd b)
= =- + 2
do b
+
+b
(8)
2
Ko
where 7a = (7,... .7). With (8), we can analyze the robust and natural errors for any intercept b (private or not, robust or natural) and any attack magnitude y.
Our next result answers the following question: fixing DP classifier for = f(; 1, bpp), or equivalently its intercept bDp, under which attack magnitude is the classifier robust? We show that, although épp is not available in the closed form, it is possible for some attack magnitude 7" that dpp by, and thus the DP classifier is the most robust classifier among all.
'Note that DP only required on trainable parameters that are learned from data; hewwiss no data privacy can leaked. Therefore this specific classifier is guarantced to be DP.

---

Theorem 2. For data distribution (x. y) in Equation (5) and for any bpp < bo, there crists y > 0 such that by- = bpp, and therefore
min ds Binar
(f)= (fx+) (f..))) = Ry (fDP).
Proof of Theorem 2. By Fact 1, by bbp is decreasing in ranging from bp - UDp to -00. By the intermediate value theorem, there exists 7 > 0 such that by. = dDP, i.e. fDP = fy- (;,1,by-).
By Theorem 2, long as the DP intercept is sulficiently small, the DP classifier must be the most robust under some attack magnitude, among all linear classifiers. We visualize in Figure at y = 0.075, that indeed dDP x by- (grey solid line).
1.25
0.1
6,7 0.075
1.20
Pop-der 0
Bur.dor
1.00
0.85
3.0
4.0
Figure 3: Intercepts (decision boundaries) for (5), same setup as Figure 2. For DP classifiers, use DP-SGD
with n 8, epochs-50, batch size=1000, sample size=10000, (c, 8) (15, le 4).
To validate the condition of Theorem 2, now demonstrate the achievability of dpp by as result of the per-sample gradient clipping in DP optimizers. In words, show that the robust intercept is stationary by the DP gradient descent but not so by the regular gradient descent.
We consider the above setting with an attack magnitude and oDr = 0. We temporarily ignore the noise because it only adds variance to the gradient but does not affect the mean (see the blue and orange boxes in Figure 3), and when learning rate small, ODP has little effect on convergence (Bu al., 2023a).
In DP gradient descent, if the clipping ge(0;, yi: 1,,,) has the same magnitude R after clipping, Therefore the positive samples, pushing the inter- cept to increase, can balance with the negative ones that pull b to descrease. Thus = by is a stationary point learnable by DP training, even though it is smaller than by by Fact However, in the non-DP gradient descent, b, is not stationary. This is because the positive class gradient E, (x, +1; b,) is larger than the negative class gradient Σ, (0 -1; 11;1,,,, so as to push the decision boundary by towards bn, where the natural classifier fo is defined. We visualize our analysis in Figure 4.
R is sufficiently small, then each per-sample gradient
Next, suppose only require the DP classifier to be more robust than the natural classifier, without requiring it to be the most robust among all linear classifiers. We can answer the question: fixing the atlack magnitude 7, under which condition is fDP more robust than the natural classifier fo?

---

+1
100
10A
0.300
0.015
0,000
0,035
DL000
0.00
0,000
6,6T0
0,100
1:30
0.75
0.50
01.25
9.00
6.55
0.30
0.75
1.00
Figure 4: Distribution of gradient with clipping (left, which is balanced) and without clipping (right, which is unbalanced) of linear classifiers for (5), 4, 0.2, 0 1, and clipping norm R = 0.1.
Theorem 3. Fizing the ullack magreilude 7, if dala distribution in Equation (5) sulisfies K²+1 then whenever by < bDP < bo. we have
2K y < 10-7|+101,
min lineer
(0) < Rg(DD)) < (FO)
Furthermore, any intercept with better natural accuracy than dpp must have worse robust accuracy:
Ro(f) < (o((f))==> R,(f> RyDFP).
Theorem 3 shows that, under some conditions, DP models are more robust than natural models, and cannot be dominated in the Pareto optimal sense. That is, DP linear classifier can be Pareto optimal in terms of the robust and natural accuracy. We visualize the premise b, < hpp < by in Figure as well as in Figure 1, and the result R,(f)) < (SD))) (fo))) in Figure
We emphasize that, results the loc attacks is generally extendable to l2 attacks. Put differently, we show that DP models can be adversarially robust and Pareto optimal under both loo and l2 attacks Corollary 1 (Extension to l2 attacks). All theorems hold for l2 attacks by changing 7 7/Vd.
Remark 3. Theorem gives sufficient and necessary condition for the DP classifier be more robust than all classifiers at one attack magnitude y"; Theorem 3 gives sufficient but not necessary condition for the DP classifier to be more robust than one classifier (the natural one) at many attack magnitudes.
4 Training Private and Robust Neural Networks
In this section, extend our investigation beyond the linear classifiers in Theorem 1, Theorem 2, and Theorem 3, and study the robustness of DP neural networks. We emphasize that several state-of-the-art (SOTA) advances are act ually achieved by linear classifiers within the deep neural networks (Mehla et al., 2022; Tramer & Boneh, 2020), i.e. by finetuning only the last linear layer of Wide ResNet, SimCLR, and vision transformers. Therefore these advances fall in the same sctting as our theorems, although the neural networks are non-linear models in general.
By experimenting with real datasets MNIST, CIFAR10 and ColebA, we corroborate our insights gained from theoretically analyzing the linear models and empirically show that the DP neural networks can be adversarially robust in practice (despite being much more challenging to analyze). We use one Nvidia GTX 1080Ti GPU and the Renyi privacy accountant to calculate the privacy loss.
'In doep learning, the clipping on gradient level under lu norm (see Footnote 1), regardless of norm adversarial attacks, which are on the sample level, i.c. on Z;.

---

In summary, we have three key observations.
By selecting the hyper-parameters carefully, we can remain exactly the same level of DP but vastly stronger robustness. Especially, we visualize the distinct landscapes of robust accuracy and natural accuracy over (), which also depend on the choice of optimizers (see Appendix E).
We observe pivacy-accurrg--rrusstess tradcoff, showing that DP models is Pareto optimal (with or without pre-training), thus extending Theorem to deep learning.
3. The robustness DP modols holds for general attacks, inchuding single-step or multi-stop (FGSM v.s. PGD), single method or ensemble (PGD v.s. APGD), and Ix or lz.
Remark 4. Our analysis also implies that DP models be resilient to data poisoning attacks, since adversarial examples can serve as strong data poisons (Fowl et al., 2021). Such resilience is empirically observed in (Yang et al., 2022; Hong et al., 2020).
4.1 Hyper-parameterr are keys to robustness
In DP deep learning, the training preer-parameters can be divided into two categories: some related to the privacy accounting, including the batch size B, the noise multiplier app, the number of iterations T;; the others are only related to the optimization but not to the privacy, including the clipping norm R and the learning rate v). That is, changing (R, v) can influence the accuracy and the robustness without affecting the DP guarantee.
On one hand, R has to be small achieve SOTA natural accuracy. Large models such as ResNet and GPT2 are optimally trained at R 1, even though the gradient's dimension is of hundreds of millions Kurakin et al. (2022); Li et al. (2021); Klause et al. (2022); Mchta et al. (2022). In fact, Bu et al. (2023b) adopts infinitely small R 0+ to achieve SOTA results, essentially applying per-sample gradient normalization instead of the clipping.
On the other hand, DP training empirically benefits from large learning rate, usually 10 times larger than the non-DP training. This pattern observed for DP-Adam (Li al., 2021, Figure 4) and for DP-SGD (Kurakin et al., 2022) over text and image datasets.
Interestingly, we also observe such choice of (R, 1) performs strongly in the adversarial robustness context (though not the same hyper-parameters). By the ablation study in Figure 5 for CIFAR10 and in Appendix B for MNIST, Fashion MNIST and CelebA, it is clear that robust accuracy and natural accuracy have distine- tively different landscapes over (R, 1)). We observe that the optimal (R, r)) should be carefully selected along the diagonal ridge for DP-SGD to obtain high robust and high natural accuracy. Otherwise, even small deviation can lead to sharp drop in the robustness, despite that the natural accuracy may remain similar (see upper right corner of 2D plots in Figure 5).
Remark 5. From Figure 5 (see also Figure 7, Figure 8 and Figure in the appendix), it is empirically sufficient to set R = 0+ and to only tune the learning rate n for both robust and natural accuracy, when using DP optimizers such as DP-SGD, Adam, DP-RMSprop and DP- Adagrad.. This approach, termed as automatic clipping by Bu et al. (2023b), reduces the 2-dimension hyperparameter search to a much cheaper I-dimension search fc.f. Table 1).
8

---

20-
55.20
00
8:78
0.770
7.2%
10
70-77
00
80
70
73.87
20:07
0.57
Pilous Areunney
T5AT
1:16
50
90.97
78.04
00:04
47.00
11.00
1L.12
0100
41
20
10
30
75,00
80.60
77.
16:
4.20
11:22
06
70
23.54
$3.99
214
73:30
79:16
19:15
NonTPP
01:77
05.00
67.10
10
42.42
91
91. 11
00/113
66.25
67/00
40.95
Nartro Acmenay
02.72
00.17
26
10.67
01.00
92. 46
01.65
00.05
67.00
20.20
46
2-
$9.44
01:54
02
09
71
24
-00
2-
64.76
N7.00
89-71
$1.11
00.97
$2.07
007-33
80.TH
$1.35
87:83
01.27
92.29
54
27
NonPP
1
Figure 5: Robust and natural accuracy by and R on CIFARIO. We use the same setting as in Tramer & Bonch (2020): pretraining SimCLR on ImageNct and then privately training using DP-SGD with momen- lum=0.9, under (c, 6) = (2, -e 5) and altacked by 20 sleps of (2 PGD.
Our ablation study demonstrates that the DP neural network, with 81.04% robust accuracy and 89.86% natural accuracy, can be more robust than the most robust of naturally trained networks (79.59% robust accuracy and 94.31% natural accuracy). If we trade some robust for the natural accuracy, we can achieve the same level of robustness (80.20%) at 91.27% natural accuracy, thus closing the gap between the natural accuracy of DP and non-DP models without sacrificing the robustness.
While Figure presents the result. of a single attack magnitude, we further study the influence of hyper- parameters under different attack magnitudes, with and without the adversarial training. We illustrate on CIFAR10 the Lx atlack performance in Table 2 and the la one in Table 3.
Table 2: Natural and robust accuracy of SimCLRv2 (Chen et al., 2020) and ResNet55 (Engstrom et al., 2019) on CIFARIO under 20 steps Loo PGD attack. Here cyan columns are adversarial training and white columns natural training. Adversarial training and robust papee-pprraeters are obtained by grid scarch over and R against Lx (2/255), and natural lyper-parameters are adopted from Tramer & Boneh (2020).
SimCLRv2 pre-trained on ualabelled ImageNet
5esNet50
DP
DP
DP
DP
DP
DP
DP
DP
DP
Non-DP Nou-DP Non-DP
Non-DP Non-DP
attack
c
2
2
4
8
8
c
8
00
magnitude
ady 9/250
robust accurate adv 9/255
robust nccurate ady 2/2ss
robust accurate ady 2/2ss
robust
accurate
ady f/2s accurate
7=0
90.46%
89.69%
92.87%
91.12%
90.91%
93.41%
91.70%
91 22%
93.64%
93.42%
94.29%
94.55%
87.03%
95.25%
2/3ss
83.83%
81.05%
33.21 %
85.28%
82.53%
57.80%
86.12%
83.02%
68.90%
89.07%
79.79%
59.56%
//ss
75.56%
68.85%
0.16%
77.73%
70.21%
9.69%
78.62%
71.08%
28.09%
83.07%
53.56%
15.99%
#/20
53.61%
39.63%
0.00%
56.90%
38.39%
0.00%
57.84%
39.28%
0.01%
66.99%
8.14%
0.00%
53.49%
0.00%
7
19/25a
8.05%
1.20%
0.00%
10.30%
0.65%
0.00%
11.31%
0.91%
0.00%
20.67%
0.00%
0.00%
18.13%
0.00%

---

Table 3: Natural and robust accuracy of SimCLRv2 (Chen et al., 2020) and ResNet50 (Engstrom et al., 2019) on CIFARIO under 20 steps l2 PGD atlack. Here cyan columns are adversarial training and white columns are natural training, Adversarial training and robust hyper-par peeerrearmmeterss are obtained by grid search over and R against (0.25), and natural hyper-parameters are adopted from Tramer & Bonch (2020).
SinCLRv2 pre-trained on unisbelled ImageNNe
ResNet50
DP
DP
DP
DP
DP
DP
DP
DP
DP
Non-DP Non-DP Non-DP Non-DP Non-DP
attack
<=
2=
c= 2
<=
4
4
c= 4
<=
8
6
8
magnitude
adv 0.25
robust accurate adv 0.25
robust accurate adv 0.25
robust accurate
adv 0.25
robust
accurate
ady 0.5
accurate
Ty ==
91.07%
89.69% 92.87%
91.19%
90.91%
93.41%
91.87%
91.22% 93.64%
93.88%
94.29%
91.55%
90.83%
95.25%
? 0.25
82.
02.69% 82.12% 59.91%
83.69%
83.35%
74.10%
84.27%
83.77%
79.03%
85.20%
82.91%
72.63%
82.34%
8.60%
7 = 0.5
70.54%
71.99%
12.76%
72.42%
72.79%
40.97%
72.00%
73.08%
54.53%
71.22%
63 63.32%
35.95%
70.17%
0.28%
7 1.0
38.57%
46.30%
9.49%
42.97%
44.46%
8.97%
40.17%
44.65%
9.68%
39.74%
33.34%
0.98%
40.47%
0.00%
We evaluate the robust and natural accuracy on the DP models in Tramer & Boneh (2020), considering two groups of hyper-parameters: the natural hyper-parameters reproduced from Tramer & Boneh (2020) that has highest natural accuracy, and the robust hyper-paramede parameters from a grid search on (Rt 17) for the highest robust accuracy. From Table and Table 3, we see that even under the same privacy constraint (including the non-DP scenario), the robustness from different. hyper-parameeers can be fundamentally different. For example, DP SimCLR at = 2 can be either very robust ( 70% accuracy at y 4/255) not robust all (0.16% accuracy). Consequently, our results may explain the mis-understanding of previous researches by the improper choice of the syper-parameters
Scrutinizing the natural training with robust hyper-parameters, we see that, across all 1o attack magnitudes Y 7= (2/255, 4/255, 8/255, 16/255] and l2 ones 7 = {0, 0.25, 0.5, 1.0,, DP SimCLR can be more robust than the non-DP SimCLR, in fact comparable to the adversarially trained ResNet50 that is benchmarked in Engstrom et al. (2019) and to the adversarially trained DP SiRCLR. To be assured, we further demonstrate that our choice of small R and large 1) is consistently robust Fashion MNIST, CIFAR10 and CelebA in Appendix C.
4.2 Pareto optimality accuracy and robustness
0.7
Pre-traio, DP-SGD, R 0.1
Pre-train, DP-SCD, R-2
0.6
Pre-train, SGD
No pre-train, DP-SGD, R 0.1 No pre-train, DP-SGD, R R=2
x
Robost Accuracy
0.5
No pre-train, SGD
0.3
0.2
00
0.1
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Clean Accuracy
Figure 6: Robust and natural accuracy on CIFAR10 at different itcrations. Dots are CNN from Papernot et al. (2020). Crosses are SimCLR from Tramer & Boneh (2020). See details in Appendix D.
In the standard non-DP regime, the tradeoff between the accuracy and the robustness is well-known (En- gstrom et al., 2019). We extend the Pareto statement in Theorem 3 to DP deep learning, thus adding the privacy dimension into the priacy-accuracc-rooiss models on CIFARIO (one pre-trained, the other not) achieve Pareto optimality with proper hyperparamters,
tradeoff. Tn Figure 6, we show that two strong DP
10

---

and thus cannot be dominated by any natural classifiers. This can be observed by the fact that no green cross (or dot) is to the top right of all blue crosses (or dots), meaning that any natural classifier may have better robustness or higher accuracy, but not both. Therefore, our observation supports the claim that DP neural nelworks can be Parelo optimal in terms of the robustness and the accuracy.
4.3 DP neural networks be robust against general attacks
Following the claim in Section 4.1 that DP neural networks can be robust against l2 and lxc PGD attacks, we now demonstrate the transferability DP neural networks' robustness against different attacks.
Table 4: Natural and robust accuracy of FGSM(Goodfellow et al., 2014), BIM(Kurakin et al., 2018), PGD (Madry et al., 2017), APGD (Croce & Hein, 2020), PGDyyMadry et al., 2017) and APGD2(Croce & Hein, 2020) on CIFAR10 under general adversarial attacks. Same model as Table 2 with the robust hyper- parameters. See detailed attack settings in Appendix D.
Natural FGSM
BIM
PGD
APGD
PGD2
APGD2
DP = 2 89.86% 69.73%
68.85%
68.85%
68.85%
72.10%
71.97%
DP c <= 4 90.91%
71.13%
70.21%
70.21%
70.21%
72.79%
72.63%
DP é = 91.22%
71.88%
71.08%
71.08%
71.06%
73.08%
72.99%
Non-DP 94.29%
56.24%
53.56%
53.56%
53.56%
63.32%
62.93%
This is interesting in the sense that DP mechanism does not intentionally defend against any adversarial attack, while the adversarial training (Goodfellow et al., 2014) usually specifically targets a particular attack, e.g. PGD attack is defensed by PGD adversarial training. In Table 4, we attack on the robust models from Table 2, with the robust lyper-paramelers. We consistently observe that DP models can be adversarially robust and more so than the non-DP ones on MNIST /Fashion MNIST/CCelebA Appendix C, if the hyper- parameters (R, 17) are set properly.
4.4 Large scale experiments on CelebA face datasets
We further validate our claims on CelebA (Liu et al., 2015), public high-rcsolution (178 x 218 pixels) image dataset, consisting of over 200,000 real human faces that are supposed be prolected against privacy risks. We train ResNet18 (He al. (2016), 11 million parameters) and Vision Transformer (VYT, Dosovitskiy al. (2020), 6 million parameters) with DP-RMSprop. Both models implemented by Wightman (2019) and pretrained on ImageNet. The experiment can be reproduced using the DP vision codebase Private Vision by (Bu et al., 2022a).
Table 5: Natural and robust accuracy on CelebA with label 'Smiling', DP-RMSprop, under 20 steps L.0 PGD attack. Here the hyper-parameters have not been carefully searched for the best robustness. See details Appendix D.
ResNet18
VIT
attack magnitude
DP
DP
DP
Non-DP
DP
DP
DP
No
e
=
= 0
=
6=8
= DO
7
80.10%
85. 10%
88.48%
91.91%
92.30%
92.33%
92.09%
92.87%
7 = 2 /255
1.26%
0.47%
1.03%
1.19%
1.42%
2.02%
10.35%
0.08%
7 4 /25
0.01%
0.01%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
7 8 /255
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
16/255
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
11

---

Table Natural and robust accuracy of FGod(Good et al., 2014), (IMakurakin al., 2018);
P el al., 2017), sCroce & Hein, 2020), PGDæ(Madry et al., 2017) and APGD2 & Hein, 2020) on CelebA with label 'Smiling' under general adversarial attacks. Same ResNet18 as Table 5, See detailed attack settings in Appendix D.
Natural FGSM
BIM
PGD
APGD
PGD2
APGD2
DP 6 80.10%
24.47%
1.24%
1.26%
1.18%
47.02%
46.25%
DP 4 85.10%
24.40%
0.45%
0.47%
0.41%
56.92%
56.09%
DP. < == 8 88.48%
29.32%
0.97%
1.03%
0.41%
57.40%
56.69%
Non-DP 91.91%
22.94%
1.09%
1.19%
0.68%
66.89%
66.13%
In Table 5 and Table 6, we observe that DP ResNet18 and ViT are almost as adversarially robust as their non-DP counterparts, not more robust. These observations are consistent with those simpler models on tiny images (c.f. Table and Table 4). We note that unlike the linear classifiers on CIFARIO in Table 2, training all layers on CelebA in Table 5 are much more vulnerable even at ? 2/255.
5 Discussion
Through the lens of theoretical analysis and extensive experiments, we have shown that differentially private models can be adversarially robust and sometimes even more robust than the naturally trained models. Morcover, DP models can be Pareto optimal in the sense that more accurate natural model must be less robust (see Theorem 3 and Figure 6). Our conclusion holds for various attacks with different magnitudes, from linear models to large vision models, from grey-scale images to real face datasets, and from SGD to adaptive optimizers. We not only are the first to reveal this possibility of achieving privacy and robustness simultancously, but also are the first to offer practical guidelines for such important goal (scc Scction 4). To be concrele, we demonstrate that hyper-parameters clipping norm R and learning rate 1) exert lot of influence on the robustness and accuracy, while remaining equally private. We hope that insights will encourage the practitioners to adopt techniques that protect the privacy and robustness in real-world applications.
For future directions, a more thorough study of private and robustness learning is desirable, by extending to language models, recommendation systems, and so on. We believe a new analysis when parameters are trainable will be challenging but enlightening. Especially, given that larger models are empirically more accurate under the fixed privacy budget, it would be interesting to understand whether the robustness also improves, at least persists, with larger model sizes. Another direction is to further investigate the adversarial training with DP optimizers, whose performance may go beyond our Pareto frontier (of the robustness and the accuracy) that is based on the natural training.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Nya Mironov, Kunal Talwar, and Li Zhang, Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on compuler and communications security, pp.
2016.
Franziska Boenisch, Philip Sperl, and Konstantin Böttinger. Gradient masking and the underestimated robustness threats of differential privacy in deep learning. arXie preprint arXiv:2105.0700 2021.
Zhiqi Bu, Jinshuo Dong, Qi Long, and Weijie Su. Deep learning with gaussian differential privacy. Harvard data science review, 2020(23), 2020.
Zhiqi Bu, Jialin Mao, and Shiyun Xu. Scalable and efficient training of large convolutional neural networks with differential privacy. Advances in Neural Information Processing Systems, 35:38305 38318, 2022a.
12

---

Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Differentially private bias-term only fine-tuning of foundation models. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022b.
Zhiqi Bu, Hua Wang, Zongyu Dai, and Qi Long. On the convergence and calibration of deep learning with diffcrential privacy. Transactions on Machine Learning Research, 2023a.
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Automatic clipping: Differentially private deep learning made easier and stronger. Advances in Neural Information Processing Systems, 2023b.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium ( (USENIX Security 21), pp. 2633-2650, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlocking high accuracy differentially private image classification through scale. arXiv preprint arXiv:2004.. 13650, 2022.
Jía Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE' Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009. doi: 10.1109 21110//CVR.2009.52666
Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint arXic:2010. 1 11929, 2020.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sei., ((3-4):211 1071111717,22100 2014.
Logan Engstrom, Andrew Tlyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https: github. con//aadtyllab/rrbutteessss
Liam Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojeiech Czaja, and Tom Goldstein. Adver- sarial examples make strong poisons. Advances in Neural Information Processing Systems, 34:30339-3000 30351,, 2021.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 1xXic:1412.671 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Sanghyun Hong, Varun Chandrasekaran, Yigitcan Kaya, Tudor Dumitraş, and Nicolas Papernot. On the effoctiveness of mitigating data poisoning attacks with gradient shaping. arXiv preprint Xiv:2002.114977 2020.
Gary Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in Real- Life Immages: detection, alignment, and recognition, 2008.
Helena Klause, Alexander Ziller, Daniel Rueckert, Kerstin Hammernik, and Georgios Kaissis. Differentially private training of residual networks with scale normalisation. arXiv preprint arXi::2203.0033 2022.
13

---

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99 112. Chapman and Hall/CRC. 2018
Alexey Kurakin, Steve Chien, Shnang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta, Toward training at imagenet scale with differential privacy. arXiv preprint arXiv:2201. 12328, 2022.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied docu- ment recognition. Proceedings of the IEEE, 86(11):2278-233) 2324, 1998. doi: 10.1109/5.726730
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Gcambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pp. TEFE, 2019.
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. arXiv preprint arxie:2110 100 05679, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) December 2015.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. Xiviv preprint ar arXiv: 1 1706006883 2017.
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. arXiv preprint arXiv:1710.00 069663,, 2017.
Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin, and Ashok Cutkosky. Large scale transfer learning for differentially private image classification. arXiv preprint arXiv:2205. 02973, 2022.
Felipe A Mejia, Paul Gamble, Zigfried Hampel-Arias, Michael Lomnitz, Nina Lopatina, Lucas Tindall, and Maria Alejandra Barrios. Robust or private? adversarial training makes models more vulnerable to privacy attacks. arXie preprint arXiv:1906.064 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Harsha Nori, Rich Caruana, Zhiqi Bu, Judy Hanwen Shen, and Janardhan Kulkarni. Accuracy, interpretabil- ity, and differential privacy via explainable boosting. In International Conference on Machine Learning, pp. 8227 8237. PMLR, 2021.
Nicolas Papernot, Abhradeep Thakurta, Shuang Song, Steve Chien, and Ulfar Erlingsson. Tempered sigmoid activations for deep learning with differential privacy. arXie preprinl 1xXi:22007.4111, pp. 10, 2020.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communi- cations Security, pp. 241-257, 2019.
Shuang Song, Thomas Steinke, Om Thakkar, and Abhradccp Thakurta. Evading the curse of dimensionality in unconstrained private glms. In International Conference on Artificial Intelligence and Statistics, pp. PMLR, 2021.
Tijmen Tieleman, Geoffrey Hinton, et al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4(2):26 31, 2012.
14

---

Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). arXiv preprint Xr::200111. 1. 1660, 2020.
Nurislam Tursynbek, Aleksandr Petiushko, and Ivan Oseledets. Robustness threats of differential privacy. arXiv preprint arXit:2012.07828, 2020.
Ross Wightman. Pytorch image models. http:///ht.....oooaaaaaa//////ta 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking
machine learning algorithms, 2017.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust to be fair: Towards fairness in adversarial training. In International Conference Machine Learning, pp. 11492 11501. PMLR, 2021.
Yu Yang, Tian Yu Liu, and Baharan Mirzasoleiman. Not all poisons are created equal: Robust training against data poisoning. In International Conference on Machine Learning, pp. 25154-25165. PMLR, 2022.
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked manguage-models. In Proceedings of the 60th Annual Meeling of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 1, 2022.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Newral Information Processing Systems, 32, 2019.
15

---

A Proofs
Proof of Theorem 1. Extended from Xu et al. (2021)) By Xu et al. (2021, Lemma 2), according to the data symmetry in (5), the optimal linear classifier has the form
1, by arg minR, (f(::,,)))
Recall that (8) proves that for such linear classifier, the robust error is
(1) $ vd(0 -d) 1
1 +
vd(0 -1)
b b) K Kv
1
V do
2
Ko
where ① is the cunulative distribution function of standard normal.
The optimal by to minimize (f) is achieved at the point that (R,(1) 0. Thus, by satisties: 8b
ò vd(o 7)
by vda
1 o
[d((0=)) (-val-=) (
by
K Kvdo 0
Vala
Ka
Kvdo
where o is the probability density function of standard normal. This equals to
Vd 7) +
by
Va(0 -?)
by Ko Kvdo
= ④
K
do
and
vd(0 77)
by
valo -7)
by
K Φ
1
+
Ka
Kvda
dda
Vile
by KVd0
by Vide
Ko
e
It not hard to see
vd((0-)) -1)
by
Vall -7) 7)
by
2 log K
+
Ka
Kvdo
vda
which re-arranges to a quadratic equation
1
2(0
1
d(O (1)-
1
by
(1
(1
2 log K.
do
K2
K2
K2
The solution is therefore explicit as
K2 + b,
4d - 7)2
K²-1 d(0 -7) -7 K
-K (K2-1)² + do²g(K),
where q(K) 21og which is positive constant and only depends on K. By incorporating b, into (8), we K²-1 can get the optimal robust error R. (f=):
1 / (B(K,n) K 3( q(K) R, (f,)
1 20
(-KB(K,n) + VB(K, 9(K)
+
where B(K,7) x²_1 vd(l-1)
16

---

Proof of Theorem 3. We denote the two roots of aR, (f(1)) 0 as by and by. Here b, = by. Clearly R, (6)
is increasing in (67 67). We hope to show bo E (by, b7 Yvy > 0, so that (b) is also increasing in (by, bo).
Note their Equation (17)
vd(o- -7)
1
vd(o 7)
(b( 1
b) + =
do b)
0
v vd
Ka
Taking derivative w.r.t. b
(b)
2v Ø1 vd(0- -7)
vd -7) 7)
+
b)
K Kvda b)
bb
0
dd
2K do
do
Ko
Setting this derivative 0:
vd(0- (7)
1
Vd(0 -7)
1
0 Ki(
+
b) 0(
K K b)
dd
Ko
which means
va(0-1)
Kvdo b)
Ka
K
6 va(e~2) + Vda
b)
Using the standard normal density ubu) = e- 21/2 and ((u) ((v)
= c -12)/2 we have
vā - 7) + 1 b)2 - (- vd(0 -7)
Kvd b)² = 2 2lo K
0
vda
Ko
= (-d(0 - 7) + b)2 - (-d(0 - 7) - -b)² = 2do K2 22log K
(K2 - 11b²² 2d(0 (K + 1)b + d( (K² 1) - -2do K log K 0
By x ===
+
4nc Anc = -a
+ (
/ 12 _/ we know
2a
2a
2a
K²+1 +1
K² - - + K2do2g(K)
b
d(0 7) + V (2
K2
K2 +1
(0 - 7)
K2-1 -1 d(0 7) K (K2 1)²
+ do²
We now derive the sufficient condition that bn < b
22 +1 1 d(0) K2-1 1
(0) (K3-1) 1)3
K2 +
4d - 7)2
V
+ do2g(K)) < K2 d(0 7) + KV
+ do2g(().
(K3 1)2
This is equivalent to
k2 +
LuP(--9) + +do²g(K)+ 1
4d202
K2 dy K
(1
(K2 1K3--)3 + +doPg(k))
(K2 1)²
Therefore, it suffices to have
K²
2K 7 < 10 - 1 |8|
Finally, it is easy to see the Pareto statement Ro(f) < Ro(SDP) - Ry(f) > (FDP)). A necessary but not sufficient condition for Ro(f) < Ro(fDe) given that by > bDp is b > bDp: since by is a minimizer which means Ro is decreasing on the interval (-o0,bo). bo). Similarly, Ry increasing on the right of by and thus has higher robust error.
17

---

Proof of Corollary 1. We can characterize the robust errors based on lu attacks in a similar fashion to (8).
We notice that
( =P(3)pp S.st. f(x + p) # y) = max P(f( +p) 4
iplls
| F(f 7a/ * y - P(f(x 7a/Va) * +
1
In short, the same analysis is in place except & +%/VV when we switch from 1. to la attacks.
B Ablation Studies
B.1 CelebA
70
21 1n-2
UL so as so au 102 101000
Cn
2 10-
1.00 45.44 auus 45.2 on in
11.00
10
13.5 17-8 4a3s ausicss sw
10
2:3 100
08.2 10.510
2-1 10.
00.02
74.10
24.21
76.50
41.012
10-1
x.7xx 10
12.22
12.52 12
12.05 12.00
10-4
2:1 100
(2/1 (6.25
10
11.0000
20
70.00
R
Nee.17 17.
90
Natural Arcuaay %
801
2' x'x110
3-3 x 10
94109% 14
1604
20
2-1 100=2
04 64.7994 4491
07714
2- 10
35.55
(1.01
10
2-1 10
71
74.00
2-12 1um2
1000 00103
9200
03.07
2
Figure 7: Robust and natural accuracy of 1) and R CelebA with label MMaee. We train 2-layer CNN using Ad-Adam and attack by 4x (2/255) PGD attack. Same in Figure 13. Here epochs 10.
2, batch size 512,
18

---

8.2
10
20
51.11
6306
11
11
80
01.94
2.8
57
0.07
00.3 00
41
80
10
96.8
35
75
05
51.7
3005
2.4
75.82
65.31
30.06
41.00
6208
64.0
0.06
00
2.0
F1.65
13-50
53.79 gies
8566
0.05
12.70
00.96
10.00
43.09
61.20
61.94
01.05
63.00
0.01
00.00
15.00
1.2
77.84
70.84
76.46
63.05
31.8
00.0
44.78 00.37
26.34
10.11
07.50
64.53
08.00
62.20
$0..0
.00
0.8
22,00
74.42
14.12
64
97577
11.08
41.05
41.30
6.02
002--7.00
71.15
09
82
69
70
72.87
07
24
07.
71.65
68.66
75.31
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Non.P
0.1
0.5
1.0
1.5
2.5
3.0
3.5
4.0
Non.DP
76.11 08.00
$6.00
84.30
16.06.680 84
12-12.77
30.20
0.28
13. 65
43.21
00.51
36. 02 42
70
74.86
68.45
29.01
60
66
0.2
24.85
10.18
51.21
51. 79 53. (6 00-55
1/5
as. 40.629
20953 206.000...0000000 40.0000.000000000.
000
0.002
3. 1965 20 6oli s2nd 2017 sa 11 22
50
0.12
74.07
74.88
61.03
68.72
11.74 58 07 12.41
39.02
603.00
0,0015
6482. unwi
s ra0 0000 10.0 14. 10
12.77
0,08
60.77
37
40
0.001
1.8872
12m
1557 s.1. 200.0000..900000..0 05 0269444.......
67
30.00
00.81
75.00
75.79
73.80
60.
03.31
18:00
15177
0.2005
51
0076,
1077 25755555588 2575. 76.00 721117
2077
10:00
2.0
NonDP
0.01
0.02
0.05
0.1
06
1.0
1.5
20
2.6
3.0
NonDP
Figure 8: Robust accuracy of CelebA with label 'Male' under different optimizer, trained with 2-layer CNN and attacked by Lo0 (2/255) PGD attack. Top left: SGD. Top right: Adagrad. Bottom left: SGD momentum. Bottom right: Adam. Here = 2, batch size 512, epochs = 10.
19

---

50
27
00.00
20.00
10.05
40.97
49.97
27 10
30.97
30.35
18.97
09.91
19.55
50003
Robsat
2"
11
19.97
19.97
10
2-1000001
11.39
11:41
13.45
11:66
71-300
9.00
10
2- 10
16:11
11:47
10:42
TM
8:37
11 64
2.03
274
10
9-5 x 10
1.43
1.36
1.70
1:73
1:30
41
1.77
x. 1001
2:
2
2-
2
2
10 10
Avcuraty %
85
10
75
89 10
10.07 40.57
40.57
40.00
40.07
40.07
88 10
19.07
40.57
40.57
40.50
40.07
49.97
40.07
70
21 10
91.77
9L.57
60
65
21 10
2"1 x 10
10
2-3 10
SW.t N8.59 18
87
2-2
10
2-5 10-
96.15
71:00
10-4
2-1
2-
2-
2-1
21
22
22
Figure 9: Robust and natural accuracy of 7 and R on CelebA with label 'Smiling'. We train ViT-tiny using DR-RMMprop and attack by La0 (2/255) PGD attack. Here 2, batch size 1024, epoch 1.
20

---

B.2 CIFAR10
21
18.97
24.12
16:31
21.11
2.52
11:235
1.200
11:11
28
70
3
27
40.39
61.11
76.00
21.87
51
10
1.31
1000
01:11
40
27-
36.90
20-10
M:.0
00.11
17.45
67-37
15:03
6.28
71-23
20
2-2
21.22
14.5
29.16
66.23
23.23
$2.00
24-0 93.33
12.23
122-12
11.14
191.15
12.71
92
Accunsy
94.10
12
6:28
90.96
01.03
92.48
02.83
11.90
1/2
94.02
90.00
02.07
00172
48.40
81:94
86.70
20.20
00.72
91.73
02.05
02.41
01.18
74.44
81:03
24.20
12.13
61.33
84.8
60-00
MLAD
12.00
92.57
NeuDP
Figure 10: Robust and clean accuracy of 7) and R on CIFAR1O, transferred from SimCLRv2 pre-trained on unlabelled ImageNet. We use DP-SGD and attack by lo (2/255) PGD attack. Here e 2, batch size 1024, epochs 50.
21

---

B.3 MNIST
8.55
131
10:30
40
27
37.00
41.24
13.344
56. M
000000
10
01.00
21122
Robuue
27.15
50.00
83.76
36. 04
47.44
50
00.113
10.02
2
00:81
555
23
2
01112
9:43
13:40
22
34.50
10.01
12.10
15:21
R
NonDP
80
2m4
41.19
62
10.10
96.91
50 21
97.10)
50 20 $1.16
59.
00.63
40
61
77
00
70
12:14
70.08
90.00
02.22
90,
00
07 (1
37.60
55.
50
-00
-30
2-2
216
11.99
24.27
65.34
30.20
70.43
90.64
00..00
96.37
97.66
-00
2-1
03.05
05.00
27
NenPP
Figure 11: Robust and clean accuracy of 7 and R on MNIST. We train the CNN from Tramer & Boneh
(2020) using DP-SGD and attack by 460 (32/255) PGD attack. Here 2, batch size 512, epochs 40.
22

---

B.4 Fashion MNIST
80
07T5. Tien noto at aots oon an onee ootole
10.00
miza 1NTs mm orgs nogss aots nors 21
1.00
27
ts. 1...100.9722 74T 2079 6er3 2.85tuu 2070 a
10.00
Robuue
7.65 1071 sats sutt-orr 20th
10
2.
11055 4007
1000 71 307 nomo a Ty 77 10r
71.
21
21
2-
2m6
K.00
24.7540
1OTO SOFD 7271 01775.0
$3.00
R
2-5
NonDP
Arcumey 5%
60
to 1200 atsa 0254 nost S00t sotu ao del Am
10.00
60
70
1en5 ARTS cast 7000 nown 7151 19t5
70
00
40
0.c0m 2015 st sosi 1300 Jres
50
3.500 7ika dsTo 8171 760. 4155 5180.00
0.005 1005
oono s1
2-4- 5.713366
6500 oan o 0 0 000 1.32
2
NonDPp
Figure 12: Robust and clean accuracy of n and R on Fashion MNIST. We train the CNN from Tramer
Boneh (2020) using DP-SGD and attack by Lx0 (2/255) PGD attack. Here e 2, batch size 2048, epochs
40.
23

---

C
More tables
Natural
FGSM
BIM
PGD
APGD
PGD2
APGD2
No
94.55%
18.71%
15.97%
15.96%
16.04%
35.95%
35.89%
DP 92.73%
10.35%
0.03%
0.03%
0.03%
12.76%
12.68%
DP 93.49%
30.10%
9.10%
9.09%
9.12%
40.97%
41.01%
DP 8
74%
31.86%
28.08%
28.09%
28.09%
54.53%
54.54%
Table 7: Natural and robust accuracy of models transferred from unlabelled ImageNet pre-trained SIMCLRv2 CIFAR10 under general adversarial attacks with Yoe 4/255 and 72 0.5. Attack steps are 20 if applicable. Model hyper-parametors are directly adopted from Tramer & Bonch (2020) for highest natural accuracy. DP models are trained using DP-SGD, R 0.1, !DPP 4, momentum 0.9, batch size = 1024. Non-DP models are trained using SGD with the same ppper-parameters except Thon-DF DP = 0.4.
Non-DP
DP
DP
DP
attack magnitude
2
= 4
6 8
? 0.0
99.24%
98.01%
98.32%
98.50%
0.25
97.57%
95.29%
95.94%
96.65%
? 0.5
93.32%
90.28%
91.71%
92.97%
7 1.0
66.58%
63.95%
73.32%
77.08%
7 2.0
36.28%
39.88%
51.48%
52.74%
Table 8: Robust accuracy on MNIST under 20 steps 12 PGD attack. Model rrper-paramcters are directly
adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R 0.1, 'pP 0.5, momentum 0.9, batch size 512. Non-DP models trained using SGD with the same dyper-parametens except MP 0.05.
No
DP
DP
DP
attack magnitude
c= 00
c=
c= 4
c=
= 0.0
99.24%
98.01%
98.32%
98.50%
= 2/255
98.73%
97.12%
97.43%
97.84%
4/255
97.88%
95.78%
96.32%
97.13%
7 = 8/255
95.32%
92.31%
93.51%
94.74%
= 16/255
82.06%
77.67%
80.28%
85.82%
Table 9: Robust accuracy on MNIST under 20 steps 1.0 PGD attack. Model byper-parameters are directly adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R 0.1, NDP = 0.5, momentum = 0.9, batch size = 512. Non-DP models are trained using SGD with the same pararpparaesters except Tho- pp 0.05.
24

---

Natural
FGSM
BIM
PGD
^PGD
PGD2
APGD2
NP
99.24%
97.92%
97.88%
97.88%
97.77%
93.32%
93.27%
DP 98.01%
95.89%
95.80%
95.79%
95.63%
90.28%
90.15%
DP 98.32%
96.45%
96.32%
96.33%
96.27%
91.71%
91.68%
DP 6
98.50%
97.19%
97.15%
97.15%
97.06%
92.97%
92.94%
Table 10: Natural and robust accuracy of CNN models on MNIST under general adversarial attacks with
Too 4/255 and ?/2 = 0.5. Attack sleps are 20 if applicable. Model hyper-parameterrs are directly adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R 0.1, TDPP 0.5, momentum 0.9, batch size 512. Non-DP models are trained using SGD with the same hyper-parameters except Mn 0.05.
No
DP
DP
DP
attack magnitude
<=
< 2
c= 4
c= 8
0.0
89.75%
85.95%
86.60%
86.74%
= 0.25
57.37%
69.24%
72.93%
75.35%
0.5
25.21%
46.09%
54.30%
59.23%
1.0
7.87%
16.77%
25.95%
29.08%
2.0
7.47%
11.77%
16.85%
17.00%
Table 11: Robust accuracy on Fashion MNIST under 20 steps 12 PGD attack. Model hyper-parameters are directly adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R = 0.1, VDP = 4, momentum = 0.9, batch size 2048. Non-DP models are trained using SGD with the same paper-parameters except Mmon DP = 0.4.
Non-DP
DP 2
DP
DP
attack magnitude
<
= 4
? 0.0
89.75%
85.95%
86.60%
86.74%
? 2/255
76.19%
78.29%
79.84%
81.47%
? 4/255
64.46%
69.75%
72.60%
74.72%
7 8/255
47.24%
54.62%
57.87%
60.52%
16/255
23.26%
28.51%
31.68%
30.90%
Table 12: Robust accuracy on Fashion MNIST under 20 steps loc PGD attack. Model hyper-parameters are directly adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R 0.1, vpp 4, momentum 0.9, batch size 2048. Non-DP models trained using SGD with the same pper-paaaeeeerss except Thon DP 0.4.
Natural
FGSM
BIM
PGD 0c
APGD
PGD2
APGD2
No
89.75%
70.41%
64.56%
64.44%
53.41%
25.21%
23.13%
DP c= 2
85.95%
72.11%
69.76%
69.71%
67.13%
46.09%
45.41%
DP ===4 4
86.60%
73.67%
72.68%
72.69%
70.84%
54.30%
53.92%
DP c 86.7
74%
75.45%
74.75%
74.74%
73.71%
59.23%
58.98%
Table 13: Natural and robust accuracy of CNN models on Fashion MNIST under general adversarial attacks with Too 4/255 and adopted from Tramer & Boneh (2020) for highest natural accuracy. DP models are trained using DP-SGD, R = 0.1, ?DP = 4, momentum = 0.9, batch size = 2048. Non-DP modcls are trained using SGD with the same hyper-parameters except M = 0.4.
0.5. Attack steps are 20 if applicable. Model hyper-parameters are directly
25

---

Natural
FGSM
BIM
PGD
APGD
PGD2
APGD2
No
94.29%
14.48%
12.02%
12.00%
12.03%
31.36%
31.28%
DP <=
92.73%
15.70%
1.59%
1.61%
1.62%
28.05%
28.06%
DP =
93.49%
30.89%
5.23%
5.27%
5.25%
35.96%
35.98%
DP 6 93.
03%74%
9.66%
4.30%
4.29%
4.31%
33.21%
33.23%
Table 14: Natural and robust accuracy of models transferred from unlabelled ImageNet pre-trained SIM-
CLRv2 on CIFARIO under general adversarial attacks with 7/o = 4/255 and 72 = 0.5. Atlack steps are 20 if applicable. Model in each row is the most accurate model obtained by simple grid search: Non-DP: 7) = 0.5;
e=4: 2P 7) = 1.0, R 0.25; DP==4: 1) 8, R 0.0625, DP v 0.5, 1.0. All models are trained using
SGD or DP-SGD, momentum 0.9 and batch size 1024.
D Hyper-parameter setup
In Table 2, SimCLRv2 models are pre-trained on unlabelled ImageNet and fine-tuned on CIFAR10. Natural models are directly adopted from Tramer & Boneh (2020) for highest natural accuracy, where optimizer is DP-SGD and SGD, R 0.1, TDP = 4, Tmon-DP 0.4, momentum 0.9, batch size 1024. Robust models are obtained by grid search over 1) and R against Lo0 (2/255), where Non-DP: v 0.0625; DP n 4, R 0.0625; DP 17 0.5, R 0.0625, DP 17 0.125, R 0.25. Similarly, adversarial training models are obtained by grid search over 17 and fixed R = 0.0625, where Non-DP: 1) = 0.25; DP 1) = 0.5:
DP 1 = 1, DPe-s: 2. Other settings are the same as the natural ones. Adversarial attack is loo. 20 steps, alpha = 0.1.
In Table 3, SimCLRv2 models are pre-trained on unlabelled ImageNet and fine-tuned on CIFAR10. Natural models are directly adopted from Tramer & Boneh (2020) for highest natural accuracy, where optimizer is DP-SGD and SGD, R 0.1, 1/P = 4, !Joon DP 0.4, momentum 0.9, batch size 1024. Robust models are obtained by grid search
1 and R against (2(0.25), where Non-DP: " 0.0625; DP = 0.125, R 0.25. Similarly, adversarial
7 0.5, R = 0.0625, DP training models are obtained by grid search over and a fixed R = 0.0625, where Non-DP: n 0.0625; DP 7 1; DP 7 = 1, DP.=s: = 2. Other settings are the same as the natural ones. Adversarial attack is 12, 20 steps, alpha- = 0.1.
n 0.0625, R = 0.25; DP
In Figure 6, models are SimCLRv2 pre- pre-trained on unlabelled ImageNet and fine-tuned on CIFAR10 using DP-SGD, with 8, batch size 1024. Adversarial attack is loc PGD, 7 = 4/255, alpha-0.1.
In Table 4, models the same as in Table 2 with robust hyper-parametters, where optimizer is DP-SGD and SGD, momentum 0.9, batch size = 1024, Non-DP: = 0.0625; DP.=2: 0.125, R = 0.25. Adversarial attack steps = 20, alpha = 0.1 if applicable.
4, R = 0.0625; DP--1:
y= 0.5, R = 0.0625, DP
In Table 5, models are ResNet18 and ViT-tiny trained on CelebA, label Smiling. Images are resized to 224 224. Optimizer is DP-RMSprop with epochs = 5, batch size 1024, 17 0.0002, 0.1, delta=5e-6. Adversarial attack is 1.x PGD, 20 steps, alpha = 1/255.
In Table 6, models are ResNet18 as in Table 5, trained on CelebA, label 'Smiling' Images are resized to
224 x 224. Optimizer is DP-RMSprop with epochs = 5, batch size = 1024, 7 0.0002, R = 0.1, delta=5e-6. Adversarial attack is Lox (2/255) with alpha = 1/255 and (2(225) with alpha, = 0.2, 20 steps, if applicable.
In Figure 13, models are 2-layer CNN trained on CelebA label 'Male' using DP-Adam, where 2, batch size 512, epochs
10. Adversarial attack La0 (2/255) PGD, 20 steps, alpha 0.1.
26

---

Bolnns Accurncy %
Narneaa Accutary
60
50
27 10-2
21 10
10
10-2
2-1
1013
2-1
x 10"3
2-2
2
10-2
10-2
2
Figure 13: Robust and natural accuracy by and R on CelebA with label 'Male' We train a simple CNN with DP-Adam and test under 20 steps of (2 (2/255) PGD attack. See details in Appendix D.
E
Extra
E.1
Robust and accuracy landscapes of DP optimizers
We note that the diagonal pattern of the accuracy landscapes observed in Figure 5 (CIFARI0 & DP- Heavyball), Figure 10 (CIFARIO & DP-SGD), Figure 11 (MNIST & DP-SGD) and Figure 12 (Fashion MNIST & DP-SGD) is not universal. For example, in Figure 7, we show that adaptive optimizers are much less sensitive to the clipping norm R, as the landscapes are characterized by the row-wise pattern instead of the diagonal pattern. This pattern particularly obvious the small R regime, where the robust and natural accuracy are high (sce right panel in Figure 7.).
To rigorously analyze the insensitivity to the clipping norm in a simplified manner, take the RM- Sprop(Tieleman et al., 2012) When R is sufficiently small, the private gradient in (3) becomes
example, similar to the analysis in (Bu et al., 2023b) on DP-Adam.
g: Σ max (1, 11: (w:)
gr (2x,)
+ +oRN(O,T) = Σ
(1g (12/ RN f)
gx,))
R
(x;))l +oN(0,1) = R 9..
With private gradient g., DP- DP-RMSprop updates the parameters 0 by
0, = 0 + gr
(9)
Vū,"
where o is the squared average of gr, written
5 = - + (1 a)gg = (1--a)a =
(1- ~2 R²
(10)
Substitute (10) into (9), obtain an updating rule that is independent of the clipping norm Rt
R g:
g:
0 = 0.- + 4t
R E1(1 - a)aaaaa-000 0.- + 1/p
As a result, DP optimizers can have fundamentally different landscapes with respect to the hyper-parameters (R, n), which in turn may affect the accuracy and robustness as illustrated in Figure 8.
27


Reviews:
[{"review_id": "6Ms6n5wxe2", "reviewer": "Reviewer_xGht", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The authors start with a very simple linear model and they provide theoretical study on that. This simple model makes it easy for the reader to understand the main message of the paper. \n\n2. The authors go beyond the linear model and they show their claim through extensive empirical study on non-linear models.\n\nWeaknesses:\n\n1. There is no theoretical analysis beyond a model with only one parameter (parameter $b$)\n2. Theorem 2 somehow is misleading. It shows that there is a $\\gamma*$ for each $b_{DP}$ such that the model is robust to  attack level $\\gamma*$. And then the authors claim that this theorem implies robustness. However, there no lowerbound on $\\gamma*$. It can be very small. If $\\gamma*$ is close to zero, technically we have a very weak robustness. \n3. In the numerical experiment part, it is not clear how the privacy loss has been calculated. For DP-SGD, the parameter $\\epsilon$ depends on batch size, number of iterations. Technically, there is no closed form for $\\epsilon$ as a function of batch size, number of iterations. The privacy curve has to be estimated numerically. The authors also do not report $\\delta$ in the tables.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper argues that a differentially private machine learning model is actually a adversarially robust as well. The author prove this for a special case where the data comes from gaussian mixture model and the model has only one parameter. The authors also show that their theoretical results can be applied to fine-tuning of deep models through empirical study.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Is it possible to provide theoretical analysis for a scenario that $w$ is not fixed before hand and needs to be trained/learned? \n\n2. Could you please clarify why theorem 2 implies robustness. I feel $\\gamma*$ can be arbitrary small. \n\n3. Please clarify how the privacy parameters $(\\epsilon,\\delta)$ have been calculated? The reported privacy loss is for the whole training process or for only one iterations?", "broader_impact_concerns": "No Concern related  to Broader Impact Concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gmQLQr8KiJ", "reviewer": "Reviewer_sCkX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n1. The authors performed the experiments in many different settings: used three other datasets (including the experiments in the Appendix), two metric norms, three target models, and at least four different attacks.\n2. The theoretical analysis seems valid and meaningful.\n\n# Weaknesses\n1. The authors should test more attacks in the experiments. While the authors used four different attacks, i.e., FGSM, BIM, PGD, and APGD, in the experiments, all those attacks work similarly. To explain, all those methods compute gradients of the objective function, then compute the sign of the gradient to find the next direction in each step: They are different only in the number of steps, or random, starts or clipping, or step sizes. They use fundamentally similar optimization methods with slight differences in the details of each step.\n2. The authors only compared differentially private training to natural training. However, it is better to contain a comparison to adversarial training methods. While I don’t expect the differentially private training to have better robustness than the state-of-the-art adversarial training methods, it is still worth presenting the robustness gap between a differentially private model and optimal robust models. (Of course, the adversarial training is not optimal, but the comparison will show the lower bound of the gap.)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides the benefit of a differential private optimizer in the model robustness. Specifically, previous research showed that differentially private models might become more vulnerable to adversarial attacks. The authors challenge this previous finding and show that differentially private classifiers can achieve optimal robustness under some conditions.\n\nThe authors first provide a formal analysis on a linear classifier. The authors first compute the intercept for the optimal robust linear classifier in a mixed Gaussian distribution setting. Then, they show that differentially private optimization can achieve this intercept under some conditions. The authors also support the validity of the conditions with several experiments. Finally, the authors show that the differentially private training has better robustness for fixed attack radius than the natural training under some conditions. \n\nThen, with experiments, the authors extend their findings on a linear classifier to more general cases. The authors first explore two parameters, i.e., clipping radius and learning rate, and check that differentially private training may show better robustness with a proper parameter setting. Then, the authors demonstrate the better robustness of differentially private models than their naturally trained counterparts. Next, the authors explore the performance of differentially private models against different attack methods. Finally, the authors present the experiments on a larger dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I suggest making the paper easier to read. The following would improve the paper presentation.\n  - Write a “Related Works” section. The Related Works section helps the readers to understand the motivation by reading an overview of the research works related to the problem.\n  - Put a summary of experiment goals at the beginning of Section 4. A short list of research questions that the experiments demonstrate will help the readers understand the purpose of the experiments.\n  - The authors should put more details about the experiment setup in Section 4, rather than putting details in the captions of figures.\n  - Reduce the redundancies in writing.\n  - Have a proper “Conclusion” section. The first paragraph in the Discussion section does not sufficiently summarize the authors’ efforts.\n2. Improve the experiments to strengthen the paper.\n  - As mentioned in the weakness, the authors can use more attacks to support the insights better. Try attacks with different optimizations, e.g., CW attack, DeepFool, or gradient-free attacks, e.g., HopSkipJump.\n  - Add comparisons to adversarial training methods and analyze the robustness gap (between a differentially private model and an optimal robust model) in practice.\n3. Minor revisions\n  - There is nearly no margin between Table 1 and the text part. Please fix.\n  - In Figure 4, change “balanced” to ”left” and “unbalanced” to ”right”, or just put titles to each subplot. Also, the x_label in the left subplot seems incorrect; the gradient is missing inside the clipping.\n  - Does Corollary 3.2 require the clipping to change from $l_\\infty$ to $l_2$? If so, include details about the necessary changes and clarify that we need different DP optimization for different clipping norms. If not, write a discussion about why such modification is unnecessary.\n  - Theorems are numbered with 1,2,3…, but all the others are numbered with section numbers. Change Definition 2.1 to Definition 1, Fact 3.1 to Fact 1, and Corollary 3.2 to Corollary 1.\n  - To the best of my knowledge, “et al.” is not supposed to be used when all the author names are listed. Check the bib file and remove the uses of “et al.” in the bibliography.", "broader_impact_concerns": "I don’t see a particular broader impact concern regarding this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ORKGKdmvt7", "reviewer": "Reviewer_MWSt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1.  This paper answers a very important question: Can a model be simultaneously private and robust? Besides, the authors also provide concrete empirical suggestions on how to train a robust DP model (e.g., using small clipping norms and large learning rates). The results are very important since they could guide the future training of DP robust models. \n2. The paper is technically sound, and the experiments are extensive. \n\nWeaknesses\n1. Technical novelty is limited. Some proofs (e.g., Theorem 1) are direct extensions from Xu et al. (2021).\n2. There is a disconnect between the theory and experiments. For example, the DP mechanism used in the theoretical analysis is bias term fine-tuning, while the experiments use DP-SGD or DP-Adam.\n3. Presentation/Clarity needs improvement. (See requested changes below)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper provides an analytic and empirical study to understand whether the DP-trained models could be adversarially robust. Theoretically, the authors study the optimal adversarially robust errors that can be achieved by private linear classifiers (where the DP noise is only applied to the bias term fine-tuning) under certain conditions. Empirically, authors find out that with proper tuning of key hyperparameters (e.g., learning rate and clipping norm), DP-trained deep neural networks could also be more adversarially robust than the non-private ones.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The deviation of the last line in Eq. (8) is still unclear to me, e.g., why there is K in the second term? Could you please explain the deviation in more detail (e.g., how to derive the function input of CDF of the standard normal)?\n\n2. Why do you choose (R, $\\eta$) as the choice of hyperparameters for your experiment? How do other hyperparamters (e.g., batch size) affect the performance? Please justify this point in the paper.\n\n3. Is this a typo “76.13% accuracy can degrade to 0.00% accuracy?”", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has mixed opinions from the reviewers, while the positive opinions slightly outweigh the negative opinions. I carefully read the entire reviews, corresponding responses, and the revised paper. As my comment in "Claims And Evidence", both the theoretical and empirical results have some flaws. I think this paper needs a minor revision to support its claim better. Although I requested additional contributions, I presume that they will not weaken the contribution of the paper because (1) the authors claimed that the extension would be possible by introducing additional assumptions (2) even though AT models perform better in terms of accuracy-robustness trade-off the main theorem does not argue about the relationship between AT models and DP models. However, if AT models perform better than DP models, I think the main argument should be fixed to "DP models are theoretically and empirically better than naturally-trained (non-DP) models in terms of the accuracy-robustness tradeoff".
Decision: Accept (poster)