Augment then Smooth: Reconciling Differential Privacy with Certified Robustness
Authors: Jiapeng Wu
Atiyeh Ashari Ghomi
David Glukhov
Jesse C. Cresswell
Franziska Boenisch
Nicolas Papernot
Venue: Accepted by TMLR ()

Abstract: Machine learning models are susceptible to a variety of attacks that can erode trust, including attacks against the privacy of training data, and adversarial examples that jeopardize model accuracy. Differential privacy and certified robustness are effective frameworks for combating these two threats respectively, as they each provide future-proof guarantees. However, we show that standard differentially private model training is insufficient for providing strong certified robustness guarantees. Indeed, combining differential privacy and certified robustness in a single system is non-trivial, leading previous works to introduce complex training schemes that lack flexibility. In this work, we present DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees simultaneously by integrating randomized smoothing into standard differentially private model training. Compared to the leading prior work, DP-CERT gives up to a 2.5x increase in certified accuracy for the same differential privacy guarantee on CIFAR10. Through in-depth per-sample metric analysis, we find that larger certifiable radii correlate with smaller local Lipschitz constants, and show that DP-CERT effectively reduces Lipschitz constants compared to other differentially private training methods. Code is available at github.com/layer6ai-labs/dp-cert.

Published in Transactions on Machine Learning Research (07/2024)
Augment then Smooth:
Reconciling Differential Privacy with Certified Robustness
Jiapeng Wu
paul@layer6.at
Layer 6 AT
Atiych Ashari Ghomi Lager 6 AT
alaych@layyer..ctt
David Glukhov University of Toronto Vector Institute
dauid.glushoon@mail.uuoooo htt.:/oooo.mmm....ttttooooooo
Jesse C. Cresswell
jesse @layer6.
Lager 6
Franziska Boenisch"
boenisch@cispa.cco de
CISPA
Nicolas Papernot
nicolas. paaalasspppeeotttttooooo.ooc
University of Toronto & Vector Institude
Reviewed on OpenReview: https: // oppenreveee.... net/ forum? YNOIcnKger
Abstract
Machine learning models are susceptible to a variety of attacks that can erode trust, includ- ing attacks against the privacy of training data, and adversarial examples that jeopardize model accuracy. Differential privacy and certified robustness are effective frameworks for combating these two threats respectively, they each provide future-proof guarantees. However, we show that standard differentially private model training is insufficient for pro- viding strong certified robustness guarantees. Indeed, combining differential privacy and certified robustness in a single system non-trivial, leading previous works to introduce complex training schemes that lack flexibility. In this work, we present DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees simultaneously by integrating randomized smoothing into standard differentially private model training. Compared to the leading prior work, DP-CERT gives up 2.5x increase in certified accuracy for the same differential privacy guarantee CIFAR10. Through in-depth per- sample metric analysis, we find that larger certifiable radii correlate with smaller local Lip- schitz constants, and show that DP-CERT effectively reduces Lipschitz constants compared to other differentially private training methods. Code available at callub.com/laycrbbee labs/ /dppcert
1 Introduction
Machine learning (ML) models are becoming incrcasingly trusted in critical settings despite incomplete understanding of their properties. This raises questions about the trastworthiness of those models, encom- passing aspects such as privacy, robustness, and more. Society at large might expect all of these aspects to be accounted for simultaneously as ML's influence on everyday life expands, but scientists and practitioners still mostly grapple with each aspect individually.
Work done while the author the University Toronto and the Vector Institute.

---

Published in Transactions on Machine Learning Research (07/2024)
We aim to reconcile two key objectives of trustworthy ML, namely privacy and robustness. Privacy in the context of MT. manifests as the requirement that a model does not leak information about the data it was trained on (Papernot et al. 2018), such as revealing whether or not certain data points were included in the training dataset (Shokri et al.] 2017) or what characteristics they exhibit (Fredrikson et al. 2015p. In our study, robustness refers to the requirement that a model's prediction should not change when its inpuls are perturbed at test-time, even in the worst case of adversarially chosen perturbations.
The current gold standard for providing privacy guarantees is differential privacy (DP) (Dwork & Roth, 2014h. In ML, DP produces mathematically rigorous privacy guarantees by limiting the impact of each training data point on the final model. This is achieved by clipping per-sample gradients, and adding well- calibrated amount of noise to all model updates. Clipping limits the sensitivily of the training algorithm, while the addition of noise ensures that training will be more likely to output similar models when any individual data point is added to or removed from the training dataset.
To quantify robustness, we focus on test-time certified robusiness (CR) (Wong & Kolter 2018 Raghunathan et al.) 2018), which provides probabilistic guarantees that perturbations of certain magnitude will not change model's prediction, regardless of what attack strategy is used to modify the inputs. A common approach for certifyyng robustness is randomized smoothing, where a classifier's outputs are averaged over a
distribution surrounding the data point (Tecuyer et al. 2019 Li et al., 2019 Cohen et al. 2019
Unfortunately, the two aims of providing DP and CR guarantees are in conflict, both empirically and concep- tually. The clipping and noise addition used in DP training can impede the convergence of models (Tramer & Boneh 2021 and yield decision boundaries that are less smooth (Hayes et al. 2022). negatively impacting robustness Fawzi ct al. 2018 . There is also substantial empirical evidence that providing privacy guar- anlees is in lension with robustness (Boenisch et al.) 2021 Tursynbek et al.) 2021 Integrating robustness measures into private training remains challenging because most methods to increase robustness use random or adversarial augmentations of training data points, which do not align well with DP training. Conceptu- ally, augmenting an input increases the sensitivity of private training to it, and thereby provides additional avenues for information leakage. From a practical viewpoint, since gradients are computed on a per-example basis for DP, augmentations drastically increase the time and memory costs of training.
Present Work. We study the possible pitfalls of combining DP and CR. Through our analysis and ablation studies combining randomized smoothing techniques with DP training, we show that standard DP training of ML. models is insufficient to provide strong CR results. Surprisingly, we find that prior complex methods used to rogularize models to improve CR are unnocessary (Phan et al. 2019 2020 Tang et al. 2022). Instead, we propose DP-CERT, a straight.forwan and adaptable framework for integraling CR into standard DP training which effectively incorporates augmentations while managing the additional privacy risks. Compared to the few existing approaches for integrating DP and CR, our proposed DP-CERT method is simpler, more flcxible, and does not require extra network components. This decreased complexity is important for the widespread adoption of private and robust MT. DP-CERT even surpasses the state-of-the-art robustness guarantecs on CIFAR10 under equivalent privacy guarantees.
To provide some insight into the mechanisms by which regularization methods improve CR for private models, we analyze CR on a per-data point basis. Using the gradient norm, Hessian spectral norm, and local Lipschitz constant, we find that the certifiable radius has a negative log-linear correlation with these quantities, and compare their distributions across methods. We conclude with concrete recommendations of best practices for the community to achieve CR and DP simultaneously.
2 Preliminaries
Problem Setup. We consider a Y-class classification task with a dataset D = {(1,, v)) where x, e Rd and y; E (1, Y). Let fa : Rd - (1,..., Y} be a neural network classifier with paramoters 6, and Fg denote the soft classifier which outputs the probability distribution, such that fo(x) arg arg.axyeet....... Fo(z) where Fo(x) denotes the model probability of being a member of class y-
2

---

Published in Transactions on Machine Learning Research (07/2024)
Differential Privacy and DPSGD. We rely on the rigorous framework of differential privacy (DP) (Dwork et al. 2006) to obtain models with privacy guarantees. DP ensures that a model's weights at the end of training will be similar in distribution whether or not a particular data point was included in the training set. More formally, let D and D' be two potential training datasets for a model fo that differ in only one data point. The training mechanism M guarantees (e, (e, if, for all possible sets of outcomes S of the training process, holds that Pr Pr[M(D) E S] I Prr [M(D") E S] + The parameter e specifies the privacy level, with smaller e yielding higher privacy, while å quantifies the probability of a catastrophic failure of privacy (Kifer et al. 2022
To obtain a diffcrentially private variant of stochastic gradient descent (SGD), two modifications need to be made (Abadi et al. 2016). First, the individual gradients of each data point are clipped to a norm C' to limit the sensitivity of the model update caused by each data point. Second, choosing a noise level p. noise from ((0,, CT is added to the aggregated gradients to prevent the changes to the model from revealing too much information about individual data points. The resulting algorithm DPSGD is detailed in Algorithm We provide more thorough introduction to DP and explanation of Algorithm 1 in Appendix A.1
Algorithm 1 Standard DPSGD, adapted from (Abadi et al. 2016
Require: Private training set D {1 vi) | E [Npr]]] loss function L(0,5z, Parameters: learning rate λτ, noise scale p, group size B, gradient norm bound C.
1: Initialize 6, randomly
2: for E [7] do
3: Sample mini-batch B, with sampling probability B/Nprv (Poisson sampling)
4: For each i E Br, compute gx(x)) + VoL(0,; {Compute per-sample gradients]
5: gx():)) + (x(z))/ max (I, [I.(a))le {Clip gradients]
C (0, p² TT) (Add noise to aggregated gradient)
6: B:
& B.
L [E Br (m.) N +N(0,
7:
0,11
+ 0, - X.gr (Gradient )escent)
8: end for
9: Output 0y and compute the overall privacy cost (c, 5) using a privacy accounting method.
Certified Robustness. Adversarial examples are a well-studied phenomenon in ML, in which an input to model at test-time is perturbed in ways that do not alter its semantics yet cause the model to misclassify the perturbed input (Biggio et al. 2013 Szegedy et al.) 2013: Goodfellow et al. 2014). Formally, for a given aims to create an adversarial example x' such that ||æ' - ællp < Ç and f(x') + y. Despite much rescarch, the most common defense against adversarial examples remains adversarial training (Goodfellow et al.] 2014: Zhang al. 2019). While adversarial training improves robustness to known algorithms for finding adversarial examples, it does not guarantee that a model will be robust to all adversarial examples. This motivates developing certifiable guarantees of robustness which provide a lower bound r on the distance between a correctly classified input and any
labeled datapoint (r. y) and classifier J, an ()-dd))avversea
adversarial example that may be misclassified (Wong & Kolter 2018 Raghunathan et al., 2018). This lower bound is also known as the certification radius.
Randomized Smoothing. A popular approach for establishing certified robustness (CR) guarantees is through probabilistic robustness verification which, with high probability, verifies that adversarial exam- ples exist within radius of the original input (Li et al.) 2023 The most common method smooths a
classifier by averaging the class predictions of ʃ using a smoothing distribution μ (Lecuyer el. al. 2019
et al. 2019 Cohen et al. 2019
((x) arg max cE(r] ((supp(u)
If(x + (()))(()d
a where I[a, b] 1 and otherwise Li et al., 2023). As computing the integral in Equation [ is intractable, Monte Carlo sampling usod. We denote the approximation of 9 given by Monte Carlo sampling as g. Smoothed classifiers are evaluated in terms of their certified accuracy- the fraction of samples correctly classified when certifying robustness at given radius
3

---

Published in Transactions on Machine Learning Research (07/2024)
A tight Ly radius was obtained by Cohen et al. (2019) when using isotropic Gaussian noise μ Na o II) where is hyperparameter that controls a oobustness/accurace tradeoff. In particular, it was proved that for any base classifier f, the Gaussian smoothed classifier is robust around an input I with radius r ( - & p-1 ((p)) where PA and pø denote the probabilitics of Cд and CB, the most and second- most probable classes returned by g(æ), and 4-1 is the inverse of the standard Gaussian CDF. The exact probabilities PA and pa are not neoded; one can use lower P4 S PA and upper PB 2 PB bounds instead, approximated by Monte Carlo sampling. The output of the smoothed classifier g(x) approximated by aggregating the predictions of base classifier f(z + 17) for n, ~ (O, o2I). As high dimensional standard Gaussian assigns almost no mass near its mean, ensuring that g(x:) is accurate at large certification radií r requires f to be accurate on Gaussian perturbed data (Gao et al. 2022
Related Work. Previous works combining DP and CR deviate significantly from the standard DPSGD template, and as result become more difficult to integrate into DP training pipelines.
Phan et al. (2019) introduced Secure-SGD, the first framework aimed at achieving both CR and DP simulta- neously. They used a PixelDP approach (Lecuyer et al. 2019) to attain CR and introduced the heterogeneous Gaussian mechanism, which adds heterogencous noise across elements of the gradient vector.
Phan et (2020) introduced StoBatch which employs an autoencoder (Hinton & Zemel 1993) and a functional DP mechanism, objective perturbation (Zhang et al., 2012), to reconstruct input examples while preserving DP. Jupseuseeu,n,ly reconstructed data is used to train deep neural nelwork, along with adversarial
training Tramer et al. 2017 to improve robustness.
Tang et al. (2022) proposed perturbing input gradients during training and introduced the multivariate Gaussian mechanism which allows them to achieve the same DP guarantee with less noise added. Their method TransDenoiser follows the architecture of denoised smoothing, adding differentially private noise to a pro-trained classifier.
We note that these prior approaches either add trainable model components increasing the overall complexity, or lack the flexibility to incorporate the latest adversarial training methods.
3 Method
Training ML models to be both differentially private and certifiably robust poses two main challenges. In this section, we describe these challenges and how our DP-CERT framework meets them by effectively improving certified robustncss while preserving privacy.
The first challenge surfaces around the use of advesaaiial training or augmentations with DPSGD. As shown by Cohen et al. (2019), data augmentations used for training can enhance a model's CR, however, augmented data points could leak private information about the originals. Previous works combining DP and CR have proposed adding noise or adversarial examples during training, but deviate from the standard DPSGD
template to address the privacy risks (Phan et al. 2019 2020: Tang et al. 2022
The second challenge is that gradient clipping and noise addition in DPSGD harms the convergence rate of training (Chen et al., 2020 Tramer & Boneh 2021 Bu et al. 2023a),, while restrictive privacy budgets may require stopping training prior to convergence. Robustness on the other hand suffers when models are not converged, as having large gradients at test points makes finding adversarial examples easier (Pawzi et all, 2018). Strategies must be employed to improve the convergence of private training, or regularize gradient size.
We aim to make CR feasible within the standard pipeline of DPSGD, with state-of-the-art convergence and proper accounting for additional privacy risks. Our DP-CERT framework effectively trains with augmented samples while preserving privacy, and integrates advancements in adversarial training and regularizers to enhance certifiable robustness (Salman et al.) 2019 Li et al. 2019 Zhai et al.) 2020). A schematic of DP- CERT is given in Figure I and the training framework described Algorithm below. We describe the components in turn.
4

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Augmentation Methods
Loss Regularizers
Stability
Adversarial
Augmentation Multiplicity
Model
Smoathing
Consistency
Gaussian
MACER
Inference
Training
Figure 1: The DP-CERT training framework for providing strong CR guarantees within DPSGD.
Augmentation Multiplicity, The first step in Figure involves training augmented data with the aim of improving robustness. For each data point (ri, Vi), we oblain K augmented data points (i), where E (1....,K K} and xę. is the j-th augmented version of z₁. For notational convenience, we use x! to denote the original data point To improve CR, augmentations may be generated adversarially, but since this is computationally expensive, we also consider Gaussian perturbations which, as shown by Cohen et al. (2019). also can enhance model's certified robustness. When using Gaussian perturbations instead adversarial training, we define z! 24 + Th. 72 -N(0, for + O.
With respect to the first challenge, important component of DP-CERT is how handle the privacy concerns of training with augmented data points. We employ augmentation multiplicity, introduced by DDe ct al. (2022) as a method for incorporating standard augmentation methods into DPSGD training. The method involves averaging the gradients of multiple input augmentations of the same training sample before clipping and noising. Since all downstream impact to the model weights from sample x; contained in this averaged gradient, clipping it provides a finite sensitivity as required for the sampled Gaussian mechanism used DPSGD (Mironov et al., 2019), and no additional privacy cost is incurred. We propose to apply a slightly modified version of this method for improving CR with DPSGD, namely, by leveraging augmentation multiplicity train the network with Gaussian noised input data, akin to classical methods of regularizing models for CR (Cohen et al. 2019).
The model updates can be expressed as follows
K V VoL(O:;4(1)) +
PC BB
Oc+1 = Oz Ae
Σ clipe
(2)
[&dl
K+1
EEB,
j=0
O, denotes the model parameters at iteration i, A the learning rate, B, the t'th batch, C' is the clipping bound, K is the number of augmentations, p is the noise multiplier, E N/ I), and VoL(0, xf w) is the gradient of the loss for data point (x.. 9:). Note that j starts from 0, which means we include the original samples along with the augmented ones in model training. We discovered that including the original samples empirically improves the robust accuracy.
Adversarial Training. When not using Gaussian augmentations to achieve better certified accuracy, we incorporate adversarial training by deploying existing attacks to create adversarial examples. Specifically, we integrate SmoothAdv (Salman et al. 2019) into private training, which, given original data (a, v), optimizes
arg max
(-
log
E
Fo (2uu))
1x'=x) 25e
find an x' e-close I that maximizes the cross entropy between the smoothed classifier go (a'), and label y. Using Monte Carlo sampling, Objective ( can be optimized by iteratively computing the approximate gradient
K Folx' + +n)by ))))
Var
log
(4)
K
j=1
5

---

Published in Transactions on Machine Learning Research (07/2024)
where ",,... 7K o0) The approximate gradient is then used to update x', with the final x' used examples within augmentation multiplicity.
Regularization. To address the second challenge, the second step in Figure | adapts stability regular- ization to private training in order to minimize the distance between the output probability of the original and augmented examples, thereby improving the robustness to input noise. Stability training (Li et al, 2019) adds a smoothed cross-entropy loss as regularization. Inspired by TRADES (Zhang al. 2019), we instead use the Kullback Leibler (KL) divergence with hyperparameter 7 controlling the strength of the regularization as:
Lutab 11, Vi) (i)= = - (DDK((((()))) Fo(z))
(5)
Additionally, we propose integrating MACER Zhai et al. 2020 directly optimize the cortified accuracy at larger robustness radii without requiring the costly process of adversarial training. MACER achieves this by decomposing the error of a smoothed classifier into a classifi- cation error term and a robustness error term, the latter reflecting whether or not the smoothed classifier is able to certify robustness for a given radius. MACER further described in Appendix B.1.3
alternative training modification to
Algorithm 2 DP-CERT Training
Require: Private training set D = { ya) i E [Nprv]) },, loss function L(fg; Parameters: learning rate X₁, noise scale p, group size B, gradient norm bound C, number of augmentations K.
1: Initialize 0 randomly
2: for & E [2] do
3: Sample mini-batch B, with sampling probability B/Npev (Poisson sampling)
4: For each i E Br, generate K augmented samples x! (Adversarial or Gaussian augmentations]
L'(0, v:) f K+I Ej= L(O;; 2t, y:) #(),, (Add regularizer, average over augmentations]
(x;))) VoI.(0,5 Ti, yi) (Per-sample gradients contain all information from 3; and its augmentations)
7: (I,))) + (xx)/ max [1.(x)))22 {Clip gradients]
8: E 1 (E+ g: (z,) + (0, PC {Add noise to aggregated gradient)
B.
O + Ar [Gradient descent)
10: end for
11: Output Or and compute the overall privacy cost (c, 5) using a privacy accounting method.
Summary of Method. DP-CERT (Figure Algo- rithm 2 combines augmentation multiplicity, which was originally introduced to improve the accuracy of models trained by DPSGD, with randomized smoothing for infer- ence, and a variety of adversarial augmentation and regu- larization approaches, which have been shown to improve certified robustness. Comparing the training algorith and 2 shows that our method follows the template of DPSGD with the following modifications: adversarial or Gaussian perturbations are generated for each data point in the batch; regularization is optionally applied to the loss function; and augmentation multiplicity is used to compute "per-sample" gradients as the average over all augmented data points associated to x;. Randomized smoothing at inference time allows certifying robustness, but improved robustness is a result of the training-time modifications.
Table 1 Instantiations of DP-CERT
Method
Augmentation
Regularization
DP- Gaussian
Gaussian
None
AmoomoothAdy
Adversarial
None
DP-Stability MA-MR
Gaussian
Stability
Gaussian
MACER
While DP-CERT a combination of known methods we emphasize simplicity compared to previous works that bring CR guarantees DP. Decreasing complexity vital for the widespread adoption of robust and private AI, and simultaneously report improved performance.

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Privacy Guarantee. As described above, augmentation multiplicity achieves the same privacy guarantees as DPSGD by averaging over all augmentations before clipping gradients. This remains true whether random or adversarial augmentations are used, and for any regularized loss function. Once a model is trained with a DP guarantee, it can be used for inference with randomized smoothing at no further privacy cost, thanks to the postprocessing guarantee of DP (Dwork & Roth 2014 Abadi al. 2016). Hence, DP-CERT requires no special accounting, and is casily implemented within any mature code package for DP. We note that Gaussian augmentations are a form of input perturbation and may contribute stronger privacy guarantees. However, imput perturbation would first require data points to be clipped for finite sensitivity, impacting performance. Also, we keep the original datapoint that does not have noise added, which is beneficial to performance but not aligned with input perturbation. Hence, we do not attempt to account for DP guarantees from this source of randommess.
In the following sections, we experimentally benchmark four instantiations of the DP-CERT framework as summarized in Table
4 Experiment Setup
We evaluate the effectiveness of DP-CERT on multiple image classification datasets, including MNIST (L--
Cun et al. 2010), Fashion-MNISTT (Xiao et al. 2017), and CIFAR10 (Krizhevsky & Hinton 2009). More detailed data statistics can be found in Appendix C.1 We demonstrate that DP-CERT consistently out- performs the undefended differentially private baselines, and establishes the state of the art for certified L-2 defense under a DP guarantee on CIFAR10.
4.1 Baseline Methods
Our comparisons include non-private training (Regular) for an accuracy baseline, along with DPSGD, and per-sample adaptive clipping (PSAC) (Xia et al. 2023 which is a variation of DPSGD with better con- vergence (see Appendix A.2 for details). For DPSGD and PSAC, we adopt the same settings as Xia et al. (2023), who exhibited state-of-the-art performance for DP optimization on various tasks. These baselines show what CR guarantees can be achieved by DP models when combined with randomized smoothing, but without training-tima time modifications designed to protect against adversarial examples.
We additionally compare against prior approaches to integrate CR with DP guarantees, namely SecureSGD (Phan et al. 2019), StoBatch (Phan al. 2020), and TransDenoiser (Tang al. 2022) CIFAR10. How- ever, the lack of open source implementations in Pytorch combined with the complexity of these approaches prevents us from rumning experiments with them. Hence, we directly quote the results published in (Tang et al. 2022) for the three baselines, and replicate their experimental setting to run our methods. We again note that our approach is much simpler, fitting the standard DPSGD training pipeline that is supported by many mature code packages, and we provide our code at (github.cco aoan//ayyrGaiilabee /dp-cert/
For all methods that guarantee privacy (i.e. all methods other than Regular), we use models that achieve the same level of privacy (e 3.0, 5 = 10-5) to ensure a fair performance comparison.
4.2 Implementation and Hyperparameters
For all experiments on MNIST and MNNSI, we train a four-layer CNN model from scratch, with the settings used by Tramèr & Boneh (2021) On CIFAR1O, fine-tune Cross ViT-Tiny (Chen et al. 2021,, pretrained on ImageNetlk (Deng et al. 2009).
For each model configuration, we consider three models trained with different noise levels a E {0.25, 0.5, 1.0] for smoothing at training time, and during inference we apply randomized smoothing with the same a as used in training.
TransDenoiser fine-tunes pretrained VGG16 model (Simonyan & Zisserman, 2014 and uses an additional denoising diffusion model. For fair comparison, fine-tune much smaller network for DP-CERT,
7

---

Published in Transactions on Machine Learning Research (07/2024)
CrossViT-Tiny (Chen et al. the reader to Appendix
2021
For additional implementation and hyperparameter settings, refer
C.3
4.3 Evaluation Metrics
First, report the clean accuracy (Aec) the test dataset without randomized smoothing for inference as mcasure of convergence. Following previous works, we report the approzimate certified accuracy, which is the fraction of the test set that can be certified to be robust at radius r using the CERTIFY procedure introduced by Cohen et al. (2019). We also include the average certified radius (ACR) (Zhai et al. 2020) returned by CERTIFY which serves as an additional metric for better comparison CR between two models (Tsipras al. 2019 Zhang et al. 2019). ACR is calculated as
ACR
Σ
CR(F.0,2) If(x)). v/.
(6)
Duest
(x.y)
where Drest is the test dataset, and CR denotes the certified radius provided by CERTIFY.
5 Experimental Evaluation
Table 2: Comparison of accuracy, ACR and the certified accuracy at radius r 0.25 between baselines and instances of the DP-CERT framework MNIST, Rashion-MNIST and CIFAR10. Higher is better for all metries.
MNIST
Pashion-MNIST
CIFAR10
Method
Acc
ACR
0.2.5
Acc
ACR
r-2.25
Ace
ACR
0.2.
Regular
99.14
0.581
83.6
89.28
0.359
55.5
91.79
0.055
9.5
DPSGD
98.11
0.606
88.3
85.87
0.343
53.2
89.74
0.023
3.3
PSAC
98.25
0.608
88.5
86.34
0.320
49.0
89.81
0.020
2.8
0.25
DP-Caussian
98.13
0.735
95.7
84.76
0.545
75.8
87.61
0,246
41.8
DP-SmonthAdy
98.08
0.742
96.0
83.97
0.554
75.9
87.89
0.275
44.3
DP-Stability
97.86
0.738
95.9
84.19
0.551
75.7
88.53
0.246
41.6
DP-MACER
98.13
0.736
95.6
84.79
75.8
87.52
0.246
41.7
Regular
99.14
0.308
31.8
89.28
0.331
34.9
94.79
0.092
9.7
DPSGD
98.13
0.344
50.0
85.87
0.309
29.8
89.74
0.057
9.8
PSAC
98.25
0.383
55.9
86.34
0.298
27.5
89.81
0.056
9.8
Gausiann
97.74
1.246
94.7
82.42
0.879
73.0
87.48
0.288
$5.5
OP-SmoothAdv
97.66
1.258
94.8
82.65
0.894
73.0
87.54
0.263
31.9
DP-Stability
97.62
1.248
82.25
0.876
72.77
DP-MACER
97.75
1.246
94.7
82.50
0.880
87.36
0.287
35.2
Regular
99.14
0.237
10.7
89.28
0.342
21.2
94.79
0.079
9.7
DPSGD
98.13
0.260
10.4
85.87
0.338
13.5
89.74
0.122
5.9
PSAC
98.25
0.213
20.0
86.34
0.328
11.5
89.81
0.023
4.2
DP-Canssian
96.33
1.262
85.6
80.96
1.101
65.4
88.55
0.299
25.4
SmoothAdv
96.54
1.249
80.93
1.096
64.7
DP-Stability
96.48
1.262
84.9
80.66
1.08-1
65.1
89.09
0.294
25.0
DP-MACER
96.31
1.262
85.5
80.83
1.102
65.3
88.40
0.255
22.3
In Table we compare our baseline methods and DP-CERT instantiations by their clean accuracy, ACR, and certified accuracy for radius 0.25. The best ACR and certified accuracy for each smoothing radius are displayed in bold, while close runner-ups underlined. In Figure 2 we plot the certified accuracy the certification radius is increased MNIST). We also compare DP-CERT to SecureSGD) StoBatch, and TransDenoiser on CIFAR10 in Figure copying the baseline results from (Tagg et al. 2022).
CIFAR10 (See Figure 7 in Appendix D.1 for MNIST and Fashion-
Discussion. Table 2 and Figure 2 show that all instantiations DP-CERT significantly outperform the bascline methods in terms of the approximate certified accuracy and ACR. Generally, DP-CERT's clean accuracy marginally lower than the PSAC baseline, but its ACR and certified accuracy do not fall off drastically is increased. Hence, for private models there is still tradeoff between clean accuracy and

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Regular
DPSGD
PSAC
DP-Gaussian
DP-SmoothAdv
DP-Stability
DP-MACER
0= 0.25
0= 0.5
0= 1.0
30
60
Certified Accuracy
Certified Accuracy
40
Certified Accuracy
25
40
30
20
20
20
10
0
0
radius
radius
radiiu
Figure 2: Approximate certified accuracy (ACR) comparison on CIFAR10.
Relative Improvement of Certified Accuracy, a=0.25
250
SecureSGD sct
Relative Improvement (%)
Secure5GD_pp
200
StoBatch
150
uauGaassiann
DP-SmoothAdy
100
DP-Stability
DR-MACER
50
0
-50
0.1
0.2
0.3
0.5
0.6
LG Radius
Figure 3: Approximate certified accuracy comparison CIFAR10. The y-axis shows the relative certified
accuracy improvement over the strongest baseline method, TransDenoiser, as percentages. Baseline results are from Tang et al. 2022).
CR, as there is for non-private models. Our well-converged baselines show that DPSGD training does not always lead to worse CR compared to non-private training, which should be contrasted with previous studies on the adversarial robustness of DPSGD (Tursybeck et al. 2021 Boenisch et al.) 2021 Zhang & Bu 2022 Still, DPSGD alone, even when well-converged, does not provide strong CR via randomized smoothing, demonstrating the need for DP-CERT's improvements.
Figure 3 shows that all variants of DP-CERT surpass the state-of-the-ar certified accuracy on CIFAR10 with much smaller pre-trained model compared to (Phan et al., 2019), (Phan et al. 2020). and TTng et al. 2022). At higher certification radii, for instance an L2 radius of 0.7, the best version of DP-CERT is over 250% better than Trooseen Among baselines, TransDenoiseer is less than 40% better than the next strongest method, Secure-SGD, showing that DP-CERT a meaningful advancement. Since we do not rely on an additional denoiser, inference is also much faster with DP-CERT.
Practical recommendations from Table Contrary prior findings in non-private training (Salman et al. 2019 Zhai et al., 2020). all methods we tested to improve certified robustness have similar perfor- mance when used with differentially private training as instantiations of DP-CERT (Table D.. Because DP-SmoothAdv incurs a significant training overhcad from adversarial perturbations, we recommend not using it in private training. Gaussian augmentations are less expensive, and give just as good performance. For training from scratch, OP-Gaussiaa is recommended since offers competitive results while being the most straightforward to implement and fastest to train. For finc-tuning pre-trained model, DP-Stability is recommended since it has the highest clean accuracy in all variants while offering competitive certified accuracy.

---

Published in Transactions on Machine Learning Research (07/2024) 2224)
100
80
Certified Accuracy
80
60
Certified Accuracy
GaGsassiia
40
SP-SmoothAdv
Num Augmentations-
GauGaan + PSAC
40
Num Augmentations=2
20
DP-SmoothAdv PSAC
Num Augmentations=4
Gau + Consistency
20
Num Augmentations-6
DP-SmoothAdv Consistency
Num Augmentations=8
0
0
0.0
0.00
0.25
0.50
0.75 1.00
1.25
1.50
1.75
2.00
La radius
La radius
Figure 4: Ablation for consistency regularization, PSAC, and augmentation mumber on Fashion-MNIST.
5.1 Impact of Model Architectures and Augmentations
In this section, as an ablation, we examine the effect of different model variants and hyperparameters. All experiments are run Fashion-MNIST with 0.5; results for MNIST and other values of are given in Appendix D.2 and confirm the trends shown here. We combine consistency regularization (reviewed in Appendix B.1.2 and PSAC with DP-Gaussian and DP-SmoothAdy to study their elfect certified accuracy and radius. Figure 4 (left) shows that neither of these techniques improves CR. We emphasize that the simplest training method, DP-Gaussian, performs just as well without specialized techniques designed to enhance convergence and robustness in other contexts. We also train models with different numbers of augmentations and compare their CR. Figure (right)) shows that the certified test accuracy is largely unchanged as the number of augmentations increases, consistent with the observations made by [Salman al.) (2019). Note that for this plot, when the number of augmentations is zero we replace the original sample with a version corrupted by Gaussian noise to benefit CR (the version without augmentation of any kind is simply DPSGD which has much worse CR, as shown in Figure 7 Since using fewer augmentations better preserves the natural accuracy and incurs less training overhead, we recommend using a minimal number of augmentations.
We provide additional hyperparameter ablation results across learning rates and clipping norms C' Appendix D.2. The results show that our settings achieve the highest clean accuracy, and that accuracy is fairly stable around these optimal parameters.
5.2 Per-sample Metric Analysis
Certified accuracy and ACR are both metrics averaged over the test set. However, robustness is inherently sample based, since it examines how resilient the model is against perturbations tailored to individual samples. Therefore, in this section of their distributions to provide insights into why certain training methods may produce more robust models than others.
per-sample metrics of robustness and conduct in-depth analysis
We consider three per-sample metrics associated with robustness: the input gradient norm, input Hessian spectral norm, and Tacsh-Lipschilz conslant. The first two metrics measure the local smoothness of the loss landscape with respect to the input space. Taylor's approximation can be used to show a direct link between these two metrics and the worst-case change in loss from small input perturbations. Due this connection, prior works directly regularized them to improve robustness (Hofmman et al. 2019 Jakubovitz & Giryes 2018 Moosavi-Dezfooli et al.) 2019).
Gradients and Hessians are highly local quantities that are only connected to robustness through Taylor's approximation at small radií around the input data point. Consequently, they may not be informative at larger radii used to certify robustness. Thus, we also compare models using an empirical estimate of the average local Lipschitz constant of the model's penultimate layer. By viewing the network as a feature extractor composed with linear classifier, using the penultimate layer captures the worst-case sensitivity of
10

---

Published in Transactions on Machine Learning Research (07 (0222
Regular
DPSGD PSAC
DP-Gaussian
350
350
350
300
300
300
250
Prequency
Frequency
200
200
200
150
150
150
100
50
50
0
-15
-10
5
-15
-10
-5
2
Log Input Spectral
Log Input
Log Lipschitz
Comparison among Regular, DPSGD, PSAC, and DP-Gaussiag training.
DP Gaussian
DP-SmoothAdv
DP-Stability
DP-MACER
400
300
300
300
Prequency
Frequency
250
200
200
150
100
100
50
-15
Log Input Hessian Spectral Norm
Log Input Grad Norm
Log Local Lipschitz Constant
(b) Comparison among Gaussiia,, DP-SmoothAdv, DP-Stability, and DP-MACER.
Figure 5: Per-sample metric comparison, MNIST, 0.5
the feature extractor to perturbations of the data, This metric was initially proposed by Yang et al. 2000
to investigate adversarial robustness and is given by
(1(x)) f(x2il
(7)
| 2o
where the maximum is approximated in the same manner as used for adversarial example generation, typically projected gradient descent (PGD) (Madry et al. 2018
RQI: How are training dynamics of various methods reflected in their metric distributions? We calculate the three metries for each test set data point, visualize their distribution in histograms, and single setting here MNIST and = 0.5 in Figure 5 The histograms for different datasets and a's in Figures T0 14 in Appendix follow a similar pattern and reinforce our analysis presented here. In Figure 5a Regular training results in an approximately log-normal distribution for input gradient norms with a mode far greater than for DPSGD variants. Meanwhile, DPSGD is bimodal with some inputs having very large gradient norms which are potentially vulnerable to adversarial examples. This likely arises as consequence of the clipping employed in the DPSGD training algorithm which effectively down-weights the contributions of hard examples and up-weights the contribution of easy examples (Shamsabadi & Papernot 2023). Rarer samples, which would dominate minibatch gradient for Regular training, are not learned and still have large input gradients the end of training. PSAC mitigates this issue slightly by explicitly up-weighting hard examples, resulting in a distribution closer to Regular training. DP-Gaussian, on the other hand, shifts the distribution towards lower norm values. Comparing variants of DP-CERT in Figure [5b] DP-Stability has significantly higher input gradient norms, input Hessian spectral norms and lower local Lipschitz constants than the other three variants. This echoes the observation that TRADES-style training (Zhang et al. 2019 results in significantly lower local Lipschitz constants (Yngg al. 2020
compare across baselines and proposed methods. For detailed analysis focus
11

---

Published in Transactions on Machine Learning Research (07/2024)
Regular
DPSGD
PSAC
3P-Gaussian
DP-SmoothAdv
DP-Stability
DP DP-MACER
Input Hessian Spectral Norm
Input Grad Norm
Local Lipschitz Constant
Log Metric Value
Log Metric Value
0
-2
-4
-4-
-6
-6
0.0
0.0
0.4
0.8
0.0
0.4
0.8
Certified Radius
Certified Radius
Certified Radius
Figure 6: The input Hessian spectral norm (left), input gradient norm (middle), and local Lipschitz constants (right) calculated at different certified radii, MNIST, 0.5.
RQ2: How do the metrics correlate with the certified radius on a per-sample basis? We visualize the correlation between certified radius and per-sample metrics Figure We first group examples by their certified radii, then for each group we compute their average metric values and take the logarithm. Across training methods, we see clear negative correlation between the log metric values and certified radius, which means that examples robust to imput noise tend to have lower metric values. However, different methods exhibit different levels of correlation, which is closcly related to their average metric values. For example, DP-Stability on average has a much higher input gradient norm and a much lower local Lipschitz constant than other methods at the same certified radii. We further analyze the distribution of individual samples basod on cortified radius in Appendix [D.] We find that the local Lipschitz constant of test examples has a closer connection to CR than the gradient norm, or Hessian spectral norm for the models trained with DP guarantees. Combining this result with Figure 5a we see that DP-CERT improves certified robustness by reducing the number of datapoints with large local Lipschitz constant, input Hessian spectral norm, input gradient norm compared to PSAC and DPSGD.
6 Conclusion
We achieve better certified robustness with DPSGD training through augmentations, regularization, and ran- domized smoothing, reconeiling two crucial objectives for trustworthy MI., namely privacy and robustness. We rely on DP training with augmentations that does not incur additional privacy costs, while employing various regularizations, and adversarial training methods to enhance robustness. Our resulting DP-CERT framework is modular and supports multiple combinations of these methods. Through our extensive ex- perimental study, we confirm that DPSGD training alone, even with state-of-the-art convergence, does not provide satisfactory certified robustness. However, roducing a small number of computationally inexpen- sive augmentations into training by adding Gaussian noise suffices to yield strong privacy protection and certified robustness, even surpassing much more complex prior methods. By thoroughly analyzing per-sample metrics, show that the certified radius correlates with the local Lipschitz constant and smoothness of the loss surface; this opens a new path to diagnosing when private models will fail to be robust. In conclusion, our proposed method greatly simplifies the existing solutions to simultaneously achieve CR and DP, and our practical recommendations provide a valuable contribution toward trustworthy ML. When training from scratch, Gaussian augmentations (not adversarial) should be used with DPSGD, and randomized smooth- ing applied at inference time. For fine-tuning pretrained models, adding stability regularization also helps accuracy, and leads to much lower local Lipschitz constants.
Acknowledgments. DG, FB, and NP would like to acknowledge sponsors who support their research with financial and in-kind contributions: CIFAR through the Canada CIFAR AI Chair, NSERC through Discovery Grant, the Ontario Early Researcher Award, and the Sloan Foundation. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.
12

---

Published in Transactions on Machine Learning Research (07/ (224))
Broader Impact Statement
Nowadays, machine learning finds extensive application in our society. As a result, ensuring the integrity of the models we build is vital. This entails safeguarding individuals' privacy while maintaining the models' robustness. Our paper focuses on examining the intersection of privacy and robustness for machine learning models, both of which are essential for establishing trustworthy ML. We introduce the DP-CERT framework, which, compared to prior work, simplifies the process for machine learning developers to create models that are both robust and private, while increasing the robustness afforded for a given privacy level.
While both differential privacy and certified robustness can provide mathematical guarantees on the privacy and robustness of trained models, the guaranlees are probabilistic and can randomly fail. For DP, failures roughly occur with probability 6, while for CR, some specific datapoints may not be certifiable for adequately large radii Practitioners should evaluate their risk tolerances for failures of privacy and robustness, and choose the parameters of these guarantees appropriately.
Furthermore, it has been noted that differential privacy can have particularly negative impacts on the fairness of models (Bagdasaryan et al. 2019 Xu et al. 2022 Esipova et al. 2023). While we do not specifically address fairness aspects in this paper, is another prong of trustworthy ML that should be taken into account and ensured for any ML model that is to be deployed.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang, Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communécations Security. ACM, 2016. doi: 10.1145/2966449.2988811
Eugene Bagdasaryan, Omid Poursaced, and Vitaly Shmatikov. Differential privacy has disparate impact model accuracy. In Advonces in Neural Information Processing Systems, volume 32, pp. 15479-15488, 2019.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Srndié, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases: European Conference, pp. 387 402, 2013.
Franziska Boenisch, Philip Sperl, and Konstantin Böttinger. Gradient masking and the underestimated robustness threats of differential privacy in deep learning. 0rXic:2105.07985 2021.
Zhiqi Bu, Ping Li, and Weijie Zhao. Practical adversarial training with differential privacy for deep learning. OpenReview Preprint, 2022a. URL httpp:://penrrevie httppp///pppppeeeee.eeeee//eeeer
Zhiqi Bu, Jialin Mao, and Shiyun Xu. Scalable and efficient training of large convolutional neural networks with differential privacy. In Advances in Neural Information Processing Systems, 2022b.
Zhiqi Bu, Hua Wang, Zongyu Dai, and Qi Long. On the convergence and calibration of deep learning with differential privacy. Transactions on Machine Learning Research, 2023a. ISSN 2835-8856. URL.
attps://openrentt
net//forum?id=KOCAGGgyY
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Automatic clipping: Differentially private deep learning made easier and stronger. In Advances in Newral Information Processing Systems, volume 36, pp.
41727-41764, 2023b.
Chun-Fu Fu (Richard) Chen, Quanfu Fan, and Rameswar Panda. Cross ViT: Cross-Attention Multi-Scale Vi- sion Transformer for Image Classification, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 357-366, October 2021.
Xiangyi Chen, Steven Z. Wu, and Mingyi Hong. Understanding Gradient Clipping in Private SGD: A Geometric Perspective. In Advances in Neural Information Processing Systems, 2020.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pp. 13100, 2019.
13

---

Available at Published in Transactions on Machine Learning Research (07/2024)
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlocking high-aceuracy differentially private image classification through scale. 2XX0:2224413660 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009. doi: R01109//VPR.22 200...060095
Cyuthia Dwork and Aaron Roth. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci. 0(3-4):211-477 August 2014. ISSN 1551-305X. doi: 10.1561/0400000044.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity Private Data Analysis. In Theory of Cryptography, pp. 265 284. Springer Berlin Heidelberg, 2006. ISBN
978-3-540-33233--5
Maria S. Esipova, Atiyeh Ashari Ghomi, Yaqiao Luo, and Jesse C Cresswell. Disparate impact in differential privacy from gradient misalignment. In The Eleventh International Conference on Learning Representa- tions, 2023.
Alhussein
Fawzi, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard, and Stefano Soatto. Empirical study of the topology and geometry of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pallern Recognition, June 2018.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322-1333) 2015. doi: 10.1145/ 201145/2810103.255
Yue Gao, Shurnailov, Kassem Fawaz, and Nicolas Papernot. On the limitations of stochastic pre- pre-processing defenses. In Aduances Neural Information Processing Systems, 2022.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. 1:Xiv:1412.6577 2014.
Jamie Hayes, Borja Balle, and M Pawan Kumar. Learning to be adversarially robust and differentially private. arXiv:2201.08 022225,, 2022.
Geoffrey E Hinton and Richard Zemel. Autoencoders, Minimum Description Length and Helmholtz Free Energy. In Advances in Neural Information Processing Systems, 1993.
Judy Hoffman, Daniel A. Roberts, and Sho Yaida. Robust Learning with Jacobian Regularization, 2019.
Miklós Z. Horváth, Mark Niklas Mueller, Mare Fischer, and Martin Vechev. Boosting randomized smoothing with variance reduced classifiers. In International Conference on Learning Representations, 2022.
Daniel Jakubovitz and Raja. Giryes. Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization. In Proceedings of the European Conference on Computer Vision, 2018.
Jongheon Jeong and Jinwoo Shin. Consistency regularization for certified robustness of smoothed classifiers. In Advances in Neural Information Processing Systems, 2020.
Daniel Kifer, John M Abowd, Robert Ashmead, Ryan Cumings-Menon, Philip Leclere, Ashwin Machanava- ijhala, William Sexton, and Pavel Zhuravlev. Bayesian and frequentist semantics for common variations of differential privacy: Applications to the 2020 census. arXiv:2209.084: 03310, 2022.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. 2, 2010.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy, pp. 656 672, 2019. doi: S0.1109/SP.20190
14

---

Published in Transactions on Machine Learning Research (07/2024)
Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. In Advances in Neural Information. Processing Systems, 2019.
L. Li, Xie, and B. Li. SoK: Certified Robustness for Deep Neural Networks. In 2023 IEEE Symposium on Security and Privacy, pp. 94 115, 2023, doi: D0.1009/SP42215.22222
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Ila Mironov, Kunal Talwar, and Li Zhang. Rényi Differential Privacy of the Sampled Gaussian Mechanism. arXiv:1908. 10530, 2019.
Seyed-Mohsen Moosavi-Dezfoolii Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robustness via curvature regularization, and vice versa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019.
Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael P. Wellman. SoK: Security and Privacy in Machine Learning. In 2018 IEEE European Symposium on Security and Privacy, pp. 1999414, 2018. doi: 10.1109 1000099/urSP20000 2018.0003..
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performande Deep Learning Library. In Advances Neural Information Processing Systems, volume 32, 2019.
Hai Phan, My T. Thai, Han Hu, Ruoming Jin, Tong Sun, and Dejing Dou. Scalable differential privacy with cortified robustness in adversarial learning. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pp. 7683 7694, 2020.
NhatHai Phan, Minh N. Vu, Yang Liu, Ruoming Jin, Dejing Dou, Xintao Wu, and My T. Thai. Hetero- geneous gaussian mechanism: Preserving differential privacy in deep learning with provable robustness. In Proceedings of the Seeenty-Figgtt International Joint Conference on Artificial Intelligenee, IJCAT-19. 2019. doi: 10.24963/ijj jjcai.2019// /660.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Cortified defenses against adversarial examples. In International Conference on Learning Representations, 2018.
Sylvestre- Pevestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. In Advances in Neurol Information Processing Systems, volume 34, 2021.
Hadi Salman, Jerry Li, llya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In Advances in Neural Information Processing Systems, 2019.
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. In Advances in Neural Information Processing Systems, 2020.
Ali Shahin Shamsabadi and Nicolas Papernot. Tosing less: A loss for differentially private deep learning. Proceedings on Privacy Enhancing Technologies, 3:307-320, 2023. doi: 10.56553 2000555/pppet-2222 2023-0083.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy, pp. 3-18, 2017. doi: 10.1109//PP.2017.41.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. 1r9 2014.
15

---

Published in Transactions on Machine Learning Research (07/2024)
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv: 1312:1199.999 2013.
Pengfei Tang, Wenjie Wang, Xiaolan Gu, Jian Lou, Li Xiong, and Ming Li. Two Birds, One Stone: Achieving both Differential Privacy and Certified Robustness for Pre-trained Classifiers via Input Perturbation. OpenReview preprint, 2022. URL https::/openrreveek http://pprreeeeee.ett.ooorrreeee ottp://wwww.tooooonod/eeeeeaaaa
Florian Tramèr and Dan Boneh. Differentially private learning needs better features (or much more data). In International Conference Learning Representations, 2021.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Tan Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. vrXix:1705.001 072004,, 2017.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations, 2019.
Nurislam Tursynbek, Aleksandr Petiushko, and Ivan Oseledets. Robustness threats of differential privacy. arXi::201..07828, 2021.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th International Conference on Machine Learning, 2018.
Tianyu Xia, Shuheng Shen, Su Yao, Xinyi Fu, Ke Xu, Xiaolong Xu, and Xing Fu. Differentially private learning with per-sample adaptive clipping. Proceedings of the AAAI Conference on Artificial Intelligence, 37(9):10444 10452, 2023. doi: 10.1609/ v37i9. 26242.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Mashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv:1708 700.0777777777 2017.
Depeng Xu, Wei Du, and Xintao Wu. Removing Disparate Impact on Model Accuracy in Differentially Private Stochastic Gradient Descent. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Dala Mining, pp. 1924-1932, 2021.
Yao- Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri. A closer look accuracy vs. robustness. In Advances in Neural Information Processing Systems, 2020.
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Ilya Mironov. Opa- cus: User-friendly differential privacy library in Py Torch. arXiv:2109. 12298, 2021.
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, and Liwei Wang. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius. Tn International Conference on Leurning Representations, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled tradc-off between robustness and accuracy. In Proceedings of the 36th International Conference Machine Learning, volume 97, pp. 7472-7482, 2019.
Jun Zhang, Zhenjic Zhang, Xiaokui Xiao, Yin Yang, and Marianne Winslett. Functional mechanism: regres- sion analysis under differential privacy. arxiv preprinl arXio: N0:::120800111 2012.
Yuan Zhang and Zhiqi Bu. Differentially private optimizers can learn adversarially robust models. 2x1002291..08442, 2022.
16

---

Published in Transactions on Machine Learning Research (07/2024)
A Differential Privacy Background
A.1 Differential Privacy and DPSGD
Differential Privacy (DP) Dwork et al. 2006 is a formal framework that aims to provide mathematical guarantees on the privacy of individual data points in a dataset, while allowing one to learn properties over the entire dataset. More formally a randomized mechanism M fulfills (e, 6)-DP if, for all pairs D and D' of neighbouring datasets (i.e. datasets differing in only one data point) and all sets of output values of M,
Pr[M(D) E S) ('Pr(M((') E S) +,
(8)
where and 8 are privacy parameters. indicates the level of obtained privacy, with smaller value of indicating a stronger privacy guarantee. ō accounts for the probability that the algorithm will suffer a catastrophic privacy failure, which occurs when the privacy loss random variable becomes infinite (Kifer) et al. 2022). This will happen when an event with zero probability for one dataset has non-zero probability for neighbouring dataset, meaning that there is no possible multiplicative factor ef that could satisfy the DP incquality Eq. with 8 = O. Hence, a larger value of increases the possibility of unmitigated privacy leakage.
The most popular algorithm for implementing DP guarantees in ML is differentially private stochastic gradient descent (DPSGD) (Abadi et al., 2016). It extends the standard SGD algorithm with two additional steps, namely gradient, clipping and noise addition. While the former bounds the sensitivity of the model update, the latter implements the privacy guarantee by preventing the gradients from revealing too much information about individual data points. More formally, let O, denote the model parameters at training iteration t. At each iteration t, DPSGD computes the gradient of the loss function with respect 6₃ individual data point x Vo). The data, point's gradient is then clipped to maximum norm C using the operation clip (VoL(0.,x), C"), which replaces the gradient with vector of the same direction but smaller magnitude if its norm exceeds C. The clipped gradient from all datapoints in batch are aggregated, then perturbed by adding random noise from (O0 p²C where is the noise scale parameter. We detail DPSGD in Algorithm
A.2 PSAC
Per-sample adaptive clipping (PSAC) (Xia et al. 2023) is one of a number of approaches that tries to reduce the bias from per-example clipping (Bu et al. 2023a b Esipova et al. 2023), and was motivated for maximiz- ing the signal to noise ratio of gradient updates. has shown the best performance for differentially private models on several datasets including MNIST, MasHIon-MNIST and CIFAR10. To complement DPSGD algorithm with better convergence, incorporated PSAC into our experiments.
PSAC is similar to DPSGD, with the exception that it employs different clipping method,
clipc,r(g...) C. gb.t C-ght
r Irr
(9)
Where Ii.t denotes the loss gradient for the ith sample at iteration The motivation of this clipping method is that per-example gradients with small norms come from data points on which the model has already converged, and these gradients are often orthogonal to mini-batch gradients. Hence, small norm gradients should not have disproportionally large contributions to the batch gradient as when using per- sample normalization methods (Bu et al. 2023b). Compared to such approaches, PSAC reduces the influence of small norm gradicnts, and empirically shows better convergence and utility.
Additional Background and Related Work
B.1 Randomized Smoothing
Previous works have tackled improving the certified robustness of randomized smoothing methods in a varicty of ways. The dominant approach for doing involves modifications to training the base classifier to
17

---

Published in Transactions on Machine Learning Research (07 (07/2024)
increase robustness and accuracy under Gaussian perturbations. The simplest approach involves adding noise to inputs during training (Cohen et al. 2019h, while other works utilize regularization (Zhai et al. 2020 Li
et al. 2019 Jeong & Shin 2020), ensembling (Horváth et al.) 2022). and adversarial training (Salman et al. 2019 While these modifications have been independently studied in the context of improving the certified accuracy of randomized smoothing classifiers, our work the first to integrate these methods with private training through augmentation multiplicity. We provide additional information on two of these training modification methods mentioned in Section as well as Consistency regularization used in one ablation experiment.
B.1.1 SmoothAdv
One of the most effective methods for improving the performance of a randomized smoothing classifier utilizes adversarial training of the classifier The method, SmoothAdy proposed in (Salman et al. 2019). motivated by the idea that to improve certified accuracy at larger certification radius one needs classifier that is more robust to local perturbations, and the best known method of achieving that is through adversarial training.
Given a soft classifier F Rd + P(Y) where P(Y) is the set of probability distributions over Y, its smoothed soft classifier G defined as:
G(æ) (F * +N( (F
(10)
The goal of SmoothAdv is to find a point & that maximizes the cross entropy loss of G in 12 ball around v. SmoothAdy uses a projected gradient descent variant to approximalely find â and define J(x') LCE(G(I'),u) y) to compute
V Vx'((1)= = Var --1g
+m)u))
(11)
Since the expectation in Equation 11 is difficult to compute exactly, Monte Carlo approximation is used by sampling noise m..... "m. ((0 o² I) to approximately compute (xz),
(e 2 V. (-log
Fa'' mibu))
(12)
m
Finally, x' is updated by taking step in the direction of Va'J(x'), and the final x' is used to train the classifier.
B.1.2 Consistency Regularization
For ablation purposes, we also compare to Consistency regularization (Jeong & Shin 2020 method very similar to Stability training which instead minimizes the KL divergence between Fo and Fo(x;), where F (zx)) = k E, Fo(2)) is the average output probability of all smoothed samples. The loss can be expressed
as
Lcons(It,, (II.b.)= =
Σ (z) v.) + 7DL (f(())))()))))
(13)
B.1.3 MACER
Similar to the Stability training method, MACER (Zhai et al. 2020 also modifies the loss for optimization so that the final model has higher certified accuracy at a larger certified radius. In contrast to SmoothAdy, regularizing the model to be more robust in this sense does not require generation of adversarial examples and instead can be optimized directly. The method achieves this by decomposing the error of the smoothed classifier into classification error term and a robustness error term. The former captures the error from the smoothed classifier misclassifying given datapoint and the latter captures the error of certified radius being too small.
18

---

Published in Transactions on Machine Learning Research (07/2024)
Robustness error, much like hard label classification error, cannot be optimized directly. To address this, MACER proposes a surrogate loss to minimize the robustness error term-4 hinge loss on the data (z, v) for which go y,
max (0, - (fo())))))))
(14)
where fo( denotes the average of softmax probabilties on Gaussian perturbations of 2, fo(2), denotes the softmax probability fo(z) assigned to the true class y, and fo(z))zz denotes the maximum softmax probability over classes other than the true class. This loss term is added as a regularization term to the cross entropy loss from the soft smoothed classifier.
B.2 DPSGD and Robustness
The adversarial robustness of differentially private models has been studied in several prior works. Tursyn- bek ct al. 2021 demonstrated that models trained with DPSGD sometimes more vulnerable to input perturbations. Boenisch et. al.] (202T) further consolidate this claim with more experiments, and showed that improper choices of hyperparameters can lead to gradient masking. Zhang & Bu (2022) found that the success of adversarially training robust models with DPSGD depends greatly on choices of hyperparameters, namely smaller clipping thresholds and learning rates, differing from those that produce the most accurate models. These works reveal interesting adversarial robustness characteristics of DP models, however, they do not endeavor to improve the robustness of DP models.
Bu et al. (202a)) propose combining adversarial training with DPSGD. They achieve this by replacing the original example with an adversarially crafted example, obtained with a FGSM or PGD attack. Their work is orthogonal to ours in that they focus on the robustness to adversarial attacks, whereas we focus on certified robustness. Additionally, they do not perform any data augmentation, which has been shown to be effective against adversarial attacks (Rebuffi et al. 2021).
Compared with existing works in DP and CR (Phan al. 2019 2020 Tang et al. 2022) reviewed in Section
_ DP-CERT 1) presents simple and effective training scheme with augmentations, allowing practitioners to introduce different adversarial training techniques through noising, regularization, and adversarially crafted examples, 2) does not rely on a denoiser at inference time, reducing inference latency, and 3) can be used for training both randomly ialized or pre-trained networks.
C Experimental Details
C.1 Datasets
We conduct experiments on MNIST, MaShIon-MNIST and CIFAR10. The MNIST database of handwritten digits has training set of 60,000 examples, and test set of 10,000 examples, does Fashion-MNIST. Each example in MNIST and Fashion-MNIST is 28x28 grayscale image, associated with one label from 10 classes.
The CIFAR10 dataset consists of 60,000 RGB images from 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images of size 32x32x3.
C.2 Code and Implementation
Our implementation of DP-CERT in code is provided supplementary material. We give credit to the original repository for the implementation of SmoothAdy, MACER, Stability, and Consistency regulariza- tion For CERTIFY and computing local Lipschitz constant: we use the code provided by the original authors. All experiments were conducted on a cluster of 8 Nvidia V100 GPUs. All the training and inference
http://gthaa..cc//aaaatttttrrr/e local
1oca1-lipccittz lipschitz
19

---

Published in Transactions on Machine Learning Research (07 (024))
Regular
DPSGD
PSAC
DP-Gaussian
0P-SmoothAdv
DP-Stability
DP-MACER
0.25
100
80
Certifee Accuracy
Certiffed Accurecy
80
Certifeed Accuracy
40
40
40
20
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.5
1.0
1.5
2.0
radius
La radiuus
radius
Regular
GP-Gaussian
DP-SmoothAdV
DP-Stability
DP-MACER
o= 0.25
a= 0.5
o= 1.0
80
Certified Accuracy
Cortriied Accuracy
60
Certifie Accuracy
60
60-
40-
20-
0.6
1.0
Is radius
La radius
1. radius
Figure Approximate certified accuracy comparison on MNIST (top), and Fashion-MNIST (bottom).
procedures are implemented based on Pytorch v1.13.0 Paszke et al. 2019 and Opacus v1.3.0 (Youscfpour
et al. 2021 and our code is provided at ggilutb.com//yyrraalllbb/////
C.3
Additional Implementation and Hyperparameters
We set the learning rate as 0.001 and train the models for 10 epochs. The rest of the hyperparameters are the same as used by Bu et al. (2022b) For evaluation, we use CERTIFY with parameters n = 10, 000, no = 100, DP-
and a = 0.001, following previous work @Cohen et al. 2019 Salman et al. 2019). Our implementations SmoothAdv, Dt-Stability and DP-MACER are adapted from the original codebases for those regularization approaches (Salman al. 2020 Li et al. 2019 Zhai et al.] 2020), and use the default hyperparameters reported in the original papers. We set the number of augmentations K to 2 for MNIST and Fashion-MNIST, and 1 for CIFAR10, as they bring better trade-off between certified accuracy and efficiency (see Figure
D Additional Results
In this section, we show the complete results and figures for our comparative study, ablations, and analysis of fine-grained metrics.
D.1 Additional Comparative Study
Figure complements Figure from the main text by comparing the approximate certified accuracy between DP-CERT and basclines on MNIST, and FaSHion-MNIST. The results confirm that Regular and DP training methods that do not specifically consider robustness as a criteria give poor CR results, but all instantiations of the DP-CERT framework do much better. We are unable to show further results similar to Figure 3 using the basclines SocureSGD, StoBatch, and TransDenoiser bocause their implementations are highly customized and compatible with our codebase based on Pytorch and Opacus. The baseline results in Figure are directly copied from (Tang et al. 2022 which provided limited experimental settings for to match.
20

---

Published in Transactions on Machine Learning Research (07/2024)
DP- Gaussian
Op-Gaassiaan + PSAC
OP-Gaussian Consistency
DP-SmoothAdv
DP-SmoothAdV PSAC
DP-SmoothAdv Consistency
0.25
0= 0.5
0= 1.0
80
Accuracy
Certified Accuracy
80
Certified Accuracy
60
60
60
40
20
0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.5
1.0
1.5
2.0
La radius
1g radiuus
radius
DP Gaussas
DP Gaussian
PSAC
DP oP-Gaussian Consistency
DP-SmoothAdV
DP-SmoothAdv PSAC
DP-SmoothAdv Consistency
1.0
80
Certified Accuracy
60
Certified Accuracy
60
Certified Accuracy
60
40
20
20
20
radius
radius
La radius
Figure 8: Certified accuracy of DP-Ga Gaussian and DP-SmoothAdy combined with consistency regularization and PSAC on MNIST (top) and Fashion-MNIST (bottom).
D.2 Additional Ablation Study
To extend Figure f from the main text, we complete the ablation study in Figures and 9 on MNIST and INSH under a E {0.25, 0.5, 1.03. Figure& the effects of adding Consistency regularization, or changing the DP clipping method PSAC. Neither modification leads to significant changes to the certified accuracy, so recommend using the simplest combination of standard DPSGD clipping without additional regularization when training models from scratch. Figure g shows the effects of changing the multiplicity of augmentations K. Since additional augmentations greatly increase computation time, must navigate the tradeoff between efficiency and performance. We see that it is beneficial to use some number of augmentations or replace the original datapoint with a single augmentation (marked line 0), but beyond K = there is little further gain in certified accuracy, We note that the case without augmentations at all, namely DPSGD, achieves much worse CR as presented Table 2 and Figures and 7 One of our main conclusions is that adding a small number of Gaussian augmentations to DPSGD sufficient to improve certified robustness low cost to efficiency.
Table and f show additional hyperparameter studies learning rale À and clipping bound C for DP- Gaussian under == 0.25. Not only does this show that our models were well-tuned for clean accuracy, but that there is no dramatic drop in performance for nearby parameter values.
21

---

Published in Transactions on Machine Learning Research (07 (0244)
0=0.25
0=0.5
0=1.0
Augmentations-
80
Num Augmantations->
Certhed Accuracy
Certifed Accuracyy
Augmentations-4
Num Augmentations-6
Sugmentations=-8
20
0-
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.5
1.0
1.5
2.0
42 redius
rediuus
radiuu
oh 0.25
0=0.55
0=010
80
70
Num Augmentations=0
Num Augmentations=2
Certifedd Accuracy
60-
Cartifed Accuracy
60-
Certiredd Accuracc
-5
Num Augmentationss
Num sugmentationss
80
Num gugeetaaionss8
40
40
30
20
20
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.5
1.5
2.0
radius
radius
Figure Certified accuracy comparison under different numbers of augmentations (0, 2, 4, and 8) on MNIST (top) and Fashion-MNTST (bottom). The line marked 0 indicates replacing the original datapoint with an augmentation.
Table 3: Effect of Learning Rate X on Clean Accuracy (%)
Dataset
Learning Rate À
Clean Accuracy (%)
MNIST
0.25
97.71
0.5
98.13
97.78
Fashion-MNIST
2
83.47
84.76
83.96
CIFARIO
0.0005
84.11
0.001
87.61
0.002
85.53
D.3 Additional Per-sample Metric Analysis
Figures 10 through 14 complete our presentation of the three proposed metrics for interpreting robustness, the input gradient norms, input Hessian spectral norms, and local Lipschitz constants, over three datasets and values of (see also Figure in the main text). We use various training methods on MNIST and Fashion-MNIST under a e {0.25, 0.5, 1.03. Echoing the analysis of RQI Section 5.2 DPSGD produces bimodal distribution for the Hessian and gradient norms, while Regular training cxhibits log-normal distribution and smaller lails for large metric values. PSAC shifts the distributions to be closer to those of Regular training by reducing clipping bias. DP-CERT methods, on the other hand, shift the distribution towards smaller metric valuos, resulting in higher certified accuracy. An exception is DP-Stability, which has significantly higher average gradient and Hessian norms, but without the mode at very high values, and with lower local Lipschitz constants than the other three variants.
22

---

Published in Transactions on Machine Learning Research (07/202) 2024)
Table 4: Effect of Clipping Norm C on Clean Accuracy (%)
Clean Accuracy (%)
Clipping Norm C
MNIST
Fashion-MNIST
CIFAR10
0.01
94.89
75.60
83.79
0.05
97.75
83.42
83.99
0.1
98.13
84.76
87.61
0.2
97.72
84.00
84.19
0.5
96.47
81.46
84.24
DR-naussian
400
250
300
200
50
-15
-10
-5
-15
-10
-5
2
Inout Ressian Spectral Nerm
Log Input Grad horrs
Log Local Lipechitz Constant
(a) Comparison of Regular, DPSGD, PSAC, and DP-Gaussian training.
DP Gauss an
DP-Smotthidy
DP-Statliit
OF NACER
400
300
Tequency
-200
200
100
100
100
-33
-10
-15
-10
-5
Log Input Ressian Spectral Norm
Log Imput Grad Norm
Log Lpsccii Constant
(b) Comparison of DP-Gaussian, Ady,, DP-Stability, and DP-MACER.
Figure 10: Per-sample metric comparison, MNIST, = 0.25
23

---

Published in Transactions on Machine Learning Research (07 (07/202) 2224)
Dog Alr
DPSGD
PSAC
(P-GGusssee
250
250
200
2.50
200
100
50
Log Imput Hessian Speciral Norm
Log impul Grad
Log Local Lipchitt
(a) Comparison of Regular, DPSGD, PSAC, and DP-Gaussiat training.
Statiliity
350
250
Requercc
200
250-
200
150
100
100
50
50
-10
Log
Log
(b) Comparison DP-Gaussian, Ad--SmotthAd, DP-Stability, and DP-MACER.
Figure 11: Per-sample metric comparison, MNIST,
1.0
Pag
DPSGD
PSAC
DP:Saussian
250
200
Dequenny
150
250
200
50
Log impur Reccian Spectral Norm
Log Inpur Grad Morm
Log Local Uppenitee Constant
(a) Comparison of Regular, DPSGD, PSAC, and DP-Gaussian training.
CP-Smosthiic
DP-Statiliy
250
250
200
Requeecc
350
Prqqeencc
230
100
200
150
30
50
-10
(b) Comparison DP-Gaussian, Ad-SmoothAdy, DP-Stability, and DP-MACER.
Figure 12: Per-sample metric comparison, Fashion-MNIST, 0.25
24

---

Published in Transactions on Machine Learning Research (07 (07/202) /022)
250
200
Requencc
200
200
150
100
50
-15
-10
-15
-10
Hussian Spectral Norm
Log Grad Norm
Log Local Lipcciitz Constant
Comparison of Regular, DPSGD, PSAC, and DP-Gaussian training.
D2 Gaussan
SmoothAdv
D7 Stabilty
MAMAEER
500
250
200
100
150
3.00
300
200
300
Log Input Hession Spectrol Norm
Log Input Grod Nome
Log Local Uppccttz Constont
(b) Comparison of DP-Gaussiat Ad-SmmothAAy,, DP-Stability, and DP-MACER.
Figure 13: Per-sample metric comparison, Rashion-MNISTT o 0.5
Regular
DPSGD
PSAC
DP Gouudan
250
Dequenty
150
250
8:00
200
100-
50
-10
Log Impul Hessian Spectral Narm
Log Imput Grad
Log Local Upachitz
(a) Comparison of Regular, DPSGD, PSAC, and OP-Gaussian training.
ro-statility
3000
250
requetic
200
Prqqaenoy
200
250
100
100
200
100
50
Imput Spectral
Log Imput.
Lag
(b) Comparison DP-Gaussian, Ady-SmothhAy, DP-Stability, and DP-MACER.
Figure 14: Per-sample metric comparison, Fashion-MNIST, o 1.0
25

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Reguiar
DPSGO
PSAC
De Gausstar
2500
2000
5500
2000
500
Certified Radius
Figure 15: Distributions of certified radii for Regular, DPSGD, PSAC, and DP-Gaussian training on MNIST with a 0.5.
Regular
DPSGD
PSAC
OP-Gaussian
DP Cp-SmoothAdv
DP-Stability
DP-MACER
a= 0.25
=
0.5
0 1.0
100
100
100
Adversarial Accuracy
Accuracy
Adversarial Accuracy
60
60
60
20
20
0.0
0.2
0.6
0.8
1.0
0.2
0.4
0.8
1.0
0.0
0.4
0.6
1.0
Lyr Attack Strength
Lay Strength
Strength
Regular
DPSGD
PSAC
Ga-usassian
Cp-omothhAdy
DP-Stabiltty
DP-MACER
0.25
0.5
1.0
100
100
100
Adversarial Accuracy
Adversarial Accuracy
Adversarial Accuracy
80
80
80
40
40
20
20
20
0.2
0.6
0.8
0.2
0.4
0.8
1.0
0.0
0.4
0.6
100
Lar Attack Strength
Lay Attack Strength
Lur Altacc Strength
Figure 16: Adversarial accuracy against a -GGMMMM attack under various attack strengths on MNIST (top) and Fashion-MNIST (bottom).
Figure 15 shows the distribution of certified radii for baseline training methods and an instance of DP-CERT for the same settings as Figure 5 Whereas Regular, DPSGD, and PSAC training all have large spike of samples that cannot be certified at any level, Ga-Gaussian achieves certified radii above 1.0 for most samples, with the mode even higher at 1.6.
While our main focus has been on certified robustness, we briefly examine adversarial robustness. Figure 16 shows the adversarial accuracy under FFGM attack, with the attack strength in (0.0005, 0.01, 0.1, 1). Consistent with the ranking of the average local Lipschitz constant from Figures I0 through 14 DP-Stability consistently outperforms other approaches, while Gaussinn, DP-SmoothAdy, and DP-MACER all achieve similar adversarial accuracy above that of the unprotected baselines.
26

---

Published in Transactions on Machine Learning Research (07 2024)
Regular
DPSGD
PSAC
DP-Gaussian
0.5
T= 1.0
1.5
400
400
400
trequency
frequency
200
200
200
100
100
100
Log Local Lipschitz Constant
Log Locall Lipschitz Constant
Log Local Lipschitz Constant
400
400
400
frequency
300
frequency
200
200
200
100
100
100
Log Local Lipschitz Constant
Log Local Lipschitz Constant
Log Local Lipschitz Constant
(a) Per-sample metric comparison among Regular, DPSGD, PSAC, and DP-Gaussian training.
ga-Gasssian
30-SmoothAAdv
De-Stability
DP-MACER
T 0.5
T= 1.0
1.5
400
400
400
300
300
rrequency
Frequency
200
200
200
100
100
100
Log Local Lipschitz Constant
Log Local Lipschitz Constant
Log Local Lipschitz Constant
400
400
400
frequency
300
frequency
300
300
100
100
100
-2
2
Log Local Lipschitz Constant
Log Local Lipschitz Constant
Log Local Lipschitz Constant
(b) Per-sample metric comparison among DP-Gaussian, DP-SmoothAdy, DP-Stability, and DP-MACER.
Figure 17: Distributions of log local Lipschitz constants among baselines and proposed methods, MNIST, 0.5. In each subfigure, data points classified into ones with certified radius above the threshold 7 (top row), and below the threshold (bottom row). We display E (0.5, 1.0, 1.5).
Finally, as shown in Figure 17 we select three certified radius thresholds - E {0.5, 1.0, 1.5), and separately plot the log local Lipschitz constants of examples above and below 7 on the top and bottom rows of the subfigures respectively. First, the examples with cortified radii below the threshold have higher average local Lipschitz constant. Second, we increase the threshold т, more examples with higher local Lipschitz constant end up below the certified radius threshold. We do similar analysis for the other two metrics in Figures 18 and 19 but since the local Lipschitz constant is derived using PGD attack, it naturally correlates with the robustness to adversarial examples better. For further comparison, we present the FGSM accuracy on MNIST and Fashion-MNIST under altack strengths in {0.0005, 0.01, 0.5. 5, 1} in Figure T6 Consistent with the ranking of the average local Lipschitz constant, Stabiiiyyyty consistently outperforms other approaches, while DP-Gaussian, DP-SmoothAdy and DP-MACER achieve similar adversarial accuracy over different attack margins.
27

---

Published in Transactions on Machine Learning Research (07 (07/2024) /204)
Regular
DPSGD
PSAC
DP-Gaussian
= 0.5
T= 1.0
T= 1.5
400
400
400
300
300
300
Frequency
Frequency
Frequency
200
100
100
100
-15
-10
-5
-15
-10
0
-15
-10
Log Input
Log Input
Log Input
300
300
300
frequency
frequency
Frequency
200
200
200
-15
-10
-15
-10
-15
-10
Log Input
Log
Log Input Norm
(a) Per-sample metric comparison among Regular, DPSGD, PSAC, and DP-Gaussian training.
QP-Gaussian
DP-SmoothAdv
DP-Stability
DP-MACER
0.5
400
300
300
300
frequency
Frequency
200
200
200
100
0
-20
-15
-10
-5
-20
-15
-10
-20
-15
-10
Log Grad
Grad Norm
Input Grad Norm
400
400
400
frequency
Frequency
200
200
200
100
-20
-15
-20
-10
-20
-15
-10
Log Input Grad Norm
Input Grad Norm
Input Grad Norm
(b) Per-sample metric comparison among DP-Gasssian, DP-SmoothAdy, DP-Stability, and DP-MACER.
Figure 18: Comparing the distribution input gradient norm of baselines and proposed methods on MNIST.
All DP-CERT mothods are trained with o 0.5. We show the logarithmic metric values for all mothods for better visualization.
28

---

Published in Transactions on Machine Learning Research (07 (07/202)) /204)
Regular
DPSGD
PSAC
DP-Gaussian
T 0.5
T= 1.0
T= 1.5
400
400
300
300
300
Frequency
Frequency
Frequency
200
100
100
0
-15
-10
-5
-15
-10
-5
-15
-10
-5
Log Input Spectral
Log Input Spectral
Log Input Spectral
300
300
300
frequency
Rrequency
Reequency
200
200
200
15
-10
-15
-10
-15
-10
-5
Log Input Spectral Norm
Log Input Hessian Norm
Log Input Hessian Spectral
(a) Per-sample metric comparison among Regular, DPSGD, PSAC, and DP-Gaussian training.
OP-Gaussian
DP-SmonthAdv
DP-Stability
DP-M
= 0.5
T=
300
300
300
frequency
Frequency
200
200
200
0
-20
-15
-10
-20
-15
-10
-20
-15
-10
-5
Log Input Spectral Norm
Log Input Hessian Spectral
Log Input Spectral
400
400
300
300
300
frequency
Frequency
Prequency
200
200
200
-20
-15
-10
-20
-15
-10
-20
-15
-10
-5
Log Input Spectral Norm
Log Input Hessian Spectral
Log Input Spectral
(b) Per-sample metric comparison among DP-Gaussian, DP-SmoothAdv, DP-Stability, and DP-MACER.
Figure 19: Comparing the distribution of input Hessian spectral norm of baselines and proposed methods on MNIST. All DP-CERT methods are trained with = 0.5. We show the logarithmic metric values for all methods for better visualization.
29


Reviews:
[{"review_id": "UC7vVMw23f", "reviewer": "Reviewer_Wp4S", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n+ The paper is easy to follow \n+ The approach is simple which can be easily implemented \n\nWeaknesses: \n+ There are no formal proof of the differential privacy and certified robustness. \n   When you do the proof, you need consider how the random smoothing can affect DP. Also, you need consider how DP may affect CR proof. \n   And when you consider the formal proof, I believe you will see the proposed may not work, or at least some detailed inference needs to be done to guarantee both. \n+ There is no comparison with the existing methods which establish DP and CR\n+ The complete algorithm DPSGD and DP-CERT should be provided", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper shows a simple solution which combines DP-SGD with random smoothing which can provide both differential privacy and certified robustness.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Formal proof of the differential privacy and certified robustness need to provided. \n   When you do the proof, you need consider how the random smoothing can affect DP. Also, you need consider how DP may affect CR proof. \n\n2. Please evaluate your solution with existing methods which establish DP and CR\n3. The complete algorithm DPSGD and DP-CERT should be provided in the main texts.", "broader_impact_concerns": "Both DP & CR are important aspects of learning. No concern on the broader impacts.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RWPAQfQGMl", "reviewer": "Reviewer_jh7m", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\nA simple and effective method, DP-CERT, that achieves both privacy and robustness guarantees simultaneously is proposed. It has been shown in the experiments that DP-CERT outperforms the existing works in the real-world datasets. \n\nWeakness:\n\n1. The paper mainly proves the effectiveness of the proposed algorithm through experiments, but lacks theoretical support. Although it is an experimental study, I think that at least the theoretical results of the privacy guarantee part should be formally given as a theorem. Specifically, what level of differential privacy does the proposed algorithm satisfy when updating parameter via (2)?\n\n2. Due to the lack of theoretical results in the privacy guarantee part, I'm not sure whether the comparison of the existing algorithms in the experiments is fair. Since even if the same noise is added in each iteration, the privacy guaranteed by each algorithm may still be different. We should compare the accuracy of different methods to achieve the same level of differential privacy (the same $\\epsilon$ and $\\delta$).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method called DP-CERT that achieves better-certified robustness with DP-SGD training through augmentations, regularization, and randomized smoothing, reconciling privacy and robustness for trustworthy ML. While DP-CERT is a combination of known methods, the experiments show that it reduces the computational complexity and increases the certified accuracy compared to previous works that bring CR guarantees to DP.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Since the paper focuses on studying differential privacy with certified robustness, I would suggest the authors move the introduction of  DP from the appendix to the main paper, e.g., Section 2. The present version is a little bit unclear if the audience is not familiar with DP. \n\n2. As mentioned in the weakness, there is a lack of rigorous proof of the main results. A formal privacy guarantee for the proposed algorithm is required.  Specifically, if the method is run with a noise scale $\\rho$, then it satisfies $(\\epsilon,\\delta)$-DP with $\\epsilon=?$. It is unclear to me.\n\n3. Experiments compare the proposed method with the state of the art. However, all methods need to be compared under the same level of privacy guarantee, e.g., the same $\\epsilon$ and $\\delta$. Since the privacy guarantee of the proposed method is missing, I’m not sure the results of the experiments are fair and reasonable. \n\n4. In Table 2, when $\\sigma$ increases, DP-stability and DP-MACER outperform the other methods, while when $\\sigma$ is small, DP-smoothadv seems better. Are there any potential reasons or explanations?", "broader_impact_concerns": "no", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yC4oTmH68o", "reviewer": "Reviewer_7N2U", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Innovative Approach: DP-CERT's integration of randomized smoothing into DP training is a significant advancement, addressing the long-standing conflict between achieving DP and CR in a unified model.\nImpressive Performance: The method sets new benchmarks in certified accuracy on CIFAR10, substantially outperforming prior works while maintaining stringent privacy guarantees.\nComprehensive Analysis: The per-sample metric analysis offers deep insights into the relationship between certifiable radii and model robustness, guiding future efforts in enhancing CR in DP models.\n\nWeakness:\n\nModel Complexity: Despite its effectiveness, the DP-CERT framework introduces additional complexity in training, potentially increasing computational resources and training time.\nGeneralizability: The experiments primarily focus on image classification tasks and CIFAR10 dataset. Further studies are required to assess the framework's performance across diverse datasets and model architectures.\nImpact on Fairness: The paper does not explore the impact of combining DP and CR on model fairness, an aspect critical for deploying AI in sensitive applications.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors presents DP-CERT, a method integrating randomized smoothing into differentially private model training. This approach significantly enhances both privacy and robustness, with a notable increase in certified accuracy on CIFAR10, compared to prior work. The paper emphasizes the importance of privacy and robustness in machine learning, challenges in achieving both, and how DP-CERT offers a simple, adaptable framework for this purpose. Through experiments, DP-CERT is shown to improve certifiable robustness without compromising privacy, offering practical guidance for trustworthy machine learning implementations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Expand Experimental Evaluation: Extend the evaluation to more datasets and model architectures, ensuring the framework's applicability and robustness across various domains.\n\nOptimize Computational Efficiency: Investigate methods to reduce the computational overhead introduced by DP-CERT, making it more accessible for real-world applications. \n\nFairness Assessment: Conduct an analysis of the impact on model fairness when applying DP-CERT, addressing potential disparate impacts.\n\nHyperparameter Sensitivity Analysis: Provide a detailed sensitivity analysis of the key hyperparameters, aiding practitioners in model tuning.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an approach called DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees. The main idea is to use augmentations that do not incur additional privacy costs, while employing regularizations and adversarial training methods to enhance robustness. All reviewers appreciated the straightforward and effective approach, supported by comprehensive experimental results. Although two reviewers suggested for theoretical proofs for explaining the empirical success of the proposed approach, the authors partially addressed this comment to justify the privacy guarantees by referencing De et al. (2022), as discussed on page 6 of the current version.

Overall, combining all reports from the reviewers and my own reading of the paper, it is evident that the paper provides contributions and observations that are novel in the literature, and the proposed method is sufficiently supported by experiments. Therefore, I conclude that it meets the standards of TMLR.
Decision: Accept (poster)