A U NIFIED FRAMEWORK FOR RANDOMIZED SMOOTHING BASED CERTIFIED DEFENSES
Anonymous authors
Paper under double-blind review
A BSTRACT
Randomized smoothing, which was recently proved to be a certiﬁed defensive technique, has received considerable attention due to its scalability to large datasets and neural networks.
However, several important questions still remain unanswered in the existing frameworks, such as (i) whether Gaussian mechanism is an optimal choice for certifying (cid:96) 2 -normed robustness, and (ii) whether randomized smoothing can certify (cid:96) ∞ -normed robustness (on high-dimensional datasets like ImageNet).
To answer these questions, we introduce a uniﬁed and selfcontained framework to study randomized smoothing-based certiﬁed defenses, where we mainly focus on the two most popular norms in adversarial machine learning, i.e., (cid:96) 2 and (cid:96) ∞ norm.
We answer the above two questions by ﬁrst demonstrating that Gaussian mechanism and Exponential mechanism are the (near) optimal options to certify the (cid:96) 2 and (cid:96) ∞ -normed robustness.
We further show that the largest (cid:96) ∞ radius certiﬁed by randomized smoothing is upper bounded by
O (1 / √ d ) , where d is the dimensionality of the data.
This theoretical ﬁnding suggests that certifying (cid:96) ∞ -normed robustness by randomized smoothing may not be scalable to high-dimensional data.
The veracity of our framework and analysis is veriﬁed by extensive evaluations on CIFAR10 and ImageNet.
I NTRODUCTION
The past decade has witnessed tremendous success of deep learning in handling various learning tasks like image classiﬁcation (Krizhevsky et al., 2012), natural language processing (Cho et al., 2014), and game playing (Silver et al., 2016).
Nevertheless, a major unresolved issue of deep learning is its vulnerability to adversarial samples that are almost indistinguishable from natural samples to humans but can mislead deep neural networks (DNNs) to make wrong predictions with high conﬁdence (Szegedy et al., 2013; Goodfellow et al., 2014).
This phenomenon, referred to as adversarial attack, is considered to be one of the biggest threats to the deployment of many deep learning systems.
Thus, a great deal of effort has been devoted to developing defensive techniques for it.
However, the majority of the existing defenses are of heuristic nature ( i.e., without any theoretical guarantees), implying that they may be ineffective against stronger attacks.
Recent works (He et al., 2017; Athalye et al., 2018; Uesato et al., 2018) have conﬁrmed this concern, and showed that most of those heuristic defenses actually fail to defend stronger adaptive attacks.
This forces us to shift our attentions to certiﬁable defenses as they can classify all the samples in a predeﬁned neighborhood of the natural samples with a theoretically-guaranteed error bound .
Among all existing certiﬁable defensive techniques, randomized smoothing emerges as the most popular one due to its scalability to large datasets and arbitrary networks.
Remarkably, using the Gaussian mechanism for randomized smoothing, Cohen et al.
(2019) successfully certify 49% accuracy on the original ImageNet dataset under adversarial perturbations with (cid:96) 2 norm less than 0 . 5 .
Despite these successes, there are still several unanswered questions regarding randomized smoothing based certiﬁed defenses.
One of such questions is, why should Gaussian noise be used for randomized smoothing to certify (cid:96) 2 -normed robustness, and is Gaussian mechanism the best option?
Another important question is regarding the generalizability of this method to other norms, especially the (cid:96) ∞ norm.
If randomized smoothing can be used to certify (cid:96) ∞ -normed robustness, what mechanism is the optimal choice?
To shed light on the above questions, we propose in this paper a uniﬁed and self-contained framework for randomized smoothing-based certiﬁed defenses.
We look at the problem from a differential privacy’s point of view and present two types of robustness in this framework.
One is motivated by
[Table]
(cid:15) -differential privacy ( (cid:15) -DP), which uses ∞ -divergence to measure the distance between the probabilities of predictions on randomized natural samples and randomized adversarial samples and is therefore called D ∞ robustness.
The other is inspired by (cid:15)
-zero concentrated differential privacy ( (cid:15) -zCDP) that uses the Maximal Relative R´enyi (MR) divergence as the probability distance measurement and is called D MR robustness.
For both of them, we focus on certifying robustness in either (cid:96) 2 or (cid:96) ∞ norm by randomized smoothing.
Speciﬁcally, our contributions are ﬁve-fold: 1.
We propose a uniﬁed and self-contained framework for certifying D ∞ and/or D MR robustness in (cid:96) 2 and (cid:96) ∞ norms by randomized smoothing.
In our framework, we demonstrate that the Gaussian mechanism is a near optimal choice for certifying D MR robustness in (cid:96) 2 norm, and the robust radius is O (1) .
We also prove that an exponential mechanism is the optimal choice for certifying D ∞ robustness in (cid:96) ∞ norm, but the robust radius is only O (1 /d ) , making it unscalable to highdimensional data.
We show that the Gaussian mechanism is also a near optimal choice for certifying D MR robustness in (cid:96) ∞ norm, but the robust radius is O (1 / √ d log d ) , making it also hardly scalable to high
-dimensional data.
The largest robust (cid:96) ∞ radius that can be certiﬁed by randomized smoothing to achieve D MR robustness is upper bounded by O (1 / √ d ) .
Table 1 summarizes the (near) optimal mechanisms of our framework for certifying the (cid:96) 2 and (cid:96) ∞ - normed robustness.
R ELATED W ORK
There are three main approaches for certiﬁed defenses.
The ﬁrst approach formulates the task of adversarial veriﬁcation as an optimization problem and solves it by relaxations (Dvijotham et al., 2018; Raghunathan et al., 2018; Wong & Kolter, 2018).
The second approach uses different techniques, such as interval analysis and abstract interpretations, to maintain an outer approximation of the output at each layer through the network. (Mirman et al., 2018;
Wang et al., 2018; Gowal et al., 2018).
The third approach uses randomized smoothing to certify robustness, and is gaining popularity recently due to its strong scalability (Lecuyer et al., 2018;
Li et al., 2018; Cohen et al., 2019) to large datasets and arbitrary networks.
For this approach, Lecuyer et al.
(2018) showed that randomized smoothing can certify the (cid:96) 2 and (cid:96) 1 -normed robustness by using inequalities from differential privacy.
Li et al.
(2018) achieved a stronger guarantee on the (cid:96) 2 -normed robustness using tools from information theory.
Cohen et al.
(2019) further obtained a tight guarantee on the (cid:96) 2 -normed robustness using Gaussian noise.
A remaining issue in all of these works is that they did not give answers to questions like why Gaussian noise is used to certify the (cid:96) 2 -normed robustness and what is the best mechanism to certify the (cid:96) ∞ -normed robustness.
To answer these questions, we present in this paper a new general framework to study randomized smoothing based certiﬁed defenses.
R OBUSTNESS M OTIVATED BY D IFFERENTIAL P RIVACY
In this section, we introduce our framework.
Let x be a data sample and
y ∈ Y be its label, where Y is the label set.
We denote by f ( · ) a deterministic classiﬁer with prediction f ( x ) for any data sample x .
If there exists an x (cid:48) in a small l p ball centered at x and with f ( x (cid:48) )
(cid:54) =
f ( x ) , x (cid:48) is viewed as an adversarial sample.
Deﬁnition 1 (Randomized Classiﬁer (Cohen et al., 2019)) .
Given an input x , the prediction of a randomized classiﬁer g ( · ) is deﬁned as
[Equation]
Speciﬁcally, for a randomized smoothing classiﬁer g ( x ) =
f ( x + Z ) , where Z is a random vector and f ( · ) is a deterministic classiﬁer, the prediction of x is the class of c whose region S (cid:44) { ˜ x ∈
R d , f (˜ x ) =
c } has the largest probability measure in the distribution of x + Z ( ˜ x ∼ p ( x + Z ) ).
Before introducing our framework, we ﬁrst recall the deﬁnition of robustness for a deterministic classiﬁer in (Diochnos et al., 2018).
Deﬁnition 2 (Robustness (Diochnos et al., 2018)) .
For a given classiﬁer f , a sample x and some norm (cid:107) · (cid:107) .
f is ( r, (cid:107) · (cid:107) ) -(error-region) robust on the sample x if
[Equation]
where B ( x, r ) is the ball centered at x and with norm (cid:107) · (cid:107) and radius r .
Note that in Deﬁnition 2, the classiﬁer is assumed to be deterministic.
To generalize the concept of robustness to randomized classiﬁers (see Deﬁnition 1), we deﬁne a relaxed version of the (errorregion) robustness.
Since g ( x ) is a random value, instead of using equality, we measure the difference between g ( x ) and g ( x (cid:48) ) by a certain divergence.
This leads us to the following deﬁnition, which is a basic concept in our framework that will be used throughout the paper.
Deﬁnition 3 (Relaxed Robustness) .
For a given (randomized) classiﬁer g ( · ) , a sample x and some norm (cid:107) · (cid:107) , the classiﬁer g is ( r, D, (cid:107) · (cid:107) , (cid:15) ) -(error-region) robust on x if
[Equation]
where D is some divergence metric between two probability distributions.
The max function is used to ensure that the measurement is symmetric.
Compared with Deﬁnition 2, there are two additional terms in Deﬁnition 3: (cid:15) represents the “distance” or difference between the distributions of g ( x ) and g ( x (cid:48) ) .
When (cid:15) is small, we expect that the distributions of predictions on x and x (cid:48) , i.e., g ( x ) and g ( x (cid:48) ) , are almost the same, which is just a generalization of the equality in Deﬁnition 2.
D is some divergence measurement between two probability distributions.
In this paper, we use two types of divergence, ∞ -Divergence and
Maximal Relative R ´ enyi Divergence, to measure the distance between two probability distributions.
Correspondingly, we have two types of robustness called D ∞ and D MR robustness.
Deﬁnition 4 ( ∞ -Divergence) .
The ∞ -Divergence D ∞ of distributions P and Q is deﬁned as
[Equation]
where supp ( Q ) is the support of the distribution Q .
Deﬁnition 5 (Maximal Relative R´enyi Divergence) .
The Maximal Relative R´enyi Divergence
D MR ( P (cid:107) Q ) of distributions P and Q is deﬁned as
[Equation]
where D α ( P (cid:107) Q ) is the R´enyi divergence between P and Q , which is deﬁned as
[Equation]
Deﬁnition 6 ( D ∞ Robustness) .
A randomized smoothing mechanism A ( · ) (including classiﬁers) is a ( r, D ∞ , (cid:107) · (cid:107) , (cid:15) )
-robust mechanism if
[Equation]
where (cid:107)·(cid:107) is the norm of the ball B ( x, r ) .
If a randomized smoothing classiﬁer g ( · ) satisﬁes Eq.
(3), it is a ( r, D ∞ , (cid:107) · (cid:107) , (cid:15) )
-robust classiﬁer or it certiﬁes D ∞
Robustness.
D ∞ Robustness is motivated by the notion of (cid:15) -differential privacy ( (cid:15) -DP)
(Dwork et al., 2006).
To achieve (cid:15) -DP for a randomized algorithm, we can use several mechanisms such as Laplacian mechanism or Exponential mechanism (see (Dwork et al., 2014) for details).
However, it is known that adding Gaussian noise often does not lead to (cid:15) -DP, but rather ( (cid:15), δ ) -DP
(Dwork et al., 2014) which has an additional parameter δ and thus is harder to be incorporated in our framework.
To alleivate this issue, we employ Maximal Relative R´enyi Divergence as the the probability distance measurement to deﬁne another type of robustness, namely D MR robustness.
Deﬁnition 7 ( D MR Robustness) .
A randomized smoothing mechanism A ( · ) is a ( r, D MR , (cid:107) · (cid:107) , (cid:15) ) - robust mechanism if
[Equation]
If a randomized smoothing classiﬁer g ( · ) satisﬁes Eq.
(4), it is a
[Equation]
D [Equation] [Equation]
) -robust classiﬁer or it certiﬁes D MR Robustness.
D MR Robustness is inspired by the notion of zero-Concentrated Differential Privacy (zCDP)
(Bun & Steinke, 2016), whose connection to DP is shown in the following theorem.
Theorem 8 ((Bun & Steinke, 2016)) .
Let P and Q be two probability distributions satisfying the conditions of D ∞ ( P (cid:107) Q ) ≤ (cid:15) and D ∞ ( Q (cid:107) P ) ≤ (cid:15) .
Then, D MR ( P (cid:107) [Equation]
Theorem 8 indicates that D MR -robustness is a relaxed version of D ∞ -robustness.
Remark (Connections between D ∞ & D MR Robustness and Standard Deﬁnitions) .
Although D ∞ & D MR Robustness are seemingly new concepts deﬁned in this paper, they actually have several connections with the existing frameworks Lecuyer et al.
and Cohen et al. Speciﬁcally, as long as D ∞ robustness is certiﬁed, the expected output stability bound in Lecuyer et al. (2018) will be guaranteed with δ (cid:48) = 0 .
And if D MR robustness is certiﬁed, the expected output stability bound in Lecuyer et al.
(2018) will be guaranteed with (cid:15) (cid:48)
=
( c + 1) √ (cid:15) and δ (cid:48) =
exp( − c 2 4 ) , according to Theorem 10.
Besides, the “scale” of the robust radius certiﬁed by our framework is similar the “scale” of the robust radius in Cohen et al.
(2019), according to Corollary 11.
Theorem 9 (Postprocessing Property) .
Let g ( x ) =
f ( A ( x )) be a randomized classiﬁer, where f ( · ) is any deterministic function (classiﬁer).
g ( · ) is ( r, D, (cid:107) · (cid:107) , (cid:15) )
-robust if A ( · ) is ( r, D, (cid:107) · (cid:107) , (cid:15) ) -robust
(where D includes D ∞ and D MR ).
The above theorem is derived from the post-processing properties of DP and zCDP.
A detailed proof (explanation) is given in Appendix B.
This property allows us to concentrate only on the randomized smoothing mechanism A without needing to consider the speciﬁc form of the deterministic function (classiﬁer) f ( · ) .
Next, we consider the cases of certifying D ∞ or D MR robustness using (cid:96) 2 and (cid:96) ∞ -norm.
3. C ERTIFYING (cid:96) 2 - NORMED R OBUSTNESS
The following theorem shows that randomized smoothing by the Gaussian mechanism is ( r, D MR , (cid:107) · (cid:107) , (cid:15) )
-robust.
Theorem 10.
Let f be any classiﬁer and g ( x ) =
f ( x + z ) be its corresponding randomized classiﬁer for samples x ∈ R d , where z ∼ N (0 , σ 2 I d ) .
Then, g ( · ) is ( r, D MR , (cid:107) · (cid:107) 2 , r 2 2 σ 2 ) -robust on any x .
Moreover, let (cid:15) denote r 2 2 σ 2 .
Then, for any λ > 0 and any measurable set S (cid:54) = ∅ , the following holds with probability at least 1 − exp( − λ 2 4 (cid:15) ) ,
[Equation]
That is, when λ = c √ (cid:15) , log P ( g ( x ) ∈ S ) P ( g ( x (cid:48) )
∈ S ) ≤ ( c + 1) √ (cid:15) with probability 1 − exp( − c 2 4 ) .
In practice, c = 3 is enough to achieve a high probability.
Corollary 11.
Adding Gaussian noise z ∈ N (0 , σ 2 I d ) can defend any x (cid:48) ∈ B ( x, r = √ 2 (cid:15)σ ) that satisﬁes the condition of D MR ( g ( x ) (cid:107) g ( x (cid:48) )) ≤ (cid:15) with probability at least 1 − exp( − c 2 4 ) .
Furthermore, √ (cid:15) can be calculated (bounded) by (log p a − log p b ) / 2(1 + c ) or (log p a / (1 − p a )) / 2(1 + c ) (binary case), where p a and p b are respectively the probabilities of the randomized classiﬁer g ( · ) returning the most probable class c a and the runner-up class c b on input x .
Detailed proofs for Theorem 10, Corollary 11, and all the following theorems are provided in
Appendix B. From Theorem 9, we can see that for classiﬁers like g ( x ) =
f ( x + z ) , we only need to prove that the randomized mechanism A ( x )
=
x
+ z ( z ∼ N (0 , σ 2 I d )) is ( r, D MR , (cid:107) · (cid:107) 2 , r 2 2 σ 2 ) - robust.
Also, the connection between (cid:15) and p a , p b can be derived for all (cid:15) or √ (cid:15) (in the certiﬁed radii) as in Corollary 11.
Note that a similar theorem has also been proved by Cohen et al.
But there are some major differences between our framework and theirs (Cohen et al., 2019).
Speciﬁcally, our framework certiﬁes the robustness with a probability of failure, and the certiﬁed radius r depends on c that controls the probability of failure.
A smaller c yields a larger r compared to those in Cohen et al.
(2019), and vice versa.
Moreover, in our framework, we show that the Gaussian mechanism is a near optimal option, by providing a lower bound below for all possible noises that can certify the (cid:96) 2 -normed D MR robustness.
Next, we consider the following unanswered question ( i.e., the ﬁrst question).
Since there are inﬁnite ways of sampling z , a natural problem is to determine whether Gaussian mechanism is the optimal option to certify the (cid:96) 2 -normed D MR robustness.
To answer this question, we ﬁrst give a lower bound on the magnitude of the noise z added in the randomized smoothing mechanism A ( x )
=
x
+ z to ensure that A ( x ) , as well as f ( A ( x )) , is ( r, D MR , (cid:107) · (cid:107) 2 , (cid:15) )
-robust.
If the magnitude of Gaussian noise is close to the lower bound, then Gaussian mechanism is considered as “near optimal”.
Theorem 12 (Lower Bound of the Noise) .
For any (cid:15) ≤ O (1) , if there is a (2 r, D MR , (cid:107) · (cid:107) 2 , (cid:15) 2 ) - robust randomized smoothing mechanism A ( x )
= x
+ z : [0 , r √ d ] d (cid:55)→ [0 , r √ d ] d such that for all x ∈ [0 , r √ d ] d ,
E [ (cid:107) z (cid:107) ∞ ] = E A (cid:107)A ( x ) − x (cid:107) ∞
≤ α, for some α ≤ O (1) , then it must be true that α ≥ Ω( r √ (cid:15) ) .
In another word, Ω( r √ (cid:15) ) is the lower bound of the expected (cid:96) ∞ norm of the random noise.
Theorem 12 indicates that the expected (cid:96) ∞ norm of the added random noise should be at least Ω( r √ (cid:15) ) to guarantee ( r, D MR , (cid:107)·(cid:107) 2 , (cid:15) )
-robustness.
For Gaussian mechanism, the expected (cid:96) ∞ norm is O ( σ √ log d ) ((Orabona & P ´ al, 2015)), which is O ( r √ (cid:15) √ log d ) according to Corollary 11.
This means that Gaussian mechanism is near optimal (i.e., up to an O ( √ log d ) factor) here.
Equivalently, if we ﬁx the magnitude of the expected (cid:96) ∞ -norm of the added noise as α , the largest radius r that can be certiﬁed by any ( r, D MR , (cid:107) · (cid:107) 2 , (cid:15) ) -robust randomized smoothing mechanisms is upper bounded by O ( α √ (cid:15) ) , which is also close to the robust radius guaranteed by Gaussian mechanism (up to an
O ( √ log d ) factor).
3. C ERTIFYING (cid:96) ∞ - NORMED R OBUSTNESS
Previous work on the randomized smoothing-based certiﬁed defenses (Cohen et al., 2019; Li et al., 2018) mainly uses Gaussian noise to certify the (cid:96) 2 -normed robustness.
Thus, another natural question ( i.e., the second question) is to determine whether randomized smoothing can use some mechanism to certify the (cid:96) ∞ -normed robustness.
In this section, we consider this question using our general framework.
Before extending our result to the (cid:96) ∞ -normed case, we ﬁrst recall the (cid:96) 2 -normed case and investigate the form of the density function of Gaussian noise: p ( z ) ∝ exp( − (cid:107) z (cid:107) 22 σ 2 ) .
Based on this, we conjecture that, to certify (cid:96) ∞ -normed robustness, we can sample the noise using an exponential mechanism:
[Equation]
We show in the following theorem that randomized smoothing by (6) certiﬁes ( r, D MR , (cid:107) · (cid:107) ∞ , · ) - robustness, which could be considered as an extension of the (cid:96) 2 -normed case.
Moreover, we can prove that it is ( r, D ∞ , (cid:107) · (cid:107) ∞ , · ) -robust.
However, the certiﬁed radius r is O (1 /d ) , which implies that it is unscalable to high-dimensional data.
Theorem 13.
Let f be any classiﬁer and g ( x ) =
f ( x + z ) be its corresponding randomized classiﬁer for sample x ∈ R d , where the noise z ∼ p ( z ) in (6).
Then, g ( · ) is ( r, D MR , (cid:107) · (cid:107) ∞ , r 2 2 σ 2 ) -robust.
Moreover, it is ( r, D ∞ , (cid:107) · (cid:107) ∞
Remark 14.
Due to the high dimensionality of samples in real world applications, directly sampling z ∼ p ( z ) by the Markov Chain Monte Carlo (MCMC) algorithm requires a large number of randomwalks that can incur high computational cost.
To alleviate this issue, we adopt an efﬁcient sampling method from (Steinke & Ullman, 2015) that ﬁrst samples R from Gamma ( d +1 , σ ) and then samples z from [ − R, R ] d uniformly.
The complexity of this sampling algorithm is only O ( d ) .
Comparing Theorems 10 and 13, we can see that randomized smoothing via (6) can certify a region that has (almost) the same radius as that of Gaussian distribution in the (cid:96) 2 -normed case, due to similarity in their density functions and the robustness guarantees.
In the following theorem we show that the magnitude of the noise added by (6) is much larger than that of Gaussian distribution in the (cid:96) 2 -normed case.
Theorem 15.
For the distribution that can guarantee Theorem 13, the following theorem holds
[Equation]
Note that compared with the Gaussian noise added in Theorem 10 which satisﬁes the condition of E z [ (cid:107) z (cid:107) ∞ ]
= O ( σ √ log d ) , the expected (cid:96) ∞ -norm of the distribution in (6) is proportional to the dimensionality d of the data, which is quite large.
This means that for any image data, at least one pixel will be perturbed by the magnitude of dσ , which will completely ruin the accuracy of the classiﬁcation network.
However, if we want the noise to have a magnitude of O (1) , σ needs to be
O (1 /d ) , and so does the robust radius.
Theorem 15 is a somewhat negative result for randomized smoothing using distribution (6) to certify the (cid:96) ∞ -normed robustness.
Thus, an immediate question is whether exponential mechanism is the right choice to certify the (cid:96) ∞ -normed robustness.
The following theorem shows that for any ( r, D ∞ , (cid:107) · (cid:107) ∞ , rσ ) -robust randomized smoothing mechanism, the expected (cid:96) ∞ -norm of the added noise is lower bounded by Ω( dσ ) .
Thus, combining the following theorem with Theorem 15, we can conclude that the exponential mechanism is actually an optimal choice to certify D ∞ robustness.
Theorem 16.
For any (2 r, D ∞ , (cid:107) · (cid:107) ∞ , (cid:15) 2 ) -robust mechanism A ( x ) = x
+ z : [0 , r ] d (cid:55)→ [0 , r ] d such that
[Equation]
it must be true that α ≥ Ω( rd(cid:15) ) .
From Theorem 16 we can see that, for any ( · , D ∞ , (cid:107) · (cid:107) ∞ , (cid:15) 2 ) -robust randomized smoothing mechanism, if we ﬁx the expectation of the (cid:96) ∞ -norm of the added noise in the exponential mechanism as α , the largest (cid:96) ∞ radius that can be certiﬁed is upper bounded by O ( α(cid:15)/d ) .
Compared with the (cid:96) 2 -normed case in Theorem 11, we can see that there is an additional factor of O (1 /d ) , which makes it unscalable to high-dimensional data.
Equivalently, if we want the same radius to be certiﬁed as in the Theorem 10, the expected (cid:96) ∞ -norm of the added noise needs to be at least Ω( rd(cid:15) ) , which will be too large for any image data.
The less than ideal lower bound in Theorem 16 is for D ∞ -robustness.
Since D MR -robustness is more relaxed than D ∞ -robustness, a natural question is thus to determine whether the lower bound can be improved by switching to D MR -robustness.
Unfortunately, the following theorem shows that a similar phenomenon still holds for D MR -robustness.
Theorem 17.
For any (2 r, D MR , (cid:107) · (cid:107) ∞ , (cid:15) 2 ) -robust mechanism A ( x ) = x + z : [0 , r ] d [Equation] [0 , r ] d such that
[Equation]
it must be true that α ≥ Ω( r √ d √ (cid:15) ) .
From Theorems 17 and 15 we can see that in the deﬁnition of (2 r, D MR , (cid:107) · (cid:107) ∞ ) -robustness, adding noise according to (6) is not near optimal.
The following theorem shows that in this case, Gaussian mechanism is actually a near optimal choice.
Theorem 18.
Let r, (cid:15) > 0 be some ﬁxed number and A ( x ) =
x + z with z ∼ N (0 , dr 2 2 (cid:15) ) .
Then,
A ( · ) is ( r, D MR , (cid:107)·(cid:107) ∞ , (cid:15) ) -robust.
E [ (cid:107) z (cid:107) ∞ ] =
E A (cid:107)A ( x ) − x (cid:107) ∞ is upper bounded by O ( r √ d log d √ (cid:15) ) .
From Theorem 17 and 18, we can conclude that for all randomized smoothing mechanisms that are ( · , 0 , D MR , (cid:107) · (cid:107) ∞ , (cid:15) 2 ) -robust, if the expected (cid:96) ∞ -norm of the added noise is ﬁxed to be α , the
Gaussian to achieve - robustness is greater than the largest radius that can be certiﬁed to achieve D ∞ -robustness by at least a factor of O ( (cid:112) d/ log d ) .
This is reasonable since the deﬁnition of D MR -robustness is more relaxed.
Obviously, there is some trade-off between the rigorousness of the notion of robustness and the largest certiﬁed robust radius, i.e., when the robustness is relaxed, the largest certiﬁed radius increases.
We will investigate this trade-off more in the future research.
E XPERIMENTS 4. D ATASETS AND M ODELS
The performance of our framework is veriﬁed on two widely-used datasets, i.e., CIFAR10 and ImageNet ∗ .
Following Cohen et al.
(2019), we use a 110-layer residual network and the classical
ResNet-50 as the base models for CIFAR10 and ImageNet respectively.
Note that it may be difﬁcult for the models to classify noisy images without seeing any noisy samples in the training stage.
Thus, we train all the models by adding appropriate Gaussian noise on the training images.
The certiﬁed accuracy for radius R is deﬁned as the fraction of the test set whose certiﬁed radii are larger than R † .
The value of (cid:15) in all our derived certiﬁed radii can be calculated by p a (or p a and p b ) as shown in the proof of Corollary 11.
It is also worth noting that we do not compare our results with (Cohen et al., 2019) in the experiments because our framework and (Cohen et al., 2019) endow robustness with different deﬁnitions.
Moreover, our work does not aim at improving the tightness of the guarantee on the (cid:96) 2 -normed robustness but aims at presenting a general and self-contained framework to study some remaining issues, such as the optimality of the Gaussian mechanism, and the speciﬁc mechanisms to certify the (cid:96) ∞ -normed robustness.
4. E MPIRICAL R ESULTS
Certifying the (cid:96) 2 -normed Robustness To certify the (cid:96) 2 -normed Robustness, as we explained in previous section, Gaussian mechanism is a near optimal option.
Thus, we mainly evaluate the performance of Gaussian mechanism in our framework.
We ﬁrst ﬁx the value of σ in Gaussian mechanism and show the certiﬁed accuracy of the classiﬁers trained by varied Gaussian noises in
Figure 1.
As shown in Figure 1, using σ = 0 .
50 Gaussian noise to train the classiﬁer is a good setting here.
So in Figure 2, we evaluate the Gaussian mechanism with different σ values on the classiﬁer trained by σ = 0 .
50 Gaussian noise.
Overall, on CIFAR-10, our framework can certify approximately 20% accuracy under (cid:96) 2 = 1 . 0 perturbation ‡ .
We also show the results on ImageNet by Figures 4 and 5 in Appendix C.
Certifying the (cid:96) ∞ -normed Robustness To certify the (cid:96) ∞ -normed robustness, we evaluate the performance of the Exponential mechanism in the deﬁnition of D ∞ -robustness and the Gaussian mechanism in the deﬁnition of D MR -robustness.
[-(Cohen et al., 2019) On CIFAR-10, (cid:96) 2 = 1 . 
 perturbation allows 4/ on every pixel mechanism in the deﬁnition of D MR -robustness.-] As shown in Figure 3, the (cid:96) ∞ radii that can be certiﬁed by Gaussian mechanism are about 10 ∼ 20 times ( i.e., O ( (cid:112) d/ log d ) with d =as shown in our theories) larger than the (cid:96) ∞ radii certiﬁed by the exponential mechanism.
On
ImageNet, as shown in Figure 6 in Appendix C, the robust radii are less than 1 / 255 (due to scaling in O (1 /d ) or O (1 / √ d log d ) ), indicating that certifying the (cid:96) ∞ -normed robustness by randomized smoothing may not be applicable to high-dimensional data.
C ONCLUSION
In this paper, we present a general framework for certifying two types of robustness ( D ∞ and D MRrobustness) in the (cid:96) 2 and (cid:96) ∞ norms by randomized smoothing.
Under our framework, we ﬁrst give the answers to the remaining questions in the previous studies on randomized smoothing-based certiﬁable defenses, i.e., the optimality of Gaussian mechanism and the possibility to certify the (cid:96) ∞ -normed robustness.
Speciﬁcally, we demonstrate that (i) Gaussian mechanism is a near optimal option to certify D MR -robustness in (cid:96) 2 norm by giving a lower bound on all D MR -robust mechanisms, with certiﬁed radii scaling in O (1) ; (ii) an exponential mechanism is the optimal choice for certifying D ∞ -robustness in (cid:96) ∞ norm, with certiﬁed radii scaling in O (1 /d ) ; (iii) Gaussian mechanism is a near optimal option to certify D MR -robustness in (cid:96) ∞ norm, with certiﬁed radii scaling in O (1 / √ d log d ) ; (iv) the largest (cid:96) ∞ radius that can be certiﬁed by randomized smoothing in our framework is upper bounded by O (1 / √ d ) , indicating that randomized smoothing may not be scalable to high-dimensional data in terms of certifying the (cid:96) ∞ -normed robustness.
A D IFFERENTIAL P RIVACY B ACKGROUND
In this section, we brieﬂy introduce the concepts of differential privacy used in this paper.
Deﬁnition 19 (Differential Privacy (DP) (Dwork et al., 2006)) .
Given a data universe X , we say that two datasets D, D (cid:48) ⊆
X are neighbors if they differ by only one entry, which is denoted by
D ∼ D (cid:48) .
A randomized algorithm A is (cid:15) -differentially private (DP) if for all neighboring datasets
D, D (cid:48)
the following holds
[Equation]
Intuitively, DP ensures that an adversary cannot infer whether or not a participant (data sample) is participating in dataset D due to the fact that the distribution of A ( D ) is almost the same as that of
A ( D (cid:48) ) , which means that DP-mechanisms are robust to 1 -sample change.
Now consider the case where D is some 1-size dataset ( i.e., one data sample).
Then, DP ensures that the distribution of
A ( D ) and A ( D (cid:48) ) are almost the same, where D (cid:48) is just any other data sample.
Inspired by notion of DP, we deﬁne D ∞ robustness in Deﬁnition 6.
Deﬁnition 20 (Zero-Concentrated Differential Privacy (zCDP)) .
A randomized mechanism A is called (cid:15)
-zCDP, if for all D ∼ D (cid:48)
[Equation]
zCDP is a relaxed version of DP according to Theorem 8.
Motivated by zCDP, we deﬁne D MR robustness in Deﬁnition 7.
B O MITTED P ROOFS
Proof of Theorem 9.
This theorem can be easily proved by the following lemma,
Lemma 21 ((Bun & Steinke, 2016)) .
Let P and Q be two distributions on Ω and let f : Ω (cid:55)→ Θ be a deterministic function.
Let f ( P ) and f ( Q ) denote the distributions on Θ induced by applying f to P and Q respectively.
Then we have
[Equation]
Similar post-processing property also holds when α = ∞ (Dwork et al., 2006).
Therefore, if A ( · ) satisﬁes Deﬁnition 6 or 7, then f ( A ( · )) will satisfy Deﬁnition 6 or 7 for any deterministic function (classiﬁer) f ( · ) .
Proof of Theorem 10.
By Theorem 9, we only need to show that the randomized smoothing mechanism A ( x ) =
x + z is ( r, D MR , (cid:107) · (cid:107) 2 ) -robust, which can be proved by the following lemma.
Lemma 22 ((Bun & Steinke, 2016)) .
Let x, x (cid:48) ∈ R d , and α ∈ [1 , ∞ ) .
Then
[Equation]
Thus for all x (cid:48) ∈ B ( x, r ) , we have D MR ( A ( x ) (cid:107)A ( x (cid:48) )) ≤ r 2 2 σ
[Equation]
Next we prove (5).
To prove this inequality, we ﬁrst deﬁne the loss random variable.
Deﬁnition 23 ((Bun & Steinke, 2016)) .
Let Y and Y (cid:48) be random variables on Ω .
We deﬁne the loss random variable between Y and Y (cid:48) , denoted by Z = Loss ( Y (cid:107) Y (cid:48) ) , as follows: Deﬁne a function
F : Ω (cid:55)→ R by F ( y ) =
log P
[ Y = y ] P [ Y (cid:48) =
y ] .
Then Z is distributed according to F ( Y ) .
By this we can write Z =
Loss ( g ( x ) (cid:107) g
( x (cid:48) ))
and rewrite D MR ( g ( x ) (cid:107) g
( x (cid:48) )) as
[Equation]
This implies that Z is sub-Gaussian.
By using the tail-bound of sub-Gaussian (Vershynin, 2018), we have
[Equation]
where r 2 2 σ 2 = (cid:15) .
Proof of Corollary 11.
Since we ﬁx (cid:15) =
r 2 2 σ 2 , the certiﬁed radius is r = √ 2 (cid:15)σ .
Now we prove the upper bound of √ (cid:15) for a classiﬁer g ( · ) .
Given Theorem 10, we should have
[Equation]
and
[Equation]
since x is also in B ( x (cid:48) , r ) .
Then we have log P ( g ( x (cid:48) )
=
c a ) P ( g ( x (cid:48) )
=
c b ) ≥ log P ( g ( x )=
c a ) P ( g ( x )=
c b ) − 2( c +1) √ (cid:15) .
According to Deﬁnition 1, as long as log P ( g ( x (cid:48) )
=
c a ) P ( g ( x (cid:48) )
=
c b ) > 0 , g ( · ) can correctly classify x (cid:48) .
Thus, as long as log P ( g ( x )=
c a ) P ( g ( x )=
c b ) − 2( c +1) √ (cid:15) > 0 ( i.e., √ (cid:15) < (log p a − log p b ) / 2(1+ c ) ), g ( · ) classiﬁes x (cid:48) as c a .
Proof of Theorem 12.
Let { x 1 , x 2 , · · · , x 2 d } = { 0 , r √ d } d .
For each x i , we use the same adversarial example x (cid:48)
=
to derive the lower bound.
Since A is (2 r, D MR , (cid:107) · (cid:107) 2 , (cid:15) 2 ) -robust, we have for all x i , x j , i, j ∈ [2 d ] ,
[Equation]
That is A is (cid:15) -zCDP on the dataset X = { 0 , r √ d } d .
Next we will prove the lower bound for all (cid:15)
-zCDP
mechanisms.
We ﬁrst consider the case where r = √ d , and then generalize it to any r .
Before that we will ﬁrst prove the lower bound of one-way marginal ( i.e., mean estimation) under (cid:15)
-zCDP.
For an n -size dataset X ∈ R n × d , the one-way marginal is just h ( D )
= 1 n (cid:80) ni
=1 X i , where X i is the i -th row of
X .
Speciﬁcally, when n = 1 , one-way marginal is just the data point itself.
We show the following theorem,
Theorem 24.
If there exists an (cid:15) -zCDP mechanism A : { 0 , 1 } d (cid:55)→ [0 , 1] d such that for all x ∈ { 0 , 1 } d
E
[Equation]
then 1 ≥ Ω( (cid:113) d(cid:15)α 2 ) .
Proof of Theorem 24.
To prove this theorem, our idea is to ﬁrst use the connection between (cid:15)
-zCDP and ( (cid:15), δ ) -DP.
Lemma 25 (Prop.1.3 in Bun & Steinke (2016)) .
If A is (cid:15) -zCDP, then it is ( (cid:15) + 2 (cid:113) (cid:15) log 1 δ , δ ) - differentially private.
Bun et al.
ﬁrst give the optimal rate of one-way marginal estimation which is improved by
Steinke & Ullman (2016).
Lemma 26 (Theorem 1.1 in Steinke & Ullman (2016)) .
For every (cid:15) ≤ O (1) , every 2 − Ω( n ) ≤ δ ≤n 1+Ω(1) and every α ≤ 110 , if A : ( { 0 , 1 } d ) n (cid:55)→ [0 , 1] d is ( (cid:15), δ ) -DP and E [ (cid:107)A ( D ) − h
( D ) (cid:107) ∞ ] ≤ α , then
[Equation]
Setting n = 1 , (cid:15) =
(cid:15) + 2 (cid:113) (cid:15) log 1 δ in Lemma 26, we can see that if E [ (cid:107)A ( x ) − x (cid:107) ∞ ] ≤ α then 1 ≥
[Equation] [Equation]
) , where the last inequality is due to the fact that
[Equation]
Ω(
[Equation]
√
[Equation]
Now we come back to the proof for any r .
If A : { 0 , r √ d } d (cid:55)→ [0 , r √ d ] d is (cid:15) -zCDP, where
E A (cid:107)A ( x i ) − x i (cid:107) ∞ ≤ α, then we have
E A (cid:107) √
dr A ( x i ) − √ dr x i (cid:107) ∞ ≤ √ dr α .
Thus, √ dr A is an (cid:15) - zCDP mechanism on { 0 , 1 } d (cid:55)→ [0 , 1] d .
By Theorem 24 with α = √ dr α ≤ O (1) , we have
[Equation]
Proof of Theorem 13.
We will ﬁrst prove that A ( x ) = x
+ Z is ( r, D ∞ , (cid:107) · (cid:107) ∞ , rσ )
-robust.
Then by Theorems 8 and 9, we can easily show that g ( · ) is ( r, D MR , (cid:107) · (cid:107) ∞ , r 2 2 σ 2 ) -robust.
Consider x, x (cid:48) , (cid:107) x (cid:48) − x (cid:107) ∞ ≤ r .
Then, for any y we have p
[Equation]
y − x )
[Equation]
Thus, for any subset S we have
[Equation]
Proof of Theorem 15.
Deﬁne the distribution D on [0 , ∞ ) to be Z ∼ D , meaning Z = (cid:107) z (cid:107) ∞ for z ∼ p ( z ) , where p ( z ) is in (6).
The probability density function of D is given by
[Equation]
which is obtained by integrating the probability density function (6) over the inﬁnity ball of radius z with surface area d 2 d z d − 1 ∝ z d − 1 .
p D is the Gamma distribution with shape d and mean σ , and thus E [ z ] = dσ .
Proof of Theorem 16.
Let X
= { x 1 , x 2 , · · · , x 2 d } = { 0 , r } d be the set of samples.
Since A is (2 r, (cid:107) · · · (cid:107) ∞ ) -robust
and (cid:107) x
i − x j (cid:107) ∞ ≤ 2 r , we know that
[Equation]
Thus, A : R d (cid:55)→ R d is (cid:15) -DP on X .
Similar to the proof for Theorem 12, we can reduce our problem to studying the lower bound of one-way marginal for 1 -size data problem in the (cid:15) -DP model.
Now we ﬁrst consider the case of r = 1 .
We have the following lemma which is given by Hardt & Talwar (2010).
Lemma 27 (Theorem 1.1 in (Hardt & Talwar, 2010)) .
If there exists an (cid:15) -DP mechanism A : { 0 , 1 } d (cid:55)→ [0 , 1] d satisfying the following inequality for all x ∈ { 0 , 1 } d
E
[Equation] [Equation]
then 1 ≥ Ω( d(cid:15)α ) .
Now we consider any (cid:15) -DP mechanism A : { 0 , r } d (cid:55)→ [0 , r ] d .
If
E [ (cid:107) A ( x ) − x (cid:107) ∞ ] ≤ α, then E [ (cid:107) 1 r A
( x ) − 1 r x (cid:107) ∞ ] ≤ α r .
That is, 1 r A ( x ) : { 0 , 1 } d (cid:55)→ [0 , 1] d .
Thus, by lemma 26 we can see that 1 ≥ Ω( dr(cid:15)α ) .
Proof of Theorem 17.
The proof is almost the same as that of Theorem 12.
Assume that we have a set of data points
X = { x 1 , x 2 · · · , x 2 d } = { 0 , r } d .
A will also be (cid:15) -zCDP on X as in the proof of
Theorem 12.
Thus, if
[Equation]
then
[Equation]
This means that 1 r A ( x ) : { 0 , 1 } d (cid:55)→ [0 , 1] d is (cid:15)
-zCDP.
Thus, by Theorem 24 we must have
[Equation]
≥
[Equation]
Proof of Theorem 18.
The proof is almost the same as that of Theorem 10.
By Lemma 22, we have
[Equation]
Therefore, A ( x ) = x + z with z ∼ N (0 , dr 2 2 (cid:15) ) is ( r, D MR , (cid:107) · (cid:107) ∞ , (cid:15) ) -robust.
The bound of E [ (cid:107) z (cid:107) ∞ ] can be easily proved by substituting σ in O ( σ √ log d ) ((Orabona & P´al, 2015)) with (cid:113) dr 2 2 (cid:15) .
C M ORE E XPERIMENTAL R ESULTS (I MAGE N ET ) C. C
[Figure] [Figure]