Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing

Abstract: Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning. However, most existing methods only leverage Gaussian smoothing noise and only work for ℓ2 perturbation. We propose a general framework of adversarial certification with non-Gaussian noise and for more general types of attacks, from a unified functional optimization perspective. Our new framework allows us to identify a key trade-off between accuracy and robustness via designing smoothing distributions, helping to design two new families of non-Gaussian smoothing distributions that work more efficiently for ℓ2 and ℓ∞ attacks, respectively. Our proposed methods achieve better results than previous works and provide a new perspective on randomized smoothing certification.

Reviews and author responses:

We fix the issue of Theorem 1 pointed out by Reviewer #3. Our lower bound is still tight (strong duality holds) for all the cases we studied.

3. (sketchy justification): 'The paper justifies a smoothing distribution that concentrates more mass around the center as follows: 'This phenomenon makes it problematic to use standard Gaussian distribution for adversarial certification, because one would expect that the smoothing distribution should concentrate around the center (the original image) in order to make the smoothed classifier close to the original classifier (and hence accurate).' I don't see why we should want more mass near the center---in the limit as we move all the mass towards the center and get the original classifier, our certified bound will be terrible, so it's not clear why moving in that direction should be expected to help. Indeed, the experimental gains are minimal (1 to 3 percentage points) and on methods that were not carefully tuned, so one could imagine that the baseline method could be improved by that much just with careful tuning.'

Reply: As we have shown in Eq (7), the certified lower bound can be interpreted as two terms: the "accuracy" and the "robustness". And a good certified lower bound is based on a good trade-off between these two terms: Too much perturbation on the original image will cause the classifier gives prediction with low accuracy while having good robustness (low lambda-total variation). On the other hand, as you said, in the extreme case, if we move all the mass to center, the certified bound would be terrible. This is because in this case, the lambda-total variation would be very large but meanwhile, we have high prediction accuracy. The reason we have better accuracy is obvious: the perturbed image is more close to the original image. 

Again, the key argument of our paper is on: using the proposed distribution to achieve a better trade-off between accuracy and robustness. It is not that meaningful to have an argument that considering the 'limit' case because the art in this area is playing with the trade-off.

Regarding your question on the experiment, we want to point out that in all the experiments, we simply use the pre-trained models provided by the paper we compare. We believe that those pre-trained models are well tuned to have good performance for the baseline method. We expect to get similar (or better) results with further tuning.

We also want to highlight that the improvement we made is not marginal at all, comparing with several recent works in this area, including the concurrent submission we mentioned (https://openreview.net/forum?id=SJlKrkSFPH). 

4. I similarly didn't understand the justification for the mixed L-inf / L-2 distribution for L-infinity verification. The main justification was "The motivation is that this allows us to allocate more probability mass along the 'pointy' directions with larger norm, and hence decrease the maximum distance term max δ∈B`∞,rDF(λπ0‖πδ)" This is at the very least too brief for justifying the main experimental innovation in the paper (here at least the empirical improvements are bigger, although still not huge).

The proposed mixed L-inf/L-2 distribution has two terms: ||z||_\infty^-k and \exp(-||z||^2_2/\sigma^2). The second term controls the tail behavior of the proposed distribution. We justify why we use \ell_2 norm for this term by Theorem 4. The first controls how the distribution shrink towards the center. As illustrated in Figure3, as the \ell_infty is a hypercube and using \ell_norm will have less total variation and thus gives better performance. 

Besides, to the best of our knowledge, we have achieved the best certified accuracy on l_infty certification on Imagenet at the time of submission.



Response to Reviewer #3

Thank you very for your time and comments. Here we provide our response. We hope you could consider raising your score if they addresses your concerns. Otherwise, please let us know and we will try our best to improve and clarify. We hope the reviewer could take the overall benefit and potentials of our new framework into account when it comes to the decision. 

As we also point to the other reviewers and AC,  there is an independent and concurrent submission to ICLR that overlaps with our work in both our basic framework and many detailed algorithmic choices. "A FRAMEWORK FOR ROBUSTNESS CERTIFICATION OF SMOOTHED CLASSIFIERS USING F-DIVERGENCES (https://openreview.net/forum?id=SJlKrkSFPH)".  Because of the high overlap with our work, we hope you could also check this paper and their reviews and calibrate your score accordingly, since if our work is rejected while their accepted only due to uncalibrated reviews, it would block the opportunity for publishing our work in the future.  

1. (main theorem is incorrect): 'Claim 3 in the appendix is wrong. The fact that (delta', f') outperforms (delta-bar, f-bar) with respect to lambda* does not imply that (delta', f', lambda*) is a better solution to the primal problem, because we must take max over lambda and the maximizing lambda need not be lambda*. In particular if f' doesn't satisfy the constraint we would instead take lambda to infinity.'

Reply: Thanks very much for pointing out this!  We have *fixed the issue*. The strong duality is the previous submission is wrong. In the newly updated version, we show that strong duality holds for the case of our experiment, that is, the proposed method is tight for the certification problem we studied. Please check the updated draft for details.

2. 'The paper makes several references (in italics) to a "fundamental trade-off between accuracy and robustness". But a fundamental trade-off means that *any* method that attains good accuracy must sacrifice robustness and vice versa; this requires a "for all" statement, i.e. a lower bound. All the paper shows is that the *particular upper bound* exhibits a trade-off (and even then, the notions of "accuracy" and "robustness" are merely interpretations of quantities in the bound; it's not clear why the robustness term in particular is tied to more standard notions of robustness).'

Theoretically, as shown by our theory, if we view the adversarial certification problem as a constraint optimization problem and set the space of classifier space as F_[0,1], the solution of the lower bound can be decomposed into the two terms in Equ(7). Under the assumption in our paper, this is a 'for all' statement and this statement holds. 

Secondly, the adversarial defense is on the trade-off between accuracy and robustness. We believe this point of view should have been widely accepted by the community, e.g. [1, 2, 3]. There are many ways to mathematically characterize this trade-off and this is exactly one of our contributions. 

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." International conference on machine learning. 2019.

[2] Raghunathan, Aditi, et al. "Adversarial Training Can Hurt Generalization." arXiv preprint arXiv:1906.06032 (2019).

[3] Shi, Yujun, et al. "Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric." arXiv preprint arXiv:1906.02494 (2019).


Thank R#1 for your time and comments. We do respectfully disagree with many of your comments, and think they are based on misunderstanding. We hope our response can help clarify the issue.

We also note an independent and concurrent submission to ICLR that overlaps with our work in many ways. 
"A FRAMEWORK FOR ROBUSTNESS CERTIFICATION OF SMOOTHED CLASSIFIERS USING F-DIVERGENCES (https://openreview.net/forum?id=SJlKrkSFPH)".  Because of the high overlap with our work, we strongly encourage these two works can be jointly considered and receives calibrated scores. 

Question: "For \ell_2 perturbations, there is no major difference between this new family of distributions (d-k \chi^2) and a Gaussian with different variance.''

Reply: Your understanding is *wrong*.  The family of distribution indexed by (k, sigma) is not mathematically equivalent to Gaussian. It strictly generalizes Gaussian. Our Figure 1 and discussion on page 6 is to illustrate this very point. 

Also as we show in the paper, the impact of k on the shape of the distribution is also very practically significant. Note that the radius distribution of our generalized family is a scaled Chi distribution, in which k is the *shape parameter*, which controls the shape. 

In addition, we also found Theorem 9 and Theorem 10 in the appendix of the concurrent submission (https://openreview.net/forum?id=SJlKrkSFPH) which illustrate the very same point.  

Question: Concerns on experiments; ''I don't see why the value of (k,\sigma) was not provided in Table 1 and only \sigma was provided. Also, the table of Cohen et al. was only calculated for specific values of \sigma for Gaussian distributions (0.12, 0.25, 0.5, 1.00). For a fair comparison, comparable values of \sigma 's must be calculated, and then the best choice should be selected.''

Reply: It looks the reviewer has missed some of our experiments. See please the "Settings and Hyper-parameters" subsection for  how k is chosen. We also do an ablation study for k in appendix, in which we empirically show that k and \sigma can lead to different results. For comparable values of \sigma, we also did so. See **Settings and Hyper-parameters**. 


Question:  ''In the light of previous arguments, I don't think the choice of Eq. (9) or Eq. (10) is well motivated. Why not smooth it with a cube of appropriate radius''

Reply:  In Theorem 4, We have already proven that Eq.(9) is not doable because of the curse of dimensionality! No matter how you choose the radius, you cannot get **any** practical result for high-dimension cases.

And we don’t quite get what you mean by ‘cube of appropriate radius’. If you mean an uniform distribution on the hyper-cude with certain radius, this smoothing distribution also suffers from the curse of dimensionality.

4. Question: "inconsistent Salmen's experiment results"

Reply: Our setting in Table 3 is **l_inf** while Salmen's whole paper is about **l_2** certification. These are totally different settings, thus we may be not aware of where does your concern come from. We 'transfer' their result on \ell_2 certification to L_infty setting using theorem 3.

5. Question: ''On p.5, why was the toy classifier sphere-based? The toughest classifier for Gaussian smoothing (the one achieving the lower bound for Gaussian smoothing) is actually a linear classifier.''

Reply: This is just a motivating example for illustration. Cohen's work derives a bound with worst case achieved by linear classifier and the space of classifier we and they concern is very general (that includes almost all nonlinear real-word case and of course the sphere-based classifier)


Response to Reviewer #2

Thank you very for your time and comments.  We hope you can re-consider your evaluation based on the new framework that we develop, which significantly generalizes and simplify the derivations in existing results. We believe our empirical results are sufficient to support and demonstrate the potential benefit of this framework (see response below). 

We also want to point out an independent and concurrent submission to ICLR that overlaps with our work in both our basic framework and many detailed algorithmic choices. "A FRAMEWORK FOR ROBUSTNESS CERTIFICATION OF SMOOTHED CLASSIFIERS USING F-DIVERGENCES (https://openreview.net/forum?id=SJlKrkSFPH)".  Because of the high overlap with our work, we hope you could also check this paper and their reviews and calibrate your score accordingly, since having our work rejected while their work accepted due to uncalibrated review would block the opportunity for publishing our work in the future.   

We response to the main arguments here:

1. You are correct that models trained with our proposed noise should be used ideally. But it is very computationally expensive to conduct (especially when training on ImageNet).  So we used Cohen's as a standard setting, which I think it forms a fair comparison. 

We used Salman et al's model for Linfty certification because we found Salman's model performs better for L_infty certification. This is likely because Cohen's model is trained with Gaussian noise data augmentation, which does not match our smoothing distribution of L_infty certification, while Salman's model is trained in a more adversarial fashion, and turns out to be more robust for L_infty distribution  (even though it was still designed for the standard Gaussian distribution).

2. For results of Cohen's method, Salman et al.'s paper (https://github.com/Hadisalman/smoothing-adversarial), Salman's blog (https://decentdescent.org/smoothadv.html) and Cohen's paper (https://arxiv.org/pdf/1902.02918.pdf) have inconsistent results. This may be due to randomness of the algorithm. For our paper, we reported numbers that came from our experiment with Cohen's github code.

3. We are not sure what do you mean by 68.2. We work on L_infty certification and there is no result for L_inf certification in Salman et al.'s paper (they work on an extension of Cohen's, which it's about L_2 setting). We 'transfer' their result on \ell_2 certification to L_infty setting using theorem 3.

Thanks for your other comments, which point out many improper descriptions and are helpful for us to revise our work. 

The paper introduces an improvement to the randomized smoothing analysis in Cohen et al. (2019), using Lagrangian relaxation to achieve a more general lower bound. Using this, it considers different adversarial smoothing distributions that yield some increase in certified adversarial accuracy.

Overall assessment: While the Lagrangian relaxation idea is interesting and could yield interesting follow-up work, the paper is sloppy in several respects and needs to be tightened before it can be considered for publication.

Key issues:
1. Proof of main theorem (strong duality) is incorrect. Likely the statement itself is also incorrect. Fortunately the most important direction (lower bound) is still true, so this isn't a fatal flaw to the approach.
2. The paper makes several references (in italics) to a "fundamental trade-off between accuracy and robustness". But a fundamental trade-off means that *any* method that attains good accuracy must sacrifice robustness and vice versa; this requires a "for all" statement, i.e. a lower bound. All the paper shows is that the *particular upper bound* exhibits a trade-off (and even then, the notions of "accuracy" and "robustness" are merely interpretations of quantities in the bound; it's not clear why the robustness term in particular is tied to more standard notions of robustness).
3. The justification for why the particular smoothing distributions are good ideas is sketchy.

I elaborate on 1 and 3 below. Addressing 1-3 effectively will improve my score.

#1 (main theorem is incorrect): Claim 3 in the appendix is wrong. The fact that (delta', f') outperforms (delta-bar, f-bar) with respect to lambda* does not imply that (delta', f', lambda*) is a better solution to the primal problem, because we must take max over lambda and the maximizing lambda need not be lambda*. In particular if f' doesn't satisfy the constraint we would instead take lambda to infinity.

#3 (sketchy justification): The paper justifies a smoothing distribution that concentrates more mass around the center as follows: "This phenomenon makes it problematic to use standard Gaussian distribution for adversarial certification, because one would expect that the smoothing distribution should concentrate around the center (the original image) in order to make the smoothed classifier close to the original classifier (and hence accurate)." I don't see why we should want more mass near the center---in the limit as we move all the mass towards the center and get the original classifier, our certified bound will be terrible, so it's not clear why moving in that direction should be expected to help. Indeed, the experimental gains are minimal (1 to 3 percentage points) and on methods that were not carefully tuned, so one could imagine that the baseline method could be improved by that much just with careful tuning.

I similarly didn't understand the justification for the mixed L-inf / L-2 distribution for L-infinity verification. The main justification was "The motivation is that this allows us to allocate more probability mass along the “pointy” directions with larger`∞norm, and hence decrease the maximum distance term max δ∈B`∞,rDF(λπ0‖πδ)." This is at the very least too brief for justifying the main experimental innovation in the paper (here at least the empirical improvements are bigger, although still not huge).

Minor but related: Why is the x-axis in Figure 4 so compressed? This is also in a regime where all 3 methods fail to certify so not clear it's meaningful.

Writing comment: Change some of the Theorems to Propositions. Theorems should be for key claims in paper (there shouldn't be 4 of them in one 8-page paper).

Summary:

This paper investigates the choice of noise distributions for smoothing an arbitrary classifier for defending against adversarial attacks.  The paper focuses on the two major adversaries: \ell_2 adversaries and \ell_\infty adversaries. Theorem 1 quantifies the tradeoff between the choice of smoothing distribution which (1) has clean accuracy close to the original classifier and (2) promotes the smoothness of smoothed classifier (and hence adversarial accuracy).  For the \ell_2 adversary, the paper argues that Gaussian distribution is not the right choice, because the distribution is concentrated on the spherical shell around the x. Instead, the authors propose using a new family of distributions, with the norm square  (p_{|z|_2^2}) following the scaled \chi^2 distribution with degree d-k (Eq. 8). This allows an extra degree of freedom, and setting k=0 recovers the Gaussian distribution. For \ell_\infty perturbations, the paper suggests another family of distributions combining the \ell_2 and \ell_\infty norm (Eq. 9), and argues that it outperforms the natural choice of \ell_\infty norm-based distributions (Eq. 10).

I think the paper should be rejected because (1) For \ell_2 perturbations, there is no major difference between this new family of distributions (d-k \chi^2) and a Gaussian with different variance. (2) For \ell_\infty distributions, the motivation of mixed norm distributions (Eq. 9) over \ell_\infty based distributions (Eq. 10) is not very clear. (3) The experimental evidence is also weak (see below).

Main arguments:

1. The distribution of the norm \|z\|_2 in Eq. (8) would be concentrated on a thin spherical shell of radius about \sqrt{d-k}\sigma. As the Gaussian distribution with standard deviation \sigma' is supported on a shell of radius about \sqrt{d} \sigma', for each (k,\sigma) in the family of Eq. 8, there is an equivalent Gaussian with appropriate \sigma' (Theorem 3 now just compares the radius of the spherical shell). Therefore, I don't see the benefit of this extra degree of freedom of k: the noise distribution is again a "soap bubble" of a different radius. Thus, a grid search over \sigma' for a Gaussian should be the same as a grid search over (k,\sigma) in Eq. 8.

Even the experimental experiments are a marginal improvement over Cohen et al.  I don't see why the value of (k,\sigma) was not provided in Table 1 and only \sigma was provided. Also, the table of Cohen et al. was only calculated for specific values of \sigma for Gaussian distributions (0.12, 0.25, 0.5, 1.00). For a fair comparison, comparable values of \sigma's must be calculated, and then the best choice should be selected. 

2. In the light of previous arguments, I don't think the choice of Eq. (9) or Eq. (10) is well motivated. 
Why not smooth it with a cube of appropriate radius? Also, not enough experimental details are provided for Table 3.  Salman et al. (2019) reports the accuracy of 68.2% for \ell_infty perturbations (Table 3, Salman et al. (2019)), whereas the value reported in your Table 3 for at the same radius is 58%. Is it a typo? In any case, the values reported for the proposed model in Table 3 are only a marginal improvement over Figure 1 (left) in Salman et al. (2019), just going by the trivial \ell_2 to \ell_\infty certificate.

Other areas for improvement:

1. The paper contains numerous grammatical errors, confusing statements, and nonstandard phrases.  For example: (i) more less robust, (ii) black start, (iii) pointy points, etc.  I suggest that the authors spend more time clarifying their manuscript.

2. The paragraph starting with "Trade-off between Accuracy and Robustness": I think this paragraph should be reworded for clarification.  It is not robustness but rather the lack thereof -- say, sensitivity.

3. On p.5, why was the toy classifier sphere-based? The toughest classifier for Gaussian smoothing (the one achieving the lower bound for Gaussian smoothing) is actually a linear classifier.




This paper presents a new method for adversarial certification using non-Gaussian noise. A new framework for certification is proposed, which allows to use different distributions compared to previous work based on Gaussian noise. From this framework, a trade-off between accuracy and robustness is identified and new distributions are proposed to obtain a better trade-off than with Gaussian noise. Using these new distributions, they re-certify models obtained in previous work.

I am hesitating between a weak reject and a weak accept. The theoretical results are interesting, showing a clear trade-off between robustness and accuracy with a new lower bound and deriving better smoothing distributions. However, the experimental results are lacking, and do not support much the proposed method. Training with this new distribution would have been a natural experiment given the argument. Moreover, the results for L_inf are partial and it would be expected to have some results for ImageNet as claimed in the introduction. I would have given an accept if the previous points had been addressed and I feel that with some more work on it, it would become an excellent paper.

Main arguments:
My main concern is about the experiments: Why were Cohen et al.’s models used instead of Salman et al.’s? Salman et al.’s have achieved better certified accuracy under the L_2 norm so it would only seem natural to use their model.
About the main results: there seems to be a discrepancy between the results reported for Cohen et al. and the original paper for both CIFAR-10 and ImageNet L2 certification. Also, the reported certified accuracy for Salman et al.’s model for L_inf on CIFAR-10 reported in the original paper is 68.2 at 2/255, which is very far from the 58 in Table 3. What is the reason for these differences?

Minor comments:
In the third paragraph, it is claimed that L_inf attacks are a stronger and more relevant type of attacks than L_2 attacks. These two different objectives cannot be compared in those terms.
Defenses such as adversarial training have not been “broken” as claimed in section 2 in the sense that the claims made in the original paper still hold true. The term broken is used for defenses in which the claimed accuracy against stronger attacks were found to be much lower than what was claimed in the original paper.
It is claimed that “if ||z||_inf is too large to exceed the region of natural images, the accuracy will be obviously rather poor”; however, the common practice is to clip to the input space bounds. How would that affect the method?

Things to improve the paper that did not impact the score:
In the first paragraph, Goodfellow et al., 2015 is cited, however, papers on adversarial attacks were published earlier than that such as Szegedy et al., 2014 or Biggio et al., 2013.
Vershynin, 2018 is cited about the distribution of a gaussian in high-dimensional spaces. However, this is a very well known result and does not need any citation (or if any, Bellman, 1961).
Typo after equation 4: ||f||_{L_p}
Typo in “Black-box Certification with Randomness” paragraph: “by convovling”
Typos in Table 2.: the columns 2.0 to 3.5 are mislabeled


Thanks a lot for your questions.

First question: We use Salman et al’s model for Linfty certification because we found Salman's model performs better for L_infty certification.
This is likely because Cohen’s model is trained with Gaussian noise data augmentation, which does not match our smoothing distribution of L_infty certification, while Salman’s model is trained in a more adversarial fashion, and tends out to be more robust for L_infty distribution  (even it was still designed for the standard Gaussian distribution).

Second question:
We agree that we may get better performance by directly training on our proposed distribution. It is an interesting direction that we plan to explore in future work.

Thanks for the clarification! This makes sense.

Some other questions:
1) Why did you use Cohen et al.'s models for l2 certification, but Salman et al.'s models for linfty certification? What if you use Salman et al.'s models for l2, and Cohen et al.'s for linfty?

2) It would seem natural to train on your proposed distribution as well, and one would perhaps expect better performance after doing so, compared to using pre-trained models trained on Gaussian noise or SmoothAdv. Do you have those results?

Thanks! Hope to hear back from you soon :)

Thanks for your concern! We want to clarify that actually we don't need to get the exact optimal lambda because *any* lambda could give a valid confidence lower bound. Besides, the optimization is smooth since the derivative of lower bound $D$ w.r.t. lambda is actually bounded by 2 with some algebra. It's our negligence of omitting these details :(  The search space is chosen heuristically, which is good enough (the optimization of lambda can be shown to convex with only one minimal point), so we just don't explore more on this. We will update relevant details as soon as the update of pdf file is allowed :)

Dear authors,

Thanks for a very interesting paper. I might have missed this in the paper, but how did you choose \lambda_start, \lambda_end, and h for your algorithm 1 and 2? Additionally, what is the guarantee you provide for those choices? For example, how do you prevent the possibility that the best \lambda is outside your interval, or the possibility that the function in lambda is very nonsmooth and h is too big in comparison?

Thanks again, and looking for your reply :)

Meta-review: The authors extend the framework of randomized smoothing to handle non-Gaussian smoothing distribution and use this to show that they can construct smoothed models that perform well against l2 and linf adversarial attacks. They show that the resulting framework can obtain state-of-the-art certified robustness results improving upon prior work.

While the paper contains several interesting ideas, the reviewers were concerned about several technical flaws and omissions from the paper:

1) A theorem on strong duality was incorrect in the initial version of the paper, though this was fixed in the rebuttal. However, the reasoning of the authors on the "fundamental trade-off" is specific to the particular framework they consider, and is not really a fundamental trade-off.

2) The justification for the new family of distributions constructed by the author is not very clear and the experiments only show marginal improvements over prior work. Thus, the significance of this contribution is not clear.

Some of the issues were clarified during the rebuttal, but the reviewers remained unconvinced about the above points.

Thus, the paper cannot be accepted in its current form.


Decision: reject