Denoised Smoothing with Sample Rejection for Robustifying Pretrained Classifiers
Abstract
Denoised smoothing is the state-of-the-art approach to defending pretrained classifiers against ℓ p adversarial attacks, where a denoiser is prepended to the pretrained classifier, and the joint system is adversarially verified via randomized smoothing.
Despite its state-of-the-art certified robustness against ℓ 2 -norm adversarial inputs, the pretrained base classifier is often quite uncertain when making its predictions on the denoised examples, which leads to lower natural accuracy.
In this work, we show that by augmenting the joint system with a “rejector” and exploiting adaptive sample rejection, (i.e., intentionally abstain from providing a prediction), we can achieve substantially improved accuracy (especially natural accuracy) over denoised smoothing alone.
That is, we show how the joint classifier-rejector can be viewed as a classification-with-rejection per sample, while the smoothed joint system can be turned into a robust smoothed classifier without rejection , against ℓ 2 -norm perturbations while retaining certifiability.
Tests on CIFAR10 dataset show considerable improvements in natural accuracy without degrading adversarial performance, with affordably-trainable rejectors, specially for medium and large values of noise parameter σ .
Introduction
Despite their success in image classification, deep learning models are known to be vulnerable against ℓ p -norm adversarial attack , where small imperceptible perturbations on the input image can considerably change model predictions Biggio et al.
[2013], Szegedy et al.
[2013], Goodfellow et al. [2014], Carlini and Wagner [2017], Uesato et al.
Many empirical defense mechanisms and training procedures have been proposed against adversarial attacks while often times stronger attacks have followed to break them [Athalye et al., 2018, Tramèr et al., 2017].
These advances have lead to certifiable defenses Levine and Feizi [2020], Wong and Kolter [2018], Salman et al.
[2020], Gowal et al.
[2018] which provide provable lower bounds of robust classification accuracy; the most relevant to our work is randomized-smoothing Lecuyer et al.
[2019], Cohen et al.
[2019], Li et al.
Denoised smoothing Salman et al.
[2020] approaches this task by prepending a denoiser to pretrained classifiers, and utilizes it jointly with randomized smoothing, i.e., taking a majority vote of the predictions from multiple copies of the input image with noise, and provides state-of-the-art certifiable defense for pretrained classifiers for ℓ 2 -norm perturbations.
Unfortunately, despite using pretrained classifiers with high accuracy, the resulting smoothed classifier still exhibits a substantial drop in clean accuracy for affordably-trainable denoisers.
On the other hand, utilization of a rejection class for improving robustness is an open area of research [Geifman and El-Yaniv, 2019, Yin et al., 2019, Stutz et al., 2020, Liu et al., 2019, Tramer, 2022], with limited work available with certifiable guarantees against ℓ ∞ -norm perturbations Sheikholeslami et al.
In fact, a certifiable utilization of detector/rejectors has shown to be necessary as carefully-designed adaptive attacks have frequently broken existing non-verifiable detectors Athalye et al.
[2018], Tramer et al.
Our contribution .
[Figure]
To the best of our knowledge, this is the first work to robustify pretrained classifiers with sample rejection while providing certifiable accuracy.
Key to our approach is to use a reject class, realized through cheaply-trainable per-class rejectors , which are trained to reject noisy samples whose prediction is inconsistent with the prediction of the clean sample.
Inevitably this can also lead to a small number of correctly classified samples to also get rejected, however, the overall certification radius with a pretrained denoiser is improved since: (a) the reject class is used to provide a lower (and tighter) upper bound on the wrong class probabilities- see Fig.
1- and subsequently (b) the lowered probability of the runner-up class leads to higher certification radius due to its non-linear dependence via the inverse Gaussian CDF function.
See Fig.
2 for a schematic of the proposed joint system.
Background and Related Work 2.1 Randomized smoothing
Consider a classification problem from R d to classes Y :
= { 1 , 2 , · · · , K } .
One can construct a “smoothed” classifier g from an arbitrary base classifier f by defining [Cohen et al., 2019] g ( x ) :
= arg max c ∈Y π c where π c :=
P ( f ( x + ϵ )
= c ) and ϵ ∼ N (0 , σ 2 I ) .
That is, the smoothed classifier g returns the class that the base classifier f is most likely to return if the input x is perturbed under
Gaussian Noise ϵ ∼ N (0 , σ 2 I ) .
Let us also denote bounds on class probabilities by π c ≤ π c ≤ π c .
Certification: The main advantage of the well-known randomized-smoothing method is its inherent capability in providing certifiable robustness against bounded ℓ 2 -norm worst-case perturbations.
Formally, for any deterministic or random function f : R d → Y , suppose c A , c B ∈ Y are the top and runner-up class respectively, and π A , π B ∈ [0 , 1] satisfy:
[Equation]
Then Cohen et al.
[2019] proved a tight verification bound as follows:
g ( x + δ ) =
c A for all ∥ δ ∥ 2 < ρ , where ρ = σ 2 (cid:0)
Φ − 1 (cid:0) π A (cid:1) − Φ − 1 ( π B ) (cid:1) and Φ − 1 ( . ) is the inverse of the standard Gaussian CDF.
In practice, Monte Carlo (MC) sampling is used to estimate the class c A and a lower bound on its class probability π A ( MC ) .
Using π B ≤ 1 − π A ( MC ) yields certification radius ρ ≥ σ Φ − 1 ( π A ( MC ) ) .
2.2 Denoised smoothing for defending pretrained classifiers
Despite its simplicity, randomized smoothing is not, in general, directly effective on pretrained classifiers.
Specifically, performance of an off-the-shelf classifier can considerably deteriorate when the input is subject to Gaussian noise (leading to small π A , and subsequently small certification radius
R ), as standard classifiers, in general, are not trained to be robust against Gaussian perturbations.
In order to construct robust classifiers without altering the underlying weights of a given network f ( . ) , Salman et al.
[2020] proposed to use an image denoisier as a pre-processing step before [Figure] passing inputs through f ( . ) , where the denoiser aims at removing the Gaussian noise added to the input.
Concretely, this is done by augmenting the classifier f with a custom-trained denoiser
D θ ( . ) :
R d → R d , rendering the entire system as the composite function f ◦ D θ : R d → Y .
Such denoisers can be trained using various objectives subject to a varying level of complexity.
Minimizing the mean-square-error (MSE) loss of the reconstructed image as the simplest option and the "stability" loss as a more expensive (to train) option are proposed, formulated respectively as L MSE ( θ ) =
E x i ,ϵ (cid:104) ∥
D θ ( x i + ϵ ) − x i ∥ 22 (cid:105) and L stability ( θ ) =
E x i ,ϵ [ ℓ CE ( F ( D θ ( x i + ϵ )) , f ( x i ))] where θ denotes trainable parameters of the denoiser, and the expectation is taken over data and noise.
Utilizing L stability has proven to be successful, while best performance is achieved by imposing up-to an order of magnitude increase in training time and complexity compared to using the MSE loss.
Denoised smoothing with sample rejection
In this work, we aim to improve certification accuracy of pretrained classifiers by incorporating an explicit ‘reject’ class into the base classifier, while preserving the certifiability against worst-case perturbations with bounded ℓ 2 -norm.
To this end, let h : R d → { 0 , 1 } denote a general function with binary outputs, which effectively ‘flags’ the input x if h ( x ) = 1 , thus assigning it to the reject class; while h ( x ) = 0 indicates allowing the input to pass and thus not rejecting it.
In this work, we introduce a novel algorithm to effectively train and operate such a ‘rejector’ in conjunction with pretrained denoised smoothing in order to improve the robust performance of a pretrained classifier.
Base classifier with a rejection class: Let us denote K distinct binary classifiers by { h k } Kk =1 , which we also refer to as rejectors in this work.
Together with the base classifier f ,
our proposed base-classifier-with-rejection f R ( x ) : R d → Y + , when queried at x , returns one of the classes in
Y + :=
Y ∪ { R } , where class R denotes the reject class.
Let us now formally define f R ( . ) .
Definition .
Consider f : R d → Y to be any given deterministic or random classifier, together with a collection of K deterministic or random binary classifiers, denoted by { h k } K k =1 where h k : R d → { 0 , 1 } .
The base-classifier-with-rejection f R ( x ) : R d → Y + is defined as
[Equation]
Smoothed classifier (without a rejection class) For a given base classifier f R ( . ) , define
[Equation]
where noise ϵ is sampled from Gaussian distribution with variance σ , i.e., ϵ ∼ N (0 , σ 2 I ) .
That is, among the original classes in Y , the smoothed classifier returns the class that is mostly likely to be returned by the base-classifier-with-rejection f R ( . ) , excluding the rejection class R .
3.1 Certified Robustness
We prove that the smoothed classifier g R ( . ) defined in 2 is robust within the ℓ 2 radius in Theorem 3.1.
Theorem 3.1.
Let f R : R d → Y + be any deterministic or random function, ϵ ∼ N (0 , σ 2 I ) , and let g R ( . ) be defined as in Eq.
Respectively, denote top and runner-up class probabilities (exc.
class R ) as p A := P ( f R ( x + ϵ )
=
c A ) , p B := max c/ ∈{ c A ,R } P ( f R ( x + ϵ ) =
c ) , and define p A
+ R :=
P ( f R ( x + ϵ ) ∈ { c A , R } ) .
Suppose for c A ∈ Y , lower bounds p A , p A + R ∈ [0 , 1] satisfy p A ≥ p
A ≥ p B and p A
+ R ≥ p
A + R .
Then g R ( x + δ ) =
c A for all ∥ δ ∥ 2 ≤ ρ , where
[Equation]
[Figure]
Details of proof is delegated to Appendix 6.1 for brevity.
It is important to note that the claim of this Theorem is tightly connected to the results in [Cohen et al., 2019]; while here by utilizing the rejection class R , the upperbound p B ≤ 1 − p A + R is employed.
Furthermore, the intuition behind why this can potentially lead to better certificatino radius is that proper training and expressiveness of the rejectors can lead to p A + p R > π A and, owing to the non-linearity of the inverse CDF especially at large values, can lead to an improved radius if
[Equation]
3. Sample rejection with denoised smoothing
The performance gap in empirical results between what an affordably trained denoiser and that of the robustly trained baseline implies the limited capability of the joint denoiser-classifier system in correctly classifying denoised inputs especially those subject to large noise.
To alleviate the performance loss in such cases, we propose to use the rejection capability in order to improve network accuracy.
Concretely, we aim at utilizing rejectors { h k } Kk =1 in blocking inputs likely to be mis-classified by the following overall classification system:
[Equation]
where ϵ ∈ N [Equation]
[Equation]
Practice .
During inference, the image x will go through the following steps: Algorithm P REDICT ( S = D, f, { h k } Kk =1 } , σ, x, N, α ) count ← S AMPLE U NDER N OISE ( S , σ, x, N ) (cid:99)
c A , (cid:99) c
B ← top two indices in count n
A , n B ← count [ (cid:99) c
A ] , count[ (cid:99) c
B ] if B INOM PV ALUE ( n A , n A + n B , 0 . 5) ≤ α then return (cid:99) c
A else return ABSTAIN end if (a) it is first perturbed by noise ϵ , (b) passes through the image preprocessing (denoising) step via D ( x + ϵ ) , and (c) the resulting denoised image goes through f R ( D ( x + ϵ )) as in Eq.
(1); (d) finally, the classification output of the overall system ( D, f, { h k } Kk =1 ) is claimed as the most likely class over the noise distribution (or it empirical realization via N iid samples).
The schematic in Fig.
2 depicts a visual placement of the components S =
{ D, f, { h k } Kk =1 } .
Algorithm 1, 2 andprovide the pseudocode for the prediction and certification (via MC-based class probability lowerbounds) of the overall system, while training the rejectors in a scalable and stable manner is detailed in Appendix 6.2. Condition p A > p B is substituted by the more restrictive condition p A ≥ p A ≥ 1 − p
A − p R ≥ p B which leads to p A > 1 2 (1 − p R ) , which if not met, the algorithm abstains from certification.
This is to be contrasted with π A > 1 / 2 in randomized smoothing [Cohen et al., 2019].
Function L OWER C ONF B OUND ( s, n 0 , 1 − α ) returns a one-sided (1 − α ) lower confidence interval for the Binomial parameter q given a sample s ∼ Binomial ( n 0 , q ) , which similar to Cohen et al. [2019], has been evaluated via Clopper–Pearson interval in the statsmodel package in Python directly.
Subscript
MC has been dropped in Algorithm 2 for brevity of notation.
Algorithm
# certify the robustness of g R around x
C ERTIFY ( S =
D, f, { h k } Kk =1 } , σ, x, N, α ) count ← S AMPLE U NDER N OISE ( S , σ, x, N ) (cid:99)
c
A ← top index in count n
A , n R ← count [ (cid:99) c
A ] , count [ R ] p
A ← L OWER C ONF B OUND ( n A , N, 1 − α ) p R ← L OWER C ONF B OUND ( n R , N, 1 − α ) p
A + R ← L OWER C ONF B OUND ( n A + n R , N, 1 − α ) if p A > 1 2(1 − p R ) then return (cid:99)
c A , ˆ ρ = σ 2 (Φ − 1 ( p A )
+ Φ − 1 ( p A + R )) else return ABSTAIN end
if
Algorithm
Sampling under noise for the overall system
S =
{ D, f, { h k } Kk =1 } function
S AMPLE U NDER N OISE ( S , x, n, σ )
Initialize count =
[0 , · · · , 0] K +1 ×for ν = 1 , · · · , n do sample noise ϵ ν ∈ N (0 , σ 2 I )
k ← f ( D ( x + ϵ ν ))
if h k ( D ( x + ϵ ν ))
{+= 
+}
then ++ counts [ k ] else ++ counts [ R ] end if end for return count
[Figure] [Figure]  Experiments In this section, we examine the performance of robustifying pretrained classifier using rejectors with denoised smoothing on CIFAR10 dataset with ResNet110 as the base classifier, and comparing our method with two baselines: randomized smoothing (serving as the upper-bound baseline) and denoised smoothing (with two choices of denoiser architectures, namely DnCNN
[Zhang et al., 2017] and MemNet [Tai et al., 2017]), both of which have network weights publicly available.
To test the proposed Sample rejection with denoised smoothing, the classifier and denoiser architectures and weights are considered given and fixed.
Rejector network is implemented with a ResNet-34-like backbone architecture with a 10-dimensional output, to realize the K = 10 rejector heads, trained with 20 epochs and SGD optimizer; see App.
6.3 for detailed description of architectures and training.
Performance is reported in terms of certified accuracy in 4 and 5, scatter plots in 6, and training time in 1 demonestrating the effective exploitation of the rejectors.
Although certification accuracy of our method is lower than expensive denoised smoothing baseline with stability objectives, our training time is less than half of training a denoiser with stability loss – training a denoisier with MSE or MSE and stability objective in addition to train the rejector is in total 2 .
78 to 6 .
8 hours while training a denoiser with stability objective is 9 . 8 to 20 .
8 hours.
Also, 6 plots the 2D-histograms corresponding [Figure] [Figure] [Figure]
[Figure]
[Figure]
[Table]
[Table]
Conclusions
In this work, we have proposed a novel algorithm for modeling and utilizing an ‘explicit reject‘ class for robustifying pre-trained classifiers with denoised smoothing with certifiable guarantees.
The reject class, effectively realized through cheaply-trainable per-class rejectors , is successfully exploited through MC sampling to reject noisy samples whose prediction is inconsistent with the prediction of the clean sample.
Thus, by reducing the number of not-rejected misclassified samples, natural accuracy as well as the certification radius is shown to have substantially improved for inexpensive denoisers.
To the best of our knowledge, this is the first work to incorporate certifiable application of rejection for robustifying pre-trained classifiers with denoised smoothing.
Appendix 6.1 Proof of Theorem 3.
Theorem [Restated] Let f R : R d → Y + be any deterministic or random function, ϵ ∼ N (0 , σ 2 I ) , and let g R ( . ) be defined as in Eq.
Suppose c A ∈ Y and p A , p A + R ∈ [0 , 1] satisfy
[Equation]
Then g R ( x + δ ) =
c A for all ∥ δ ∥ ≤ R , where
[Equation]
Proof.
We need to prove that for ∥ δ ∥ 2 ≤ R , the classification outcome of g R ( . ) will not flip from c A to any other class.
According to the definition of g R ( . ) in Eq.
(2), class R is never selected as its outcome, thus it suffices to show
[Equation]
where the probability is computed over the randomness of noise ϵ .
As a direct result from the certification guarantee in Cohen et al.
[2019], this would hold for
[Equation]
where p B ≥ p B = max c/ ∈{ c A ,R } P ( f R ( x + δ )
= c ) and c B = arg max c/ ∈{ c A ,R } P ( f R ( x + δ ) =
c ) .
Since we have
[Equation]
it holds for c B = arg max c/ ∈{ c A ,R } P ( f R ( x + δ ) =
c ) that
[Equation]
and so the right-hand-side of the inequality serves as an upper bound on p B .
Substituting this into Eq. 5 one can get
[Equation]
where we have used Φ − 1 (1 − z )
= − Φ − 1 ( z ) .
[-= 
-]
6.2 Training the rejectors
In this work, we have assumed that a pretrained classifier along with an affordably pretrained denoiser are given to enable robustness and certification of the given classifier.
The goal is to utilize rejectors { h k } k with inexpensive/affordable training to increase the robustness certification of the joint system.
As the proposed procedure suggests, an ideal rejector h k () is the one that can successfully discriminate between the correctly classified and mis-classified denoised inputs that are assigned to class k by the base classifier f ( . ) .
Thus, we propose to train these networks according to this objective.
Concretely, with ℓ BCE denoting binary cross-entropy loss, define the classification loss for rejector h k as
[Equation]
(7) where H k is the sigmoid outputs of rejector k , and the target label b i for image x i is defined as
[Equation]
8) where 1 { . } is the indicator function.
That is b i =
if the classifier f has classified denoised input
D ( x i + ϵ ) to the same class as that of the noise-free image f ( x i ) , and b i = 1 otherwise, thus rejecting the noisy images whose classification outcome has changed.
2 Capturing the set of parameters of all the K rejectors by Ω := { Ω 1 , · · · , Ω K } , total loss aggregated over the entire set of data with all possible K classes yields
[Equation]
In order to make the training more affordable, we propose to tie the K rejectors through a shared backbone h BB , parameterized by Ω BB and define each h k by adding a fully-connected layer parameterized by ω k to the features extracted via the backbone network; see Fig.
This renders the minimization of L rejection w.r.t.
Ω =
{ Ω BB , ω 1 , · · · , ω K } upon approximation by empirical mean over the dataset and T ≥ 1 realizations of Gaussian noise to simplify to
[Equation]
6.3 Experiment details
Randomized smoothing with training classifiers against Gaussian perturbations.
The work of [Cohen et al., 2019] obtains the classifier by training its weights with augmented input images with
Gaussian noise.
This method serves as the upper-bound baseline in robustifying pretrained networks, as the pretrained classifier’s weights are given and cannot be altered while randomized smoothing optimizes those.
Denoised smoothing .
[Salman et al., 2020] uses a trainable denoiser and trains it with the (i)
MSE loss that is fast to train ( 0 . 78 to 3 hours of training), (2) MSE loss then fine-tuning by the ‘classification-stability’ loss (total of 1 to 3 .
8 hours of training), and (3) the ‘classification-stability’ loss ( 9 . 8 to 20 .
8 hours of training).
While the last objective gives the highest performance, it takes almost an order of magnitude longer to train compared to other objectives, with up to trainingepochs as proposed by the authors.
Two choices of recent denoiser architecture, namely DnCNN [Zhang et al., 2017] and MemNet [Tai et al., 2017], have been tested, the choice of which influences the training time.
We utilize the weights shared by the authors under MIT License and take the best performing selection of training hyperparameters per denoiser architecture and noise parameter σ .
Our method .
In this setup, we assume that the classifier and denoiser architectures and weights are given and cannot be changed.
In order to verify the effectiveness of the proposed method with sample rejection, we augment the system consisting of the classifier and denoiser with a rejector network h Ω ( . ) with a ResNet-34-like backbone architecture, followed by 3 layers of fully connected (FC) layers with a 10-dimensional output, to realize the K = 10 rejector heads 3 , given the 10 classes in the CIFAR10 dataset.
Additionally, we have adjusted the architecture of the backbone slightly so that mid-layer features of the classification network f ( . ) are concatenated with the mid-layer features of the rejector right after each of the 4 "blocks" in the Resnet architectures, and are fed forward into the rejector network.
We have found that explicit use of classifier features yields empirical advantage, while it is important to note that the gradient does not flow backwards into the classifier architecture as the weights of network f ( . ) are fixed.
We train the h Ω ( . ) network with a total of 20 epochs where the first 5 epochs utilize Adam optimizer with 1 e − 4 learning rate, followed by an SGD optimizer that starts with learning rate of 1e-3 and drops by a factor of 10 every 5 epochs.
We train a rejector for every denoiser given by denoised-smoothing [Salman et al., 2020] as discussed above
Certification accuracy and training time
For a given (classifier f , denoiser D , rejector h Ω ), the certified radii of CIFAR10 test set are calculated using the Theorem 3.1 for α = 0 .
01 and N = 10 , 000 , and the certification curves are plotted by calculating the percentage of the data points whose radii are larger than the given ℓ 2 -radius.
The experiments are carried out for σ ∈ { 0 . 25 , 0 .
50 , 1 . 00 } , and the results are plotted in Figuresand 5 for DnCNN and MemNet denoiser architectures, respectively.
For higher resolution copies of these plots together with additional choice of σ = 0 .
12 see Appendix 6.4.
We chose T = 20 , leading to a per-epoch training-time of 350 seconds for the DnCNN-based denoisers and 540 seconds for the
MemNet-based denoisers, respectively; See Table.
(1) for detailed training-time comparisons, andfor ablation study on T .
Fig.
4 and 5 demonestrate that for both denoiser architectures, the proposed method with rejection has better certification accuracy (except for σ = 1 . 0 with MemNet and MSE objective).
With σ being 0.25 or 0.5, the gain of certified accuracy with rejection is more significant especially at radius 0.
This shows that by adding the rejector, the clean accuracy of the smoothed classifier g R ( . ) in Eq.
(2) is better than the base classifier f ( . ) .
Such increase of clean accuracy also helps increase certification radii since larger p
A + R will lead to larger radii as in Theorem 3.1.
Since the proposed randomized smoothing with sample rejection utilizes pretrained denoisers, making direct comparison difficult, we emphasize that the point to demonstrate through empirical results is the additional gain that can be obtained by utilizing a reject class on a denoised classifier.
Our method can specifically improve cheaply trained denoisers and improve their performance without imposing a large computational cost; see Table (1) for training-times.
Although certification accuracy of our method is lower than denoised smoothing baseline with stability objectives, our training time is less than half of training a denoiser with stability loss – training a denoisier with MSE or MSE and stability objective in addition to train the rejector is in total 2 .
78 to 6 .
8 hours while training a denoiser with stability objective is 9 . 8 to 20 .
8 hours.
Furthermore, Fig.
6 plots the 2D-histograms corresponding to the overall denoiser-classifier-rejector system in terms of number-of-rejected vs. top-class not-rejected samples for the publicly available
DnCNN denoiser architecture trained with the MSE-loss and fine-tuned with the stability loss, for various values of σ .
The plots shows that indeed, by utilizing the rejector, as expected and discussed in Figure 1 and Remark in Section 3.1„ the total number of the samples in the top-class (not-rejected) and the reject-class lie close to the diagonal line x 1 + x 2 = N where N = 10 , 000 , especially for larger σ , giving high values for the (lower-bounds of)
p A + R , leading to an improved certified radius.
6.4 Certification radius plots
Figures 7 and 8 plot the results on CIFAR10 as discussed in Section 4 with higher resolution together with the additional choice of σ = 0 .
12 (which was omitted from the main body due to space limitation).
6.5 Experiments- Ablation on varying T
Ablation study on number of noise realizations T in empirical approximation of the training loss for training the rejector is presented here in Fig.
For these experiments, the best DnCNNbased denoiser from [Salman et al., 2020] trained with the MSE-loss and fine-tuned by the stability loss was selected.
training parameters of the rejectors is selected as explained in Section 5 while
T ∈ 4 , 6 , 10 , 20 .
[Figure] [Figure]
f i
[Figure]
a
[Figure]
y
[Figure]
a cc u r a c y
[Figure]
[Table]
[Figure] [Figure]
cc
[Figure]
y
[Figure]