B OOSTING R ANDOMIZED S MOOTHING WITH V ARIANCE R EDUCED C LASSIFIERS
A BSTRACT
Randomized Smoothing (RS) is a promising method for obtaining robustness certiﬁcates by evaluating a base model under noise.
In this work, we: (i) theoretically motivate why ensembles are a particularly suitable choice as base models for RS, and (ii) empirically conﬁrm this choice, obtaining state-of-the-art results in multiple settings.
The key insight of our work is that the reduced variance of ensembles over the perturbations introduced in RS leads to signiﬁcantly more consistent classiﬁcations for a given input.
This, in turn, leads to substantially increased certiﬁable radii for samples close to the decision boundary.
Additionally, we introduce key optimizations which enable an up to 55-fold decrease in sample complexity of RS for predetermined radii, thus drastically reducing its computational overhead.
Experimentally, we show that ensembles of only 3 to 10 classiﬁers consistently improve on their strongest constituting model with respect to their average certiﬁed radius (ACR) by 5% to 21% on both CIFAR10 and ImageNet, achieving a new state-of-theart ACR of 0 .
86 and 1 . 11 , respectively.
We release all code and models required to reproduce our results at https://github.com/eth-sri/smoothing-ensembles .
I NTRODUCTION
Modern deep neural networks are successfully applied to an ever-increasing range of applications. However, while they often achieve excellent accuracy on the data distribution they were trained on, they have been shown to be very sensitive to slightly perturbed inputs, called adversarial examples (Biggio et al., 2013; Szegedy et al., 2014).
This limits their applicability to safety-critical domains.
Heuristic defenses against this vulnerability have been shown to be breakable (Carlini & Wagner, 2017; Tramèr et al., 2020), highlighting the need for provable robustness guarantees.
A promising method providing such guarantees for large networks is Randomized Smoothing (RS) (Cohen et al., 2019).
The core idea is to provide probabilistic robustness guarantees with arbitrarily high conﬁdence by adding noise to the input of a base classiﬁer and computing the expected classiﬁcation over the perturbed inputs using Monte Carlo sampling.
The key to obtaining high robust accuracies is a base classiﬁer that remains consistently accurate even under high levels of noise, i.e., has low variance with respect to these perturbations.
Existing works use different regularization and loss terms to encourage such behavior (Salman et al., 2019; Zhai et al., 2020; Jeong & Shin, 2020) but are ultimately all limited by the bias-variance trade-off of individual models.
We show ﬁrst theoretically and then empirically how ensembles can be constructed to signiﬁcantly reduce this variance component and thereby increase certiﬁed radii for individual samples, and consequently, certiﬁed accuracy across the whole dataset.
We illustrate this in Fig.
Ensembles are a well-known tool for reducing classiﬁer variance at the cost of increased computational cost (Hansen & Salamon, 1990).
However, in the face of modern architectures (He et al., 2016; Huang et al., 2017) allowing the stable training of large models, they have been considered computationally inefﬁcient.
Yet, recent work (Wasay & Idreos, 2021) shows ensembles to be more efﬁcient than single monolithic networks in many regimes.
In light of this, we develop a theoretical framework analyzing this variance reducing property of ensembles under the perturbations introduced by RS.
Further, we show how this reduced variance can signiﬁcantly increase the majority class’s prediction probability, leading to much larger certiﬁed radii than evaluating more perturbations with an individual model. [Figure]
Certiﬁcation with RS is computationally costly as the base classiﬁer has to be evaluated many thousand times.
To avoid exacerbating these costs by using ensembles as base models, we develop two techniques: (i) an adaptive sampling scheme for RS, which certiﬁes samples for predetermined certiﬁcation radii in stages, reducing the mean certiﬁcation time up to 55-fold, and (ii) a special aggregation mechanism for ensembles which only evaluates the full ensemble on challenging samples, for which there is no consensus between a predeﬁned subset of the constituting models.
Main Contributions
Our key contributions are:
• A novel, theoretically motivated, and statistically sound soft-ensemble scheme for Randomized Smoothing, reducing perturbation variance and increasing certiﬁed radii (§4 and §5).
• A data-dependent adaptive sampling scheme for RS that reduces the sample complexity for predetermined certiﬁcation radii in a statistically sound manner (§6).
• An extensive evaluation, examining the effects and interactions of ensemble size, training method, and perturbation size.
We obtain state-of-the-art results on ImageNet and CIFAR for a wide range of settings, including denoised smoothing (§7).
R ELATED W ORK
Adversarial Robustness
Following the discovery of adversarial examples (Biggio et al., 2013;
Szegedy et al., 2014), adversarial defenses aiming to robustify networks were proposed (Madry et al., 2018).
Particularly relevant to this work are approaches that certify or enforce robustness properties.
We distinguish deterministic and probabilistic methods.
Deterministic certiﬁcation methods compute the reachable set for given input speciﬁcations using convex relaxations (Singh et al., 2019; Xu et al., 2020), mixed-integer linear programming (Tjeng et al., 2019), semideﬁnite programming (Dathathri et al., 2020), or SMT (Ehlers, 2017), to reason about properties of the output.
To obtain networks amenable to such approaches, specialized training methods have been proposed (Mirman et al., 2018; Balunovic & Vechev, 2020; Xu et al., 2020).
Probabilistic certiﬁcation (Li et al., 2019; Lécuyer et al., 2019) introduces noise to the classiﬁcation process to obtain probabilistic robustness guarantees, allowing the certiﬁcation of larger models than deterministic methods.
We review Randomized
Smoothing (RS) (Cohen et al., 2019) in §3 and associated training methods (Jeong & Shin, 2020; Zhai et al., 2020;
Salman et al., 2019) in App.
G.3.
Orthogonally to training, RS has been extended in numerous ways (Lee et al., 2019; Dvijotham et al., 2020), which we review in App.
B.
Ensembles Ensembles have been extensively analyzed with respect to different aggregation methods (Kittler et al., 1998;
Inoue, 2019), diversiﬁcation (Dietterich, 2000), and the reduction of generalization errors (Tumer & Ghosh, 1996a;b).
Randomized Smoothing and ensembles were ﬁrst combined in Liu et al.
(2020) as ensembles of smoothed classiﬁers.
However, the method does not retain strong certiﬁcates for individual inputs; thus, we consider the work to be in a different setting from ours.
We discuss this in App. A.
While similar at ﬁrst glance, Qin et al.
(2021) randomly sample models to an ensemble, evaluating them under noise to obtain an empirical defense against adversarial attacks.
They, however, do not provide robustness guarantees.
R ANDOMIZED S MOOTHING
Here, we review the relevant background on Randomized Smoothing (RS) as introduced in Cohen et al.
We let f : R d (cid:55)→ R m denote a base classiﬁer that takes a d -dimensional input and produces m numerical scores (pre-softmax logits), one for each class.
Further, we let F ( x ) :=
arg max q f q ( x ) denote a function R d (cid:55)→ [1 , . . .
, m ] that directly outputs the class with the highest score.
For a random variable (cid:15) ∼ N (0 , σ 2 (cid:15) I ) , we deﬁne a smoothed classiﬁer G : R d (cid:55)→ [1 , . . .
, m ] as
[Equation]
This classiﬁer G is then robust to adversarial perturbations as follows:
Theorem 3.1 (From Cohen et al.
Let c
A ∈ [1 , . . .
, m ] , p A , p B ∈ [0 , 1] .
If
[Equation]
, then G ( x + δ ) =
c A for all δ satisfying (cid:107) δ (cid:107) 2 <
R with R :
= σ (cid:15) 2 (Φ − 1 ( p A ) − Φ − 1 ( p B )) .
Algorithm 1 Certify from (Cohen et al., 2019)
[Table]
A
Here, Φ − 1 denotes the inverse Gaussian
CDF.
Computing the exact probabilities
P (cid:15) ( F ( x + (cid:15) ) =
c ) is generally intractable.
Thus, to allow practical application, C ERTIFY (Cohen et al., 2019) (see Algorithm 1) utilizes sampling: First, n 0 samples to determine the majority class, then n samples to compute a lower bound p A to the success probability with conﬁdence 1 − α via the Clopper-Pearson lemma (Clopper & Pearson, 1934).
If p A > 0 . 5 , we set p B = 1 − p A and obtain radius R = σ (cid:15) Φ − 1 ( p A ) via Theorem 3.1, else we abstain (return (cid:11) ).
See App.
F for exact deﬁnitions of the sampling and lower bounding procedures.
To obtain high certiﬁed radii, the base model F has to be trained to cope with the added Gaussian noise (cid:15) .
To achieve this, several training methods, discussed in App.
G.3, have been introduced.
We also see this in Fig.
1, where various models obtain different p A , and thereby different radii R .
R ANDOMIZED S MOOTHING FOR E NSEMBLE C LASSIFIERS
In this section, we extend the methods discussed in §3 from single models to ensembles.
For a set of k classiﬁers { f l : R d (cid:55)→ R m } kl =1 , we construct an ensemble ¯ f via weighted aggregation, ¯ f ( x ) = (cid:80) kl
=1 w l γ ( f l ( x )) , where w l are the weights, γ : R m (cid:55)→ R m is a post-processing function, and f l ( x ) are the pre-softmax outputs of an individual model.
Soft-voting (where γ denotes identity) and equal weights w l = 1 k perform experimentally well (see App.
H.3.1) while being mathematically simple.
Thus, we consider soft-ensembling via the averaging of the logits:
[Equation]
The ensemble ¯ f and its corresponding hard-classiﬁer ¯ F ( x ) :
= arg max q ¯ f q ( x ) , can be used without further modiﬁcation as base classiﬁers for RS.
We ﬁnd that classiﬁers of identical architecture and trained with the same method but different random seeds are sufﬁciently diverse to, when ensembled with k ∈ [3 , 50] , exhibit a notably reduced variance with respect to the perturbations (cid:15) .
As we will show in the next section, this increases both the true majority class probability p A and its lower conﬁdence bound p A , raising the certiﬁed radius as per Theorem 3.1.
V ARIANCE R EDUCTION VIA E NSEMBLES FOR R ANDOMIZED S MOOTHING
We now show how ensembling even similar classiﬁers f l (cf.
Eq.
(2)) reduces the variance over the perturbations introduced in RS signiﬁcantly, thereby increasing the majority class probability p A of resulting ensemble ¯ f and making it particularly well-suited as a base classiﬁer for RS.
To this end, we ﬁrst model network outputs with general distributions before investigating our theory empirically for Gaussian distributions.
We defer algebra and empirical justiﬁcations to App.
C and D, respectively.
Individual Classiﬁer We consider individual classiﬁers f l : R d (cid:55)→ R m and perturbed inputs x + (cid:15) for a single arbitrary but ﬁxed x and Gaussian perturbations (cid:15) ∼ N (0 , σ 2 (cid:15) I ) .
We model the presoftmax logits f l ( x ) =:
y l ∈ R m as the sum of two random variables y l = y lp + y lc .
Here, y lc corresponds to the classiﬁer’s behavior on the unperturbed sample and models the stochasticity in weight initialization and training with random noise augmentation.
y lp describes the effect of the random perturbations (cid:15) applied during RS.
Note that this split will become essential when analyzing the ensembles.
We drop the superscript l when discussing an individual classiﬁer to avoid clutter.
We model the distribution of the clean component y c over classiﬁers with mean c = E l [ f l ( x )] , the expectation for a ﬁxed sample x over the randomness in the training process, and covariance Σ c ∈ R m × m , characterizing this randomness.
We assume the distribution of the perturbation effect y p to be zero-mean (following from local linearization and zero mean perturbations) and to have covariance Σ p ∈
R m × m .
While Σ p might depend on the noise level σ (cid:15) , it is distinct from it.
We do not restrict the structure of either covariance matrix and denote Σ ii = σ 2 i and Σ ij = σ i σ j ρ ij , for standard deviations σ i and correlations ρ ij .
As y c models the global training effects and y p models the local behavior under small perturbations, we assume them to be independent.
We thus obtain logits y with mean c , and covariance matrix Σ = Σ c + Σ p .
A classiﬁer’s prediction F l ( x ) = arg max q y q is not determined by the absolute values of its logits but rather by the differences between them.
Thus, to analyze the classiﬁer’s behavior, we consider the differences between the majority class logit and others, referring to them as classiﬁcation margins.
During certiﬁcation with RS, the ﬁrst step is to determine the majority class.
Without loss of generality, let the class with index 1 be the majority class, i.e., A = 1 in Theorem 3.1, leading to the classiﬁcation margin
z i =
y 1 − y i .
Note that if z i > 0 for all i (cid:54) = 1 , then the majority class logit y 1 is larger than the logits of all other classes
y i .
Under the above assumptions, the classiﬁcation margin’s statistics for an individual classiﬁer are:
[Equation]
Ensemble Now, we construct an ensemble of k such classiﬁers.
We use soft-voting (cf. Eq.
(2)) to compute the ensemble output ¯ y = 1 k (cid:80) kl =1 y l
and then the corresponding classiﬁcation margins ¯ z i =
¯ y 1 −
¯ y i .
We consider similar classiﬁers, differing only in the random seed used for training.
Hence, we assume that the correlation between the logits of different classiﬁers has a similar structure but smaller magnitude than the correlation between logits of one classiﬁer.
Correspondingly, we parametrize the covariance between y ic and y jc for classiﬁers i (cid:54) =
j with ζ c Σ c and similarly between y ip and y jp with ζ p Σ p for ζ c , ζ p ∈ [0 , 1] .
Note that, as we capture the randomness introduced in the training process with Σ c , we use the same Σ c , Σ p for each individual model.
With these correlation coefﬁcients ζ c and ζ p , this construction captures the range from no correlation ( ζ = 0 ) to perfect correlation ( ζ = 1 ).
By the linearity of expectation and this construction, respectively, we obtain:
[Equation]
Variance Reduction We can split Var[¯ z i ] into the components associated with the clean prediction σ 2 c ( k ) and the perturbation effect σ 2 p ( k ) , both as functions of the ensemble size k .
We now compare these variance terms independently to the corresponding terms of an individual classiﬁer dropping the subscripts p and c from σ 2 c ( k ) /σ 2 c (1) and σ 2 p ( k ) /σ 2 p (1) as they follow the same structure:
[Equation]
We observe that both variance component ratios go towards their corresponding correlation coefﬁcients ζ c and ζ p as ensemble size grows. As we now proceed to show (see Fig. 2a, explained later), this corresponds to an increase in success probability p 1 and thereby certiﬁed radius. [Figure]
40Probability density k=1, n=
[Figure] k=3, n= [Figure] k=
[Figure]
n
[Figure]
[Figure]
k
[Figure]
k
[Figure]
[Figure]
n
[Figure]
k
[Figure]
[Figure]
Our modeling approach is mathematically closely related to Tumer & Ghosh (1996a), which focuses on analyzing ensemble over the whole dataset.
However, we condition on a single input to study the interplay between the stochasticity in training and random perturbations encountered in RS.
Effect on Success Probability We can compute the probability of an ensemble of k classiﬁers predicting the majority class by integrating the probability distribution of the classiﬁcation margin over the orthant 1 ¯ z i > 0 ∀ i ≥ 2 where majority class 1 is predicted:
[Equation]
Assuming Gaussian distributions for z , we observe the increase in success probability shown in Fig.
2b.
Without assuming a speciﬁc distribution of z , we can still lower-bound the success probability using Chebyshev’s inequality and the union bound.
Given that a mean classiﬁcation yields the majority class and hence c 1 − c i > 0 , we let t i
=
c
1 − c i σ i ( k ) and have:
[Equation]
where σ i ( k ) 2 = σ c,i ( k ) 2 + σ p,i ( k ) 2 is the variance of classiﬁcation margin
¯ z i .
We observe that as σ i ( k ) decreases with increasing ensemble size k
(see Eq.
(3)), the lower bound to the success probability approaches 1 quadratically.
The further we are away from the decision boundary, i.e., the larger c i , the smaller the absolute increase in success probability for the same variance reduction.
Given a concrete success probability p 1 , we compute the probability distribution over the certiﬁable radii reported by C ERTIFY (Algorithm 1) for a given conﬁdence α , sample number n , and perturbation variance σ 2 (cid:15) (up to choosing an incorrect majority class ˆ c A ) as:
[Equation]
where B ( s, r, p ) is the probability of drawing s successes in r trials from a Binomial distribution with success probability p , and p ( s, r, α ) is the lower conﬁdence bound to the success probability of a
Bernoulli experiment given s successes in r trials with conﬁdence α according to the Clopper-Pearson interval (Clopper & Pearson, 1934).
We illustrate the resulting effect assuming Gaussian distributions in Fig. 2c.
Empirical Analysis via Gaussian Assumption To investigate our theory, we now assume y lc and y lp and hence also ¯ z i to be multivariate Gaussians.
This choice is empirically well-ﬁtting (see
App. D) and follows from the central limit theorem for y lc and from Gaussian perturbations and local linearization for y lp .
To estimate the free parameters c , Σ c , Σ p , ζ c , and ζ p , we evaluate ensembles of
up to k = 50 G AUSSIAN trained ResNet20 (for details, see §7) at σ (cid:15) = 0 .
We obtain c and Σ c as the mean and covariance of the output on a randomly chosen sample x .
Subtracting the clean outputs from those for the perturbed samples, we estimate the covariance matrix Σ p .
We determine ζ c ≈and ζ p ≈ 0 .
82 as the median ratio of the inter- and intra-classiﬁer covariance.
ζ c ≈ 0 implies that our models can be treated as independent conditioned on a single ﬁxed and unperturbed input.
[Figure]
over the number n of samples for the true success probabilities p A .
Plugging these estimates into our construction under the Gaussian assumption, we observe a signiﬁcant decrease in the variance of the classiﬁcation margin to the runner-up class as ensemble size k increases (see Fig. 2a).
This generally leads to more polarized success probabilities: the majority class’ probability is increased, while the other classes’ probabilities are decreased because the probability mass concentrates around the mean, and consequently on one side of the decision threshold ¯ z = 0 , which determines the success probability via Eq.
An increase, as in our case for a correct mean prediction (see Fig. 2b), leads to much larger expected certiﬁed radii (see Fig. 2c) for a given number of sampled perturbations (here n = 10 3 ) via Eq.
In contrast, sampling more perturbations will, in the limit, only recover the true success probability.
In our example, going from one classiﬁer to an ensemble of 50 increases the expected certiﬁed radius by 191% , while drawing 50 times more perturbations for a single model only yields a 28% increase (see Fig. 2c).
As illustrated in Fig.
3, these effects are strongest close to decision boundaries ( p A (cid:28) 1 ), where small increases in p A impact the certiﬁed radius much more than the number of samples n , which dominated at p A ≈ 1 .
C OMPUTATIONAL OVERHEAD REDUCTION
To avoid exacerbating the high computational cost of certiﬁcation via RS with the need to evaluate many models constituting an ensemble, we propose two orthogonal and synergizing approaches to reduce sample complexity signiﬁcantly.
First, an adaptive sampling scheme reducing the average number of samples required for certiﬁcation.
Second, a more efﬁcient aggregation mechanism for ensembles, reducing the number of models evaluated per sample, also applicable to inference.
Adaptive Sampling We assume a setting where a target certiﬁcation radius r is known a priori, as is the standard in deterministic certiﬁcation, and aim to show R ≥ r via Theorem 3.1.
After choosing the majority class ˆ c A as in C ERTIFY , C ERTIFY A DP (see Algorithm 2) leverages the insight that relatively few samples are often sufﬁcient to certify robustness at radius r .
We perform an s ≥ 1 step procedure:
At step i , we evaluate n i fresh samples and, depending on these, either certify radius r as in C ERTIFY , abort certiﬁcation if an upper conﬁdence bound suggests that certiﬁcation will not be fruitful, or continue with n
i +1 (cid:29) n i .
After s steps, we abstain.
To obtain the same strong statistical guarantees as with C ERTIFY , it is essential to correct for multiple testing by applying Bonferroni correction (Bonferroni, 1936), i.e., use conﬁdence 1 − αs rather than 1 − α .
The following theorem summarizes the key properties of the algorithm:
Theorem 6.1.
For α, β ∈ [0 , 1] , s ∈ N + , n 1 < · · · < n s , C ERTIFY A DP :
returns ˆ c A if at least 1 − α conﬁdent that G is robust with a radius of at least r . 2.
returns (cid:11) before stage s only if at least 1 − β conﬁdent that robustness of G at radius r can not be shown.
for n s ≥(cid:100) n (1 − log α ( s )) (cid:101) has maximum certiﬁable radii at least as large as C ERTIFY for n .
We provide a proof in App.
E.
For well-chosen parameters, this allows certifying radius r with, in expectation, signiﬁcantly fewer evaluations of F than C ERTIFY as often n i (cid:28) n samples sufﬁce to certify or to show that certiﬁcation will not succeed.
At the same time, we choose n s in accordance with (3) in Theorem 6.1 to retain the ability to certify large radii (compared to C ERTIFY ) despite the increased conﬁdence level required due to Bonferroni correction. For an example, see Fig. 4.
1: function C ERTIFY A DP ( F,σ (cid:15) , x ,n 0 , { n j } sj =1 ,s,α,β,r ) 2: cnts0 ← S AMPLE U NDER N OISE ( F, x ,n 0 ,σ (cid:15) ) 3:
ˆ c A ← top index in cnts0 4: for i ← 1 to s do 5: cnts ← S AMPLE U NDER N OISE
( F, x ,n i ,σ (cid:15) ) 6: p
A ← L OWER C ONF B ND ( cnts [ˆ c A ] , n i , 1 − αs ) 7:
If σ (cid:15) Φ − 1 ( p A ) ≥ r then return ˆ c A 8: p A ← U PPER C ONF B ND ( cnts [ˆ c A ] , n i , 1 − βs − 1 ) 9:
If σ (cid:15) Φ − 1 ( p A ) <
r then return (cid:11) 10: return (cid:11)
[Figure]
guarantee as
Both use α = 0 .
001 , σ (cid:15) =
r = 0 .
K -Consensus Aggregation To reduce both inference and certiﬁcation times, we can adapt the ensemble aggregation process to return an output early when there is consensus.
Concretely, ﬁrst, we order the classiﬁers in an ensemble by their accuracy (under noisy inputs) on a holdout dataset.
Then, when evaluating ¯ f , we query classiﬁers in this order.
If the ﬁrst K individual classiﬁers agree on the predicted class, we perform soft-voting on these and return the result without evaluating the remaining classiﬁers.
Especially for large ensembles, this approach can signiﬁcantly reduce inference time without hampering performance, as the ratio kK can be large.
We note that similar approaches for partial ensemble evaluation have been proposed (Inoue, 2019; Wang et al., 2018; Soto et al., 2016), and that this does not affect the mathematical guarantees of RS.
E XPERIMENTAL E VALUATION
In line with prior work, we evaluate the proposed methods on the CIFAR10 (Krizhevsky et al., 2009) and ImageNet (Russakovsky et al., 2015) datasets with respect to two key metrics: (i) the certiﬁed accuracy at predetermined radii
r and (ii) the average certiﬁed radius (ACR).
We show that on both datasets, all ensembles outperform their strongest constituting model and obtain a new state-of-the-art.
On CIFAR10, we demonstrate that ensembles outperform individual models even when correcting for computational cost via model size or the number of used perturbations.
Further, we ﬁnd that using adaptive sampling and K-Consensus aggregation speeds up certiﬁcation up to 55 -fold for ensembles and up to 33 -fold for individual models.
Experimental Setup We implement our approach in PyTorch (Paszke et al., 2019) and evaluate on
CIFAR10 with ensembles of ResNet20 and ResNet110 and on ImageNet with ensembles of ResNet (He et al., 2016), using 1 and 2 GeForce RTX 2080 Ti, respectively.
Due to the high computational cost of C ERTIFY , we follow previous work (Cohen et al., 2019) and evaluate every 20 th image of the
CIFAR10 test set and every 100 th of the ImageNet test set ( 500 samples total).
Training and Certiﬁcation We train models with G AUSSIAN
(Cohen et al., 2019), C ONSISTENCY
(Jeong & Shin, 2020), and S MOOTH A DV (Salman et al., 2019) training and utilize pre-trained S MOOTH A DV and M ACER (Zhai et al., 2020) models.
More details are provided in App.
G.
If not declared differently, we use n 0 = 100 , n = 100 (cid:
) 000 , α = 0 .
001 , no C ERTIFY A DP or K -Consensus and evaluate and train with the same σ (cid:15) .
7. M AIN R ESULTS
Results on CIFAR
In Table 1, we compare ensembles of 10 ResNet110 against individual networks at σ (cid:15)
= 0 .
50 w.r.t.
average certiﬁed radius (ACR) and certiﬁed accuracy at various radii.
We consistently observe that ensembles outperform their constituting models by up to 21 % in ACR and 45 % in certiﬁed accuracy, yielding a new state-of-the-art at every radius.
Improvements are more pronounced for larger ensembles (see Fig. 5) and at larger radii,
where larger p A are required, which agrees well with our theoretical observations in §5.
We present more extensive results in App.
H.1, including experiments for σ (cid:15) = 0 .
25 and σ (cid:15)
= 1 . 0 yielding new state-of-the-art ACR in all settings.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
Results on ImageNet In Table 2, we compare ensembles of 3 ResNet50 with individual models at σ (cid:15)
= 1 . 0 , w.r.t.
ACR and certiﬁed accuracy.
We observe similar trends to CIFAR10, with ensembles outperforming their constituting models and the ensemble of 3 C ONSISTENCY trained obtaining a new state-of-the-art ACR of 1 . 11 , implying that ensembles are effective on a wide range of datasets.
We visualize this in Fig.
6 and present more detailed results in App.
H.2, where we also provide experiments for σ (cid:15) = 0 .
25 and σ (cid:15) = 0 .
5 , yielding new state-of-the-art ACR in both cases.
[Table]
[Table]
Computational Overhead Reduction We evaluate C ERTIFY A DP in conjunction with KConsensus aggregation using β = 0 .
001 , { n j } = { 100 , 1 (cid:48) 000 , 10 (cid:48) 000 , 120 (cid:48) 000 } , and K = 5 in Table 3 and report KCR, the percentage of inputs for which only K classiﬁers were evaluated and SampleRF and TimeRF, the factors by which sample complexity and certiﬁcation time are reduced, respectively.
We observe up to 67-fold certiﬁcation speed-ups and up to 55-fold sample complexity reductions for individual predetermined radii, without incurring any accuracy penalties.
This way, certifying an ensemble of k = 10 networks with C ERTIFY A DP can be signiﬁcantly faster than certifying an individual model with C ERTIFY while yielding notably higher certiﬁed accuracies.
We observe that adaptive sampling and K -Consensus aggregation complement each other well: At larger radii r , more samples pass on to the later stages of C ERTIFY A DP , requiring more model evaluations.
However, only samples with high success probabilities reach these later stages, making it more likely for K -Consensus aggregation to terminate an evaluation early.
We provide extensive additional experiments in App.
H.4.
7. A BLATION S TUDY
Ensemble Size and Model Size Ablation In Fig.
5, we illustrate the effect of ensemble size on
ACR for various training methods and model architectures, providing more extensive results in
App.
H.3.2.
We observe that in all considered settings, even relatively small ensembles of just 3models realize most of the improvements to be made.
In fact, as few as 3 and 5 ResNet20 are enough to outperform a single ResNet110 under G AUSSIAN and C ONSISTENCY training, respectively, while only having 47% and 79% of the parameters.
[Table]
[Figure]
Equal Number of Inferences In Fig.
7, we compare the effect of evaluating a larger number of perturbations n for an individual model compared to an ensemble at the same computational cost.
We observe that increasing the sample count for a single model only enables marginal improvements at most radii and only outperforms the ensemble at radii, which can mathematically not be certiﬁed using the smaller sample count.
See also Fig.
3 for a theoretical explanation and App.
H.3.4 for more experiments.
Other Base Classiﬁers While our approach works exceptionally well when ensembling similar classiﬁers, it can also be successfully extended to base models trained with different training methods (see
App.
H.3.5)
and denoised classiﬁers (see App. H.3.6), yielding ACR improvements of up to 5% and 17% over their strongest constituting models, respectively.
This way, existing models can be reused to avoid the high cost of training an ensemble.
Adaptive Sampling Ablation
In Table 4, we show the effectiveness of C ERTIFY A DP for certifying robustness at predetermined radii using β = 0 .
001 , different target radii, training methods, and sampling count sets { n j } .
We compare ASR j , the portion of samples returned in stage j , SampleRF, and TimeRF.
Generally, we observe larger speed-ups for smaller radii (up to a factor of 34 at r = 0 . 25 ).
There, certiﬁcation only requires small p A , which can be obtained even with few samples.
For large radii (w.r.t σ (cid:15) ), higher p A must be shown, requiring more samples even at true success rates of p A = 1 .
There, the ﬁrst phase only yields early abstentions.
The more phases we use, the more often we certify or abstain early, but also the bigger the costs due to Bonferroni correction.
We observe that exponentially increasing phase sizes and in particular { n j } = { 100 , 1 (cid:48) 000 , 10 (cid:
) 000 , 120 (cid:48) 000 } perform very well across different settings.
See App.
H.4.1 for additional results.
[Table]
K -Consensus Aggregation Ablation In Table 5, we compare ACR and certiﬁcation time reduction for ensembles of 10 ResNet110 and 50 ResNet20 using K -Consensus aggregation in isolation across different K .
Even when using only K = 2 , we already obtain 81% and 70% of the
ACR improvement obtainable by always evaluating the full ensembles ( k = 10 and k = 50 ), respectively.
The more conservative K = 10 for ResNet20 still reduces certiﬁcation times by a factor of 2 without losing accuracy.
See
App.
H.4.2 for more detailed experiments.
C ONCLUSION
We propose a theoretically motivated and statistically sound approach to construct low variance base classiﬁers for Randomized Smoothing by ensembling.
We show theoretically and empirically why this approach signiﬁcantly increases certiﬁed accuracy yielding state-of-the-art results.
To offset the computational overhead of ensembles, we develop a generally applicable adaptive sampling mechanism, reducing certiﬁcation costs up to 55-fold for predetermined radii and an ensemble aggregation mechanism, complementing it and reducing evaluation costs on its own up to 6-fold.
E THICS S TATEMENT
Most machine learning techniques can be applied both in ethical and unethical ways.
Techniques to increase the certiﬁed robustness of models do not change this but only make the underlying models more robust, again allowing beneﬁcial and malicious applications, e.g., more robust medical models versus more robust weaponized AI.
As our contributions improve certiﬁed accuracy, certiﬁcation radii, and inference speed, both of these facets of robust AI are ampliﬁed.
Furthermore, while we achieve state-of-the-art results, these do not yet extend to realistic perturbations.
Malicious actors may aim to convince regulators that methods such as the proposed approach are sufﬁcient to provide guarantees for general real-world threat scenarios, leading to insufﬁcient safeguards.
R EPRODUCIBILITY S TATEMENT
We make all code and pre-trained models required to reproduce our results publicly available at https://github.com/eth-sri/smoothing-ensembles .
There, we also provide detailed instructions and scripts facilitating the reproduction of our results.
We explain the basic experimental setup in §7 and provide exact experimental details in App.
G, where we extensively describe the datasets (App. G.1), model architectures (App.
G.2), training methods (App. G.3), hyper-parameter choices (App. G.4), and experiment timings (App.
G.5).
We remark that the datasets, architectures, and training methods we use are all publicly available.
Section §7 and App.
H contain numerous experiments for various datasets, model architectures, training methods, noise levels, and ensemble compositions.
The consistent trends we observe over this wide range of settings highlight the general applicability of our approach.
In App.
H.3.3, we analyze the variability of our results, reporting standard deviations for a range of metrics, ﬁnd that our ensembling approach reduces variability, and note that our results are statistically signiﬁcant.
We include complete proofs of all theoretical contributions (§5 and §6) in App.
C and E.
In App.
D, we theoretically motivate the modelling assumptions made in §5 and validate them empirically.
A C OMPARISON TO L IU ET AL . (2020)
Liu et al.
(2020) introduce Smoothed Weighted Ensembling (SWEEN), which are weighted ensembles of smoothed base models.
They focus on deriving optimal ensemble weights for generalization rather than on analyzing the particularities of ensembles in the RS setting.
As they do not consider multiple testing correction and the conﬁdence of the applied Monte Carlo sampling, they obtain statements about empirical, distributional robustness of their classiﬁers, rather than individual certiﬁcates with high conﬁdence.
Due to this difference setting, we do not compare numerically.
B A DDITIONAL R ELATED W ORK
Extensions As outlined in §3, Cohen et al.
(2019) focuses on the addition of Gaussian noise to drive (cid:96) 2 -robustness results.
However, various extensions with other types of noise and guarantees have been proposed.
By using different types of noise distributions and radius calculations, Yang et al.
; Zhang et al.
(2020) derive recipes to determine certiﬁcates for general (cid:96) p -balls and speciﬁcally showcase results for p = 1 , 2 , ∞ .
Numerous works have shown extensions to discrete perturbations such as (cid:96)
-perturbations (Lee et al., 2019;
Wang et al., 2021; Schuchardt et al., 2021), graphs (Bojchevski et al., 2020; Gao et al., 2020), patches Levine & Feizi (2020) or manipulations of points in a point cloud Liu et al. (2021).
Dvijotham et al.
(2020) provide theoretical derivations for the application of both continuous and discrete smoothing measures.
Mohapatra et al.
(2020) improve certiﬁcates by considering gradient information.
Beyond norm-balls certiﬁcates, Fischer et al.
; Li et al.
(2021) show how geometric operations such as rotation or translation can be certiﬁed via Randomized Smoothing.
Chiang et al.
Fischer et al.
(2021) show how the certiﬁcates can be extended from the setting of classiﬁcation to regression (and object detection) and segmentation, respectively.
In the classiﬁcation setting, Jia et al.
(2020) extend certiﬁcates from just the top-1 class to the top-k classes.
Similarly,
Kumar et al.
(2020a) certify the conﬁdence of the classiﬁer not, just the top class prediction.
Beyond the inference-time evasion attacks, Rosenfeld et al.
(2020) showcase RS as a defense against data poisoning attacks.
Many of these extensions are orthogonal to the standard Randomized Smoothing approach and thus orthogonal to our improvements.
We showcase that these improvements carry over from the (cid:96) 2 case to other (cid:96) p norms in App.
H.3.7.
Limitations Recently, some works also have investigated the limitations of Randomized Smoothing:
Mohapatra et al.
(2021) point out the limitations of training for Randomized Smoothing as well as the impact on class-wise accuracy.
Kumar et al.
(2020b)
; Wu et al.
(2021) show that for p > 2 , the achievable certiﬁcation radius quickly diminishes in high-dimensional spaces.
C M ATHEMATICAL D ERIVATION OF V ARIANCE R EDUCTION FOR E NSEMBLES
In this section, we present the algebraic derivations for §5, skipped in the main part due to space constraints.
Individual Classiﬁer In §5 deﬁne f l ( x ) =:
y l ∈ R m as the sum of two random variables y l = y lp + y lc .
The behavior on a speciﬁc clean sample x is modeled by y lc with mean c ∈ R m , the expectation of the logits for this sample over the randomization in the training process, and corresponding covariance
[Equation]
We note that: (i) we omit the lower triangular part of covariance matrices due to symmetry, (ii) c and Σ c are constant across the different f l for a ﬁxed x and (iii) that due to (ii) and our modelling assumptions all means, (co)variances and probabilities are conditioned on x , e.g., E [ y lc ] =
E [ y lc | x ] = c .
However, as we deﬁne all random variables only for a ﬁxed x , we omit this.
Similarly, we model the impact of the perturbations (cid:15) introduced during RS by y lp with mean zero and covariance Σ p :
[Equation]

That is, we assume c l to be the expected classiﬁcation a model learns for the clean sample while the covariances Σ c and Σ p encode the stochasticity of the training process and perturbations, respectively.
As y lp models the local behavior under small perturbations and y lc models the global training effects, we assume them to be independent:
[Equation]
The classiﬁer prediction arg max q y q is determined by the differences between logits.
We call the difference between the target logit and others the classiﬁcation margin.
During certiﬁcation with RS, the ﬁrst step is to determine the majority class.
Without loss of generality, we assume that it has been determined to be index 1 , leading to the classiﬁcation margin
z i =
y 1 − y i .
If z i > 0 for all i (cid:54) = 1 , the majority class logit y 1 is larger than those of all other classes
y i .
We deﬁne z :=
[ z 2 , .., z n ] T , skipping the margin of y 1 to itself.
Under the above assumptions, the statistics of the classiﬁcation margin for a single classiﬁer are:
[Equation]
Ensemble Now, we construct an ensemble of k of these classiﬁers.
Using soft-voting (cf. Eq.
(2)) to compute the ensemble output ¯ y = 1 k (cid:80) kl =1 y l .
We assume the y ip and y jp to be correlated with ζ p Σ p for classiﬁers
i (cid:54) =
j and similarly model the correlation of y lc with ζ c Σ c for ζ p , ζ c ∈ [0 , 1] .
Letting y ∗ :
= [ y 1 (cid:62) , ..., y k (cid:62) ] (cid:62) denote the concatenation of the logit vectors of all classiﬁers, we assemble their joint covariance matrix in a block-wise manner from the classiﬁer individual covariance matrices as:
[Equation]

We then write the corresponding classiﬁcation margins ¯ z i =
¯ y 1 −
¯ y i .
or in vector notation ¯ z :=
[¯ z 2 , .., ¯ z m ] T , again skipping the margin of ¯ y 1 to itself. By linearity of expectation we obtain E [¯ z i ] = E [ z i ] = c
1 − c i or equivalently
[Equation]
We deﬁne the ensemble difference matrix D ∈ R m − 1 × mk with elements d
i,j such that ¯ z = Dy ∗ :
[Equation] [Equation]
, else if j mod
[Equation]
[Equation]
This allows us to write the covariance matrix of the ensemble classiﬁcation margins
[Equation]
Now we can evaluate its diagonal elements or use the multinomial theorem and rule on the variance of correlated sums to obtain the variance of individual terms:
[Equation]
Variance Reduction We can split Var[¯ z i ] into the components associated with the perturbation effect σ 2 p ( k ) and the clean prediction σ 2 c ( k ) , all as functions of the ensemble element number k .
Now
[Equation]
we analyze these variance components independently by normalizing them with the corresponding components of an individual classiﬁer:
[Equation]
We observe that both variance components go towards their corresponding correlation coefﬁcients ζ p and ζ c as ensemble size grows, highlighting the importance of non-identical classiﬁers.
Especially for samples that are near a decision boundary, this variance reduction will lead to much more consistent predictions, in turn signiﬁcantly increasing the lower conﬁdence bound p 1 and thereby the certiﬁed radius as per Theorem 3.1.
D T HEORY V ALIDATION
In this section, we present experiments validating the modeling assumptions made in §5 to derive the reduced variance over perturbations in RS resulting from using ensembles of similar classiﬁers.
There are three main assumptions we make:
• The (pre-softmax) output of a single model can be modeled as f l ( x ) =
y p + y c , where y c captures the prediction on clean samples and y p the effect of perturbations with mean zero.
• The clean and perturbation components y c and y p are independent.
• The covariance of y lp and y lc between different classiﬁers
i (cid:54) =
j can be parametrized as Cov( y ip , y jp )
= ζ p Σ p and similarly Cov( y ic , y jc )
= ζ c Σ c .
To show that these assumptions are valid, we additionally assume multivariate Gaussian distributions and evaluate an ensemble of k = 50 ResNet20 on 100 clean samples x , each perturbed with n =different error terms (cid:15) , and compare the obtained distributions and covariance matrices with our modeling.
We generally observe excellent agreement between observation and model at low noise level σ (cid:15)
= 0 .
25 and good agreement at a high noise level of σ (cid:15)
= 1 . 0 .
Individual classiﬁer predictions We model the predictions of individual classiﬁers f : R d (cid:55)→ R m on perturbed inputs x + (cid:15) for a single arbitrary but ﬁxed x and Gaussian perturbations (cid:15) ∼ N (0 , σ 2 (cid:15) I ) with f l ( x ) =
y p + y c .
[Figure]
[Figure]
We can interpret the clean component y c in two different ways: either as the prediction y c :
= f ( x ) on the clean sample x or as the mean prediction over perturbations of this sample y c := E (cid:15) ∼N (0 ,σ 2 (cid:15) I ) [ f ( x + (cid:15) )] .
In both cases, we assume y c to be distributed with mean c ∈ R m and covariance Σ c .
The ﬁrst case is illustrated in Fig.
8, where we rescale both our model and the predictions with the sample-wise mean and variance over classiﬁers and observe excellent agreement between model and observed distribution.
The second case is illustrated in the leftmost column in Fig.
9 for two different σ (cid:15) , where we again observe excellent agreement between the true distribution and our model.
Here we denote with µ f ( x (cid:48) )
the mean prediction over perturbed samples E (cid:15) ∼N (0 ,σ 2 (cid:15) I )
[ f ( x + (cid:15) )] , corresponding to the setting of y c := E (cid:15) ∼N (0 ,σ 2 (cid:15) I )
[ f ( x + (cid:15) )] and similarly with denote with µ f ( x ) the mean prediction over classiﬁers for an unperturbed sample E l [ f l ( x )] , corresponding to the setting of y c :
= f ( x ) .
The perturbation effect y p is assumed to be mean zero, either following from local linearization and zero mean perturbations when deﬁning the clean component as y c :
= f ( x ) or directly from the deﬁnition of the clean component as y c := E (cid:15) ∼N (0 ,σ (cid:15) )
[ f ( x + (cid:15) )] .
For small perturbations ( σ (cid:15) = 0 . 25 ), the assumption of local linearity holds, and we observe a mean perturbation effect distributed very tightly around
(see top right pane in Fig. 9).
For larger perturbations ( σ (cid:15) = 1 . 00 ), this assumption of local linearity begins to break down, and we observe a much wider spread of mean perturbation effect (see bottom right pane in Fig. 9).
At both perturbation levels, we observe an excellent agreement of the perturbation effect model with the observed data when using the deﬁnition of y c := E (cid:15) ∼N (0 ,σ (cid:15) )
[ f ( x + (cid:15) )]
(green in the middle column in Fig. 9).
Using y c :
= f ( x ) , we still observe great agreement at the lower perturbation magnitude but a notable disagreement at high perturbation levels.
While both deﬁnitions for y c are fully compatible with our derivation in §5, we choose to present a version based on y c :
= f ( x ) as it allows for a better intuitive understanding.
[Figure]
[Figure]
Correlation between clean and error component We model y c and y p separately to distinguish between randomness over perturbations, which are introduced during RS, and over the models introduced during training with different random seeds.
As y c models the large scale effects of training and y p the smaller local scale local effects of inference time perturbations, we assume them to be independent, corresponding to a covariance Cov( y p,i , y c,j )
≈ 0 ∀
i, j .
Depending on which deﬁnition of y c we use, we obtain either of the two covariance matrices in Fig.
[Figure]
where the upper left-hand block is the covariance of y c , the lower righthand block the covariance of y p and the off-diagonal blocks
Cov( y p,i , y c,j ) .
In the setting of y c := µ f ( x (cid:48) ) , the two components y c and y p are perfectly independent as can be seen by the 0 off-diagonal blocks in the covariance matrix shown in Fig. 10a.
In the setting of y c :
= f ( x ) , very small terms can be seen in the off-diagonal blocks shown in Fig. 10b.
Overall, the agreement with our model is excellent in both cases.
Inter classiﬁer correlation structure
In our analysis, we consider ensembles of similar classiﬁers which are based on the same architecture and trained using the same methods, just using different random seeds.
Hence, we assume that the correlation between the logits of different classiﬁers has a similar structure but smaller magnitude than the correlation between the logits of the same classiﬁer.
Correspondingly, we parametrize the covariance between y ip and y jp for classiﬁers i (cid:54) =
j with ζ p Σ p and similarly between y ic and y jc with ζ c Σ c for ζ p , ζ c ∈ [0 , 1] .
To evaluate this assumption, we consider all (cid:0) 502 (cid:1) pairings of classiﬁers and compute the resulting pairwise covariance matrices for the clean components
y lc
=
f l ( x ) and perturbation effects y lp .
In Fig.
11 we show the resulting covariance matrices (left column), our model (right column), and the delta between the two (center column).
Again we observe excellent agreement to the point that even given the delta maps, it is hard to spot the differences in the covariance matrices.
Model ﬁt conclusion We conclude that our model is able to capture the relevant behavior even of deep networks (here ResNet20 ) with great accuracy that goes beyond what is required for a theoretical analysis to allow for interesting insights into the underlying mechanics.
E P ROOF OF T HEOREM 6.
We ﬁrst restate the theorem:
Theorem (Theorem 6.1 restated) .
For α, β ∈ [0 , 1] , s ∈ N + , n 1 < · · · < n s , C ERTIFY A DP :
returns ˆ c A if at least 1 − α conﬁdent that G is robust with a radius of at least r . 2.
returns (cid:11) before stage s only if at least 1 − β conﬁdent that robustness of G at radius r can not be shown.
for n s ≥(cid:100) n (1 − log α ( s )) (cid:101) has maximum certiﬁable radii at least as large as C ERTIFY for
n .
Proof.
We show statements 1, 2, and 3 individually.
Proof of 1 .
If a phase in C ERTIFY A DP returns ˆ c A , then via Theorem 3.1, G will be not robust with probability at most α/s .
Via Bonferroni correction (Bonferroni, 1936), if C ERTIFY A DP returns ˆ c A , then G is robust with radius at least r with conﬁdence 1 − s ( α/s )
=
1 − α .
Proof of 2 .
If C ERTIFY A DP returns (cid:11) in phase s , we have p
A < p (cid:48)
A = Φ( r/σ (cid:15) ) the minimum success probability for certiﬁcation at r. By the deﬁnition of the U PPER C ONFIDENCE B OUND the true success probability of G will be p A ≤ p A with conﬁdence
at least β/
( s − 1) .
Hence, if phase j returns (cid:11) , robustness of G cannot be shown at radius r with conﬁdence− β/
( s − 1) , even with access to the true success probability or inﬁnitely many samples.
Again with Bonferroni correction (Bonferroni, 1936), the overall probability that G is robust at r despite C ERTIFY A DP abstaining early is at most ( s − 1)( β/ ( s − 1))
=
β .
Proof of 3 .
Finally, to prove the last part, we assume cnts[ ˆ c A ] = n , yielding the largest certiﬁable radii for any given n and α .
Now, the largest radius provable via Theorem 3.1 with n samples at α is α 1 /n
(Cohen et al., 2019).
Similarly, for n s samples at α/s , it is (cid:0) αs (cid:1) 1 ns .
Then we have the following equivalences:
[Equation]
Hence, if we choose n s = (cid:100) n (1 − log α ( s )) (cid:101) , then we can certify at least the same maximum radius with n s samples at α/s as with n samples at α .
Thus, overall, we can certify the same maximum radius at α .
F A DDITIONAL R ANDOMIZED S MOOTHING D ETAILS
This section contains deﬁnitions needed for the standard certiﬁcation via Randomized Smoothing,
C ERTIFY , proposed by Cohen et al.
S AMPLE U NDER N OISE
( F, x, n, σ (cid:15) )
ﬁrst samples n inputs x 1 , . .
, x n with x 1 = x + (cid:15) 1 , . . .
, x n = x + (cid:15) n for (cid:15) 1 , . .
, (cid:15) n ∼ N (0 , σ (cid:15) ) .
Then it counts how often F predicts which class for these x 1 , . . .
, x n and returns the corresponding m dimensional array.
U PPER C ONF B ND ( k, n, 1 − α ) returns an upper bound on p with conﬁdence at least 1 − α where p is an unknown probability such that k ∼ B ( n, p ) .
Here, B ( n, p ) is a binomial distribution with parameters n and p .
L OWER C ONF B ND ( k, n, 1 − α ) is analogous to U PPER C ONF B ND but returns a lower bound on p instead of an upper bound with conﬁdence 1 − α .
We point the interested reader to Cohen et al.
for more details about Randomized Smoothing.
G E XPERIMENTAL D ETAILS
In this section, we provide greater detail on the datasets, architectures, and training and evaluation methods used for our experiments.
G. D ATASET D ETAILS
We use the MNIST, CIFAR10, and ImageNet datasets in our experiments.
MNIST (Deng, 2012) contains 60’000 training and 10’000 test set images partitioned into 10 classes for the 10 digits.
We evaluate all images from the test set.
CIFAR10 (Krizhevsky et al., 2009) (MIT License) contains 50’000 training and 10’000 test set images partitioned into 10 classes.
We evaluate every 20 th image of the test set starting with index(0, 20, 40, ..., 9980) in our experiments, following previous work (Cohen et al., 2019).
ImageNet (Russakovsky et al., 2015) contains 1’287’167 training and 50’000 validation images, partitioned into 1000 classes.
We evaluate every 100 th image of the validation set starting with index
(0, 100, 200, ..., 49900) in our experiments, following previous work (Cohen et al., 2019).
G. A RCHITECTURE D ETAILS
[Table]
We use different versions of ResNet (He et al., 2016) for our experiments.
Concretely, we evaluate ensembles of ResNet and ResNet110 on CIFAR10 and ensembles of ResNet50 on
ImageNet.
ResNet110 has about 6.35 times as many parameters as
ResNet20 (see Table 6).
ResNet50 instantiated for ImageNet has substantially more parameters than ResNet110 instantiated for CIFAR10 because of the signiﬁcantly larger input dimension of ImageNet samples.
G. T RAINING M ETHODS
To obtain high certiﬁed radii via RS, the base model F must be trained to cope with the added
Gaussian noise (cid:15) .
To this end, Cohen et al.
(2019) propose data augmentation with Gaussian noise during training, referred to as G AUSSIAN in the following.
Building on this, Salman et al.
(2019) suggest S MOOTH A DV , a combination of adversarial training (Madry et al., 2018; Kurakin et al., 2017; Rony et al., 2019) with the data augmentation from G AUSSIAN .
(Here, we always consider the PGD version.)
While improving accuracy, this training procedure is computationally very expensive.
M ACER (Zhai et al., 2020) achieves similar performance with a cheaper training procedure, adding a loss term directly optimizing a surrogate certiﬁcation radius.
Jeong & Shin (2020), called C ONSISTENCY in the following, replace this term with a more easily optimizable loss, further decreasing training time and improving performance.
Depending on the setting, the current state-of-
Further, Salman et al.
(2020) present denoised smoothing, where the base classiﬁer f = h ◦ f (cid:48) is the composition of a denoiser h that removes the Gaussian noise from the input and an underlying classiﬁer f (cid:48) that is not specially adapted to Gaussian noise, e.g., accessed via an API.
G. T RAINING D ETAILS
All models are implemented in PyTorch (Paszke et al., 2019) (customized BSD license).
For G AUSSIAN training (No license speciﬁed), we use the published code by Cohen et al.
For each network, we chose a different random seed, otherwise using identical parameters.
Overall, we trained 50 ResNet20 and 10 ResNet110 for each σ (cid:15) ∈ { 0 .
on CIFAR10.
For ImageNet, we trained 3 ResNet50 for σ (cid:15) = 1 . 0 , and took the results for σ (cid:15) ∈ { 0 . 25 , 0 . 50 } from their GitHub.
For C ONSISTENCY training (MIT License), we use the code published by Jeong & Shin (2020) and
C ONSISTENCY instantiation built on G AUSSIAN training.
Similarly to G AUSSIAN , we use a different random seed and otherwise identical parameters for all networks.
We chose the parameters reported to yield the largest ACR by Jeong & Shin (2020) for any given σ (cid:15) , except for σ (cid:15) = 0 .
25 on ImageNet, for which no parameters were reported.
In detail, for CIFAR10 we generally use η = 0 .
5 , using λ = 20 for σ (cid:15) = 0 .
25 and λ = 10 for σ (cid:15) = 0 .
5 and σ (cid:15)
= 1 .
(all for both ResNet20 and ResNet110 ).
In this way, we train 50 ResNet20 and 10 ResNet110 for each σ (cid:15) ∈ { 0 . 25 , 0 .
For ImageNet, we use η = 0 .
1 and λ = 5 for σ (cid:15)
= 1 . 0 .
For σ (cid:15) = 0 .
25 and σ (cid:15) = 0 .
5 , we use η = 0 .
5 with λ =respectively λ
= 5 .
For S MOOTH A DV training (MIT License), we use the PGD based instantiation of the code published by Salman et al.
Note that due to the long training times, we only train S
MOOTH
A DV models ourselves on CIFAR10.
The individual models we train only differ by random seed, all using PGD attacks.
For σ (cid:15) = 0 .
25 , we train with T = 10 steps, an (cid:15) = 255 / 255 , 10 epochs of warm-up, and m train = 8 noise terms per sample during training.
For σ (cid:15) = 0 .
5 , we train with T = 2 steps, an (cid:15) = 512 / 255 , 10 epochs of warm-up, and m train = 2 noise terms per sample during training.
Finally, σ (cid:15) = 1 . 0 , we train with T = 10 steps, an (cid:15) = 512 / 255 , 10 epochs of warm-up, and m train = 2 noise terms per sample during training.
We use the same training schedule and optimizer for all models, i.e., stochastic gradient descent with Nesterov momentum (weight = 0 .
9 , no dampening), with an (cid:96) 2 weight decay of 0 . 0001 .
For
CIFAR10, we use a batch size of 256 and an initial learning rate of 0 . 1 , reducing it by a factor ofevery 50 epochs and training for a total of 150 epochs.
For ImageNet, we use the same settings, only reducing the total epoch number to 90 and decreasing the learning rate every 30 epochs.
All single M ACER (No license speciﬁed) trained models for CIFAR10 are taken directly from Zhai et al.
For our experiments on denoised smoothing, we use the 4 white box denoisers with DNCNN-WIDE architecture trained with learning schedules 1, 3, 4, and 5 for STAB ResNet110 , and the ResNet trained on unperturbed samples for 90 epochs from Salman et al.
(2020) (MIT License).
To rank the single models for CIFAR10, we have evaluated them on a disjunct hold-out portion of the CIFAR10 test set.
Concretely, we use the test images with indices 1 , 21 , 41 , ..., 9981 to rank the single models for G AUSSIAN , C ONSISTENCY , and
S MOOTH A DV trained models.
The performances of individual models values we report are those of the models with the best score on this validation set for CIFAR10 and the best score on the test set for ImageNet (favoring individual models in this setting).
For ensembles of size k < 10 and k < 50 for ResNet110 and ResNet20 , respectively, we ensemble the k models according to their performance on the hold-out set.
We note that other combinations might yield stronger ensembles, but an exhaustive search of all combinatorially many possibilities is computationally infeasible.
CIFAR10 models were trained on single GeForce RTX 2080 Ti and ImageNet models on quadruple 2080 Tis.
We report the epoch-wise and total training times for individual models in Table 7 (when trained sequentially one at a time).
[Table]
G. E XPERIMENT T IMINGS
To evaluate the ensembles of ResNet20 and ResNet110 on CIFAR10, we use single GeForce RTX 2080 Ti, and to evaluate ensembles of ResNet50 on ImageNet, we use double 2080 Tis.
For both datasets, 500 samples are evaluated per experiment as discussed in App. G.1.
Below we list the time required for certiﬁcation using C ERTIFY and no K -Consensus aggregation:
• ResNet110 on CIFAR10 (1 GPU): 4.12h per single model; 41.2h per ensemble of• ResNet20 on CIFAR10 (1 GPU): 0.825h per single model; 41.3h per ensemble of• ResNet50 on ImageNet (2 GPUs): 8.75h per single model; 26.2h per ensemble of
The timing for different ensemble sizes scales linearly between full size and individual model timings.
The time required for certiﬁcation using C ERTIFY A DP or K -Consensus aggregation or both can be obtained by dividing the timings reported above with the speed-up factor TimeRF reported for the corresponding experiments.
H A DDITIONAL E XPERIMENTS
In the following, we present numerous additional experiments and more detailed results for some of the experiments presented in §7.
H. A DDITIONAL R ESULTS ON CIFAR
In this section, we present more experimental results on CIFAR10.
Concretely, Table 8 presents the performance of ensembles of ResNet20 ( k ∈ { 1 , 5 , 50 } ) and ResNet110 ( k ∈ { 1 , 10 } ) trained using a wide range of of methods for σ (cid:15)
= 0 .
25 , σ (cid:15) = 0 .
5 , and σ (cid:15) = 1 . 0 .
We again consistently observe that ensembles signiﬁcantly outperform their constituting models, implying that ensembles are effective for various training methods and model architectures, as well as different σ (cid:15) .
In particular, they lead to a new state-of-the-art both in ACR and at most radii at σ (cid:15) = 0 .
25 , σ (cid:15) = 0 .
5 , and σ (cid:15) = 1 . 0 .
Fig.
12 visualizes the evolution of ACR and certiﬁed accuracy with ensemble size for σ (cid:15)
= 0 .
In
Fig.
13, we visualize the evolution of certiﬁed accuracy over radii for G AUSSIAN trained models.
H. A DDITIONAL R ESULTS ON I MAGE N ET
Extending the results from §7, Table 9 also provides ensemble results on ImageNet for σ (cid:15)
= 0 .
25 , and σ (cid:15)
= 0 .
5 with C ONSISTENCY trained ResNet50 .
Similarly to CIFAR10, we consistently observe that ensembles signiﬁcantly outperform their constituting models, implying that ensembles are effective on various datasets.
In particular, we achieve a new state-of-the-art both in ACR and certiﬁed accuracy at most radii for σ (cid:15)
= 0 .
25 , σ (cid:15) = 0 .
50 , and σ (cid:15)
= 1 . 00 .
Additionally, we report the performance of all individual classiﬁers which constitute the ensembles, where k = 3 combines all three and k
= 2 the ﬁrst two for G AUSSIAN and C ONSISTENCY , and the last two for S MOOTH A DV .
the S MOOTH A DV models are taken directly from Salman et al.
and were trained with a different (cid:15) parameter ( 256 , 512 , and 1024 ).
We observe that if there is a large discrepancy between model performance at some radii (e.g., the S MOOTH A DV models at
[Table]
C ONSISTENCY
[Table]
[Figure]
[Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
[Figure]
small radii)
, the ensemble might perform worse than the strongest constituting model.
Overall and when performance is homogenous, however, ensembles once again obtain signiﬁcant performance increases over individual models.
H. A DDITIONAL A BLATION S TUDY OF E NSEMBLES
This section presents a range of experiments on different different aspects of variance reduction through ensembles for Randomized Smoothing.
In particular, we cover aggregation approaches, the effect of ensemble size, the variability of our results, certiﬁcations with an equal number of inferences, ensembles of differently trained models, and ensembles of denoisers.
H.3. A GGREGATION A PPROACHES
We experiment with various instantiations of the general aggregation approach described in §4.
In particular, we consider soft-voting, hard-voting, soft-voting after softmax, and weighted soft-voting.
Soft-voting We simply average the logits to obtain
[Equation]
Hard-voting We process the outputs of single models with the post-processing function γ HV ( y l ) = 1 j =argmax i ( y li ) that provides a one-hot encoding of the arg max of f l ( x ) before averaging to obtain
[Equation]
Soft-voting after softmax We process the outputs of the single models by applying the softmax function as post-processing function γ softmax to obtain
[Equation]
Weighted soft-voting We consider soft-voting with a classiﬁer-wise weights w l learned on a separate holdout set and obtain
[Equation]
We compare these approaches in Table 10 and observe that the two soft-voting schemes perform very similarly and outperform the other approaches.
We decide to use soft-voting without soft-max for its conceptual simplicity for all other experiments.
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
C ONSISTENCY
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] H.3. E NSEMBLE S IZE In Table 11, we present extended results on the effect of ensemble size for different training methods and σ (cid:15) .
Generally, we observe that ensembles of sizes 3 and 5 already signiﬁcantly improve over the performance of a single model, while further increasing the ensemble size mostly leads to marginal gains.
This aligns well with our theory from §5.
H.3. V ARIABILITY OF R ESULTS
[Figure]
In Table 12, we report mean and standard deviation of ACR and certiﬁed accuracy for 50 G AUSSIAN and C ONSISTENCY trained
ResNet20 with different random seeds, either combined toensembles with k = 5 or evaluated individually.
Generally, we observe a notably smaller standard deviation of the ensembles compared to individual models.
We visualize this in Fig.
14, where we show the ± 3 σ region.
H.3. E QUAL N UMBER OF I NFERENCES
In Table 13, we compare the certiﬁed accuracy of an ensemble of k = 10 ResNet110 obtained with n = 100 (cid:48)
000 samples
[Table] [Table]
on CIFAR10 with that of an individual model obtained with n = 1 (cid:48) 000 (cid:48) 000 , i.e. at the same computational cost.
For both G AUSSIAN and C ONSISTENCY training, we observe that at all radii which can mathematically be certiﬁed using n = 100 (cid:48)
samples the ensemble it signiﬁcantly outperforms the individual model at equal computational cost.
Note that the individual models with n = 1 (cid:48) 000 (cid:48)
samples have the same or just a marginally larger certiﬁed accuracy at all radii ≤ 1 .
75 than the same individual model with n = 100 (cid:48)
000 samples.
Only at radius 2 .
where certiﬁcation is mathematically impossible with n = 100 (cid:48)
000 samples does using additional samples increase the certiﬁed accuracy.
Note however, that, in contrast to our ensembles, this does not inﬂuence the actual accuracy of the underlying model in any way, but simply allows to derive tighter conﬁdence bounds.
Table 14 shows similar results for ImageNet and ensembles of k = 3 ResNet110 , implying that these observations hold for various datasets.
H.3. E NSEMBLES OF D IFFERENTLY T RAINED M ODELS
In Table 15 we show how ensembles built from differently trained base models (S MOOTH A DV ,
M ACER , and C ONSISTENCY ) perform.
We observe that the ensemble of all three models is strictly better than any individual model, suggesting that ensembles also work for models trained with different training methods.
However, the improvements are signiﬁcantly smaller than the improvements of ensembling similar classiﬁers in the same setting (see also Table 8 for comparison).
This is again due to a very heterogeneous model performance at some radii, where the strongest models are weighed down by weaker ones.
H.3. E NSEMBLES OF D ENOISERS
[Table]
It was shown (Salman et al., 2020) that standard neural networks can be robustiﬁed with respect to Gaussian noise by training a suitable denoiser and evaluating the original model on the denoised sample.
Here, we aim to overcome the drawback of any ensemble, the need to train multiple diverse models, by instead training different denoisers and building an ensemble by combining these denoisers with a single underlying model.
We show the general applicability of our ensembling approach by
[Table] [Table]
applying it to the setting of denoised smoothing (Salman et al., 2020).
We consider 4 pre-trained denoisers and combine them with a single standard ResNet110 (see App.
G.3 for details).
Comparing the strongest resulting model with an ensemble of all 4 at σ (cid:15) = 0 .
25 , we observe a 17% ACR improvement.
Training strong denoisers with current methods is challenging.
However, based on these results we are conﬁdent that advances in denoiser training combined with ensembles can present an efﬁcient way to obtain strong provable models.
H.3. A DDITIONAL N ORMS
Below, we demonstrate that our approach generalizes beyond the (cid:96) 2 -norm setting and generalizes well to improve the robust accuracy against (cid:96) 0 -, (cid:96) 1 - and (cid:96) ∞ -norm bounded attacks.
(cid:96) ∞ -norm robustness is typically derived directly from (cid:96) 2 -norm bounds (Yang et al., 2020;
Salman et al., 2019), and hence our results for the (cid:96) 2 -norm are directly applicable.
In Table 17 and Table 18, we show the effectiveness of our approach for (cid:96) 0 -norm bounded perturbations using the training and certiﬁcation method from Lee et al.
For both, we use the default hyper-parameter α (which is not the same as the α in standard Randomized Smoothing from Cohen et al.
, i.e. α = 0 .
8 for MNIST and α = 0 .
2 for ImageNet.
We observe that even an ensemble of just 3 models outperforms the individual models in every setting.
In Table 19, we similarly show the effectiveness of our approach for (cid:96) 1 -norm bounded perturbations using training with uniform noise and the uniform noise based certiﬁcation method from Yang et al.
(2020) with σ (cid:15) =
λ = 1 . 0 .
We observe again that even an ensemble of just 3 models outperforms the single model in every setting.
H. A DDITIONAL C OMPUTATIONAL O VERHEAD R EDUCTION E XPERIMENTS
This section contains additional experiments for the computational overhead methods introduced in §6.
Whenever we use C ERTIFY A DP , we set β = 0 . 001 .
Adaptive sampling and K -consensus Table 20 is an extensive version of Table 3.
In Table 21, we show the effect of adaptive sampling and K -Consensus aggregation for G AUSSIAN trained ResNet on CIFAR10, analogously to Table 20.
Certifying an ensemble with C ERTIFY A DP and K -Consensus aggregation achieves approximately the same certiﬁed accuracy as C ERTIFY while the certiﬁcation time is up to 67 times faster.
In particular, for both training methods and for each radius, the certifying the ensemble of k = 10 networks with C ERTIFY A DP and K -Consensus aggregation is faster than certifying an individual model with C ERTIFY while the certiﬁed accuracy is signiﬁcantly improved
[Table] [Table]
in most cases.
In addition [Table] we conﬁrm our previous observation that the sample reduction is most prominent for small radii (for a given σ (cid:15) ) while KCR increases with radius.
H.4. A DDITIONAL A DAPTIVE S AMPLING E XPERIMENTS
This section contains other experiments with C ERTIFY A DP , highlighting various aspects of adaptive sampling for ensembles and also individual models.
Adaptive sampling for ensembles on CIFAR10 For comparison with Table 3 (respectively Table 20 and Table 21), where we show the combined effect of K -Consensus aggregation and adaptive sampling, we show the results for just applying adaptive sampling to the same setting in Table 22, using { n j } = { 100 , 1 (cid:48) 000 , 10 (cid:48) 000 , 120 (cid:48) 000 } , σ (cid:15)
= 0 .
25 , and ensembles of 10 C ONSISTENCY trained ResNet110 .
We observe that the speed-ups for small radii are comparable to applying both
K -Consensus aggregation and adaptive sampling, as only a few samples have to be evaluated.
The additional beneﬁt due to K -Consensus aggregation grows as the later certiﬁcation stages are entered more often for larger radii, highlighting the complementary nature of the two methods.
Note that for radius r = 0 .
75 , 100 samples, and even 1 (cid:48) 000 are never sufﬁcient for certiﬁcation but can still yield early abstentions.
Adaptive sampling for ImageNet ensembles In Table 23, we demonstrate the applicability of our adaptive sampling approach to ensembles of ResNet50 on ImageNet.
In particular, we achieve sampling reduction factors of up to 42 without incurring a signiﬁcant accuracy penalty.
For a ﬁxed σ (cid:15) , we empirically observe larger speed-ups at smaller radii.
This effect, however, depends on the underlying distribution of success probabilities and would be expected to invert for a model that has true success probabilities close to p A ≈ 0 .
5 for most samples.
Effect of sampling stage sizes Additionally, in Table 24, we investigate how the choice of the number of phases and the number of samples in each phase inﬂuence the sampling gain for various radii and models.
We observe that as few as 100 samples can be sufﬁcient to certify or abstain from up to 80% of samples, making schedules with many phases attractive.
However, when the true success probability is close to the one required for certiﬁcation, many samples are required to obtain sufﬁciently tight conﬁdence bounds on the success probability to decide either way.
This leads to many samples being discarded for schedules with many phases.
Additionally, the higher per phase conﬁdence required due to Bonferroni correction requires more samples to be drawn to be able to obtain the same maximum lower conﬁdence bound to the success probability.
This latter effect becomes especially pronounced for larger radii, where higher success probabilities are required.
Ablation of α and β
In Fig.
15, we illustrate the effect of different α and β on the ﬁrst phase of certiﬁcation with C ERTIFY A DP where we consider just 100 perturbations.
Increasing α reduces [Table] [Table] the conﬁdence 1 − α required for certiﬁcation and hence makes early certiﬁcation more likely for the same underlying true success probabilities (see Fig. 15a).
Increasing β reduces the conﬁdence 1 − β that certiﬁcation will not be possible required for early termination and hence makes early termination more likely for the same underlying true success probabilities (see Fig. 15b).
Overall, we observe that evaluating even only 100 perturbations allows us to, with high probability, abstain from or certify samples with a wide range of true success probabilities.
We only have to continue with the costly certiﬁcation process for samples in a narrow band of true success probabilities (see Fig. 15c), greatly reducing the expected certiﬁcation cost.
H.4. A DDITIONAL K-C ONSENSUS E XPERIMENTS
In Table 25, we present a more detailed version of Table 5.
We emphasize that for K = 5 and K =for ResNet110 and ResNet20 , respectively, we achieve the same ACR as with the full ensemble while reducing the certiﬁcation time by a factor of 1 . 59 and 2 . 01 , respectively.
[Table] [Table]
[Table]
[Table] [Table]
[Table] [Table]
α
[Table]
α
[Table]
(a)
[Table]
[-= 0 .(a) 0 .0 .0 .
-][Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
of
[Table] β = [Table]
β = [-
 .β-][Table]
β = 0 .(b) 0 [-.0 .0 .
-].
[-Underlying Probability of Majority Class 0 .0 .0 .
-][Table]
[Table] [Table]
[Table]
[Table]
Underlying Probability
[Table]
Majority Class 0 .0 .0 .0 .0 .1 .
Expected Probability of Continuing α = β = 0 .
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Figure] [Figure] [Figure]
[Table]