On the Perils of Cascading Robust Classifiers
Authors: Ravi Mangal
Zifan Wang
Chi Zhang
Klas Leino
Corina Pasareanu
Matt Fredrikson
Venue: ICLR 2023 poster (2023)
Keywords: Certifiable Robustness
Ensemble
Adversarial Attack
Soundness

Abstract: Ensembling certifiably robust neural networks is a promising approach for improving the \emph{certified robust accuracy} of neural models. 
Black-box ensembles that assume only query-access to the constituent models (and their robustness certifiers) during prediction are particularly attractive due to their modular structure. Cascading ensembles are a popular instance of black-box ensembles that appear to improve certified robust accuracies in practice. However, we show that the robustness certifier used by a cascading ensemble is unsound. That is, when a cascading ensemble is certified as locally robust at an input $x$ (with respect to $\epsilon$), there can be inputs $x'$ in the $\epsilon$-ball centered at $x$, such that the cascade's prediction at $x'$ is different from $x$ and thus the ensemble is not locally robust. Our theoretical findings are accompanied by empirical results that further demonstrate this unsoundness. We present a new attack against cascading ensembles and show that: (1) there exists an adversarial input for up to 88\% of the samples where the ensemble claims to be certifiably robust and accurate; and (2) the accuracy of a cascading ensemble under our attack is as low as 11\% when it claims to be certifiably robust and accurate on 97\% of the test set. Our work reveals a critical pitfall of cascading certifiably robust models by showing that the seemingly beneficial strategy of cascading can actually hurt the robustness of the resulting ensemble. Our code is available at https://github.com/TristaChi/ensembleKW.

Published as a conference paper at ICLR 2023
ON THE PERILS OF CASCADING ROBUST CLASSIFIERS
Ravi Mangal', Zifan Wang", Chi Zhang" Electrical and Computer Engincering
Carnegie Mellon University Pittsburgh, PA 15213 (rmangal, z zifanw, chiz5) @awdrew.cmu.ee.
Klas Leino
Corina Pasäreanu Carnegie Mellon University and NASA Ames Moffett Field, CA 94043 pcor pcorinale moorin@@andree emu.edi
Matt Fredrikson
School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 kleino@cs, cmu edu
School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 mfredri 1 cmu edu
ABSTRACT
Ensembling certifiably robust neural networks is promising approach for improving the certified robust accuracy of neural models. Black-box ensembles that assume only query-access to the constituent models (and their robustness certifiers) during prediction particularly attractive due their modular structure. Cascading ensembles are a popular instance of black-box ensembles that appear to improve certified robust accuracies in practice. However, we show that the robustness certifier used by cascading ensemble is unsound. That is, when cascading ensemble is certified as locally robust at an input x (with respect to c), there can be inputs x' in the e-ball centered at 2, such that the cascade's prediction at x' is different from and thus the ensemble is not locally robust. Our theoretical findings are accompanied by empirical results that further demonstrate this unsoundness. We present cascade attack (Cas.A), adversarial attack against cascading ensembles, and show that: (1) there exists an adversarial input for up to 88% of the samples where the ensemble claims to be certifiably robust and accurate; and (2) the accuracy of a cascading ensemble under
attack is as low 11 % when it claims to be certifiably robust and accurate on 97% of the test set. Our work reveals a critical pitfall of cascading certifiably robust models by showing that the seemingly beneficial strategy of cascading can actually hurt the robustness of the resulting ensemble. Our code is available at ht https: //////// /githhb...../... com Tr i sesissaahhhi aChi www.chhheeemmeeeie.aeae
INTRODUCTION
Local robustness has emerged as important requirement of classifier models. It ensures that models are not susceptible misclassification caused by small perturbations to correctly classified inputs. lack of robustness can be exploited by not only malicious actors (in the form of adversarial examples (Szegedy et al., 2014)) but can also lead incorrect behavior in the presence of natural noise (Gilmer at., 2019). However, ensuring local robustness of neural network classifiers has turned out be hard challenge. Although neural networks can achieve state-of-the-an classification accuracies on variety of important tasks, neural classifiers with comparable certified robust accuracies' (CRA, Def. 2.2) remain elusive, even when trained in a robustness aware manner (Madry et al., 2018; Wong & Kolter, 2018; Cohen al., 2019; Leino al., 2021). In light of the limitations of robustness-awar training, en- sembling certifiably robust neural classifiers has been shown to be a promising approach for improving certified robust accuracies (Wong et al., 2018; Yang et al., 2022). An ensemble combines the outputs of multiple base classifiers to make a prediction, and is well-known mechanism for improving classi- fication accuracy when one only has access to weak learners (Dietterich, 2000; Bauer & Kohavi, 1999).
Ensembles designed to improve CRA take one of two forms. White-box ensembles (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) assume white-box access to the constituent models. They
"qqul Contribution
Percentage of inputs where the classifier is accurate and certified as locally robust.

---

Published as a conference paper at ICLR 2023
Input
Constituent Models
(a)
(b)
(c)
Ensembles
imppl Made
(d)
Figure 1: Visualizing classification results of 2D points for constituent models (a-c) and the corre- sponding Cascading Ensemble (d, Def. 2.7) and Uniform Voting Ensemble (e, Def. 5.3). Regions with colors correspond to predictions (0: red, 1: blue, 2: green) made by the underlying model (or ensemble). Darker colors indicate that the accompanying robustness certification of the underlying model (or ensemble) returns 1 and lighter colors for cases when the certification returns O. All points receiving for certifications (darker regions) are at least c-away from the other classes in (a)-(c), i.e. certification sound (Def. 2.3). This property is violated in (d), e.g. points from dark red regions are not e-away from the blue region in the zoomed-in view on the left, but preserved in (e). Namely, voting ensembles are soundnesx-preservee (Def. 2.6) while cascading ensembles are not.
calculate new logits by averaging the corresponding logits of the constituent classifiers. For local robustness certification, they treat the ensemble as single, large model and then use off-the-shelf techniques (Cohen et al., 2019; Weng al., 2018; Wong & Kolter, 2018; Zhang et al., 2018) for certification. Black-box ensembles (Wong et al., 2018; Blum et al., 2022), on the other hand, assume only query-access the constituent classifiers during prediction, and are, therefore, agnostic their internal details. They re-use the prediction and certification outcomes of the constituent models to calculate the ensemble's 's prediction and certificate. Their black-box nature lends them modularity and permits any combination of constituent classifiers, irrespective of their individual certification mechanism, so we focus our efforts on them in this paper.
Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are particularly popular instance of black-box ensembles that appear improve CRA in practice. They evaluate the constituent classifiers (and their certifiers) in a fixed sequence. The ensemble's prediction is the output the first constituent classifier in the sequence that is certified locally robust, defaulting to the last classifier's 's output if no model can be certified. Importantly, the cascading ensemble itself certified locally robust only when at least one of the constituent classifiers is certified locally robust.
Our contributions. We show in this paper that the local robustness certification mechanism used by cascading ensembles unsound even when the certifiers used by each of the constituent classifiers are sound (Theorem 2.8). In other words, when a cascading ensemble is certified as locally robust at an input there can, in fact, be inputs x' in the e-ball centered at ær such that the cascade's prediction x' is different from x. Figure demonstrates this visually on a toy dataset. The cascading ensemble can have points that are less than e away from the decision boundary, yet the ensemble is certified locally robust at such points (Figure (d))). As consequence of our result, use of a cascading ensemble any scenario requiring local robustness guarantees is unsafe and existing empirical results that report the CRA of cascading ensembles are not valid.
Guided by theoretical construction, propose cascade attack (CasA, Algorithm 3.1), an adversarial attack against cascading ensembles, and conduct an empirical evaluation with the cascading ensembles trained by Wong et al. (2018) for MNIST and CIFAR- 10 datasets. With CasA, we show that: (1) there exists adversarial example for up 88% of the samples where the ensemble claims
2

---

Published as a conference paper at ICLR 2023
to be certifiably robust and accurate; (2) the empirical robust accuracy of a cascading ensemble is as low as 11% while it claims to be certifiably robust and accurate on 97% of the test set; and (3) viewing all experiments as a whole, the empirical robust accuracy of a cascading ensemble is almost always lower than even the CRA of the single best model in the ensemble. Namely, a cascading ensemble is often less robust. Our results conclusively demonstrate that the unsoundness of the cascading ensemble certification mechanism can be exploited in practice, and cause the ensemble to perform markedly worse than the single best constituent model.
We also present an alternate ensembling mechanism based on weighted voting that, like cascading ensembles, assumes only query-access to the constituent classifiers but comes with provably sound local robustness certification procedure (Section 5). We show through thought experiment that it is possible for a voting ensemble to improve upon the CRA of its constituent models (Section 5.2), and observe that the key ingredient for the success of voting ensembles is a suitable balance between diversity and similarity of their constituents. We leave the design of training algorithms that balance the diversity and similarity of the constituent classifiers as future work.
CASCADING ENSEMBLES
In this section, we introduce our notation and required definitions. We then show the local robustness certification procedure used by cascading ensembles is unsound.
2.1 CERTIFIABLE CLASSIFIERS AND ENSEMBLERS
We begin with our notation and necessary definitions. Suppose neural network f: -R" takes an input and outputs the probability of m different classes. The subscript xj denotes the j-th element of a vector x. When discussing multiple networks, we differentiate them with superscript, e.g. , Throughout the paper we use the upper-case letter F to denote the prediction of f such that F(x)= argmax,e>/ (x)) where fj is the logit for class j and y = [m]]² 2 The prediction F F(x) is considered e-locally robust at x if all neighbors within an e-ball centered at a receive the same predictions, which is formally stated in Def. 2.1.
Definition 2.1 (c-Local Robustness). A network, F, is c-locally robust at x w.r.t norm, || ||, if Vx' || = S = F(x))
Though local robustness certification of ReLU networks is NP-Complete (Katz et al., 2017), due to its importance, the problem has been receiving increasing attention from the community. Proposed certification methods rely variety of algorithmic approaches like solving corresponding linear (Jordan et al., 2019) or semi-definite programs (Raghunathan et al., 2018), interval propaga- tions (Gowal et al., 2018; Lee et al., 2020; Zhang et al., 2018 18), abstract interpretation (Singh et al., 2019a), geometric projections (Fromherz et al., 2021), 1), dual networks (Wong & Kolter, 2018), or Lipschitz approximations (Leino et al., 2021; ; Weng et al., 2018).
If certification method provided for a network F. we use Fe: Rd y x 1) to denote a certifiable neural classifier that returns prediction according to F and the outcome of the certification method applied to F with respect to robustness radius We use Fiabe (z) to refer to the prediction and (x) to refer to the certification outcome. If Fx cen (1) = 0, the accompanying robustness certification unable certify F (i.e., Faabe) as e-locally robust at æ. When is clear from the context, we directly write F. One popular metric to evaluate the performance of any F is Certified Robust Accuracy (CRA).
Definition 2.2 (Certified Robust Accuracy). The certified robust accuracy (CRA) of certifiable given dataset Sy C Rdd x y with k samples is given by
classifier F E I y (0. CRA(F.S)):=10 := (E..))s. (x.))(.,,11)
For F and its CRA to be useful in practice without providing false robustness guarantees, it must be sound to begin with (Def. 2.3).
Definition 2.3 (Certification Soundness), A certifiable classifier, F:R. + x (0.1), is sound V Foon(x)= =1 = Flabel is c-locally robust at x.
mm] := (0:0..,..... (0.1,....m
3

---

Published as a conference paper at ICLR 2023
Notice that if there exist c-close inputs z,æ' where F(x)= and F(x')=(()) (v',0), where y v'. then it still means that F is not sound. We define ensembler (Def. 2.4) as function that combines multiple certifiable classifiers into a single certifiable classifier (i.c., an ensemble).
Definition 2.4 (Ensembler). Let = : + x {0.1} represent the set of all certifiable classifiers. An ensembler : FN F is function over N certifiable classifiers that returns certifiable classifier.
A query-access ensembler formalizes our notion of box ensemble.
Definition 2.5 (Query-Aceess Ensembler). Let (===(xxx) (0.1)) +V (0.1). & is query-access ensembler if, V F() F...)) eF " 3G VrER"
El fr0)) F ,.... F..-)) (x) G ((F (x), (t)
Def. 2.5 says that if & query-access its output F can always be re-written a function over the outputs of the certifiable classifiers ((0) F(1) ,.... F(N-1),
((N)))..((NNI)) Put differently, F only has black-box or query-access to classifiers F()) ((N-1)
Finally, soundness-per prressrvsnnp ensembler (Def. 2.6) ensures that if the constituent certifiable classifiers are sound (as defined in Def. 2.3), the ensemble output by the ensembler is also sound.
Definition 2.6 - Ensembler). An ensembler & is soundness soundness-peeserving if, VF(0), F(I) F(N e F(0) FFI) ,..., F(N are sound = {(F()) ((..,.((--1) is sound.
2.2 CASCADING ENSEMBLER Is NOT SOUNDNESS- PRESEESINI
Cascading ensembles (Wong et al., 2018: Blum et al., 2022) are a popular instance of black-box ensembles that appear to be practically effective in improving certified robust accuracies. However, show that cascading ensembles are not sound.
We define a cascading ensemble be the output of cascading ensembler (Def. 2.7). A cascading ensemble evaluates its constituent certifiable classifiers in a fixed sequence. For given input v, the ensemble either returns the prediction and certification outcomes of the first constituent classifier FG) such that F(J) be certified. Clearly, cascading ensemblers are query-access (formal proof Appendix A).
of the last constituent classifier in case none of the constituent classifiers can
Definition 2.7 (Cascading Ensembler). Let ((0) F()) "F.,....FFNNE F(N-1) be N certifiable classifiers. A
cascading ensembler & :::FN. + FFi is defined as follows
Ec (T:0)) fo(1) ,..., .(.,))()N1)) (x) FONN- (x)
(z)
ifj c(j=
otherwise
where c(j):= if ((1) cen (x)==) 1) and (Ai <.j cen (x) ((
0), and otherwise.
Theorem 2.8 shows that cascading ensemblers are not moundness-preservinn and so a cascading ensemble can be unsound. We show this by means of counterexample. Theorem 2.8. The cascading ensembler &c is not soundness-preservvee
Proof. We can re-write the theorem statement as, 37(0), f(1)) (N- E F such that for F: F:=Ec(F()) ((N-)) Red
+ Flabel is e-lccally robust at æ.
We prove by constructing the following counterexample. Consider cascading ensemble F constituted of certifiable classifiers ((0) and F(1) F(() and ((1) are such there exists an r where
( (I) (x)= ^ (z)
(1)
Using Def. 2.7, it is true that F(x) (y, Without violating (1), we can have another point x'
such that,
'The use of cascade of certification methods for single classifier in (Gowal et al., 2018; Singh al., 2019h) is orthogonal and sound.
4

---

Published as a conference paper at ICLR 2023
Algorithm 3.1: Cascade Attack (CasA)
models Inputs: Ensemble FE constituunt models plo), F(N-1) E F,, input E Rd, attack bound and distance metric Ep
Output: An attack input x' ER"
Attack (F,
P(N-1) epl
y:= Flabel (x)
idxs = |
(O)
(x) (
scel
foreach iE idxs do
if = N - then
L ( one-hot(y)) + > 101-Foot (x+ (x+)))
+ argmax (0,e(0,e)
else
F(o)
Focl (x+6) + (1- (z++))
+ argmax (0.c)
kci
F sbar x 4y then return x."
10
return
(1l=~2|1<e) (z') ()',1) A (F') ( ^ (e' Fy)
Using Def. 2.7, it is true that F(x') (v Thus, for two points x' constructed above, show that 3r,,', s.t. || -x1 x|| e, Foen (x) =1 + Flabel (x') Flabel which violates the condition of local robustness (Def. 2.1).
The counterexample constructed in Thm. 2.8 is not just hypothetical, but something that materializes on real models (see Figure for a toy example and Section 4 for our empirical evaluation).
ATTACKING CASCADING ENSEMBLES
Section 2.2 shows that cascading ensemble does not provide robustness guarantee. We further show here how one can attack the cascading ensemble and find an adversarial example within the e-ball centered at the input v.
Overview of A Attack. Algorithm 3.1 describes the attack algorithm, cascade attack (CasA), inspired by the proof of Theorem 2.8. Given an input æ, the goal of the algorithm to find an input x'' in the e-ball centered at such that the predictions of the cascade at x and x' are different. The inputs to the algorithm are an ensemble É, its constituent classifiers fi(0)
((N-1), the input æ to be attacked, well as distance metric Ep. The algorithm either returns successful
and the attack distance bound adversarial input x' such that | =/'lp Se and Fiabel (x) Fia
returns the original input a if no adversarial input was found. We use the following notations: e stands for gross-entropy loss, mee-oot is the one-hot encoding function, and (0,e) is the ba-ball of radius centered at O.
Preparing Targets. CasA gets the label predicted by the ensemble F input (line 2) to select the constituent models may attack. The attacker only interested a constituent model (by remembering its index i) if predicts a label other than at I or it is the last one (line 3). We are not interested in attacking model F(i) that predicts y at because such an effort is bound to fail. FG) is still sound even though the ensemble not; therefore, no point assigned label other than y by p() is such that is both less than e-away from x and ((0) is also certifiably robust at (the second condition is necessary for P() to be used for prediction x' by the ensemble). However, the last model F(N-I) is exception and always remembered, i.e. i idxs always includes N -. The reason is that, given an input z', if all models F() < N - fail to be certifiably robust at x', the ensemble uses ((N-1) for prediction at x' irrespective of whether (NN-1) itself certifiably robust at or not.
*In implementation,
surrogate version of this objective (see Section 3).
5

---

Published as a conference paper at ICLR 2023
Attacker's Steps. For each model index in i idxs, we try find adversarial example (lines 4-10). An attacker stops as soon as they find a valid adversarial example (lines 9-1 Lines 6 and 8 describe the objective an attacker minimizes to find the adversarial examples. If index i N - 11, the attacker optimizes 6 such that, at input x + 8, the model F() is certified robust whereas all other models F(();k < i are not certified robust. This ensures that model FP() is used for prediction at input x a-+ ó as it is certifiably robust at æ. If index i = N - 1, we still require that all models F(K); < i are not certified robust at 2 00-- But instead of requiring that F() is certified robust at x + 6. we only require that the predicted label at z ô be different from y. We solve the optimization problems on lines 6 and using projected gradient descent (PGD) (Madry et. 2018).
cent (I++)) , is differenttlell or too expensive to run multiple times, provide the following cheap surrogate replacements. The intuition underlying the surrogate versions is that, given model, the distance to the decision boundary from an input is correlated with the margin between the top logit scores of the model at that input. To use the surrogate objectives for the attack, need to assume access to the logit scores of the models.
Surrogate Objectives. For cases when the certification procedure, i.e. ((()
For the problem argmax$eB,,CC (O.c) F(A) (z 8), we try to increase the logit score associated with the desired prediction as much as possible. Then, a surrogate version of the problem is as follows (where ((()) represents the logit scores produced by model F())):
llogn
argmax- (F (1+ one-hot(Fial (i())
label (z)))
(3)
(0,c)
For the problem argmaxseR, (O.c) Eke /e (2r + 8)), we want the input + ô to be as close as possible to the decision boundaries for each of the models by F(k), ,k < i so that the robustness certifications will fail. The specific predictions Fi (x++)) of these models do not matter. Towards that end, we aim to make the margin between the logit scores of any model (() be as small as possible. This leads to the following surrogate problem (where unif is a discrete uniform distribution):
Ecoe (Fiogh (x+ unif)
argmax- (0,e) kec
4 EMPIRICAL EVALUATION
The goal of our empirical evaluation is demonstrate the extent which the unsoundness of the cascading ensembles manifests in practice and can be exploited by an adversary, i.e. CasA. For our measurements, we use the Eo and Eg robust cascading ensembles constructed by Wong et al. (2018) for MNIST (LeCun et al., 1998) and CIFAR- 10 (Krizhevsky, 2009) datasets. The constituent classifiers in these cascades use local robustness certification mechanism based on dual networks (Wong et al., 2018). Each cascade includes between 2-7 certifiable classifiers with the same architecture (except for the &o robust, CIFAR- 10 Resnet cascades that include only a single constituent model, and are hence not considered in our evaluation). The training code and all the constituent models in the ensembles made available by Wong et al. (2018) 8) in a public repository (Wong& & Kolter).
We report the certified robust accuracy (CRA) and standard accuracy (Acc) for the cascading ensemble as well as the single best constituent model in the ensemble. While the certifier for single model is sound, the ensemble certifier is unsound and the reported ensemble CRA is over-estimate. We therefore measure the empirical robustness of the ensemble under CasA. Certifying with dual networks (Wong et al., 2018) is differentiable but extremely expensive. To run the attack more efficiently, we use the surrogate replacements in Section and take 100 steps using PGD (other hyper-pane parameters to follow in Appendix B) to empirically minimize the objectives. After the attack, we report the false positive rate (FPR), i.e. & of test inputs for which an adversarial example is found within the c-ball, and the empirical robust accuracy (ERA), i.e. of test set where the cascade is empirically robust (i.e., our attack failed). All our experiments were run on pair of NVIDIA TITAN RTX GPUs with 24 GB of RAM each, and 4.2GHz Intel Core i7-7700K with 64 GB of RAM.
Table shows the results for t. robustness (top) and & robustness (bottom). Each row in the table represents a specific combination of dataset (MNIST or CIFAR- architecture (Small or Large convolutional networks), and value used for local robustness certification. The structure of the table is the same as Tables and 4 in (Wong et al., 2018), except we add the columns reporting FPR and ERA.
We see from Table I that, irrespective of the dataset, model, value, or lp metric under consideration, our attack can find false positives, with false positive rates (FFR) as
Summary of Results.

---

Published as a conference paper at ICLR 2023
Table 1: Results models pre-trained by Wong et. (2018) for to (top) and lg (bottom) robustness.
CRA: % of test set where model certified robust and accurate. Acc: % of test set where model is accurate. FPR: among all test inputs where cascade is certified robust and accurate, % of inputs for which an adversarial example is found within the e-ball using our ensemble attack (i.e., false positive rate). ERA: % of test set where the cascading is empirically robust (i.e., our attack failed) and accurate (ERA of a single model is always greater equal to its CRA because of soundness and therefore not included). The unsoundness of cascade certification is shown by the high false positive rates (FPR).
Single Model Acc(%)
Cascading Ensemble FPR(%)
Dataset
Model
CRA(%)
unsound CRA(S)
Acc(%)
SRA %b)
MNIST
Small, Exact
0.1
95.54
98.96
96.33
88.71
96.62
11.17
MNIST
Small
0.1
94.94
98.79 79
96.07
81.93
96.24
17.51
MNIST
Large
0.1
95.55
98.81
96.27
86.37
96.42
13.27
MNIST
Small
0.3
56.21
85.23
65.41
88.87
65.80
7.67
MNIST
Large
0.3
58.02
88.84
65.50
85,27
65.50
9.65
CIFARIO
Small
2/255
46.43
60.86
56,65
11.51
56.65
50.13
CIFARIO
Large
2/255
52.65
67.70
64.87
10.47
65.13
58.15
CIFARIO CIFARIO
Small
8/255
20.58
27.60
28.32
16.00
28.32
23.79
Large
8/255
16.04
19.01
20.83
17.67
20.83
17.15
E2
Single Model CRA(S))
Cascading PPR(%)
Dataset
Model
Acc(%)
CRA(
Acc(%)
RRA(%)
MNIST
Small, Exact
1.58
43.52
88.14
75.58
44.72
80.43
43.46
MNIST
Small
1.58
43.34
87.73
74.66
40.93
79.07
45.73
MNIST
Large
1.58
43.96
88.39
74,50
51.95
74.99
35.81
CIFARIO
Small
36/255
46.05
54.39
49.89
3.61
51.37
49.27
CIFARIO
Large
36/255
50.26
60.14
58.72
2.70
58.76
57.17
CIFARIO
Resnet
36/255
51.65
60.70
58.65
3.41
58.69
56.68
Table 2: Run time and peak memory usage of CasA. Results are reported on one Titan RTX.
Dataset
Model
of Models
Objective
unsound CRA(%)
(B2(%)
Batch Size
Mem//Bach (MB)
Time/Batch (Min)
MNIST
Small
0.3 (f
dual networkks
4.9%
32
12565
4.34
MNIST
Small
7
(f.))
our sunogates
32
CIFARIO CIFARIO
Small
5
2/255 (C.,)
dual networks
28.32
23.79
32
10124
2.43
Small
5
2/255
our surrogattes
28.32
22.92
32
4898
0.06
high as 88.87%. In other words, there always exist test inputs where the ensemble is accurate and declares itself to be certified robust, but our attack is able to find an adversarial example. This result demonstrates that the unsoundness of the cascading ensemble certification mechanism not just problem in theory but it can be exploited by adversaries in practice. More strikingly, the empirical robust accuracy (ERA) of the ensemble is often significantly lower than the certified robust accuracy (CRA) the best constituent model. Since the ERA of a model is an upper-bound actual CRA of the ensemble can be no larger than the reported ERA. This result shows that the use of cascading ensemble can actually hurt the robustness instead of improving it. Note that there are small differences between the Acc and CRA numbers reported in Table and those in ( (Wong et al., 2018). Though we use the evaluation code and pre-trained models made available by Wong et al. (2018), the hardware and PyTorch versions we use in our experiments are different.
its CRA, the
Attack Efficiency. In Table 2, we compare the attack results of CasA using the original objectives, iee. dual networks (Wong et al., 2018), and using surrogate replacements. Because the ensemble on MNIST contains more constituent models, it uses more memory with dual networks compared to CIFAR10. Our report of run time and memory usage shows that using surrogate replacements allows us to run attacks with larger batch size, less memory and time to reach the same level of performance.
A QUERY-ACCESS, SOUNDNEESS-PRESERVINNG ENSEMBLER
We present query-access, soundness-preservvee preserving ensembler based on weighted voting in this section. Voting a natural choice the design of query-access ensembler ensuring that the ensembler soundness-presse preserving can be subtle. Section 5. defines our ensembler and proves that is soundness- preserving. In Section 5.2, we present thought experiment demonstrating that it possible for voting
7

---

Published as a conference paper at ICLR 2023
ensemble to significantly improve upon the CRA its constituent models. Appendix D describes our al- gorithm for learning the weights to be used in weighted voting, and Appendix presents initial empirical results with the voting ensemble. Our results show that improving the CRA via a voting ensemble can be difficult realistic datasets since it requires the ensemble to demonstrate suitable balance between di- versity and similarity its constituents, but we believe that this a fruitful direction for future research.
5.1 WEIGHTED VOTING ENSEMBLE
Voting ensembles run a vote amongst their constituent classifiers to make a prediction. In the simplest case, each constituent has a single vote that gets assigned to their predicted label. The label with the maximum number of votes is chosen as the ensemble's s prediction. More generally, weighted voting allows a single constituent to be allocated more than one vote. The decimal number of votes allocated to each constituent is referred to as its weight. For simplicity, we assume that weights of the constituents in an ensemble are normalized so that they sum up 1. We use weighted voting to not only choose the ensemble's prediction but to also decide its certification outcome. The interaction between voting and certification is subtle and needs careful design to ensure that the certification procedure is sound.
Extra Notations. Let of (j,c) denote the total number of votes allocated to certifiable classifiers, F(0, in the ensemble that output (j.c). More formally, for imput label j y. certification outcome 1]. and set of constituent certifiable classifiers, F((), ,.... F(N-1) ((N",,,.FFFF. let "h (j,c) = Σx w, * I [ (x) = (ie)). We find useful to use vn (*,c) to denote the number of votes for any class with certificate, c; i.e., v/ (+,c) = Jey o (j,z). Likewise, we will use oz (j) to denote the number of votes for class j regardless of certificate; i.e. 2/ (f)== = o/r (j.0)+ (j.1).
C {0, 10,11..ww weightt
Definition 5.1 (Weighted Voting Ensembler). Let P()),... F(N) be N certifiable classifiers. A weighted voting ensembler, EV IFN - F is defined as follows
E( (F)(0) fo()) P (z) =
: =aaagaa) argmaax ()) j
{
i (la otherwise
1.2 (Fabe of (x,0)+ (x,00) -. (.1)
where Feen (z)
0
The prediction of the weighted voting ensemble is the label receiving the maximum number of votes regardless of the certificate. However, for the certification outcome, the ensemble has to consider the certificates of the constituent models. The ensemble should be certified robust only if its prediction outcome, i.e., the label receiving the maximum number of votes (regardless of the certificate). can be guaranteed not change in an e-ball. The condition under which (r) 1 ensures this the case, and allows us to prove that weighted voting ensemblers are soundness- soundness-preeevvig preserving (Theorem 5.2). A key observation underlying the condition is that only constituent classifiers that are not certified robust at the current input can change their predicted label in the e-ball, and, in the worst case, transfer all their votes (*."(.00 (+,0) to the label with the second highest number of votes at æ. We believe that our proof of soundness preservvaionnnnnonn is of independent interest. We also note that weighted-voting ensemblers are query-access access (formal proofs in Appendix A).
Theorem 5.2. The weighted voting ensembler EYy is srandddess--reeerving
Definition 5.3 (Uniform Voting Ensembler). Let fr() ((N-))) be N certifiable classifiers. The uniform voting ensembler, &g: ; i.N - f is weighted voting ensembler that assigns equal weights to each classifier, i.e. Eい Ey where Vi (......--1l.uu 1/N.
5.2 EFFECTIVENESS OF VOTING: A THOUGHT EXPERIMENT
Voting ensembles require the constituents to strike the right balance between diversity and similarity to be effective. In other words, while the constituents should be accurate and robust in different regions of the input space (diversity), these regions should also have some overlap (similarity). We conduct a thought experiment using simple hypothetical example (Example. 5.4) where such a balance is struck. The existence of this example provides evidence and hope that voting ensembles can improve the CRA.We present the example informally here. The detailed, rigorous argument is in Appendix A.
In case of tic, we assume that the lahel corresponding to the logit with the lowest index returned.
8

---

Published as a conference paper at ICLR 2023
Example 5.4. Assume that have uniform voting ensemble F with three constituent classifiers F(0), F(), and F+22. Assume that on given dataset Sş with 100 samples, each of the constituent classifiers has CRA equal to 0.5. Let's say that the samples in Sy. are ordered such that F is accurate and robust on the first 50 samples (i.e., samples 0-49), F(1) is accurate and robust on samples 25-74, and (22 on samples 0-24 and 50-74. Then, for each of the first 75 samples, two out of three constituents in the ensemble are accurate and robust. Therefore, by Def. 5.1, the ensemble F is accurate and robust on samples 0-74, and has CRA equal to 0.75.
RELATED WORK
Ensembling is well-known approach for improving the accuracy of models long as the constituent models are suitably diverse (Dietterich, 2000). In recent years, with the growing focus on robust accuracy as metric of model quality, a number of ensembling techniques have been proposed for improving this metric. Depending on whether ensemble is query-access or not (i.e., does not or does require access to the internal details of the constituent models for prediction and certification). it can be classified as a white-box or black-box ensemble. The modularity of black-box ensembles is attractive as the constituent classifiers can each be from a different model family (i.e., neural networks, decision trees, support vector machines, etc.) and each use a different mechanism for robustness certification. The constituents of white-box ensembles, the other hand, tend to be from the same model family but this provides the benefit of tuning the ensembling strategy to the model family being used.
White-box ensembles. Several works (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) present certifiable ensembles where the ensemble logits are calculated by averaging the corresponding logits of the constituent classifiers. Needing access to the logits of the constituent classifiers, and not just their predictions, is one aspect that makes these ensembles white-box. More importantly, the approaches used by these ensembles for local robustness certification are also in violation of our definition of query-access ensembles (Def. 2.5). For instance, randomized smoothing (Cohen et al., 2019) is used in (Yang et al., 2022; Liu et al., 2020) to certify the ensemble, which requires evaluating the constituent models on a large number of inputs for each prediction, and not just one input. Other approaches (Zhang et al., 2019) use interval bound propagation (IBP) (Gowal et al., 2018; Zhang et al., 2018) to certify the ensemble. Calculating the interval bounds requires access to the architecture and weights of each of the constituent models. violating the requirements of query-access ensemble. A number of white-box ensembling techniques (Pang et al., 2019; Yang et al., 2020; Kariyappa & Qureshi, 2019; Sen et al., 2020; Zhang et al., 2022) only aim improve empirical robust accuracy, i.e., these ensembles do not provide robustness certificate. As before, the ensemble logits are calculated by averaging the corresponding logits of the constituent models. These approaches differ from each other in the training interventions used to promote diversity in the constituent models.
Black-bor ensembles. Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are the most popular example of certifiably robust black-box ensembles. While Wong et al. (2018) empirically evaluate their cascading ensemble, the results of Blum et al. (2022) are purely theoretical. However, as we show in this work, the certification mechanism used by cascading ensembles is unsound. Devvrit et al. (2020); Sen et al. (2020) present black-box voting ensemble but, unlike our voting ensemble, their ensemble does not provide robustness certificates. Nevertheless, they are able to show improvements in the empirical robust accuracy with the voting ensemble.
7 CONCLUSION
In this paper, we showed that the local robustness certification mechanism used by cascading ensembles is unsound. As consequence, existing empirical results that report the certified robust accuracies (CRA) of cascading ensembles are not valid. Guided by our theoretical results, we designed an attack algorithm against cascading ensembles and demonstrated that their unsoundness can be easily exploited in practice. In fact, the performance of the ensembles is markedly worse than their single best constituent model. Finally, we presented an alternate black-box ensembling mechanism based on weighted voting that we prove to be sound, and, via a thought experiment, showed that voting ensembles can significantly improve the CRA if the constituent models have the right balance between diversity and similarity.

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
Our work sheds light on existing vulnerabilities state-of-the-art certifiably robust neural classifiers. The presented attacks can be used by malicious entities to adversarially attack deployed cascading ensembles of certifiably robust models. However, by putting this knowledge and making practitioners aware of the existence of the problem, we hope that precautions can be taken to protect existing systems. Moreover, it highlights the need to harden future systems against such attacks.
the public domain
REPRODUCIBILITY STATEMENT
To examine our theoretical results, the proof of Theorem 2.8 directly follows the body of the theorem in Section 2.2 while the proof of Theorem 5.2 is delayed Appendix A and F, together with the proof's of other theorems that only appear in the appendix, i.c. Theorem A. A.2 (Appendix A) and Theorem F.2 (Appendix F). All the datasets used in our work are publicly available with links in their corresponding reference. Our experimental code is uploaded in the supplementary material (and also at hstps: / /pss://gttttttt /githbu...... com/Tri /wrrrsrss attssttahhiCAA wichhheenembbaaee e eK) with a detailed README file and weights of models to reproduce the results in Table 1, 5, 6, 7, and 8. Moreover, hyper- parameters used in these table are also documented in Appendix and D. The hardware information used in all experiments is reported in Section 4.
ACKNOWLEDGEMENTS
We would like to thank the reviewers for their comments which helped us improve this article. The work described in this paper has been supported by the Software Engineering Institute under its with the U.S. Department of Defense, DARPA and the Air Force Research Laboratory under agreement number FA8750-15---0277, as well as DARPA GARD Contract HR001 12020006.
FERDC Contract No. FA8702-15-D-ON
REFERENCES
Eric Bauer and Ron Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36()):10511)13 1999.
Avrim Blum, Omar Montasser, Greg Shakhnarovich, and Hongyang Zhang. Boosting barely robust learners: new perspective on adversarial robustness. arXiv preprini arXiv:2202.06 05920, 2022.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via ran-
domized smoothing, In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Pro- ceedings of the 36th International Conference on Machine Learning, volume 97 of Pro-
ceedings of Machine Learning Research, pp. 1310-1320. PMLR, 09-15 Jun 2019. URL v27 cohen1 html..
https:/ htppss//proceedinge
Devvrit, Minhao Cheng, Cho-Jui Hsieh, and Inderjit S. Dhillon. Voting based ensemble improves robustness of defensive models. ArXiv, abs/2001..4403...
Thomas G. Dietterich. Ensemble methods in machine learning In Multiple Classifier Systems, pp. 1-15, Berlin, Heidelherg, 2000. Springer Berlin Heidelberg. iSBN 978-3-540-455
Aymeric Fromherz, Klas Leino, Matt Fredrikson, Bryan Parno, and Corina Päsäreanu. Fast geometric projections for local robustness certification. In International Conference on Learning Representations (ICLR), 2021.
Justin Gilmer, Nicolas Ford, Nicholas Carlini, and Ekin Cubuk. Adversarial examples are natural consequence of eπrо in noise. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2280-2289, PMLR, 09-15 Jun 2019. URL .ll...r.....
htttps:/pproceedinn www.:sswww........eeeeememmereee wwwpss/vv...... mmer html. htmml.
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv: 1810. 127/5, 2018
10

---

Published as a conference paper at ICLR 2023
Matt Jordan, Justin Lewis, and Alexandros G Dimakis. Provable certificates for adversarial examples: Fitting a ball the union of polytopes. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: httpps////p/ccciinggg.s... paper/2019 ott.e///eeeeeeooooeeeeeeeeeddddd
www.nng....uuii.... neurips. cc/ pdf.
Sanjay Kariyappa and Moinuddin K Qureshi. Improving adversarial robustness of ensembles with diversity training. arXiv preprint arXiv: 1XI.:v:99100999/ 2019.
Guy Katz, Clark Barrett, David Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: efficient smi solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pp. 97-117. Springer, 2017.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 36(11):2278-2324, 1998.
Sungyoon Lee, Jaewook Lee, and Saerom Park. Lipschitz-certffiable training with a tight outer bound. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 16891-16902. Curran Associates, Inc., 2020. URL file/c4 (e1.82225533742200bbe
neurips cc/paper/2020/ 920048e---eeeee Paper. paf.r.
Klas Leino, Zifan Wang, and Matt Fredrikson. Globally-robust neural networks. In International Conference on Machine Learning (ICML), 2021.
Chizhou Liu. Yunzhen Feng, Ranran Wang. and Bin Dong. Enhancing certified robustness via smoothed weighted ensembling. arXiv preprint arXiv:2005.09336 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2018.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970-4979. PMLR, 2019.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adver- sarial examples. International Conference on Learning Representations, 2018. URL .nt. https: http://pwananeeeeeeoeeoooooaaaaa
Sanchari Sen, Balaraman Ravindran, and Anand Raghunathan. Empir: Ensembles of mixed precision deep networks for increased robustness against adversarial at- tacks. In International Conference on Learning Representations, 2020. URL petps://cpenreee
Gagandeep Singh, Timon Gehr, Markus Püishell, and Martin Vechev. An abstract for certifying neural networks. Proc. ACM Program. Lang., (PPPL), January 2019a.
Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. Robustness certification with refinement. In International Conference on Learning Representations, 2019b.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference Learning Representations, ICLR2014, 2014, Banff. AB, Canada, April 14- 16, 2014, Conference Track Proceedings, 2014. URL http: / arxiv. org/ aaaa/asl11 6199 6199.
Lily Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, and Inderjit Dhillon. Towards fast computation of certified robustness relu networks. In International Conference on Machine Learning, pp. 5276-5285. PMLR, 2018.
Eric Wong and J Zico Kolter. adversarial.
URL https: http:/tubb..om//ooossaaaaa//asao
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adver- sarial polytope. In International Conference Machine Learning, pp. 5286 2286-5295. PMLR, 2018.
11

---

Published as a conference paper at ICLR 2023
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. Advances Neural Information Processing Systems, 31, 2018.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for enhanced robust generation of ensembles. Advances in Neural Information Processing Systems, 13:5505-5515, 2020.
Zhuolin Yang, Linyi Li, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, and Bo On the certified robustness for ensemble models and beyond. In International Conference on Learning Representations, 2022.
URL https:tpp......c / opwpreveeeeeeeeeeeeeeeeee net / Eorum? LaReLdddttUUAGGGG11
Dinghuai Zhang, Hongyang Zhang, Aaron Courville, Yoshua Bengio, Pradeep Ravikumar, and Arun Sai Suggala. Building robust ensembles via margin boosting. In Kamalika Chaud- huri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Proceedings of Machine Learning Research, pp. 26669-26692. PMLR, 17-23 Jul 2022. URL https: pro.eedincggg mlr...r.............. w.w..reess.ss..eeommmmm..em.m..e / v1 62/ zhang22a html.
Machine Learning, volume 162 of
Huan Zhang. Tsui- Weng, Pin- Yu Chen, Cho-Jui Hsich, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 4944-4953, Red Hook, NY, USA, 2018. Curran Associates Inc.
Huan Zhang, Minhao Cheng, and Cho-Jui Hsieh. Enhancing certifiable robustness via a deep model ensemble. arXiv preprint arXiv: 1910. 14655, 2019.
12

---

Published as a conference paper at ICLR 2023
A PROOFS
Theorem 5.2. The weighted voting ensembler Ey is soundness pressevvs--
Proof. Let F(0) F(N-1) be N certifiable classifiers, which we assume are sound. Let F = Ew (F F(N.)))) F given by Definition 5.1.
Assume for the sake of contradiction 3x,,' s.t. || - x' S. F(x) (i1,1), and Flabol j2 where j2 øJi-
Since ((x) = (ii, 1). by Definition 5.1., Vj ji. 2 (j 1) 1 (x,0) 0 (i,1, and thus, in particular, Equation 5 holds.
(1.1)>= (+,0) (j2.1)
Consider the votes on x', The models that contribute to v= (j1,1) are all locally robust at x. so each of these models must output the label f1 x', which distance no greater than from æ; thus Equation holds.
a ( 2 x (J1,1)
(6)
Conversely, only points that are non-robust x can change labels on x', thus we obtain Equation 7. (7) 2 u." (*.0) {+ (J2.1)
Putting things together we have va (f1) 2 ox (j1.1)
by (6)
ur (s,0)+ 2 (f2,1)
by (5)
2 2 (jz)
by (7)
Thus, since o (i1) v (j2), Flabol (x') cannot be j2 5.
Example 5.4. We want to show that, 3F(0), F() F() FO),.., P(N-1) 1) F, w E [0, 1 F = En ((0), F(I) F(N-1)),
such that for
CS X.. Vie (O........ CRA(F,SM) CRA(F() .Sk)
Consider a weighted voting ensemble F constituted of certifiable classifiers F(0), F(1), and F(2)
with weights = ( i.e. F is uniform voting ensemble.
Suppose = 100, i.e. S₃ is a dataset with 100 samples. Moreover, lets say that CRA(F ((0), Sk) CRA(F(1), Sx) CRA(F(2), Sy) = 0.5. Also, suppose that the samples in Sy are arranged in a fixed sequence S = (T. such such that,
vie [0,49). F()) (x.) (J1,1)
(8)
(25,74.7) F(1) (x;)= (y;,1) ViE [,244]]]
(y;,1)
(10)
where [i.]] is the set of integers from to j. and included. 8, 9, and 10 are consistent with the fact that certified robust accuracy of each model is 0.5.
From 8, 9, 10, and Definition 5.1,
vie [0,74], Flaber( (x:) argmax { 0", (1) -
(11)
Vie (0,74) Vj v. (1r, (+,0)+ (j.1)
(12)
From 12 and Definition 5.1,
Vi Forr].)
(13)
From 11, 13, and Definition 2.2,
CRA(F (5.) = 0.75
(14)
Theorem A.1. The cascading ensembler & is query-access.
TThe models known be locally robust because they sound and their output matches (+,1).
13

---

Published as a conference paper at ICLR 2023
Table 3: Hyper-parameters Used for CasA in Table 1
Dataset
Norm
Normalization
Max Steps
Step Size
MNIST
L00
0.1
[0.1]
0.004
MNIST
Eg0
0.3
[0.1]
100
0.012
MNIST
E2
1,58
[0.1]
100
0.03
CIFARIO
800
2/255
[0.485,
0.456,
0,406], 0.2225
100
0.0003
CIFARIO
8/255
(0.485 0.456, a
100
0.00124
CIFARIO
82
36/255
==[0.485, 0.456, 0.446),
100
0.0003
Proof. Let f= E= fa) , (.. Let o =)0 We use 92 refer
(N
the second element in the pair g Define G as follows,
G(g _g(2) ...... (.. = (
1 c(j)=1
g (N1- otherwise
where c(j) 1f (f 11/ (0) =0) and 0 otherwise.
Then, by Def. 2.7, F(x) = (z) F(N,,, (x)). Then, by Def. 2.5, cascading ensembler is query-access,
Theorem A.2. The weighted voting ensembler EVi is query-access.
Proof. Let F: E F(N-)))))))) Let (0
g ......
g (N EEE {0.1). We use & to refer
to the set (g g ......( (N-1)) Define G as follows,
G(g (0) . (N (Gi
N G2 (N.....((-1111
where
Gi (9) (1) g....g argmax 1/ (.+t)
{
ifjjj}
+
"2(g ..... (N)
otherwise
and
0y (i.))) Σ
w 9 -0)=
1=0
Then, by Def. 5.1, F(x)= = ensembler is query-access.
(
F(N-1) ((N...,.
Then, by Def. 2.5, weighted voting
B HYPER-PARAMETIE OF TABLE 1
In Table 3, report hyper- parameters used to CasA to reach the statistics reported in Table 1. Notice that if normalization [0.400 0.456, 0.406), a =0,225' we divide the and step size by a during the experiment. We use SGD as the optimizer for all experiments.
C ATTACKING
TRAINED CASCADING ENSEMBLES
Wong et al. (2018) train cascading ensembles in sequential manner, i.e., each model in the sequence only trained on those training samples that could not be certified robust by any of the previous models. The training algorithm is described by Wong et al. (2018) in appendix C (Algorithm 2) of their paper. We evaluate the efficacy of attack algorithm (CasA) on cascade ensembles trained in non-sequential manner. That is, each constituent model is trained independently on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training. Besides this difference, the code, architecture, hyperparameters, and data used for training are the same as that used by Wong et al. (2018). For every combination of dataset, architecture, and value, we train three constituent models, and them construct non-sequentially trained cascading ensembles.
Table shows the results of running our attack on such cascades with non non-sequentially trained constituents. We make the following observations:
14

---

Published as a conference paper at ICLR 2023
Table 4: Attack results non-sequentially trained cascade ensembles.
Dataset
Model
Ep
unsound CRA(%)
FPR(%)
Acc(%)
ERA(%)
MNIST
Exact
Loo
0.1
97.07
0.32
99.14
98.1
MNIST
Small
loo
0.1
96.25
0.27
98.69
97.4
MNIST
Small
foo
0.3
66.09
3.60
84.77
72.97
CIFARIO
Small
loo
2/255
55.53
6.43
62.05
54.59
CIFAR1O
Small
8/255
25.3
9.05
27.44
23.35
MNIST
Small,Exact
82
1.58
47.42
0.0002
87.78
73.56
MNIST
Small
L2
1.58
47.4
0.0
87.5
72.93
CIFARIO
Small
82
36/255
52.33
3.29
55.88
53.71
The non-sqquentially trained ensembles continue to be unsound and our attack is able to find adversarial inputs (demonstrated by non-zero FPR).
The success rate of our attack is much lower than on the sequentially-trann traineed models shared by Wong al. (2018).
The unsound CRA of these ensembles is comparable to that of the sequentially eequentially-trained -rained models.
We hypothesize that attack demonstrates much higher success rates on sequentially-train trrinnedd models because, when trained sequentially, it is likely that the later models in the cascade are degenerate, i.e., are very robust but with low accuracy (similar to constant functions). Then, to attack the ensemble, we just need to find an attack input where the initial models cannot be certified, since the remaining degenerate models are typically robust and inaccurate.
The degeneracy of the later models in the sequentially-traine ensemble may also explain why the un- sound CRAs of the two kinds of ensembles are comparable. Models are trained in sequential manner enhance their "diversity". However, due to the degeneracy of the later models, the sequentially-trainn ensembles likely end up being about only as diverse as the non-eqquentially trained ensembles.
Finally, we note that the sequential style of training cascade ensembles is quite natural. In fact, both, Wong et al. (2018) and Blum et al. (2022) train models a sequential manner. But these results suggest that sequentially training may make it casier to exploit the unsoundness of cascading ensembles.
D WEIGHTED VOTING ENSEMBLE: LEARNING WEIGHTS
The weights w in EY determines the importance of each constituent classifier in the ensemble. Given of k labeled inputs, S& (e.g. the training set), we would like to learn the optimal weights that maximize the ensembler's CRA (Def. 2.2) over St. When S& resembles the true distribution of the test points, the learned u is expected to be close to the optimal weights that maximizes the CRA of the test set. Weight optimization over Ss naturally takes the following form.
max I](
Σ
1EE ....., FIN- (INN (x.,))
(15)
& (A..)))
For the indicator to output 1, it is required that the margin of votes be greater than 0, i.c. A w. nn (y:) = th (y:; o (+,0)= maxjáu; [u2, (j,1)) >> Namely, the votes for the class yıs i.e. ".. 11), must be greater than the votes for all other classes i.c. max {v"", (j.1)) plus the votes for non-robust predictions 1 (x,0) as discussed in Def. 5.1. (q5( then becomes:
max WE
Σ
I[ (v))
(16)
K
(a..y:)ES,
The indicator function is not differentiable we replace with differentiable and monotonically increasing function s, which leads to Eq. 17.
1
max WE[O.1] N
Σ (AA
(17)
K
(#4.9))
In this paper, choose be the sigmoid function o, where is the temperature only for negative inputs, i.e., (x(x) == (x) if > and (x/tt) otherwise, where a is the standard sigmoid function. Sigmoid is non-ne non-nveative so margins with opposite signs do not cancel, and it also avoids biasing training towards producing larger margins on small number of points. Indeed, vanishing gradients
15

---

Published as a conference paper at ICLR 2023
Table 5: Results models pre-trained by Wong et al. (2018) for Lo robustness.
Single Model
Cascuding
Unifarm Voting
Weighted Voring
Dataset
Model
CRA( Acc(%)
unsound CRA(%)
Aco(%)
ERM(%)
CRA(5)
Aco(%)
CRA(%)
Aco(%))
MNIST
Small,
0.1
96.33
96.62
61.68
MNIST
Small
0.1
94.94
98.79
96.07
96.24
17.51
9.29
65.85
94.94
98.79
MINSTT
Large
0.1
95.55
98.81
96.27
96.42
13.27
10.12
63.89
95.55
98.81
MNIST
Small
0.3
56.21
85
65,4
65.80
7.67
56.46
56.21
85.23
MNIST
Large
0.3
58.02
65.50
65.50
26.95
65.97
58.02
88.84
CIFARIO
Small
2/258
46.43
60.86
56.65
56.65
50.13
18.58
40.88
46.41
60.86
CIFARIO
2/255
52.65
67.70
64,88
65.14
58.15
18.07
48.92
52.65
67.70
CIFARIO CIFARIO
Small
8/255
20.58
27.60
28.32
28.32
23.79
10.78
24.11
19.00
23.78
Large
8/255
16.04
19.01
20.83
20.83
17.15
5.18
21.01
16.04
19.01
Table 6: Results on models pre-trained by Wong et al. (2018) for & robustness.
Single Model CRA(S)
Cascading unsound CRA(S)
Uniffrmm Vating CRA(S)
Weighted Voxing
Dataset
Aco(S)
Acc(%)
ERA(S)
Acc(5)
CRA(S))
Acc(%)
MNIST
Small, Exact
1.58
43.52
88. 14
75.58
80.43
43.46
6.42
74.25
43.52
88.14
MNIST
Small
1,58
43.34
87.73
74.66
79:07
45.73
6.74
74.24
43.34
87:73
MNIST
Large
43,96
88.39
74.50
74.99
35.81
65.99
43.96
88.39
CIFARIO
Small
36/255
46.05
54.39
49.89
51.37
49.27
11,47
38.74
46.05
54.39
CITARIO
Large
36/255
50.26
60.14
58.72
58.76
57.17
10.56
39.74
50.26
60.14
CIFARIO
Resnet
36/255
51.65
60.7
58.65
58.69
56.28
22.74
46.71
51.65
60.7
useful on points around large positive margins, so the temperature only applied on negative inputs. This leads us to Eq. 18, the optimization objective we solve for optimal weights w".
w" = argmax
Σ
or( (v:)
(18)
wE[0,I]N (2..8))ESk
E WEIGHTED VOTING ENSEMBLE: EMPIRICAL RESULTS
The goal of these experiments is to evaluate the efficacy of our sound voting ensemble. For our experiments, we use the pre-trained ensemble constituent models made available by Wong al. (2018) to construct three kinds of ensembles, namely, cascading ensembles, uniform voting ensembles, and weighted voting ensembles. The weights for the weighted voting ensemble are learned in the manner described in Appendix D. We report certified robust accuracy (CRA) and standard accuracy (Acc) for each ensemble as well as for the best constituent model. Note that all these ensembles are query-access but only the uniform voting and weighted voting ensembles the CRA reported for the cascading ensemble grossly overestimates the actual CRA as demonstrated by our attack results. We al always set the temperature to le5 and learning rate to 1e-2 when learning the weights as described in Appendix D.
soundness prreeeernn.......ngg............ Consequently,
Table shows the results for foo robustness. Each row in the table represents a specific combination of dataset (MNIST or CIFAR-10), architecture (Small or Large convolutional networks or Resnet), and value used for local robustness certification. Table shows the results for & robustness using constituent models pre-trained by Wong et al. (2018).
Summary of Results. We see from Tables 5 and 6 that while the cascading ensemble appears to improve upon the CRA of the single best model in the ensemble, these numbers are misleading due to the unsoundness of the certification mechanism. The CRA for the uniform voting and weighted ensembles are consistently lower than that reported by the cascading ensemble, and in many cases, significantly so. Uniform voting ensembles stand-out for their low CRA but there explanation for these results. The constituent models are trained by Wong et al. (2018) in a cascading manner, i.c., later constituent models are trained on only those points that cannot be certified by the previous models. This strategy causes the subset of inputs labeled correct and certifiably robust by each constituent model to have minimal overlap. However, voting ensembles need these input subsets to strike the right balance between diversity and overlap for improving the CRA.
simple
Another interesting observation is that, most cases, the CRA of the weighted voting ensemble and the single best constituent model are the same. This is again a consequence of the cascaded manner in which the constituent models trained. The first model in the cascade typically vastly outperforms
16

---

Published as a conference paper at ICLR 2023
Table 7: Results on non-sequentially trained models for loc robustness.
Single Model
Cascuding
Uniform Voting CRA(5)
Weighted Vorin
Dataset
Model
CRA(
Acc(%)
unsound CRA(%)
Aco(%)
FRM(%)
Aco(%)
CRA(%)
Aco(%)
MNIST MNNST
Small,
0.1
97.07
99.14
95.56
99.16
Small
0.1
94.94
98.79
96.25
98.69
97.4
94,46
98.78
94.94
98.79
MNIST
Small
0.3
56.21
85.23
66.09
84.77
72.97
55.24
85.02
56.21
85.23
CIFARIO
Small
2/255
46.43
55.49
62.06
62.79
42.35
CIFARIO
Small
8/255
21.04
28.29
25.11
27.73
23,35
20.44
28.32
21.04
28.29
Table 8: Results on non-sequentially trained models for & robustness.
Single Model CRM())
Cascading unsound CRM(%)
Uniform Voting CRA(S))
Weighted Voting CRA(R))
Dataset
Model
Acc(%))
Acu(%)
(RA(%)
Acu(f))
Acc(%)
MNIST
Small, Exact
1.58
43.52
88. 88.14
47.42
87.78
73.56
42.71
88. 89.19
43.52
88.14
MNIST
Small
1.58
43.34
87.73
47.40
87.50
72.93
42,87
87.99
43,34
87.73
CI
Small
36/255
46.05
54.39
52.33
58.88
53.71
37.07
57.40
42.12
54.65
the subsequent models. Moreover, as already mentioned, the constituent models have almost no overlap in the input regions where they perform well, and their presence only ends up harming the performance of the voting ensemble. As a consequence, the optimal normalized weights, learned by solving the optimization problem described in Appendix D, typically assign all the mass to the first model. The detailed weights for each of the weighted voting ensemble are given in Tables 9, 10, 11, and 12.
These results suggest two takeaway messages. First, the cascaded strategy of Wong et al. (2018) for training constituent models is in conflict with the requirement that constituent models overlap in their behavior for voting ensembles to be effective. This gives up hope that if the constituent models suitably trained, voting ensembles
improve the CRA. We leave this exploration for future work. Second, even if the constituent models do not show the right balance between diversity and similarity, our weight learning procedure ensures that the performance of the weighted voting ensemble no worse than the single best constituent model. Idcally, we would like the weights to be equally distributed since this conveys that every constituent in the ensemble has something to contribute. But, in the worst case, the weights play the role of a model selection procedure, assigning zero weights to constituent models that do not contribute the ensemble.
Non-Sequential Training. We conduct another set of experiments where instead of using the con- stituent models pre-trained by Wong et al. (2018), we train them ourselves in a non-sequential manner. That is, each constituent model is trained on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training. Besides this dif- ference, the code, architecture, hyperparameters, and data used for training are the same that used by Wong al. (2018), For every combination of dataset, architecture, and e value, we train three constituent models, and use them to construct cascading, uniform voting, and weighted voting ensembles.
Table 7 shows the results for fo robustness using sequennitllllyy trained constituent model ands Table 8 shows the results for & robustness. We observe that, for non-sequentially trained models, the CRA of uniform voting and weighted voting ensembles are comparable, and similar to the CRA of the single best constituent model in the ensemble. In this case, the constituent models have too much overlap and almost no diversity. These results reaffirm our observation that voting ensembles require a balance between diversity and similarity to be effective.
F AN ALTERNATE FORMULATION OF UNIFORM VOTING ENSEMBLER
Definition F.1 (Permutation- bbased Cascading Ensembler). Let P()), F()) ... F(N-1) be N certifiable classifiers and N odd. Suppose II the set of all permutations of [0.1,..,,N-111 A permutation- based cascading ensembler Ep FN + is defined as follows
F((Ta) (I)
Il. c2(
(p() (())))) FE,,,,,,,EEM ..... (x)=
(Mttee (z))0) (v.0)
if 7h' ET.. c2(x')=1 EII. (TT)- otherwise
17

---

Published as a conference paper at ICLR 2023
Table 9: Learned weights for weighted voting ensemble with models pre-trained by Wong et al. (2018) for oo robustness.
number of models
Dataset
Model
weights
MNIST
Small, Exact
0.1
6
10.996, 0.003, 0.000, 0.001, 0.000, 0.000]
MNIST
Small
0.1
7
[0.996, 0.000, 0.000, 0.003, 0.000, 0.000, 0.001]
MNIST
Large
0.1
5
[0.996, 0.002, 0.001, 0.000, 0.001]
MNIST
Small
0.3
3
[0.995, 0.003, 0.002]
MNIST
Large
0.3
3
[0.948, 0.008, 0.044]
CIFAR10
Small
2/255
5
[0.995, 0.002, 0.001, 0.001, 0.001] [0.994, 0.003, 0.001, 0.002]
CIFARIO
Large
2/255
4
CIFAR1O
Small
8/255
3
[0.003, 0.995, 0.002] [0.995, 0.002, 0.003]
CIFARI0
Large
8/255
3
Table 10: Learned weights for weighted voting ensembles with models pre-trained by Wong et al. (2018) for &g robustness.
number of models
Dataset
Model
weights
MNIST
Small, Exact
1.58
6
[0.995, 0.001, 0.001, 0.000, 0.003, 0.001]
MNIST
Small
1.58
6
[0.995, 0.001, 0.002, 0.001, 0.001, 0.000]
MNIST
Large
1.58
6
[0.996, 0.000, 0.001, 0.002, 0.000, 0.001]
CIFAR10
Small
36/255
2
[0.994, 0.006]
CIFARIO
Large
36/255
6
[0.994, 0.002, 0.001, 0.001, 0.001, 0.0011 [0.994, 0.001,
CIFAR10
Resnet
36/255
4
Table 11: Learned weights for weighted voting ensemble with non-sequentially trained models for lox. robustness.
number models
Dataset
Model
weights
MNIST
Small, Exact
0.1
3
[0.710, 0.131, 0.159] [0.694, 0.154, 0. 0512
MNIST
Small
0.1
3
MNIST
Small
0.3
3
[0.908, 0.061, 0.031]
CIFARIO
Small
2/255
3
[0.011, 0.956, 0.0323]
CIFARIO
Small
8/255
3
[0.042, 0.087, 0.871]
Table 12: Learned weights for weighted voting ensembles with non-sequentially trained models for l robustness.
number of models
Dataset
Model
E
weights
MNIST
Small, Exact
1.58
[0.695, 0,9, 0.146]
MNIST
Small
1.58
3
[0.660, 1966 0444
CIFAR1O
Small
36/255
[0.003. 0.002, 0.995]
18

---

Published as a conference paper at ICLR 2023
where is random label selected from y" ,To refers the first element of the permutation n,
if ( SjSN-1)A(Miss
label (x)- - label (x))
cl(T):
(19)
0 otherwise
if
(Flort (x) -1)
c2(+):= =
(20)
0 otherwise
and (O,,
Theorem F.2. The permutatiin--aaeet cascading ensembler Ep is soundhexss ensembler.
Proof. Let E EF(00), ((N, For F we want to show that, VrER Fon (x) - Flabel is e-locally robust
(21)
W.L.O.G suppose Flabel x) = y. If ( let us assume that i is the permutation such that c2(x)= = Let & be the inteerr that makes c2 ( = to be true. Thus
N+ <KSN
1) - (
(v.1))
(22)
2
By our assumptions that [r(0)_ f(() ....
(N- are sound, which invariant the permutation
of these models. Therefore, by Def. 2.3, Vi
(Vz' || -
F((ee) (x')= (z) yy)
(23)
Eq. (23) implies that Vz' s.t. || S the following statement is true
N+1 SK (x)
(xt)) label (x') =y)
(24)
Plug the condition (24) into Def. F.1, we find that c1 1 for x'. Moreover, there cannot p(
be permutation such that cl (T') = c2(x') = A label # y since 2 N+1
Therefore,
2
Flabel (a') = label (a') y and we arrive at the following statement, Va' || - - Faabel (x') Fabel =y
(25)
which completes the proof for the soundness of Flabol at any x.
"One also return the plurality prediction of all models for the consideration of clean accuracy but the choice of will change the relevant theorems.
19


Reviews:
[{"review_id": "rjuUMIXf1BC", "reviewer": "Reviewer_VBmo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Cascading ensembling of certifiably adversarially robust classifiers, which has been used by previous works, is shown to be unsound, which is demonstrated by theoretical proof as as well as experimental results on real test set data.", "summary_of_the_review": "- The paper points out an important mathematical issue with the method of cascading ensembling of certifiably adversarially robust classifiers and shows with convincing experiments that this issue is relevant in practice.\n- This makes it an interesting contribution in my opinion.\n- The proposed weighted voting ensembling method is to my understanding not convincing by the discussed theoretical properties or experimental results, but it does not really hurt the paper and its efficacy is not overclaimed.", "strength_and_weaknesses": "Strengths:\n\n- The theoretical introduction and methodology are clearly presented and appear to be sound.\n\n- The experimental evaluations are very strong, as (contrary to other counterexamples against similar certifications) they work with fixed input points and fixed radii. I.e. not finding specific, potentially non-sensical input images but evaluatin the actual test set.\n\nWeaknesses/Issues/Suggestions/Questions:\n\n- The paper is mostly about showing that a previously presented improvement of adversarial robustness certification is invalid.\nI think it is very important to point out crucial issues with previously published and well cited papers, so I think that the submission is a good contribution for ICLR.\nHowever, I am not sure how popular/relevant the cascading methods in [1] and [2] are currently and if they are nominally relevant when applied to ensembles of and compared with state of the art certifyable robusntess techniques. A bit more evidence for them being used would underline the importance of the submission.\n\n- A more concrete overview of the attacked ensemble from [1] would be helpful. Particularly on how the sequence of models has been trained.\n\t- I would assume that if the models were trained independently (e.g. just from different seeds), the attack success rate on the cascading ensemble would be much smaller. Is that correct?\n\n- It would be helpful to emphasize that cascade ensembling over different certification methods for one classifier, as e.g. done in [3], is sound.\n\n- \"We solve the optimization problems on lines 6 and 8 using projected gradient descent (PGD)\" -- Why do you not use a stronger adversarial attack scheme, like AutoAttack [0]? Intuitively, the same considerations as for adversarials should apply for optimizing the objective for $\\tilde{F}(x+\\delta)$, right? Since and attack and not a defense is presented, this is not a major issue since the claims made by the paper could only be improved, but it could lead to improvements and might be the correct thing to do.\n\n- The paragraph on \"Surrogate Objectives\" is very confusing to me and should potentially be rewritten.\n\t - Optimizing the indicator functions does not seem to make sense: Would $argmax_\\delta \\mathbb{1}[\\tilde{F}_{cert}^{(i)}(x+\\delta)]$ not be realized for too many points, often including $\\delta = \\vec{0}$ since the function value is either $0$ or $1$?\n\t - Before, completely black box models were considered, but now access to the logits is assumed; this new assumption should be stated.\n\t - For the sake of demonstrating wrong certifications, it might make sense to use a specific objective for each model that appears in the counterexample. \n\n- There are very small differences between the evaluations in Table 1 and the original numbers from [1]. How can they be explained? Potentially, the explanation should be included in the paper.\n\n- The proposed weighted does not seem to be a convincing contribution.\n\t - Its results are not better than using the single first model (correct?).\n\t - Its construction and its theoretical soundness are not particularly surprising or insightful. \n\t - The case presented in Example 5.4. shows that there are situations where a voting ensemble does better than a single constituent model. However it doesn't seem to be possible to decide if this is the case in a given situation (without knowing the test labels). There are certainly also many situations where it is the other way around, and the voting ensemble is worse than a single model.\n\t - Still, it is a baseline alternative to cascading ensembling, and thus it is interesting to have it discussed and evaluated in this paper, I would just not see it as a recommendable alternative to using a single model, as I understand the results.\n\n[0] Croce and Hein, ICML 2020: \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks\"\n\n[1] Wong et al., NeurIPS 2018: \"Scaling provable adversarial defenses\"\n\n[2] Blum et al. 2022: \"Boosting barely robust learners: A new perspective on adversarial robustness.\"\n\n[3] Gowal et al. 2018: \"On the effectiveness of interval bound propagation for training verifiably robust models.\"", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "otWKZ5F2Fay", "reviewer": "Reviewer_YgY8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a counterexample to prove that certified cascading robust classifiers does not hold up to its certified robustness claim. Then, they proposed an attack method to show that on real data, models certified for cascading robust classifiers failed to held up to their certification. In other words, there are adversarial examples that makes the certified robust accuracy significantly lower than the certification. Finally, they proposed an alternative ensemble method, weighted voting ensemble, that is provably sound (can correctly provide a certified robust radius).", "summary_of_the_review": "I think this is a well-written paper. However, the paper seems to have a limited scope and the proposed method seems not so useful.", "strength_and_weaknesses": "Strength\n- The paper is straight forward, rigorous, and easy to follow.\n\nWeaknesses\n- I am not confident how important/widespread cascade robust classifiers are. If it is barely used, it may limit the scope of this paper.\n- In Table 4 and 5, is the CRA for cascading ensemble the correct CRA or the overestimated CRA? It seems it is the latter one. If that's the case, I think it is necessary to compare the ERA with the CRA of weighted voting ensemble so that we can understand how effective is the weighted voting ensemble.\n- It also seems that weighted voting have CRA and Acc less than or equal to a single model. So what could be the use case of this ensemble over a single model.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "oLE9cqz3ty", "reviewer": "Reviewer_XiH9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper pointed out a fallacy that cascading robust classifier is provably robust to epsilon perturbation. The authors first pointed out a counter example of such method and then provided a systematical algorithm to generate the adversarial examples from a sequence of provably sound classifiers. This paper essentially showed that chaining the epsilon-robust classifiers are by itself not epsilon-robust.", "summary_of_the_review": "There aren't a lot to comment for this submission, after all this is a correction of previous misbelief. ", "strength_and_weaknesses": "Strengthes:\n- The layout of this paper is good and easy to follow, pleasant to read and the idea is interesting. In fact, I am surprised such mistake in cascading classifiers are pointed out until now. \n- The completeness of this paper is good: the authors first provided a theoretically sound proof; followed by a practical algorithm to find the adversarial examples; and finally intensive experiments are made to show that such algorithm is indeed able to \n\n\nWeaknesses:\n- Does (Blum et al. 2022) also employ the erroneous cascading ensemble? I roughly checked their paper and it seems not the case.\n- It looks like even in (Wong 2018), the cascading ensemble was still claimed as one of the smaller contributions (only mentioned in one paragraph). So I think the negative impact of the unsoundness is limited ==> not sure whether publishing in a workshop is a better option (but I left this for authors and ACs to decide).\n- Another question is -- instead of defining $\\tilde{F}_{\\text{cert}}=1$ \n\nas  $\\tilde{F}_{label}$ to be epsilon-locally robust, can we define \n\n$\\tilde{F}_{cert}=1$ \n\nto be \n$\\tilde{F}_{label}\\text{ is } \\epsilon$-locally robust AND\n\n$\\tilde{F}_{label}$ equals to the ground truth $y$? \n\nBecause if the classifier makes a mistake, then we don’t need to care about its local stability at all. By strengthening the definition, the cascading classifier seems to be correct again.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster