Original: O N THE P ERILS OF C ASCADING R OBUST C LASSIFIERS
Revised: O N THE P ERILS OF C ASCADING R OBUST C LASSIFIERS

Original: A BSTRACT
Revised: A BSTRACT

Original: Ensembling certiﬁably robust neural networks is a promising approach for improving the certiﬁed robust accuracy of neural models.
Revised: Ensembling certiﬁably robust neural networks is a promising approach for improving the certiﬁed robust accuracy of neural models.

Original: Black-box ensembles that assume only query-access to the constituent models (and their robustness certiﬁers) during prediction are particularly attractive due to their modular structure.
Revised: Black-box ensembles that assume only query-access to the constituent models (and their robustness certiﬁers) during prediction are particularly attractive due to their modular structure.

Original: Cascading ensembles are a popular instance of black-box ensembles that appear to improve certiﬁed robust accuracies in practice.
Revised: Cascading ensembles are a popular instance of black-box ensembles that appear to improve certiﬁed robust accuracies in practice.

Original: However, we show that the robustness certiﬁer used by a cascading ensemble is unsound.
Revised: However, we show that the robustness certiﬁer used by a cascading ensemble is unsound.

Original: That is, when a cascading ensemble is certiﬁed as locally robust at an input x (with respect to (cid:15) ), there can be inputs x (cid:48) in the (cid:15) -ball centered at x , such that the cascade’s prediction at x (cid:48) is different from x and thus the ensemble is not locally robust.
Revised: That is, when a cascading ensemble is certiﬁed as locally robust at an input x (with respect to (cid:15) ), there can be inputs x (cid:48) in the (cid:15) -ball centered at x , such that the cascade’s prediction at x (cid:48) is different from x and thus the ensemble is not locally robust.

Original: Our theoretical ﬁndings are accompanied by empirical results that further demonstrate this unsoundness.
Revised: Our theoretical ﬁndings are accompanied by empirical results that further demonstrate this unsoundness.

Original: We present cascade attack (CasA), an adversarial attack against cascading ensembles, and show that: (1) there exists an adversarial input for up to 88% of the samples where the ensemble claims to be certiﬁably robust and accurate; and (2) the accuracy of a cascading ensemble under our attack is as low as 11% when it claims to be certiﬁably robust and accurate on 97% of the test set.
Revised: We present cascade attack (CasA), an adversarial attack against cascading ensembles, and show that: (1) there exists an adversarial input for up to 88% of the samples where the ensemble claims to be certiﬁably robust and accurate; and (2) the accuracy of a cascading ensemble under our attack is as low as 11% when it claims to be certiﬁably robust and accurate on 97% of the test set.

Original: Our work reveals a critical pitfall of cascading certiﬁably robust models by showing that the seemingly beneﬁcial strategy of cascading can actually hurt the robustness of the resulting ensemble.
Revised: Our work reveals a critical pitfall of cascading certiﬁably robust models by showing that the seemingly beneﬁcial strategy of cascading can actually hurt the robustness of the resulting ensemble.

Original: Our code is available at https://github.com/TristaChi/ensembleKW .
Revised: Our code is available at https://github.com/TristaChi/ensembleKW .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Local robustness has emerged as an important requirement of classiﬁer models.
Revised: Local robustness has emerged as an important requirement of classiﬁer models.

Original: It ensures that models are not susceptible to misclassiﬁcations caused by small perturbations to correctly classiﬁed inputs.
Revised: It ensures that models are not susceptible to misclassiﬁcations caused by small perturbations to correctly classiﬁed inputs.

Original: A lack of robustness can be exploited by not only malicious actors (in the form of adversarial examples (Szegedy et al., 2014)) but can also lead to incorrect behavior in the presence of natural noise (Gilmer et al., 2019).
Revised: A lack of robustness can be exploited by not only malicious actors (in the form of adversarial examples (Szegedy et al., 2014)) but can also lead to incorrect behavior in the presence of natural noise (Gilmer et al., 2019).

Original: However, ensuring local robustness of neural network classiﬁers has turned out to be a hard challenge.
Revised: However, ensuring local robustness of neural network classiﬁers has turned out to be a hard challenge.

Original: Although neural networks can achieve state-of-the-art classiﬁcation accuracies on a variety of important tasks, neural classiﬁers with comparable certiﬁed robust accuracies 1 ( CRA , Def. 2.2) remain elusive, even when trained in a robustness-aware manner (Madry et al., 2018; Wong & Kolter, 2018;
Revised: Although neural networks can achieve state-of-the-art classiﬁcation accuracies on a variety of important tasks, neural classiﬁers with comparable certiﬁed robust accuracies 1 ( CRA , Def. 2.2) remain elusive, even when trained in a robustness-aware manner (Madry et al., 2018; Wong & Kolter, 2018;

Original: Cohen et al., 2019; Leino et al., 2021).
Revised: Cohen et al., 2019; Leino et al., 2021).

Original: In light of the limitations of robustness-aware training, ensembling certiﬁably robust neural classiﬁers has been shown to be a promising approach for improving certiﬁed robust accuracies (Wong et al., 2018; Yang et al., 2022).
Revised: In light of the limitations of robustness-aware training, ensembling certiﬁably robust neural classiﬁers has been shown to be a promising approach for improving certiﬁed robust accuracies (Wong et al., 2018; Yang et al., 2022).

Original: An ensemble combines the outputs of multiple base classiﬁers to make a prediction, and is a well-known mechanism for improving classiﬁcation accuracy when one only has access to weak learners (Dietterich, 2000; Bauer & Kohavi, 1999).[Table] [Table] [Table]
Revised: An ensemble combines the outputs of multiple base classiﬁers to make a prediction, and is a well-known mechanism for improving classiﬁcation accuracy when one only has access to weak learners (Dietterich, 2000; Bauer & Kohavi, 1999).

Original: [Figure]

Original: Ensembles designed to improve CRA take one of two forms.
Revised: Ensembles designed to improve CRA take one of two forms.

Original: White-box ensembles (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) assume white-box access to the  models. 
Revised: White-box ensembles (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) assume white-box access to the constituent models. They [Figure]

Original: They calculate new logits by averaging the corresponding logits of the constituent classiﬁers.
Revised: calculate new logits by averaging the corresponding logits of the constituent classiﬁers.

Original: For  certiﬁcation,  the ensemble as a single, large  and then off- techniques (Cohen et al., 2019; Weng et al., 2018; Wong & Kolter, 2018; Zhang et al., 2018) for certiﬁcation.
Revised: For local robustness certiﬁcation, they treat the ensemble as a single, large model and then use off-the-shelf techniques (Cohen et al., 2019; Weng et al., 2018; Wong & Kolter, 2018; Zhang et al., 2018) for certiﬁcation.

Original: Black-box ensembles (Wong et al., 2018;
Revised: Black-box ensembles (Wong et al., 2018;

Original: Blum et al., 2022), on the  hand, assume only query-access to the constituent classiﬁers during prediction, and are, therefore, agnostic to their internal details.
Revised: Blum et al., 2022), on the other hand, assume only query-access to the constituent classiﬁers during prediction, and are, therefore, agnostic to their internal details.

Original: They re-use the prediction and certiﬁcation outcomes of the constituent models to calculate the ensemble’s prediction and certiﬁcate.
Revised: They re-use the prediction and certiﬁcation outcomes of the constituent models to calculate the ensemble’s prediction and certiﬁcate.

Original: Their black-box nature lends them modularity and permits any combination of constituent classiﬁers, irrespective of their individual certiﬁcation mechanism, so we focus our efforts on them in this paper.
Revised: Their black-box nature lends them modularity and permits any combination of constituent classiﬁers, irrespective of their individual certiﬁcation mechanism, so we focus our efforts on them in this paper.

Original: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are a particularly popular instance of black-box ensembles that appear to improve CRA in practice.
Revised: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are a particularly popular instance of black-box ensembles that appear to improve CRA in practice.

Original: They evaluate the constituent classiﬁers (and their certiﬁers) in a ﬁxed sequence.
Revised: They evaluate the constituent classiﬁers (and their certiﬁers) in a ﬁxed sequence.

Original: The ensemble’s prediction is the output of the ﬁrst constituent classiﬁer in the sequence that is certiﬁed locally robust, defaulting to the last classiﬁer’s output if no model can be certiﬁed.
Revised: The ensemble’s prediction is the output of the ﬁrst constituent classiﬁer in the sequence that is certiﬁed locally robust, defaulting to the last classiﬁer’s output if no model can be certiﬁed.

Original: Importantly, the cascading ensemble is itself certiﬁed locally robust only when at least one of the constituent classiﬁers is certiﬁed locally robust.
Revised: Importantly, the cascading ensemble is itself certiﬁed locally robust only when at least one of the constituent classiﬁers is certiﬁed locally robust.

Original: Our contributions.
Revised: Our contributions.

Original: We show in this paper that the local robustness certiﬁcation mechanism used by cascading ensembles is unsound even when the certiﬁers used by each of the constituent classiﬁers are sound (Theorem 2.8).
Revised: We show in this paper that the local robustness certiﬁcation mechanism used by cascading ensembles is unsound even when the certiﬁers used by each of the constituent classiﬁers are sound (Theorem 2.8).

Original: In other words, when a cascading ensemble is certiﬁed as locally robust at an input x , there can, in fact, be inputs x (cid:48) in the (cid:15) -ball centered at x , such that the cascade’s prediction at x (cid:48) is different from x .
Revised: In other words, when a cascading ensemble is certiﬁed as locally robust at an input x , there can, in fact, be inputs x (cid:48) in the (cid:15) -ball centered at x , such that the cascade’s prediction at x (cid:48) is different from x .

Original: Figure 1 demonstrates this visually on a toy dataset.
Revised: Figure 1 demonstrates this visually on a toy dataset.

Original: The cascading ensemble can have points that are less than (cid:15) away from the decision boundary, yet the ensemble is certiﬁed locally robust at such points (Figure 1(d)).
Revised: The cascading ensemble can have points that are less than (cid:15) away from the decision boundary, yet the ensemble is certiﬁed locally robust at such points (Figure 1(d)).

Original: As a consequence of our result, use of a cascading ensemble in any scenario requiring local robustness guarantees is unsafe and existing empirical results that report the CRA of cascading ensembles are not valid.
Revised: As a consequence of our result, use of a cascading ensemble in any scenario requiring local robustness guarantees is unsafe and existing empirical results that report the CRA of cascading ensembles are not valid.

Original: Guided by our theoretical construction, we propose cascade attack (CasA, Algorithm 3.1), an adversarial attack against cascading ensembles, and conduct an empirical evaluation with the cascading ensembles trained by Wong et al.
Revised: Guided by our theoretical construction, we propose cascade attack (CasA, Algorithm 3.1), an adversarial attack against cascading ensembles, and conduct an empirical evaluation with the cascading ensembles trained by Wong et al.

Original: (2018) for MNIST and CIFAR-10 datasets.
Revised: (2018) for MNIST and CIFAR-10 datasets.

Original: With CasA, we show that: (1) there exists an adversarial example for up to 88% of the samples where the ensemble claims to be certiﬁably robust and accurate; (2) the empirical robust accuracy of a cascading ensemble is as low as 11% while it claims to be certiﬁably robust and accurate on 97% of the test set; and (3) viewing all experiments as a whole, the empirical robust accuracy of a cascading ensemble is almost always lower than even the CRA of the single best model in the ensemble.
Revised: With CasA, we show that: (1) there exists an adversarial example for up to 88% of the samples where the ensemble claims to be certiﬁably robust and accurate; (2) the empirical robust accuracy of a cascading ensemble is as low as 11% while it claims to be certiﬁably robust and accurate on 97% of the test set; and (3) viewing all experiments as a whole, the empirical robust accuracy of a cascading ensemble is almost always lower than even the CRA of the single best model in the ensemble.

Original: Namely, a cascading ensemble is often less robust.
Revised: Namely, a cascading ensemble is often less robust.

Original: Our results conclusively demonstrate that the unsoundness of the cascading ensemble certiﬁcation mechanism can be exploited in practice, and cause the ensemble to perform markedly worse than the single best constituent model.
Revised: Our results conclusively demonstrate that the unsoundness of the cascading ensemble certiﬁcation mechanism can be exploited in practice, and cause the ensemble to perform markedly worse than the single best constituent model.

Original: We also present an alternate ensembling mechanism based on weighted voting that, like cascading ensembles, assumes only query-access to the constituent classiﬁers but comes with a provably sound local robustness certiﬁcation procedure (Section 5).
Revised: We also present an alternate ensembling mechanism based on weighted voting that, like cascading ensembles, assumes only query-access to the constituent classiﬁers but comes with a provably sound local robustness certiﬁcation procedure (Section 5).

Original: We show through a thought experiment that it is possible for a voting ensemble to improve upon the CRA of its constituent models (Section 5.2), and observe that the key ingredient for the success of voting ensembles is a suitable balance between diversity and similarity of their constituents.
Revised: We show through a thought experiment that it is possible for a voting ensemble to improve upon the CRA of its constituent models (Section 5.2), and observe that the key ingredient for the success of voting ensembles is a suitable balance between diversity and similarity of their constituents.

Original: We leave the design of training algorithms that balance the diversity and similarity of the constituent classiﬁers as future work.
Revised: We leave the design of training algorithms that balance the diversity and similarity of the constituent classiﬁers as future work.

Original:  C ASCADING E NSEMBLES
Revised:  C ASCADING E NSEMBLES

Original: In this section, we introduce our notation and required deﬁnitions.
Revised: In this section, we introduce our notation and required deﬁnitions.

Original: We then show the local robustness certiﬁcation procedure used by cascading ensembles is unsound.
Revised: We then show the local robustness certiﬁcation procedure used by cascading ensembles is unsound.

Original: 2. C ERTIFIABLE C LASSIFIERS AND E NSEMBLERS
Revised: 2. C ERTIFIABLE C LASSIFIERS AND E NSEMBLERS

Original: We begin with our notation and necessary deﬁnitions.
Revised: We begin with our notation and necessary deﬁnitions.

Original: Suppose a neural network f : R d → R m takes an input and outputs the probability of m different classes.
Revised: Suppose a neural network f : R d → R m takes an input and outputs the probability of m different classes.

Original: The subscript x j denotes the j -th element of a vector x .
Revised: The subscript x j denotes the j -th element of a vector x .

Original: When discussing multiple networks, we differentiate them with a superscript, e.g. f (1) ,f (2) .
Revised: When discussing multiple networks, we differentiate them with a superscript, e.g. f (1) ,f (2) .

Original: Throughout the paper we use the upper-case letter F to denote the prediction of f such that F ( x )=argmax j ∈Y { f j ( x ) } where f j is the logit for class j and Y =[ m ] .
Revised: Throughout the paper we use the upper-case letter F to denote the prediction of f such that F ( x )=argmax j ∈Y { f j ( x ) } where f j is the logit for class j and Y =[ m ] .

Original: The prediction F ( x ) is considered (cid:15) -locally robust at x if all neighbors within an (cid:15) -ball centered at x receive the same predictions, which is formally stated in Def. 2.1.
Revised: The prediction F ( x ) is considered (cid:15) -locally robust at x if all neighbors within an (cid:15) -ball centered at x receive the same predictions, which is formally stated in Def. 2.1.

Original: Deﬁnition 2.1 ( (cid:15) -Local Robustness) .
Revised: Deﬁnition 2.1 ( (cid:15) -Local Robustness) .

Original: A network, F , is (cid:15) -locally robust at x w.r.t to norm, || · || , if ∀ x (cid:48) .
Revised: A network, F , is (cid:15) -locally robust at x w.r.t to norm, || · || , if ∀ x (cid:48) .

Original: || x (cid:48) − x ||≤ (cid:15) =
Revised: || x (cid:48) − x ||≤ (cid:15) =

Original: ⇒ F ( x (cid:48) )
Revised: ⇒ F ( x (cid:48) )

Original: =
Revised: =

Original: F ( x ) .
Revised: F ( x ) .

Original: Though local robustness certiﬁcation of ReLU networks is NP-Complete (Katz et al., 2017), due to its importance, the problem has been receiving increasing attention from the community.
Revised: Though local robustness certiﬁcation of ReLU networks is NP-Complete (Katz et al., 2017), due to its importance, the problem has been receiving increasing attention from the community.

Original: Proposed certiﬁcation methods rely on a variety of algorithmic approaches like solving corresponding linear (Jordan et al., 2019) or semi-deﬁnite programs (Raghunathan et al., 2018), interval propagations (Gowal et al., 2018; Lee et al., 2020; Zhang et al., 2018), abstract interpretation (Singh et al., 2019a), geometric projections (Fromherz et al., 2021), dual networks (Wong & Kolter, 2018), or Lipschitz approximations (Leino et al., 2021; Weng et al., 2018).
Revised: Proposed certiﬁcation methods rely on a variety of algorithmic approaches like solving corresponding linear (Jordan et al., 2019) or semi-deﬁnite programs (Raghunathan et al., 2018), interval propagations (Gowal et al., 2018; Lee et al., 2020; Zhang et al., 2018), abstract interpretation (Singh et al., 2019a), geometric projections (Fromherz et al., 2021), dual networks (Wong & Kolter, 2018), or Lipschitz approximations (Leino et al., 2021; Weng et al., 2018).

Original: If a certiﬁcation method is provided for a network F , we use ˜ F (cid:15) : R d →Y×{ 0 , 1 } to denote a certiﬁable neural classiﬁer that returns a prediction according to F and the outcome of the certiﬁcation method applied to F with respect to robustness radius (cid:15) .
Revised: If a certiﬁcation method is provided for a network F , we use ˜ F (cid:15) : R d → Y × { 0 , 1 } to denote a certiﬁable neural classiﬁer that returns a prediction according to F and the outcome of the certiﬁcation method applied to F with respect to robustness radius (cid:15) .

Original: We use ˜ F (cid:15) label ( x ) to refer to the prediction and ˜ F (cid:15) cert ( x ) to refer to the certiﬁcation outcome.
Revised: We use ˜ F (cid:15) label ( x ) to refer to the prediction and ˜ F (cid:15) cert ( x ) to refer to the certiﬁcation outcome.

Original: If ˜ F (cid:15) cert ( x ) = 0 , the accompanying robustness certiﬁcation is unable to certify F (i.e., ˜ F label ) as (cid:15) -locally robust at x .
Revised: If ˜ F (cid:15) cert ( x )=0 , the accompanying robustness certiﬁcation is unable to certify F (i.e., ˜ F label ) as (cid:15) -locally robust at x .

Original: When (cid:15) is clear from the context, we directly write ˜ F .
Revised: When (cid:15) is clear from the context, we directly write ˜ F .

Original: One popular metric to evaluate the performance of any ˜ F is Certiﬁed Robust Accuracy ( CRA ).
Revised: One popular metric to evaluate the performance of any ˜ F is Certiﬁed Robust Accuracy ( CRA ).

Original: Deﬁnition 2.2 (Certiﬁed Robust Accuracy) .
Revised: Deﬁnition 2.2 (Certiﬁed Robust Accuracy) .

Original: The certiﬁed robust accuracy ( CRA ) of a certiﬁable classiﬁer ˜ F ∈ R d → Y × { 0 , 1 } on a given dataset S k ⊆
Revised: The certiﬁed robust accuracy ( CRA ) of a certiﬁable classiﬁer ˜ F ∈ R d → Y × { 0 , 1 } on a given dataset S k ⊆

Original: R
Revised: R

Original: d × Y with k samples is given by
Revised: d × Y with k samples is given by

Original: CRA ( ˜ F,S k ):=1 /k (cid:80) ( x i ,y i ) ∈
Revised: CRA ( ˜ F,S k ):=1 /k (cid:80) ( x i ,y i ) ∈

Original: S k 

Revised: S k

Original: [ ˜ F ( x i )=(
Revised: [ ˜ F ( x i )=(

Original: y i , 1)] .
Revised: y i , 1)] .

Original: 2 [ m ]:= { 0 , 1 ,...,m − 1 }

Original: For ˜ F and its CRA to be useful in practice without providing false robustness guarantees, it must be sound to begin with (Def. 2.3).
Revised: For ˜ F and its CRA to be useful in practice without providing false robustness guarantees, it must be sound to begin with (Def. 2.3).

Original: Deﬁnition 2.3 (Certiﬁcation Soundness) .
Revised: Deﬁnition 2.3 (Certiﬁcation Soundness) .

Original: A certiﬁable classiﬁer, ˜ F : R d → Y ×{ 0 , 1 } , is sound if ∀ x ∈ R d .
Revised: A certiﬁable classiﬁer, ˜ F : R d → Y ×{ 0 , 1 } , is sound if ∀ x ∈ R d .

Original: ˜ F cert ( x )=1 = ⇒ ˜
Revised: ˜ F cert ( x )=1 = ⇒ ˜

Original: F label is (cid:15) -locally robust at x .
Revised: F label is (cid:15) -locally robust at x .

Revised: 2 [ m ]:= { 0 , 1 ,...,m − 1 }

Original: Notice that if there exist (cid:15) -close inputs x,x (cid:48) where ˜ F ( x ) = ( y, 1) and ˜ F ( x (cid:48) ) = ( y (cid:48) , 0) , where y (cid:54) =
Revised: Notice that if there exist (cid:15) -close inputs x,x (cid:48) where ˜ F ( x )=( y, 1) and ˜ F ( x (cid:48) )=( y (cid:48) , 0) , where y (cid:54) =

Original: y (cid:48) , then it still means that ˜ F is not sound.
Revised: y (cid:48) , then it still means that ˜ F is not sound.

Original: We deﬁne an ensembler (Def. 2.4) as a function that combines multiple certiﬁable classiﬁers into a single certiﬁable classiﬁer (i.e., an ensemble ).
Revised: We deﬁne an ensembler (Def. 2.4) as a function that combines multiple certiﬁable classiﬁers into a single certiﬁable classiﬁer (i.e., an ensemble ).

Original: Deﬁnition 2.4 (Ensembler) .
Revised: Deﬁnition 2.4 (Ensembler) .

Original: Let ˜ F := R d → Y ×{ 0 , 1 } represent the set of all certiﬁable classiﬁers.
Revised: Let ˜ F := R d → Y ×{ 0 , 1 } represent the set of all certiﬁable classiﬁers.

Original: An ensembler E : ˜ F N → ˜ F is a function over N certiﬁable classiﬁers that returns a certiﬁable classiﬁer.
Revised: An ensembler E : ˜ F N → ˜ F is a function over N certiﬁable classiﬁers that returns a certiﬁable classiﬁer.

Original: A query-access ensembler formalizes our notion of a black-box ensemble.
Revised: A query-access ensembler formalizes our notion of a black-box ensemble.

Original: Deﬁnition 2.5 (Query-Access Ensembler) .
Revised: Deﬁnition 2.5 (Query-Access Ensembler) .

Original: Let G :=(
Revised: Let G :=(

Original: Y×{ 0 , 1 } ) N →Y×{ 0 , 1 } .
Revised: Y×{ 0 , 1 } ) N →Y×{ 0 , 1 } .

Original: E is a query-access ensembler if, ∀ ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1)
Revised: E is a query-access ensembler if, ∀ ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1)

Original: ∈ ˜ F .
Revised: ∈ ˜ F .

Original: ∃ G ∈ G .
Revised: ∃ G ∈ G .

Original: ∀ x ∈ R d .
Revised: ∀ x ∈ R d .

Original: E (cid:16) ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) (cid:17) ( x ) = G (cid:16) ˜ F (0) ( x ) , ˜ F (1) ( x ) ,..., ˜ F ( N − 1) ( x ) (cid:17)
Revised: E (cid:16) ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) (cid:17) ( x ) = G (cid:16) ˜ F (0) ( x ) , ˜ F (1) ( x ) ,..., ˜ F ( N − 1) ( x ) (cid:17)

Original: Def. 2.5 says that if E is query-access its output ˜ F can always be re-written as a function over the outputs of the certiﬁable classiﬁers ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) .
Revised: Def. 2.5 says that if E is query-access its output ˜ F can always be re-written as a function over the outputs of the certiﬁable classiﬁers ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) .

Original: Put differently, ˜ F only has black-box or query-access to classiﬁers ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) .
Revised: Put differently, ˜ F only has black-box or query-access to classiﬁers ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) .

Original: Finally, a soundness-preserving ensembler (Def. 2.6) ensures that if the constituent certiﬁable classiﬁers are sound (as deﬁned in Def. 2.3), the ensemble output by the ensembler is also sound.
Revised: Finally, a soundness-preserving ensembler (Def. 2.6) ensures that if the constituent certiﬁable classiﬁers are sound (as deﬁned in Def. 2.3), the ensemble output by the ensembler is also sound.

Original: Deﬁnition 2.6 (Soundness-Preserving Ensembler) .
Revised: Deﬁnition 2.6 (Soundness-Preserving Ensembler) .

Original: An ensembler E is soundness-preserving if, ∀ ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1)
Revised: An ensembler E is soundness-preserving if, ∀ ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1)

Original: ∈ ˜ F , ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) are sound =
Revised: ∈ ˜ F , ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) are sound =

Original: ⇒ E ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) is sound.
Revised: ⇒ E ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) is sound.

Original: 2. C ASCADING E NSEMBLER I S N OT S OUNDNESS -P RESERVING
Revised: 2. C ASCADING E NSEMBLER I S N OT S OUNDNESS -P RESERVING

Original: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are a popular instance of black-box ensembles that appear to be practically effective in improving certiﬁed robust accuracies.
Revised: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are a popular instance of black-box ensembles that appear to be practically effective in improving certiﬁed robust accuracies.

Original: However, we show that cascading ensembles are not sound.
Revised: However, we show that cascading ensembles are not sound.

Original: We deﬁne a cascading ensemble to be the output of a cascading ensembler (Def. 2.7).
Revised: We deﬁne a cascading ensemble to be the output of a cascading ensembler (Def. 2.7).

Original: A cascading ensemble evaluates its constituent certiﬁable classiﬁers in a ﬁxed sequence.
Revised: A cascading ensemble evaluates its constituent certiﬁable classiﬁers in a ﬁxed sequence.

Original: For a given input x , the ensemble either returns the prediction and certiﬁcation outcomes of the ﬁrst constituent classiﬁer ˜ F ( j ) such that ˜ F ( j ) cert
Revised: For a given input x , the ensemble either returns the prediction and certiﬁcation outcomes of the ﬁrst constituent classiﬁer ˜ F ( j ) such that ˜ F ( j ) cert

Original: =
Revised: =

Original: 1 or of the last constituent classiﬁer in case none of the constituent classiﬁers can be certiﬁed.
Revised: 1 or of the last constituent classiﬁer in case none of the constituent classiﬁers can be certiﬁed.

Original: Clearly, cascading ensemblers are query-access (formal proof in Appendix A).
Revised: Clearly, cascading ensemblers are query-access (formal proof in Appendix A).

Original: Deﬁnition 2.7 (Cascading Ensembler) .
Revised: Deﬁnition 2.7 (Cascading Ensembler) .

Original: Let ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) be N certiﬁable classiﬁers.
Revised: Let ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) be N certiﬁable classiﬁers.

Original: A cascading ensembler E C : ˜ F N → ˜ F is deﬁned as follows
Revised: A cascading ensembler E C : ˜ F N → ˜ F is deﬁned as follows

Original: [Equation]
Revised: [Equation]

Revised: ˜ F ( N − 1) (cid:17) ( x ) := (cid:26) ˜ F ( j ) ( x ) if ∃ j ≤ N − 1 .

Original:  where c ( j ):=1 if ( ˜ F ( j ) cert ( x )=1) and ( ∀ i<j, ˜ F ( i ) cert ( x )=0) , and 0 otherwise.
Revised: c ( j )= ˜ F ( N − 1) ( x ) otherwise where c ( j ):=1 if ( ˜ F ( j ) cert ( x )=1) and ( ∀ i<j, ˜ F ( i ) cert ( x )=0) , and 0 otherwise.

Original: Theorem 2.8 shows that cascading ensemblers are not soundness-preserving, and so a cascading ensemble can be unsound.
Revised: Theorem 2.8 shows that cascading ensemblers are not soundness-preserving, and so a cascading ensemble can be unsound.

Original: We show this by means of a counterexample.
Revised: We show this by means of a counterexample.

Original: Theorem 2.8.
Revised: Theorem 2.8.

Original: The cascading ensembler E C is not soundness-preserving.
Revised: The cascading ensembler E C is not soundness-preserving.

Original: Proof.
Revised: Proof.

Original: We can re-write the theorem statement as, ∃ ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1)
Revised: We can re-write the theorem statement as, ∃ ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1)

Original: ∈ ˜ F such that for ˜ F := E C ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) , ∃ x ∈ R d , ˜ F cert ( x )=1 (cid:54) =
Revised: ∈ ˜ F such that for ˜ F := E C ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) , ∃ x ∈ R d , ˜ F cert ( x )=1 (cid:54) =

Original: ⇒ ˜
Revised: ⇒ ˜

Original: F label is (cid:15) -locally robust at x .
Revised: F label is (cid:15) -locally robust at x .

Original: We prove by constructing the following counterexample.
Revised: We prove by constructing the following counterexample.

Original: Consider a cascading ensemble ˜ F constituted of certiﬁable classiﬁers ˜ F (0) and ˜ F (1) .
Revised: Consider a cascading ensemble ˜ F constituted of certiﬁable classiﬁers ˜ F (0) and ˜ F (1) .

Original: ˜ F (0) and ˜ F (1) are such there exists an x where
Revised: ˜ F (0) and ˜ F (1) are such there exists an x where

Revised: [Equation]

Revised: Using Def. 2.7, it is true that ˜ F ( x ) =

Revised: ( y, 1) .

Revised: Without violating (1), we can have another point x (cid:48) such that,

Original: Inputs: Ensemble ˜ F ∈ ˜ F , constituent models ˜ F (0) , ... , ˜ F ( N − 1)
Revised: Inputs: Ensemble ˜ F ∈ ˜ F , constituent models ˜ F (0) , ... , ˜ F ( N − 1)

Original: ∈ ˜ F , input x ∈ R d , attack bound (cid:15) ∈ R , and distance metric (cid:96) p
Revised: ∈ ˜ F , input x ∈ R d , attack bound (cid:15) ∈ R , and distance metric (cid:96) p

Original: Output: An attack input x (cid:48) ∈
Revised: Output: An attack input x (cid:48) ∈

Original: R d 1 Attack( ˜ F, ˜ F (0) , ... , ˜ F ( N − 1) , x , (cid:15) , (cid:96) p ) :
Revised: R d 1 Attack( ˜ F, ˜ F (0) , ... , ˜ F ( N − 1) , x , (cid:15) , (cid:96) p ) :

Original: y :
Revised: y :

Original: =
Revised: =

Original: ˜ F label ( x ) idxs :=
Revised: ˜ F label ( x ) idxs :=

Original: { i | i ∈
Revised: { i | i ∈

Original: [ N ] ∧ ˜ F ( i ) label ( x ) (cid:54) =
Revised: [ N ] ∧ ˜ F ( i ) label ( x ) (cid:54) =

Original: y } ∪ { N − 1 } foreach i ∈ idxs do if i = N − 1 then
Revised: y } ∪ { N − 1 } foreach i ∈ idxs do if i = N − 1 then

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Revised: [Figure]

Original: else
Revised: else

Original: [Figure]
Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: if ˜ F label ( x ∗ ) (cid:54) =
Revised: [Figure] [Table] [Figure] [Table]

Original: y then return x ∗ return x (cid:16) ˜ F(0) cert (
Revised: [Figure] [Table] [Figure] if ˜ F

Original: [Equation]
Revised: [Figure]

Original: )=
Revised: ( x

Original: [Equation]
Revised: [Figure]

Original: (cid:17)
Revised: ) (cid:54) =

Original: [Equation] F [Equation]
Revised: y then return x ∗ return x

Original: [Equation] [Equation] (cid:17) (1)
Revised: ( || x − x (cid:48) ||≤ (cid:15) )

Original: Using Def. 2.7, it is true that ˜ F  ( x  )=( y, 1) . Without violating (1), we can have another point x (cid:48) such that, (|| x − x 
Revised: ∧ ( ˜ F (0) ( x (cid:48) )=( y (cid:48) , 1)) ∧ ( ˜ F (1) ( x (cid:48) )=( y, 0)) ∧ ( y (cid:48) (cid:54) =

Original: [Equation] [Equation]
Revised: y ) (2)

Original: Using Def. 2.7, it is true that ˜ F ( x (cid:48) )=( y (cid:48) , 1) .
Revised: Using Def. 2.7, it is true that ˜ F ( x (cid:48) ) = ( y (cid:48) , 1) .

Original: Thus, for two points x,x (cid:48) constructed as above, we show that ∃ x,x (cid:48) , s.t. ||
Revised: Thus, for two points x,x (cid:48) constructed as above, we show that ∃ x,x (cid:48) , s.t. ||

Original: x (cid:48) − x || ≤ (cid:15) , ˜ F cert ( x ) =(cid:54) =
Revised: x (cid:48) − x || ≤ (cid:15) , ˜ F cert ( x ) =(cid:54) =

Original: ⇒ ˜
Revised: ⇒ ˜

Original: F [Equation] ( x [Equation]
Revised: F label ( x (cid:48) )

Original: ) =
Revised: =

Original: ˜ F label ( x ) , which violates the condition of local robustness (Def. 2.1).
Revised: ˜ F label ( x ) , which violates the condition of local robustness (Def. 2.1).

Original: The counterexample constructed in Thm.
Revised: The counterexample constructed in Thm.

Original: 2.8 is not just hypothetical, but something that materializes on real models (see Figure 1 for a toy example and Section 4 for our empirical evaluation).
Revised: 2.8 is not just hypothetical, but something that materializes on real models (see Figure 1 for a toy example and Section 4 for our empirical evaluation).

Original:  A TTACKING C ASCADING E NSEMBLES
Revised:  A TTACKING C ASCADING E NSEMBLES

Original: Section 2.2 shows that a cascading ensemble does not provide a robustness guarantee.
Revised: Section 2.2 shows that a cascading ensemble does not provide a robustness guarantee.

Original: We further show here how one can attack the cascading ensemble and ﬁnd an adversarial example within the (cid:15) -ball centered at the input x .
Revised: We further show here how one can attack the cascading ensemble and ﬁnd an adversarial example within the (cid:15) -ball centered at the input x .

Original: Overview of Attack.
Revised: Overview of Attack.

Original: Algorithm 3.1 describes the attack algorithm, cascade attack (CasA), inspired by the proof of Theorem 2.8.
Revised: Algorithm 3.1 describes the attack algorithm, cascade attack (CasA), inspired by the proof of Theorem 2.8.

Original: Given an input x , the goal of the algorithm is to ﬁnd an input x (cid:48) in the (cid:15) -ball centered at x such that the predictions of the cascade at x and x (cid:48) are different.
Revised: Given an input x , the goal of the algorithm is to ﬁnd an input x (cid:48) in the (cid:15) -ball centered at x such that the predictions of the cascade at x and x (cid:48) are different.

Original: The inputs to the algorithm are an ensemble ˜ F , its constituent classiﬁers ˜ F (0) , ... , ˜ F ( N − 1) , the input x to be attacked, and the attack distance bound (cid:15) as well as distance metric (cid:96) p .
Revised: The inputs to the algorithm are an ensemble ˜ F , its constituent classiﬁers ˜ F (0) , ... , ˜ F ( N − 1) , the input x to be attacked, and the attack distance bound (cid:15) as well as distance metric (cid:96) p .

Original: The algorithm either returns a successful adversarial input x (cid:48) such that || x − x (cid:48)
Revised: The algorithm either returns a successful adversarial input x (cid:48) such that || x − x (cid:48)

Original: || p ≤ (cid:15) and ˜ F label ( x ) (cid:54) =
Revised: || p ≤ (cid:15) and ˜ F label ( x ) (cid:54) =

Original: ˜ F label ( x (cid:48) ) or it returns the original input 
Revised: ˜ F label ( x (cid:48) ) or it returns the original input x if no adversarial input was found.

Revised: We use the following notations: L ce stands for cross-entropy loss, one-hot is the one-hot encoding function, and B p (0 ,(cid:15) ) is the (cid:96) p -ball of radius (cid:15) centered at 0 .

Original: Preparing Targets.
Revised: Preparing Targets.

Original: CasA gets the label y predicted by the ensemble ˜ F at input x (line 2) to select the constituent models it may attack.
Revised: CasA gets the label y predicted by the ensemble ˜ F at input x (line 2) to select the constituent models it may attack.

Original: The attacker is only interested in a constituent model (by remembering its index i ) if it predicts a label other than y at x or it is the last one (line 3).
Revised: The attacker is only interested in a constituent model (by remembering its index i ) if it predicts a label other than y at x or it is the last one (line 3).

Original: We are not interested in attacking a model ˜ F ( j ) that predicts y at x because such an effort is bound to fail.
Revised: We are not interested in attacking a model ˜ F ( j ) that predicts y at x because such an effort is bound to fail.

Original: ˜ F ( j ) is still sound even though the ensemble is not; therefore, no point x (cid:48) assigned a label other than y by ˜
Revised: ˜ F ( j ) is still sound even though the ensemble is not; therefore, no point x (cid:48) assigned a label other than y by ˜

Original: F ( j ) is such that it is both less than (cid:15) -away from x and ˜ F ( j ) is also certiﬁably robust at x (cid:48)
Revised: F ( j ) is such that it is both less than (cid:15) -away from x and ˜ F ( j ) is also certiﬁably robust at x (cid:48)

Original: (the second condition is necessary for ˜ F ( j ) to be used for prediction at x (cid:48) by the ensemble).
Revised: (the second condition is necessary for ˜ F ( j ) to be used for prediction at x (cid:48) by the ensemble).

Original: However, the last model ˜ F ( N − 1) is an exception and always remembered, i.e. idxs always includes N − 1 .
Revised: However, the last model ˜ F ( N − 1) is an exception and always remembered, i.e. idxs always includes N − 1 .

Original: The reason is that, given an input x (cid:48) , if all models ˜ F ( i ) ; i<N − 1 fail to be certiﬁably robust at x (cid:48) , the ensemble uses ˜ F ( N − 1) for prediction at x (cid:48) irrespective of whether ˜ F ( N − 1) is itself certiﬁably robust at x (cid:48) or not.
Revised: The reason is that, given an input x (cid:48) , if all models ˜ F ( i ) ; i<N − 1 fail to be certiﬁably robust at x (cid:48) , the ensemble uses ˜ F ( N − 1) for prediction at x (cid:48) irrespective of whether ˜ F ( N − 1) is itself certiﬁably robust at x (cid:48) or not.

Original: Attacker’s Steps.
Revised: Attacker’s Steps.

Original: For each model index in idxs , we try to ﬁnd an adversarial example (lines 4-10).
Revised: For each model index in idxs , we try to ﬁnd an adversarial example (lines 4-10).

Original: An attacker stops as soon as they ﬁnd a valid adversarial example (lines 9-10).
Revised: An attacker stops as soon as they ﬁnd a valid adversarial example (lines 9-10).

Original: Lines 6 and 8 describe the objective an attacker minimizes to ﬁnd the adversarial examples.
Revised: Lines 6 and 8 describe the objective an attacker minimizes to ﬁnd the adversarial examples.

Original: If index i (cid:54) =
Revised: If index i (cid:54) =

Original: N − 1 , the attacker optimizes δ such that, at input x + δ , the model ˜ F ( i ) is certiﬁed robust whereas all other models ˜ F ( k ) ; k <i are not certiﬁed robust.
Revised: N − 1 , the attacker optimizes δ such that, at input x + δ , the model ˜ F ( i ) is certiﬁed robust whereas all other models ˜ F ( k ) ; k <i are not certiﬁed robust.

Original: This ensures that model ˜ F ( i ) is used for prediction at input x + δ as it is certiﬁably robust at x .
Revised: This ensures that model ˜ F ( i ) is used for prediction at input x + δ as it is certiﬁably robust at x .

Original: If index i = N − 1 , we still require that all models ˜ F ( k ) ; k < i are not certiﬁed robust at x + δ .
Revised: If index i = N − 1 , we still require that all models ˜ F ( k ) ; k < i are not certiﬁed robust at x + δ .

Original: But instead of requiring that ˜ F ( i ) is certiﬁed robust at x + δ , we only require that the predicted label at x + δ be different from y .
Revised: But instead of requiring that ˜ F ( i ) is certiﬁed robust at x + δ , we only require that the predicted label at x + δ be different from y .

Original: We solve the optimization problems on linesand 8 using projected gradient descent (PGD) (Madry et al., 2018).
Revised: We solve the optimization problems on linesand 8 using projected gradient descent (PGD) (Madry et al., 2018).

Original: Surrogate Objectives.
Revised: Surrogate Objectives.

Original: For cases when the certiﬁcation procedure, i.e. ˜ F ( i ) cert ( x + δ ) , is not differentiable or too expensive to run multiple times, we provide the following cheap surrogate replacements.
Revised: For cases when the certiﬁcation procedure, i.e. ˜ F ( i ) cert ( x + δ ) , is not differentiable or too expensive to run multiple times, we provide the following cheap surrogate replacements.

Original: The intuition underlying the surrogate versions is that, given a model, the distance to the decision boundary from an input is correlated with the margin between the top logit scores of the model at that input.
Revised: The intuition underlying the surrogate versions is that, given a model, the distance to the decision boundary from an input is correlated with the margin between the top logit scores of the model at that input.

Original: To use the surrogate objectives for the attack, we need to assume access to the logit scores of the models.
Revised: To use the surrogate objectives for the attack, we need to assume access to the logit scores of the models.

Original: For the problem argmax δ ∈ B p (0 ,(cid:15) )
Revised: For the problem argmax δ ∈ B p (0 ,(cid:15) )

Original: ˜ F ( i ) cert ( x + δ ) , we try to increase the logit score associated with the desired prediction as much as possible.
Revised: ˜ F ( i ) cert ( x + δ ) , we try to increase the logit score associated with the desired prediction as much as possible.

Original: Then, a surrogate version of the problem is as follows (where ˜ F ( i ) logit represents the logit scores produced by model ˜ F ( i ) ):
Revised: Then, a surrogate version of the problem is as follows (where ˜ F ( i ) logit represents the logit scores produced by model ˜ F ( i ) ):

Original: [Equation]
Revised: [Equation]

Original: For the problem argmax δ ∈ B p [Equation] [Equation] k<i
Revised: For the problem argmax δ ∈ B p (0 ,(cid:15) ) (cid:80) k

Original: [Equation]

Original: −

Original: [Equation] F [Equation] k ) cert ( x + δ )) , we want the input x + δ to be as close as possible to the decision boundaries for each of the models by ˜ F ( k ) ,k < i so that the robustness certiﬁcations will fail.
Revised: <i (1 − ˜ F ( k ) cert ( x + δ )) , we want the input x + δ to be as close as possible to the decision boundaries for each of the models by ˜ F ( k ) ,k < i so that the robustness certiﬁcations will fail.

Original: The speciﬁc predictions ˜ F ( k ) label ( x + δ ) of these models do not matter.
Revised: The speciﬁc predictions ˜ F ( k ) label ( x + δ ) of these models do not matter.

Original: Towards that end, we aim to make the margin between the logit scores of any model ˜ F ( k ) be as small as possible.
Revised: Towards that end, we aim to make the margin between the logit scores of any model ˜ F ( k ) be as small as possible.

Original: This leads to the following surrogate problem (where unif is a discrete uniform distribution):
Revised: This leads to the following surrogate problem (where unif is a discrete uniform distribution):

Original: [Equation]
Revised: [Equation]

Original:  E MPIRICAL E VALUATION
Revised:  E MPIRICAL E VALUATION

Original: The goal of our empirical evaluation is to demonstrate the extent to which the unsoundness of the cascading ensembles manifests in practice and can be exploited by an adversary, i.e. CasA. For our measurements, we use the (cid:96) ∞ and (cid:96) 2 robust cascading ensembles constructed by Wong et al.
Revised: The goal of our empirical evaluation is to demonstrate the extent to which the unsoundness of the cascading ensembles manifests in practice and can be exploited by an adversary, i.e. CasA. For our measurements, we use the (cid:96) ∞ and (cid:96) 2 robust cascading ensembles constructed by Wong et al.

Original: (2018) for
Revised: (2018) for

Original: MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky, 2009) datasets.
Revised: MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky, 2009) datasets.

Original: The constituent classiﬁers in these cascades use a local robustness certiﬁcation mechanism based on dual networks (Wong et al., 2018).
Revised: The constituent classiﬁers in these cascades use a local robustness certiﬁcation mechanism based on dual networks (Wong et al., 2018).

Original: Each cascade includes between 2-7 certiﬁable classiﬁers with the same architecture (except for the (cid:96) ∞ robust, CIFAR-10 Resnet cascades that include only a single constituent model, and are hence not considered in our evaluation).
Revised: Each cascade includes between 2-7 certiﬁable classiﬁers with the same architecture (except for the (cid:96) ∞ robust, CIFAR-10 Resnet cascades that include only a single constituent model, and are hence not considered in our evaluation).

Original: The training code and all the constituent models in the ensembles are made available by Wong et al.
Revised: The training code and all the constituent models in the ensembles are made available by Wong et al.

Original: (2018) in a public repository (Wong & Kolter).
Revised: (2018) in a public repository (Wong & Kolter).

Original: We report the certiﬁed robust accuracy ( CRA ) and standard accuracy ( Acc ) for the cascading ensemble as well as the single best constituent model in the ensemble.
Revised: We report the certiﬁed robust accuracy ( CRA ) and standard accuracy ( Acc ) for the cascading ensemble as well as the single best constituent model in the ensemble.

Original: While the certiﬁer for a single model is sound, the ensemble certiﬁer is unsound and the reported ensemble CRA is an over-estimate.
Revised: While the certiﬁer for a single model is sound, the ensemble certiﬁer is unsound and the reported ensemble CRA is an over-estimate.

Original: We therefore measure the empirical robustness of the ensemble under CasA. Certifying with dual networks (Wong et al., 2018) is differentiable but extremely expensive.
Revised: We therefore measure the empirical robustness of the ensemble under CasA. Certifying with dual networks (Wong et al., 2018) is differentiable but extremely expensive.

Original: To run the attack more efﬁciently, we use the surrogate replacements in Section 3 and take 100 steps using PGD (other hyper-parameters to follow in Appendix B) to empirically minimize the objectives.
Revised: To run the attack more efﬁciently, we use the surrogate replacements in Section 3 and take 100 steps using PGD (other hyper-parameters to follow in Appendix B) to empirically minimize the objectives.

Original: After the attack, we report the false positive rate ( FPR ), i.e. % of test inputs for which an adversarial example is found

Original: [Table]

Original: of set where model is certiﬁed robust

Original: [Table]

Original: accurate.

Original: Acc :

Original: [Table]

Original: test set where model is accurate.

Original: FPR

Original: [Table]

Original: among all test inputs where cascade is certiﬁed robust and accurate of inputs an adversarial example

Original: [Table]

Original: [Table] [Table] [Table] within the (cid:15) -ball, and the empirical robust accuracy ( ERA ), i.e. % of test set where the cascade is empirically robust (i.e., our attack failed).
Revised: After the attack, we report the false positive rate ( FPR ), i.e. % of test inputs for which an adversarial example is found within the (cid:15) -ball, and the empirical robust accuracy ( ERA ), i.e. % of test set where the cascade is empirically robust (i.e., our attack failed).

Original: All our experiments were run on a pair of NVIDIA TITAN
Revised: All our experiments were run on a pair of NVIDIA TITAN

Original: RTX GPUs with 24 GB of RAM each, and a 4.2GHz Intel Core i7-7700K with 64 GB of RAM.
Revised: RTX GPUs with 24 GB of RAM each, and a 4.2GHz Intel Core i7-7700K with 64 GB of RAM.

Original: Table 1 shows the results for (cid:96) ∞ robustness (top) and (cid:96) 2 robustness (bottom).
Revised: Table 1 shows the results for (cid:96) ∞ robustness (top) and (cid:96) 2 robustness (bottom).

Original: Each row in the table represents a speciﬁc combination of dataset (MNIST or CIFAR-10), architecture (Small or Large convolutional networks), and (cid:15) value used for local robustness certiﬁcation.
Revised: Each row in the table represents a speciﬁc combination of dataset (MNIST or CIFAR-10), architecture (Small or Large convolutional networks), and (cid:15) value used for local robustness certiﬁcation.

Original: The structure of the table is the same as Tables 2 and 4 in (Wong et al., 2018), except we add the columns reporting FPR and ERA .
Revised: The structure of the table is the same as Tables 2 and 4 in (Wong et al., 2018), except we add the columns reporting FPR and ERA .

Original: Summary of Results.
Revised: Summary of Results.

Original: We see from Table 1 that, irrespective of the dataset, model, (cid:15) value, or (cid:96) p metric under consideration, our attack can ﬁnd false positives, with false positive rates ( FPR ) as high as 88.87%.
Revised: We see from Table 1 that, irrespective of the dataset, model, (cid:15) value, or (cid:96) p metric under consideration, our attack can ﬁnd false positives, with false positive rates ( FPR ) as [Table] [Table] [Table]

Revised: [Table]

Revised: high as 88.87%.

Original: In other words, there always exist test inputs where the ensemble is accurate and declares itself to be certiﬁed robust, but our attack is able to ﬁnd an adversarial example.
Revised: In other words, there always exist test inputs where the ensemble is accurate and declares itself to be certiﬁed robust, but our attack is able to ﬁnd an adversarial example.

Original: This result demonstrates that the unsoundness of the cascading ensemble certiﬁcation mechanism is not just a problem in theory but it can be exploited by adversaries in practice.
Revised: This result demonstrates that the unsoundness of the cascading ensemble certiﬁcation mechanism is not just a problem in theory but it can be exploited by adversaries in practice.

Original: More strikingly, the empirical robust accuracy ( ERA ) of the ensemble is often signiﬁcantly lower than the certiﬁed robust accuracy ( CRA ) of the best constituent model.
Revised: More strikingly, the empirical robust accuracy ( ERA ) of the ensemble is often signiﬁcantly lower than the certiﬁed robust accuracy ( CRA ) of the best constituent model.

Original: Since the ERA of a model is an upper-bound of its CRA , the actual CRA of the ensemble can be no larger than the reported ERA .
Revised: Since the ERA of a model is an upper-bound of its CRA , the actual CRA of the ensemble can be no larger than the reported ERA .

Original: This result shows that the use of a cascading ensemble can actually hurt the robustness instead of improving it.
Revised: This result shows that the use of a cascading ensemble can actually hurt the robustness instead of improving it.

Original: Note that there are small differences between the Acc and CRA numbers reported in Table 1 and those in (Wong et al., 2018).
Revised: Note that there are small differences between the Acc and CRA numbers reported in Table 1 and those in (Wong et al., 2018).

Original: Though we use the evaluation code and pre-trained models made available by Wong et al.
Revised: Though we use the evaluation code and pre-trained models made available by Wong et al.

Original: (2018), the hardware and PyTorch versions we use in our experiments are different.
Revised: (2018), the hardware and PyTorch versions we use in our experiments are different.

Original: Attack Efﬁciency.
Revised: Attack Efﬁciency.

Original: In Table 2, we compare the attack results of CasA using the original objectives, i.e. dual networks (Wong et al., 2018), and using surrogate replacements.
Revised: In Table 2, we compare the attack results of CasA using the original objectives, i.e. dual networks (Wong et al., 2018), and using surrogate replacements.

Original: Because the ensemble on MNIST contains more constituent models, it uses more memory with dual networks compared to CIFAR10.
Revised: Because the ensemble on MNIST contains more constituent models, it uses more memory with dual networks compared to CIFAR10.

Original: Our report of run time and memory usage shows that using surrogate replacements allows us to run attacks with larger batch size, less memory and time to reach the same level of performance.
Revised: Our report of run time and memory usage shows that using surrogate replacements allows us to run attacks with larger batch size, less memory and time to reach the same level of performance.

Original:  A Q UERY -A CCESS , S OUNDNESS -P RESERVING E NSEMBLER
Revised:  A Q UERY -A CCESS , S OUNDNESS -P RESERVING E NSEMBLER

Original: We present a query-access, soundness-preserving ensembler based on weighted voting in this section.
Revised: We present a query-access, soundness-preserving ensembler based on weighted voting in this section.

Original: Voting is a natural choice for the design of a query-access ensembler but ensuring that the ensembler is soundness-preserving can be subtle.
Revised: Voting is a natural choice for the design of a query-access ensembler but ensuring that the ensembler is soundness-preserving can be subtle.

Original: Section 5.1 deﬁnes our ensembler and proves that it is soundnesspreserving.
Revised: Section 5.1 deﬁnes our ensembler and proves that it is soundnesspreserving.

Original: In Section 5.2, we present a thought experiment demonstrating that it is possible for a voting ensemble to signiﬁcantly improve upon the CRA of its constituent models.
Revised: In Section 5.2, we present a thought experiment demonstrating that it is possible for a voting ensemble to signiﬁcantly improve upon the CRA of its constituent models.

Original: Appendix D describes our algorithmforlearningtheweightstobeusedinweightedvoting, andAppendixEpresentsinitialempirical results with the voting ensemble.
Revised: Appendix D describes our algorithmforlearningtheweightstobeusedinweightedvoting, andAppendixEpresentsinitialempirical results with the voting ensemble.

Original: Our results show that improving the CRA via a voting ensemble can be difﬁcult on realistic datasets since it requires the ensemble to demonstrate a suitable balance between diversity and similarity of its constituents, but we believe that this is a fruitful direction for future research.
Revised: Our results show that improving the CRA via a voting ensemble can be difﬁcult on realistic datasets since it requires the ensemble to demonstrate a suitable balance between diversity and similarity of its constituents, but we believe that this is a fruitful direction for future research.

Original: 5. W EIGHTED V OTING E NSEMBLE
Revised: 5. W EIGHTED V OTING E NSEMBLE

Original: Voting ensembles run a vote amongst their constituent classiﬁers to make a prediction.
Revised: Voting ensembles run a vote amongst their constituent classiﬁers to make a prediction.

Original: In the simplest case, each constituent has a single vote that gets assigned to their predicted label.
Revised: In the simplest case, each constituent has a single vote that gets assigned to their predicted label.

Original: The label with the maximum number of votes is chosen as the ensemble’s prediction.
Revised: The label with the maximum number of votes is chosen as the ensemble’s prediction.

Original: More generally, weighted voting allows a single constituent to be allocated more than one vote.
Revised: More generally, weighted voting allows a single constituent to be allocated more than one vote.

Original: The decimal number of votes allocated to each constituent is referred to as its weight .
Revised: The decimal number of votes allocated to each constituent is referred to as its weight .

Original: For simplicity, we assume that weights of the constituents in an ensemble are normalized so that they sum up to 1.
Revised: For simplicity, we assume that weights of the constituents in an ensemble are normalized so that they sum up to 1.

Original: We use weighted voting to not only choose the ensemble’s prediction but to also decide its certiﬁcation outcome.
Revised: We use weighted voting to not only choose the ensemble’s prediction but to also decide its certiﬁcation outcome.

Original: The interaction between voting and certiﬁcation is subtle and needs careful design to ensure that the certiﬁcation procedure is sound.
Revised: The interaction between voting and certiﬁcation is subtle and needs careful design to ensure that the certiﬁcation procedure is sound.

Original: Extra Notations.
Revised: Extra Notations.

Original: Let v wx ( j,c ) denote the total number of votes allocated to certiﬁable classiﬁers, ˜
Revised: Let v wx ( j,c ) denote the total number of votes allocated to certiﬁable classiﬁers, ˜

Original: F ( i ) , in the ensemble that output ( j,c ) .
Revised: F ( i ) , in the ensemble that output ( j,c ) .

Original: More formally, for an input x , label j ∈Y , certiﬁcation outcome c ∈ { 0 , 1 } ,weight w ∈ [0 , 1] N , and a set of constituent certiﬁable classiﬁers, ˜ F (0) ,..., ˜ F ( N − 1) , let v wx ( j,c ) :
Revised: More formally, for an input x , label j ∈Y , certiﬁcation outcome c ∈ { 0 , 1 } ,weight w ∈ [0 , 1] N , and a set of constituent certiﬁable classiﬁers, ˜ F (0) ,..., ˜ F ( N − 1) , let v wx ( j,c ) :

Original: = (cid:80) N − 1 i =

Revised: = (cid:80) N − 1 i =

Original: w i ∗ 

Revised: w i ∗

Original: [ ˜ F ( i ) ( x ) =
Revised: [ ˜ F ( i ) ( x ) =

Original: ( j,c )] .
Revised: ( j,c )] .

Original: We ﬁnd it useful to use v wx ( ∗ ,c ) to denote the number of votes for any class with certiﬁcate, c ; i.e., v wx ( ∗ ,c )=
Revised: We ﬁnd it useful to use v wx ( ∗ ,c ) to denote the number of votes for any class with certiﬁcate, c ; i.e., v wx ( ∗ ,c )=

Original: (cid:80) j ∈Y v wx ( j,x ) .
Revised: (cid:80) j ∈Y v wx ( j,x ) .

Original: Likewise, we will use v wx ( j ) to denote the number of votes for class
Revised: Likewise, we will use v wx ( j ) to denote the number of votes for class

Original: j regardless of certiﬁcate; i.e., v wx ( j )= v wx ( j, 0)+ v wx ( j, 1) .
Revised: j regardless of certiﬁcate; i.e., v wx ( j )= v wx ( j, 0)+ v wx ( j, 1) .

Original: Deﬁnition 5.1 (Weighted Voting Ensembler) .
Revised: Deﬁnition 5.1 (Weighted Voting Ensembler) .

Original: Let ˜ F (1) ,..., ˜ F ( N ) be N certiﬁable classiﬁers.
Revised: Let ˜ F (1) ,..., ˜ F ( N ) be N certiﬁable classiﬁers.

Original: A weighted voting ensembler, E wV : ˜ F N → ˜ F is deﬁned as follows
Revised: A weighted voting ensembler, E wV : ˜ F N → ˜ F is deﬁned as follows

Original: [Equation]
Revised: [Equation]

Original: where ˜ F cert ( x ):= (cid:26) 1 if ∀ j (cid:54) =
Revised: where ˜ F cert ( x ):= (cid:26) 1 if ∀ j (cid:54) =

Original: ˜ F label ( x ) .
Revised: ˜ F label ( x ) .

Original: [Equation]{+v wx ( ˜ F label ( x ) , 1) > v wx ( ∗ , 0)+ v wx ( j, 1) 
+}
Revised: {+v wx ( ˜ F label ( x ) , 1) > v wx ( ∗ , 0)+ v wx ( j, 1) 
+}

Original: otherwise
Revised: otherwise

Original: The prediction of the weighted voting ensemble is the label receiving the maximum number of votes regardless of the certiﬁcate.
Revised: The prediction of the weighted voting ensemble is the label receiving the maximum number of votes regardless of the certiﬁcate.

Original: However, for the certiﬁcation outcome, the ensemble has to consider the certiﬁcates of the constituent models.
Revised: However, for the certiﬁcation outcome, the ensemble has to consider the certiﬁcates of the constituent models.

Original: The ensemble should be certiﬁed robust only if its prediction outcome, i.e., the label receiving the maximum number of votes (regardless of the certiﬁcate), can be guaranteed to not change in an (cid:15) -ball.
Revised: The ensemble should be certiﬁed robust only if its prediction outcome, i.e., the label receiving the maximum number of votes (regardless of the certiﬁcate), can be guaranteed to not change in an (cid:15) -ball.

Original: The condition under which ˜ F cert ( x )=1 ensures this is the case, and allows us to prove that weighted voting ensemblers are soundness-preserving (Theorem 5.2).
Revised: The condition under which ˜ F cert ( x )=1 ensures this is the case, and allows us to prove that weighted voting ensemblers are soundness-preserving (Theorem 5.2).

Original: A key observation underlying the condition is that only constituent classiﬁers that are not certiﬁed robust at the current input can change their predicted label in the (cid:15) -ball, and, in the worst case, transfer all their votes ( v wx ( ∗ , 0) ) to the label with the second highest number of votes at x .
Revised: A key observation underlying the condition is that only constituent classiﬁers that are not certiﬁed robust at the current input can change their predicted label in the (cid:15) -ball, and, in the worst case, transfer all their votes ( v w x ( ∗ , 0) ) to the label with the second highest number of votes at x .

Original: We believe that our proof of soundness-preservation is of independent interest.
Revised: We believe that our proof of soundness-preservation is of independent interest.

Original: We also note that weighted-voting ensemblers are query-access (formal proofs in Appendix A).
Revised: We also note that weighted-voting ensemblers are query-access (formal proofs in Appendix A).

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: The weighted voting ensembler E wV is soundness-preserving.
Revised: The weighted voting ensembler E wV is soundness-preserving.

Original: Deﬁnition 5.3 (Uniform Voting Ensembler) .
Revised: Deﬁnition 5.3 (Uniform Voting Ensembler) .

Original: Let ˜ F (0) ,..., ˜ F ( N − 1) be N certiﬁable classiﬁers.
Revised: Let ˜ F (0) ,..., ˜ F ( N − 1) be N certiﬁable classiﬁers.

Original: The uniform voting ensembler, E U : ˜ F N → ˜ F is a weighted voting ensembler that assigns equal weights to each classiﬁer, i.e. E U =
Revised: The uniform voting ensembler, E U : ˜ F N → ˜ F is a weighted voting ensembler that assigns equal weights to each classiﬁer, i.e. E U =

Original: E wV where ∀ i ∈{ 0 ,...,N − 1 } .w
Revised: E wV where ∀ i ∈{ 0 ,...,N − 1 } .w

Original: i =1 /N .
Revised: i =1 /N .

Original: 5. E FFECTIVENESS OF V OTING : A T HOUGHT E XPERIMENT
Revised: 5. E FFECTIVENESS OF V OTING : A T HOUGHT E XPERIMENT

Original: Voting ensembles require the constituents to strike the right balance between diversity and similarity to be effective.
Revised: Voting ensembles require the constituents to strike the right balance between diversity and similarity to be effective.

Original: In other words, while the constituents should be accurate and robust in different regions of the input space (diversity), these regions should also have some overlap (similarity).
Revised: In other words, while the constituents should be accurate and robust in different regions of the input space (diversity), these regions should also have some overlap (similarity).

Original: We conduct a thought experiment using a simple hypothetical example (Example. 5.4) where such a balance is struck.
Revised: We conduct a thought experiment using a simple hypothetical example (Example. 5.4) where such a balance is struck.

Original: The existence of this example provides evidence and hope that voting ensembles can improve the CRA .We
Revised: The existence of this example provides evidence and hope that voting ensembles can improve the CRA .We

Original: present the example informally here.
Revised: present the example informally here.

Original: The detailed, rigorous argument is in Appendix A.
Revised: The detailed, rigorous argument is in Appendix A.

Original: Example 5.4.
Revised: Example 5.4.

Original: Assume that we have a uniform voting ensemble ˜ F with three constituent classiﬁers ˜
Revised: Assume that we have a uniform voting ensemble ˜ F with three constituent classiﬁers ˜

Original: F (0) , ˜ F (1) , and ˜ F (2) .
Revised: F (0) , ˜ F (1) , and ˜ F (2) .

Original: Assume that on a given dataset S k with 100 samples, each of the constituent classiﬁers has CRA equal to 0.5.
Revised: Assume that on a given dataset S k with 100 samples, each of the constituent classiﬁers has CRA equal to 0.5.

Original: Let’s say that the samples in S k are ordered such that ˜ F (0) is accurate and robust on the ﬁrst 50 samples (i.e., samples 0-49), ˜ F (1) is accurate and robust on samples 25-74, and ˜ F (2) on samples 0-24 and 50-74.
Revised: Let’s say that the samples in S k are ordered such that ˜ F (0) is accurate and robust on the ﬁrst 50 samples (i.e., samples 0-49), ˜ F (1) is accurate and robust on samples 25-74, and ˜ F (2) on samples 0-24 and 50-74.

Original: Then, for each of the ﬁrst 75 samples, two out of three constituents in the ensemble are accurate and robust.
Revised: Then, for each of the ﬁrst 75 samples, two out of three constituents in the ensemble are accurate and robust.

Original: Therefore, by Def. 5.1, the ensemble ˜ F is accurate and robust on samples 0-74, and has CRA equal to 0.75.
Revised: Therefore, by Def. 5.1, the ensemble ˜ F is accurate and robust on samples 0-74, and has CRA equal to 0.75.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Ensembling is a well-known approach for improving the accuracy of models as long as the constituent models are suitably diverse (Dietterich, 2000).
Revised: Ensembling is a well-known approach for improving the accuracy of models as long as the constituent models are suitably diverse (Dietterich, 2000).

Original: In recent years, with the growing focus on robust accuracy as a metric of model quality, a number of ensembling techniques have been proposed for improving this metric.
Revised: In recent years, with the growing focus on robust accuracy as a metric of model quality, a number of ensembling techniques have been proposed for improving this metric.

Original: Depending on whether an ensemble is query-access or not (i.e., does not or does require access to the internal details of the constituent models for prediction and certiﬁcation), it can be classiﬁed as a white-box or a black-box ensemble.
Revised: Depending on whether an ensemble is query-access or not (i.e., does not or does require access to the internal details of the constituent models for prediction and certiﬁcation), it can be classiﬁed as a white-box or a black-box ensemble.

Original: The modularity of black-box ensembles is attractive as the constituent classiﬁers can each be from a different model family (i.e., neural networks, decision trees, support vector machines, etc.) and each use a different mechanism for robustness certiﬁcation.
Revised: The modularity of black-box ensembles is attractive as the constituent classiﬁers can each be from a different model family (i.e., neural networks, decision trees, support vector machines, etc.) and each use a different mechanism for robustness certiﬁcation.

Original: The constituents of white-box ensembles, on the other hand, tend to be from the same model family but this provides the beneﬁt of tuning the ensembling strategy to the model family being used.
Revised: The constituents of white-box ensembles, on the other hand, tend to be from the same model family but this provides the beneﬁt of tuning the ensembling strategy to the model family being used.

Original: White-box ensembles.
Revised: White-box ensembles.

Original: Several works (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) present certiﬁable ensembles where the ensemble logits are calculated by averaging the corresponding logits of the constituent classiﬁers.
Revised: Several works (Yang et al., 2022; Zhang et al., 2019; Liu et al., 2020) present certiﬁable ensembles where the ensemble logits are calculated by averaging the corresponding logits of the constituent classiﬁers.

Original: Needing access to the logits of the constituent classiﬁers, and not just their predictions, is one aspect that makes these ensembles white-box.
Revised: Needing access to the logits of the constituent classiﬁers, and not just their predictions, is one aspect that makes these ensembles white-box.

Original: More importantly, the approaches used by these ensembles for local robustness certiﬁcation are also in violation of our deﬁnition of query-access ensembles (Def. 2.5).
Revised: More importantly, the approaches used by these ensembles for local robustness certiﬁcation are also in violation of our deﬁnition of query-access ensembles (Def. 2.5).

Original: For instance, randomized smoothing (Cohen et al., 2019) is used in (Yang et al., 2022; Liu et al., 2020) to certify the ensemble, which requires evaluating the constituent models on a large number of inputs for each prediction, and not just one input.
Revised: For instance, randomized smoothing (Cohen et al., 2019) is used in (Yang et al., 2022; Liu et al., 2020) to certify the ensemble, which requires evaluating the constituent models on a large number of inputs for each prediction, and not just one input.

Original: Other approaches (Zhang et al., 2019) use interval bound propagation (IBP)
Revised: Other approaches (Zhang et al., 2019) use interval bound propagation (IBP)

Original: (Gowal et al., 2018; Zhang et al., 2018) to certify the ensemble.
Revised: (Gowal et al., 2018; Zhang et al., 2018) to certify the ensemble.

Original: Calculating the interval bounds requires access to the architecture and weights of each of the constituent models, violating the requirements of a query-access ensemble.
Revised: Calculating the interval bounds requires access to the architecture and weights of each of the constituent models, violating the requirements of a query-access ensemble.

Original: A number of white-box ensembling techniques (Pang et al., 2019; Yang et al., 2020; Kariyappa & Qureshi, 2019; Sen et al., 2020; Zhang et al., 2022) only aim to improve empirical robust accuracy, i.e., these ensembles do not provide a robustness certiﬁcate.
Revised: A number of white-box ensembling techniques (Pang et al., 2019; Yang et al., 2020; Kariyappa & Qureshi, 2019; Sen et al., 2020; Zhang et al., 2022) only aim to improve empirical robust accuracy, i.e., these ensembles do not provide a robustness certiﬁcate.

Original: As before, the ensemble logits are calculated by averaging the corresponding logits of the constituent models.
Revised: As before, the ensemble logits are calculated by averaging the corresponding logits of the constituent models.

Original: These approaches differ from each other in the training interventions used to promote diversity in the constituent models.
Revised: These approaches differ from each other in the training interventions used to promote diversity in the constituent models.

Original: Black-box ensembles.
Revised: Black-box ensembles.

Original: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are the most popular example of certiﬁably robust black-box ensembles.
Revised: Cascading ensembles (Wong et al., 2018; Blum et al., 2022) are the most popular example of certiﬁably robust black-box ensembles.

Original: While Wong et al.
Revised: While Wong et al.

Original: (2018) empirically evaluate their cascading ensemble, the results of Blum et al.
Revised: (2018) empirically evaluate their cascading ensemble, the results of Blum et al.

Original: (2022) are purely theoretical.
Revised: (2022) are purely theoretical.

Original: However, as we show in this work, the certiﬁcation mechanism used by cascading ensembles is unsound.
Revised: However, as we show in this work, the certiﬁcation mechanism used by cascading ensembles is unsound.

Original: Devvrit et al.
Revised: Devvrit et al.

Original: Sen et al.
Revised: Sen et al.

Original: (2020) present a black-box voting ensemble but, unlike our voting ensemble, their ensemble does not provide robustness certiﬁcates.
Revised: (2020) present a black-box voting ensemble but, unlike our voting ensemble, their ensemble does not provide robustness certiﬁcates.

Original: Nevertheless, they are able to show improvements in the empirical robust accuracy with the voting ensemble.
Revised: Nevertheless, they are able to show improvements in the empirical robust accuracy with the voting ensemble.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we showed that the local robustness certiﬁcation mechanism used by cascading ensembles is unsound.
Revised: In this paper, we showed that the local robustness certiﬁcation mechanism used by cascading ensembles is unsound.

Original: As a consequence, existing empirical results that report the certiﬁed robust accuracies ( CRA ) of cascading ensembles are not valid.
Revised: As a consequence, existing empirical results that report the certiﬁed robust accuracies ( CRA ) of cascading ensembles are not valid.

Original: Guided by our theoretical results, we designed an attack algorithm against cascading ensembles and demonstrated that their unsoundness can be easily exploited inpractice.
Revised: Guided by our theoretical results, we designed an attack algorithm against cascading ensembles and demonstrated that their unsoundness can be easily exploited inpractice.

Original: Infact, theperformanceoftheensemblesismarkedlyworsethantheirsinglebestconstituent model.
Revised: Infact, theperformanceoftheensemblesismarkedlyworsethantheirsinglebestconstituent model.

Original: Finally, we presented an alternate black-box ensembling mechanism based on weighted voting that we prove to be sound, and, via a thought experiment, showed that voting ensembles can signiﬁcantly improve the CRA if the constituent models have the right balance between diversity and similarity.
Revised: Finally, we presented an alternate black-box ensembling mechanism based on weighted voting that we prove to be sound, and, via a thought experiment, showed that voting ensembles can signiﬁcantly improve the CRA if the constituent models have the right balance between diversity and similarity.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: Our work sheds light on existing vulnerabilities in state-of-the-art certiﬁably robust neural classiﬁers.
Revised: Our work sheds light on existing vulnerabilities in state-of-the-art certiﬁably robust neural classiﬁers.

Original: The presented attacks can be used by malicious entities to adversarially attack deployed cascading ensembles of certiﬁably robust models.
Revised: The presented attacks can be used by malicious entities to adversarially attack deployed cascading ensembles of certiﬁably robust models.

Original: However, by putting this knowledge out in the public domain and making practitioners aware of the existence of the problem, we hope that precautions can be taken to protect existing systems.
Revised: However, by putting this knowledge out in the public domain and making practitioners aware of the existence of the problem, we hope that precautions can be taken to protect existing systems.

Original: Moreover, it highlights the need to harden future systems against such attacks.
Revised: Moreover, it highlights the need to harden future systems against such attacks.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: To examine our theoretical results, the proof of Theorem 2.8 directly follows the body of the theorem in Section 2.2 while the proof of Theorem 5.2 is delayed to Appendix A and F, together with the proofs of other theorems that only appear in the appendix, i.e. Theorem A.1, A.2 (Appendix A) and Theorem F.2 (Appendix F).
Revised: To examine our theoretical results, the proof of Theorem 2.8 directly follows the body of the theorem in Section 2.2 while the proof of Theorem 5.2 is delayed to Appendix A and F, together with the proofs of other theorems that only appear in the appendix, i.e. Theorem A.1, A.2 (Appendix A) and Theorem F.2 (Appendix F).

Original: All the datasets used in our work are publicly available with links in their corresponding reference.
Revised: All the datasets used in our work are publicly available with links in their corresponding reference.

Original: Our experimental code is uploaded in the supplementary material with a detailed README ﬁle and weights of models to reproduce the results in Table 1, 2, 5, 6, 7, and 8.
Revised: Our experimental code is uploaded in the supplementary material with a detailed README ﬁle and weights of models to reproduce the results in Table 1, 2, 5, 6, 7, and 8.

Original: Moreover, hyper-parameters used in these table are also documented in Appendix B and D.
Revised: Moreover, hyper-parameters used in these table are also documented in Appendix B and D.

Original: The hardware information used in all experiments is reported in Section 4.
Revised: The hardware information used in all experiments is reported in Section 4.

Original: A P ROOFS
Revised: A P ROOFS

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: The weighted voting ensembler E wV is soundness-preserving.
Revised: The weighted voting ensembler E wV is soundness-preserving.

Original: Proof.
Revised: Proof.

Original: Let ˜ F (0) , ..., ˜ F ( N − 1) be N certiﬁable classiﬁers, which we assume are sound.
Revised: Let ˜ F (0) , ..., ˜ F ( N − 1) be N certiﬁable classiﬁers, which we assume are sound.

Original: Let ˜ F := E wV ( ˜ F (0) ,..., ˜ F ( N − 1) ) ; i.e., ˜ F is given by Deﬁnition 5.1.
Revised: Let ˜ F := E wV ( ˜ F (0) ,..., ˜ F ( N − 1) ) ; i.e., ˜ F is given by Deﬁnition 5.1.

Original: Assume for the sake of contradiction ∃ x,x (cid:48)
Revised: Assume for the sake of contradiction ∃ x,x (cid:48)

Original: s.t.
Revised: s.t.

Original: || x − x (cid:48) ||≤ (cid:15) , ˜ F ( x )=( j 1 , 1) , and ˜ F label ( x (cid:48) )=
Revised: || x − x (cid:48) || ≤ (cid:15) , ˜ F ( x ) = ( j 1 , 1) , and ˜ F label ( x (cid:48) ) =

Original: j 2 where j 2 (cid:54) =
Revised: jwhere j 2 (cid:54) =

Original: j 1 .
Revised: j 1 .

Original: Since ˜ F ( x )=( j 1 , 1) , by Deﬁnition 5.1., ∀ j (cid:54) =
Revised: Since ˜ F ( x ) = ( j 1 , 1) , by Deﬁnition 5.1., ∀ j (cid:54) =

Original: j 1 , v wx ( j 1 , 1) >v wx ( ∗ , 0)+ v wx ( j, 1) , and thus, in particular, Equation 5 holds.
Revised: j 1 , v wx ( j 1 , 1) > v wx ( ∗ , 0) + v wx ( j, 1) , and thus, in particular, Equation 5 holds.

Original: [Equation]
Revised: [Equation]

Original: Consider the votes on x (cid:48) .
Revised: Consider the votes on x (cid:48) .

Original: The models that contribute to v wx ( j 1 , 1) are all locally robust 6 at x , so each of these models must output the label j 1 on x (cid:48) , which is at distance no greater than (cid:15) from x ; thus
Revised: The models that contribute to v wx ( j 1 , 1) are all locally robust 6 at x , so each of these models must output the label j 1 on x (cid:48) , which is at distance no greater than (cid:15) from x ; thus

Original: Equation 6 holds.
Revised: Equation 6 holds.

Original: [Equation]
Revised: [Equation]

Original: Conversely, only points that are non-robust at x can change labels on x (cid:48) , thus we obtain Equation 7.
Revised: Conversely, only points that are non-robust at x can change labels on x (cid:48) , thus we obtain Equation 7.

Original: [Equation]
Revised: [Equation]

Original: Putting things together we have
Revised: Putting things together we have

Original: [Equation]
Revised: [Equation]

Revised: >v wx (

Revised: [Equation]

Revised: [Equation]

Original: >v wx ( ∗ , 0)+ v wx ( j 2 , 1) by (5) ≥ v wx (cid:48)
Revised: wx [Equation] [Equation] by (5) ≥ v wx (cid:48)

Original: ( j 2 ) by (7)
Revised: ( j 2 ) by (7)

Original: Thus, since v wx (cid:48) ( j 1 ) >v wx (cid:48)
Revised: Thus, since v wx (cid:48) ( j 1 ) >v wx (cid:48)

Original: ( j 2 ) , ˜ F label ( x (cid:48) ) cannot be j(cid:18) .
Revised: ( j 2 ) , ˜ F label ( x (cid:48) ) cannot be j(cid:18) .

Original: Example 5.4.
Revised: Example 5.4.

Original: We want to show that, ∃ ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1) ∈ ˜ F , w ∈ [0 , 1] N such that for ˜ F 
Revised: We want to show that, ∃ ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1) ∈ ˜ F , w ∈ [0 , 1] N such that for ˜ F :=

Original: [Equation]
Revised: [Equation]

Original: Consider a weighted voting ensemble ˜ F constituted of certiﬁable classiﬁers ˜ F (0) , ˜ F (1) , and ˜ F (2) with weights w =( 13 , 13 , 13 ) , i.e., ˜ F is a uniform voting ensemble.
Revised: Consider a weighted voting ensemble ˜ F constituted of certiﬁable classiﬁers ˜ F (0) , ˜ F (1) , and ˜ F (2) with weights w =( 13 , 13 , 13 ) , i.e., ˜ F is a uniform voting ensemble.

Original: Suppose k = 100 , i.e., S k is a dataset with 100 samples.
Revised: Suppose k = 100 , i.e., S k is a dataset with 100 samples.

Original: Moreover, lets say that
Revised: Moreover, lets say that

Original: CRA
Revised: CRA

Original: [Equation]
Revised: [Equation]

Revised: = CRA

Revised: [Equation]

Original: 2) , S k ) = 0 .
Revised: ˜ F (2) , S k ) = 0 .

Original: Also, suppose that the samples in
Revised: Also, suppose that the samples in

Original: S k are arranged in a ﬁxed sequence S k =( x 0 ,y 0 ) ,..., ( x k − 1 ,y k − 1 ) such that,
Revised: S k are arranged in a ﬁxed sequence S k =( x 0 ,y 0 ) ,..., ( x k − 1 ,y k − 1 ) such that,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (10) where [ i,j ] is the set of integers from i to j , i and j included.
Revised: where [ i,j ] is the set of integers from i to j , i and j included.

Original: 8, 9, and 10 are consistent with the fact that certiﬁed robust accuracy of each model is 0.5.
Revised: 8, 9, and 10 are consistent with the fact that certiﬁed robust accuracy of each model is 0.5.

Original: From 8, 9, 10, and Deﬁnition 5.1,
Revised: From 8, 9, 10, and Deﬁnition 5.1,

Original: [Equation]
Revised: [Equation]

Original: From 12 and Deﬁnition 5.1,
Revised: From 12 and Deﬁnition 5.1,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: From 11, 13, and Deﬁnition 2.2,
Revised: From 11, 13, and Deﬁnition 2.2,

Original: [Equation]
Revised: [Equation]

Original: Theorem A.1.
Revised: Theorem A.1.

Original: The cascading ensembler E C is query-access.
Revised: The cascading ensembler E C is query-access.

Revised: [Table]

Original: Proof.
Revised: Proof.

Original: Let ˜ F := E C ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .
Revised: Let ˜ F := E C ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .

Original: Let g (0) ,g (1) ,...,g ( N − 1) ∈Y×{ 0 , 1 } .
Revised: Let g (0) ,g (1) ,...,g ( N − 1) ∈Y×{ 0 , 1 } .

Original: We use g ( j ) 2 to refer to the second element in the pair g ( j ) .
Revised: We use g ( j ) 2 to refer to the second element in the pair g ( j ) .

Original: Deﬁne G as follows,
Revised: Deﬁne G as follows,

Original: [Equation]
Revised: [Equation]

Original: if ∃ j ≤ N − 1 .

Original: c

Original: [Equation] = g [Equation] otherwise where c ( j ):=1 if ( g ( j ) 2 =1) ∧
Revised: otherwise where c ( j ):=1 if ( g ( j ) 2 =1) ∧

Original: ( ∀ i<
Revised: ( ∀ i<

Original: j,g ( i ) 2 =0) and 0 otherwise.
Revised: j,g ( i ) 2 =0) and 0 otherwise.

Original: Then, by Def. 2.7, ˜ F ( x ) = G ( ˜ F (0) ( x ) , ˜ F (1) ( x ) , ... , ˜ F ( N − 1) ( x )) .
Revised: Then, by Def. 2.7, ˜ F ( x ) = G ( ˜ F (0) ( x ) , ˜ F (1) ( x ) ,..., ˜ F ( N − 1) ( x )) .

Original: Then, by Def. 2.5, cascading ensembler is query-access.
Revised: Then, by Def. 2.5, cascading ensembler is query-access.

Original: Theorem A.2.
Revised: Theorem A.2.

Original: The weighted voting ensembler E wV is query-access.
Revised: The weighted voting ensembler E wV is query-access.

Original: Proof.
Revised: Proof.

Original: Let ˜ F := E wV ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .
Revised: Let ˜ F := E wV ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .

Original: Let g (0) ,g (1) ,...,g ( N − 1) ∈Y ×{ 0 , 1 } .
Revised: Let g (0) ,g (1) ,...,g ( N − 1) ∈Y ×{ 0 , 1 } .

Original: We use ¯ g to refer to the set { g (0) ,g (1) ,...,g ( N − 1) } .
Revised: We use ¯ g to refer to the set { g (0) ,g (1) ,...,g ( N − 1) } .

Original: Deﬁne G as follows, 
Revised: Deﬁne G as follows, G

Original: [Equation]
Revised: [Equation]

Original: ) ):= (cid:0) G 1 ( g (0) ,g (1) ,...,g ( N − 1) ) ,G 2 ( g (0) ,g (1) ,...,g ( N − 1) ))
Revised: N [Equation] [Equation] ) where ˆ j = G

Revised: [Equation]

Revised: ( g

Revised: [Equation]

Revised: ,g

Revised: [Equation]

Revised: ,g

Revised: [Equation]

Revised: N

Original: where ˆ
Revised: [Equation] argmax j (cid:110) v [Equation] g

Original: [Equation]
Revised: [Equation]

Original: G
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  1 if ∀ 
Revised: G 2 ( g (0) ,g (1) ,...,g ( N − 1) ):= (cid:26) 1 if ∀ j (cid:54) = ˆ j . v w ¯ g

Original: [Equation] [Equation] (ˆ j, 1) >v w ¯ {+g ( ∗ , 0)+ v w ¯ g ( j, 1) 
+}
Revised: (ˆ j, 1) >v w ¯ {+g ( ∗ , 0)+ v w ¯ g ( j, 1) 
+}

Revised: otherwise and v w

Original: [Equation]
Revised: [Equation]

Original: ( ∗ , 0)+ v w ¯ g ( j, 1) 0 otherwise and
Revised: i [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then, by Def. 5.1, ˜ F ( x ) = G ( ˜ F (0) ( x ) , ˜ F (1) ( x ) ,..., ˜ F ( N − 1) ( x )) .
Revised: Then, by Def. 5.1, ˜ F ( x ) = G ( ˜ F (0) ( x ) , ˜ F (1) ( x ) ,..., ˜ F ( N − 1) ( x )) .

Original: Then, by Def. 2.5, weighted voting ensembler is query-access.
Revised: Then, by Def. 2.5, weighted voting ensembler is query-access.

Original: B H YPER - PARAMETERS OF T ABLE
Revised: B H YPER - PARAMETERS OF T ABLE

Original: In Table 3, we report hyper-parameters used to run CasA to reach the statistics reported in Table 1.
Revised: In Table 3, we report hyper-parameters used to run CasA to reach the statistics reported in Table 1.

Original: Notice that if a normalization is ‘ µ = [0.485, 0.456, 0.406], σ = 0.225’, we divide the (cid:15) and step size by σ during the experiment.
Revised: Notice that if a normalization is ‘ µ = [0.485, 0.456, 0.406], σ = 0.225’, we divide the (cid:15) and step size by σ during the experiment.

Original: We use SGD as the optimizer for all experiments.
Revised: We use SGD as the optimizer for all experiments.

Original: C A TTACKING N ON -S EQUENTIALLY T RAINED C ASCADING E NSEMBLES
Revised: C A TTACKING N ON -S EQUENTIALLY T RAINED C ASCADING E NSEMBLES

Original: Wong et al.
Revised: Wong et al.

Original: (2018) train cascading ensembles in a sequential manner, i.e., each model in the sequence is only trained on those training samples that could not be certiﬁed robust by any of the previous models.
Revised: (2018) train cascading ensembles in a sequential manner, i.e., each model in the sequence is only trained on those training samples that could not be certiﬁed robust by any of the previous models.

Original: The training algorithm is described by Wong et al.
Revised: The training algorithm is described by Wong et al.

Original: (2018) in appendix C (Algorithm 2) of their paper.
Revised: (2018) in appendix C (Algorithm 2) of their paper.

Original: We evaluate the efﬁcacy of our attack algorithm (CasA) on cascade ensembles trained in a non-sequential manner.
Revised: We evaluate the efﬁcacy of our attack algorithm (CasA) on cascade ensembles trained in a non-sequential manner.

Original: That is, each constituent model is trained independently on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training.
Revised: That is, each constituent model is trained independently on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training.

Original: Besides this difference, the code, architecture, hyperparameters, and data used for training are the same as that used by Wong et al.
Revised: Besides this difference, the code, architecture, hyperparameters, and data used for training are the same as that used by Wong et al.

Original: For every combination of dataset,[Table] [Table] architecture, and (cid:15) value, we train three constituent models, and use them to construct non-sequentially trained cascading ensembles.
Revised: For every combination of dataset, architecture, and (cid:15) value, we train three constituent models, and use them to construct non-sequentially trained cascading ensembles.

Original: Table 4 shows the results of running our attack on such cascades with non-sequentially trained constituents.
Revised: Table 4 shows the results of running our attack on such cascades with non-sequentially trained constituents.

Original: We make the following observations:
Revised: We make the following observations:

Original:  • The non-sequentially trained ensembles continue to be unsound and our attack is able to ﬁnd adversarial inputs (demonstrated by non-zero FPR).
Revised: [Table] • The non-sequentially trained ensembles continue to be unsound and our attack is able to ﬁnd adversarial inputs (demonstrated by non-zero FPR).

Original: • The success rate of our attack is much lower than on the sequentially-trained models shared by Wong et al.
Revised: • The success rate of our attack is much lower than on the sequentially-trained models shared by Wong et al.

Original: • The unsound CRA of these ensembles is comparable to that of the sequentially-trained models.
Revised: • The unsound CRA of these ensembles is comparable to that of the sequentially-trained models.

Original: We hypothesize that our attack demonstrates much higher success rates on sequentially-trained models because, when trained sequentially, it is likely that the later models in the cascade are degenerate, i.e., are very robust but with low accuracy (similar to constant functions).
Revised: We hypothesize that our attack demonstrates much higher success rates on sequentially-trained models because, when trained sequentially, it is likely that the later models in the cascade are degenerate, i [Table] e., are very robust but with low accuracy (similar to constant functions).

Original: Then, to attack the ensemble, we just need to ﬁnd an attack input where the initial models cannot be certiﬁed, since the remaining degenerate models are typically robust and inaccurate.
Revised: Then, to attack the ensemble, we just need to ﬁnd an attack input where the initial models cannot be certiﬁed, since the remaining degenerate models are typically robust and inaccurate.

Original: The degeneracy of the later models in the sequentially-trained ensemble may also explain why the unsound CRAs of the two kinds of ensembles are comparable.
Revised: The degeneracy of the later models in the sequentially-trained ensemble may also explain why the unsound CRAs of the two kinds of ensembles are comparable.

Original: Models are trained in a sequential manner to enhance their "diversity".
Revised: Models are trained in a sequential manner to enhance their "diversity".

Original: However, due to the degeneracy of the later models, the sequentially-trained ensembles likely end up being about only as diverse as the non-sequentially trained ensembles.
Revised: However, due to the degeneracy of the later models, the sequentially-trained ensembles likely end up being about only as diverse as the non-sequentially trained ensembles.

Original: Finally, we note that the sequential style of training cascade ensembles is quite natural.
Revised: Finally, we note that the sequential style of training cascade ensembles is quite natural.

Original: In fact, both,
Revised: In fact, both,

Original: Wong et al.
Revised: Wong et al.

Original: (2018) and Blum et al.
Revised: (2018) and Blum et al.

Original: (2022) train models in a sequential manner.
Revised: (2022) train models in a sequential manner.

Original: But these results suggest that sequentially training may make it easier to exploit the unsoundness of cascading ensembles.
Revised: But these results suggest that sequentially training may make it easier to exploit the unsoundness of cascading ensembles.

Original: D W EIGHTED V OTING E NSEMBLE : L EARNING W EIGHTS
Revised: D W EIGHTED V OTING E NSEMBLE : L EARNING W EIGHTS

Original: The weights w in E wV determines the importance of each constituent classiﬁer in the ensemble.
Revised: The weights w in E wV determines the importance of each constituent classiﬁer in the ensemble.

Original: Given a set of k labeled inputs,
Revised: Given a set of k labeled inputs,

Original: S k
Revised: S k

Original: (e.g. the training set), we would like to learn the optimal weights w that maximize the ensembler’s CRA (Def. 2.2) over S k .
Revised: (e.g. the training set), we would like to learn the optimal weights w that maximize the ensembler’s CRA (Def. 2.2) over S k .

Original: When S k resembles the true distribution of the[Table] test points, the learned w is expected to be close to the optimal weights that maximizes the CRA of the test set.
Revised: When S k resembles the true distribution of the test points, the learned w is expected to be close to the optimal weights that maximizes the CRA of the test set.

Original: Weight optimization over S k naturally takes the following form.
Revised: Weight optimization over S k naturally takes the following form.

Original: [Equation]
Revised: [Equation]

Original: For the indicator to output 1, it is required that the margin of votes be greater than 0, i.e.
Revised: For the indicator to output 1, it is required that the margin of votes be greater than 0, i.e.

Original: ∆ wx i ( y i ) := v wx i ( y i , 1) − v wx i ( ∗ , 0)
Revised: ∆ wx i ( y i ) := v wx i ( y i , 1) − v wx i ( ∗ , 0)

Original: − max j (cid:54) =
Revised: − max j (cid:54) =

Original: y i { v wx i ( j, 1) } > 0 .
Revised: y i { v wx i ( j, 1) } > 0 .

Original: Namely, the votes for the class
Revised: Namely, the votes for the class

Original: y i , i.e. v wx i
Revised: y i , i.e. v wx i

Original: ( y i , 1) , must be greater than the votes for all other classes i.e. max j (cid:54) =
Revised: ( y i , 1) , must be greater than the votes for all other classes i.e. max j (cid:54) =

Original: y i { v wx i ( j, 1) } plus the votes for non-robust predictions v wx i ( ∗ , 0) as discussed in Def. 5.1.
Revised: y i { v wx i ( j, 1) } plus the votes for non-robust predictions v wx i ( ∗ , 0) as discussed in Def. 5.1.

Original: Eq.(15) then becomes:
Revised: Eq.(15) then becomes:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The indicator function is not differentiable so we replace it with a differentiable and monotonically increasing function s , which leads to Eq.
Revised: The indicator function is not differentiable so we replace it with a differentiable and monotonically increasing function s , which leads to Eq.

Original: [Equation]
Revised: [Equation]

Original: In this paper, we choose s to be the sigmoid function σ t where t is the temperature only for negative inputs, i.e., σ t ( x ) :
Revised: In this paper, we choose s to be the sigmoid function σ t where t is the temperature only for negative inputs, i.e., σ t ( x ) :

Original: = σ ( x ) if x > 0 and σ ( x/t ) otherwise, where σ is the standard sigmoid function.
Revised: = σ ( x ) if x > 0 and σ ( x/t ) otherwise, where σ is the standard sigmoid function.

Original: Sigmoid is non-negative so margins with opposite signs do not cancel, and it also avoids biasing training towards producing larger margins on a small number of points. 
Revised: Sigmoid is non-negative so margins with opposite signs do not cancel, and it also avoids biasing training towards producing larger margins on a small number of points. Indeed, vanishing gradients are [Table]

Original: Indeed, vanishing gradients are useful on points around large positive margins, so the temperature is only applied on negative inputs.
Revised: [Table] useful on points around large positive margins [Table] so the temperature is only applied on negative inputs.

Original: This leads us to Eq.
Revised: This leads us to Eq.

Original: 18, the optimization objective we solve for optimal weights w ∗.
Revised: 18, the optimization objective we solve for optimal weights w ∗

Revised: [Table]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: E W EIGHTED V OTING E NSEMBLE : E MPIRICAL R ESULTS
Revised: E W EIGHTED V OTING E NSEMBLE : E MPIRICAL R ESULTS

Original: The goal of these experiments is to evaluate the efﬁcacy of our sound voting ensemble.
Revised: The goal of these experiments is to evaluate the efﬁcacy of our sound voting ensemble.

Original: For our experiments, we use the pre-trained ensemble constituent models made available by Wong et al.
Revised: For our experiments, we use the pre-trained ensemble constituent models made available by Wong et al.

Original: (2018) to construct three kinds of ensembles, namely, cascading ensembles, uniform voting ensembles, and weighted voting ensembles.
Revised: (2018) to construct three kinds of ensembles, namely, cascading ensembles, uniform voting ensembles, and weighted voting ensembles.

Original: The weights for the weighted voting ensemble are learned in the manner described in Appendix D. We report certiﬁed robust accuracy ( CRA ) and standard accuracy ( Acc ) for each ensemble as well as for the best constituent model.
Revised: The weights for the weighted voting ensemble are learned in the manner described in Appendix D. We report certiﬁed robust accuracy ( CRA ) and standard accuracy ( Acc ) for each ensemble as well as for the best constituent model.

Original: Note that all these ensembles are query-access but only the uniform voting and weighted voting ensembles are soundness-preserving.
Revised: Note that all these ensembles are query-access but only the uniform voting and weighted voting ensembles are soundness-preserving.

Original: Consequently, the CRA reported for the cascading ensemble grossly overestimates the actual CRA as demonstrated by our attack results.
Revised: Consequently, the CRA reported for the cascading ensemble grossly overestimates the actual CRA as demonstrated by our attack results.

Original: We always set the temperature to 1e5 and learning rate to 1e-2 when learning the weights as described in Appendix D.
Revised: We always set the temperature to 1e5 and learning rate to 1e-2 when learning the weights as described in Appendix D.

Original: Table 5 shows the results for (cid:96) ∞ robustness.
Revised: Table 5 shows the results for (cid:96) ∞ robustness.

Original: Each row in the table represents a speciﬁc combination of dataset (MNIST or CIFAR-10), architecture (Small or Large convolutional networks or Resnet),[Table] and (cid:15) value used for local robustness certiﬁcation.
Revised: Each row in the table represents a speciﬁc combination of dataset (MNIST or CIFAR-10), architecture (Small or Large convolutional networks or Resnet), and (cid:15) value used for local robustness certiﬁcation.

Original: Table 6 shows the results for (cid:96) 2 robustness using constituent models pre-trained by Wong et al.[Table]
Revised: Table 6 shows the results for (cid:96) 2 robustness using constituent models pre-trained by Wong et al.

Original: Summary of Results.
Revised: Summary of Results.

Original: We see from Tables 5 and 6 that while the cascading ensemble appears to improve upon the CRA of the single best model in the ensemble, these numbers are misleading due to the unsoundness of the certiﬁcation mechanism.
Revised: We see from Tables 5 and 6 that while the cascading ensemble appears to improve upon the CRA of the single best model in the ensemble, these numbers are misleading due to the unsoundness of the certiﬁcation mechanism.

Original: The CRA for the uniform voting and weighted ensembles are consistently lower than that reported by the cascading ensemble [Table] and in many cases, signiﬁcantly so.
Revised: The CRA for the uniform voting and weighted ensembles are consistently lower than that reported by the cascading ensemble, and in many cases, signiﬁcantly so.

Original: Uniform voting ensembles stand-out for their low CRA but there is a simple explanation for these results.
Revised: Uniform voting ensembles stand-out for their low CRA but there is a simple explanation for these results.

Original: The constituent models are trained by Wong et al.
Revised: The constituent models are trained by Wong et al.

Original: (2018) in a cascading manner, i.e., later constituent models are trained on only those points that cannot be certiﬁed by the previous models.
Revised: (2018) in a cascading manner, i.e., later constituent models are trained on only those points that cannot be certiﬁed by the previous models.

Original: This strategy causes the subset of inputs labeled correct and certiﬁably robust by each constituent model to have minimal overlap.
Revised: This strategy causes the subset of inputs labeled correct and certiﬁably robust by each constituent model to have minimal overlap.

Original: However, voting ensembles need these input subsets to strike the right balance between diversity and overlap for improving the CRA .
Revised: However, voting ensembles need these input subsets to strike the right balance between diversity and overlap for improving the CRA .

Original: Another interesting observation is that, in most cases, the CRA of the weighted voting ensemble and the single best constituent model are the same.
Revised: Another interesting observation is that, in most cases, the CRA of the weighted voting ensemble and the single best constituent model are the same.

Original: This is again a consequence of the cascaded manner in which the constituent models are trained.
Revised: This is again a consequence of the cascaded manner in which the constituent models are trained.

Original: The ﬁrst model in the cascade typically vastly outperforms  the subsequent models.
Revised: The ﬁrst model in the cascade typically vastly outperforms [Table] [Table] the subsequent models.

Original: Moreover, as already mentioned, the constituent models have almost no overlap in the input regions where they perform well, and their presence only ends up harming the performance of the voting ensemble.
Revised: Moreover, as already mentioned, the constituent models have almost no overlap in the input regions where they perform well, and their presence only ends up harming the performance of the voting ensemble.

Original: As a consequence, the optimal normalized weights, learned by solving the optimization problem described in Appendix D, typically assign all the mass to the ﬁrst model.
Revised: As a consequence, the optimal normalized weights, learned by solving the optimization problem described in Appendix D, typically assign all the mass to the ﬁrst model.

Original: The detailed weights for each of the weighted voting ensemble are given in Tables 9, 10, 11, and 12.
Revised: The detailed weights for each of the weighted voting ensemble are given in Tables 9, 10, 11, and 12.

Original: These results suggest two takeaway messages.
Revised: These results suggest two takeaway messages.

Original: First, the cascaded strategy of Wong et al.
Revised: First, the cascaded strategy of Wong et al.

Original: (2018) for training constituent models is in conﬂict with the requirement that constituent models overlap in their behavior for voting ensembles to be effective.
Revised: (2018) for training constituent models is in conﬂict with the requirement that constituent models overlap in their behavior for voting ensembles to be effective.

Original: This gives up hope that if the constituent models are suitably trained, voting ensembles can improve the CRA .
Revised: This gives up hope that if the constituent models are suitably trained, voting ensembles can improve the CRA .

Original: We leave this exploration for future work.
Revised: We leave this exploration for future work.

Original: Second, even if the constituent models do not show the right balance between diversity and similarity, our weight learning procedure ensures that the performance of the weighted voting ensemble is no worse than the single best constituent model.
Revised: Second, even if the constituent models do not show the right balance between diversity and similarity, our weight learning procedure ensures that the performance of the weighted voting ensemble is no worse than the single best constituent model.

Original: Ideally, we would like the weights to be equally distributed since this conveys that every constituent in the ensemble has something to contribute.
Revised: Ideally, we would like the weights to be equally distributed since this conveys that every constituent in the ensemble has something to contribute.

Original: But, in the worst case, the weights play the role of a model selection procedure, assigning zero weights to constituent models that do not contribute to the ensemble.
Revised: But, in the worst case, the weights play the role of a model selection procedure, assigning zero weights to constituent models that do not contribute to the ensemble.

Original: Non-Sequential Training.
Revised: Non-Sequential Training.

Original: We conduct another set of experiments where instead of using the constituent models pre-trained by Wong et al.
Revised: We conduct another set of experiments where instead of using the constituent models pre-trained by Wong et al.

Original: (2018), we train them ourselves in a non-sequential manner.
Revised: (2018), we train them ourselves in a non-sequential manner.

Original: That is, each constituent model is trained on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training.
Revised: That is, each constituent model is trained on the entire train dataset, and each constituent only differs due to the randomness of initialization and of stochastic gradient descent during training.

Original: Besides this difference, the code, architecture, hyperparameters, and data used for training are the same as that used by
Revised: Besides this difference, the code, architecture, hyperparameters, and data used for training are the same as that used by

Original: Wong et al.
Revised: Wong et al.

Original: For every combination of dataset, architecture, and (cid:15) value, we train three constituent models, and use them to construct cascading, uniform voting, and weighted voting ensembles.
Revised: For every combination of dataset, architecture, and (cid:15) value, we train three constituent models, and use them to construct cascading, uniform voting, and weighted voting ensembles.

Original: Table 7 shows the results for (cid:96) ∞ robustness using non-sequentially trained constituent model ands
Revised: shows the results for (cid:96) ∞ robustness using non-sequentially trained constituent model ands

Original: Table 8 shows the results for (cid:96) 2 robustness.
Revised: Table 8 shows the results for (cid:96) 2 robustness.

Original: We observe that, for non-sequentially trained models, the CRA of uniform voting and weighted voting ensembles are comparable, and similar to the CRA of the single best constituent model in the ensemble.
Revised: We observe that, for non-sequentially trained models, the CRA of uniform voting and weighted voting ensembles are comparable, and similar to the CRA of the single best constituent model in the ensemble.

Original: In this case, the constituent models have too much overlap and almost no diversity.
Revised: In this case, the constituent models have too much overlap and almost no diversity.

Original: These results reafﬁrm our observation that voting ensembles require a balance between diversity and similarity to be effective.[Table] [Table] [Table]
Revised: These results reafﬁrm our observation that voting ensembles require a balance between diversity and similarity to be effective.

Original: [Table]

Original: F A N A LTERNATE F ORMULATION OF U NIFORM V OTING E NSEMBLERDeﬁnition
Revised: F A N A LTERNATE F ORMULATION OF U NIFORM V OTING E NSEMBLER

Original:  F.1 (Permutation-based Cascading Ensembler) .
Revised: Deﬁnition F.1 (Permutation-based Cascading Ensembler) .

Original: Let ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1) be N certiﬁable classiﬁers and N is odd.
Revised: Let ˜ F (0) , ˜ F (1) , ... , ˜ F ( N − 1) be N certiﬁable classiﬁers and N is odd.

Original: Suppose Π is the set of all permutations of { 0 , 1 ,...,N − 1 } .A [Table] [Table]
Revised: Suppose Π is the set of all permutations of { 0 , 1 ,...,N − 1 } .

Original:  permutation-based cascading ensembler E P : ˜ F N → ˜ F is deﬁned as follows
Revised: A permutation-based cascading ensembler E P : ˜ F N → ˜ F is deﬁned as follows

Original: E P ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) )( x ):= 
Revised: E P ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) )( x ):=  

Original:  ˜ F ( π 0 ) ( x ) if∃ π ∈ Π .
Revised: [Equation] [Equation] ) ( x ) if

Original: c2 ( π )=
Revised: [Equation]

Original: ( ˜ F ( π 0 ) label ( x ) , 0) if (cid:54)∃ π (cid:48) ∈ Π .
Revised: [Equation] x ) , 0) if (cid:54)∃ π (cid:48) [Equation]

Original: c2 ( π (cid:48) )
Revised: c2 ( [Equation]

Original: =
Revised: [Equation]

Original: 1 ∧ 
Revised: 1 ∧ ∃

Original: ∃ π ∈ Π . c1 ( π )=
Revised: [Equation] Π . c1 ( π )=

Original: ( ∗ , 0) otherwise where ∗ is a random label selected from Y 7 , π 0 refers to the ﬁrst element of the permutation π , c1 ( π ):= (cid:40)
Revised: ( ∗ , 0) otherwise [Table] [Table] [Table]

Original: [Equation]
Revised: [Table]

Original: otherwise
Revised: where ∗ is a random label selected from Y 7 , π 0 refers to the ﬁrst element of the permutation π ,

Original: [Equation]
Revised: [Equation]

Original: and i,j ∈{ 0 , 1 ,...,N − 1 } .
Revised: and i,j ∈{ 0 , 1 ,...,N − 1 } .

Original: Theorem F.2.
Revised: Theorem F.2.

Original: The permutation-based cascading ensembler E P is a soundness-preserving ensembler.
Revised: The permutation-based cascading ensembler E P is a soundness-preserving ensembler.

Original: Proof.
Revised: Proof.

Original: Let ˜ F := E P ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .
Revised: Let ˜ F := E P ( ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) ) .

Original: For ˜ F we want to show that, 
Revised: For ˜ F we want to show that, ∀ x ∈ R d , ˜ F cert ( x )= 1= ⇒ ˜

Revised: F label is (cid:15) -locally robust at x.

Original: [Equation]
Revised: [Equation]

Original: W.L.O.G suppose ˜ F label ( x ) =
Revised: W.L.O.G suppose ˜ F label ( x )=

Original: y .
Revised: y .

Original: If ˜ F ( x ) = ( y, 1) , let us assume that π is the permutation such that c2 ( π )
Revised: If ˜ F ( x )=( y, 1) , let us assume that π is the permutation such that c ( π )

Original: =1 .
Revised: =1 .

Original: Let k be the integer that makes c2 ( π )=1 to be true.
Revised: Let k be the integer that makes c2 ( π )=1 to be true.

Original: Thus
Revised: Thus

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: By our assumptions that ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) are sound, which are invariant to the permutation of these models.
Revised: By our assumptions that ˜ F (0) , ˜ F (1) ,..., ˜ F ( N − 1) are sound, which are invariant to the permutation of these models.

Original: Therefore, by Def. 2.3, ∀ i ≤ k ,
Revised: Therefore, by Def. 2.3, ∀ i ≤ k ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Eq.
Revised: Eq.

Original: (23) implies that ∀ x (cid:48)
Revised: (23) implies that ∀ x (cid:48)

Original: s.t.
Revised: s.t.

Original: || x (cid:48) − x ||≤ (cid:15) , the following statement is true
Revised: || x (cid:48) − x ||≤ (cid:15) , the following statement is true

Original: [Equation]
Revised: [Equation]

Original: Plug the condition (24) into Def.
Revised: Plug the condition (24) into Def.

Original: F.1, we ﬁnd that c1 ( π )
Revised: F.1, we ﬁnd that c1 ( π )

Original: = 1 for x (cid:48) .
Revised: = 1 for x (cid:48) .

Original: Moreover, there cannot be a permutation π (cid:48)
Revised: Moreover, there cannot be a permutation π (cid:48)

Original: such that c1 ( π (cid:48) )
Revised: such that c1 ( π (cid:48) )

Original: =
Revised: =

Original: c2 ( π (cid:48) )
Revised: c2 ( π (cid:48) )

Original: = 1 ∧ ˜ F ( π (cid:48) 0 ) label (cid:54) =
Revised: = 1 ∧ ˜ F ( π (cid:48) 0 ) label (cid:54) =

Original: y since k ≥ N +12 .
Revised: y since k ≥ N +12 .

Original: Therefore, ˜ F label [Equation] x [Equation]
Revised: Therefore, ˜ F label ( x (cid:48) )=

Original: = ˜ F ( [Equation] 0 ) label ( x 
Revised: ˜ F ( π 0 ) label ( x (cid:48) )

Original: [Equation]

Original: =
Revised: =

Original: y , and we arrive at the following statement,
Revised: y , and we arrive at the following statement,

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which completes the proof for the soundness of ˜ F label at any x .
Revised: which completes the proof for the soundness of ˜ F label at any x .