Original: C ERTIFIED R OBUSTNESS FOR T OP - k P REDICTIONS AGAINST A DVERSARIAL P ERTURBATIONS VIA R ANDOMIZED S MOOTHING
Revised: C ERTIFIED R OBUSTNESS FOR T OP - k P REDICTIONS AGAINST A DVERSARIAL P ERTURBATIONS VIA R ANDOMIZED S MOOTHING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: It is well-known that classiﬁers are vulnerable to adversarial perturbations.
Revised: It is well-known that classiﬁers are vulnerable to adversarial perturbations.

Original: To defend against adversarial perturbations, various certiﬁed robustness results have been derived.
Revised: To defend against adversarial perturbations, various certiﬁed robustness results have been derived.

Original: However, existing certiﬁed robustnesses are limited to top-1 predictions.
Revised: However, existing certiﬁed robustnesses are limited to top-1 predictions.

Original: In many real-world applications, top- k predictions are more relevant.
Revised: In many real-world applications, top- k predictions are more relevant.

Original: In this work, we aim to derive certiﬁed robustness for top- k predictions.
Revised: In this work, we aim to derive certiﬁed robustness for top- k predictions.

Original: In particular, our certiﬁed robustness is based on randomized smoothing , which turns any classiﬁer to a new classiﬁer via adding noise to an input example.
Revised: In particular, our certiﬁed robustness is based on randomized smoothing , which turns any classiﬁer to a new classiﬁer via adding noise to an input example.

Original: We adopt randomized smoothing because it is scalable to large-scale neural networks and applicable to any classiﬁer.
Revised: We adopt randomized smoothing because it is scalable to large-scale neural networks and applicable to any classiﬁer.

Original: We derive a tight robustness in (cid:96) 2 norm for top- k predictions when using randomized smoothing with Gaussian noise.
Revised: We derive a tight robustness in (cid:96) 2 norm for topk predictions when using randomized smoothing with Gaussian noise.

Original: We ﬁnd that generalizing the certiﬁed robustness from top-1 to top-
Revised: We ﬁnd that generalizing the certiﬁed robustness from top-1 to top-

Original: k predictions faces signiﬁcant technical challenges.
Revised: k predictions faces signiﬁcant technical challenges.

Original: We also empirically evaluate our method on CIFAR and ImageNet.
Revised: We also empirically evaluate our method on CIFAR10 and ImageNet.

Original: For example, our method can obtain an ImageNet classiﬁer with a certiﬁed top-5 accuracy of 62.8% when the (cid:96) 2 -norms of the adversarial perturbations are less than 0.5 (=127/255). 
Revised: For example, our method can obtain an ImageNet classiﬁer with a certiﬁed top-5 accuracy of 62.8% when the (cid:96) 2 -norms of the adversarial perturbations are less than 0.5 (=127/255). Our code is publicly available at: https://github.com/jjy1994/Certify_Topk .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Classiﬁers are vulnerable to adversarial perturbations (Szegedy et al., 2014; Goodfellow et al., 2015;
Revised: Classiﬁers are vulnerable to adversarial perturbations (Szegedy et al., 2014; Goodfellow et al., 2015;

Original: Carlini & Wagner, 2017b).
Revised: Carlini & Wagner, 2017b; Jia & Gong, 2018).

Original: Speciﬁcally, given an example x and a classiﬁer f , an attacker can carefully craft a perturbation δ such that f makes predictions for x + δ as the attacker desires.
Revised: Speciﬁcally, given an example x and a classiﬁer f , an attacker can carefully craft a perturbation δ such that f makes predictions for x + δ as the attacker desires.

Original: Various empirical defenses (e.g., Goodfellow et al. (2015); Svoboda et al.
Revised: Various empirical defenses (e.g., Goodfellow et al. ; Svoboda et al.

Original: ; Buckman et al.
Revised: ; Buckman et al.

Original: ; Ma et al.
Revised: ; Ma et al.

Original: ; Guo et al.
Revised: ; Guo et al.

Original: ; Dhillon et al.
Revised: ; Dhillon et al.

Original: ; Xie et al.
Revised: ; Xie et al.

Original: ; Song et al.
Revised: ; Song et al.

Original: (2018)) have been proposed to defend against adversarial perturbations.
Revised: (2018)) have been proposed to defend against adversarial perturbations.

Original: However, these empirical defenses were often soon broken by adaptive adversaries (Carlini & Wagner, 2017a; Athalye et al., 2018).
Revised: However, these empirical defenses were often soon broken by adaptive adversaries (Carlini & Wagner, 2017a; Athalye et al., 2018).

Original: As a response, certiﬁed robustness (e.g., Wong & Kolter (2018)
Revised: As a response, certiﬁed robustness (e.g., Wong & Kolter (2018)

Original: ; Raghunathan et al.
Revised: ; Raghunathan et al.

Original: (2018a);
Revised: (2018a);

Original: Liu et al.
Revised: Liu et al.

Original: Lecuyer et al.
Revised: Lecuyer et al.

Original: ; Cohen et al.
Revised: ; Cohen et al.

Original: against adversarial perturbations has been developed.
Revised: against adversarial perturbations has been developed.

Original: In particular, a robust classiﬁer veriﬁably predicts the same top-1 label for data points in a certain region around any example x .
Revised: In particular, a robust classiﬁer veriﬁably predicts the same top-1 label for data points in a certain region around any example x .

Original: In many applications such as  image classiﬁcation cloud service (Clarifai; Google Cloud Vision),recommender systems, and web search, top- k predictions are more relevant.
Revised: In many applications such as recommender systems, web search, and image classiﬁcation cloud service (Clarifai; Google Cloud Vision), top- k predictions are more relevant.

Original: In particular, given an example, a set of k most likely labels are predicted for the example.
Revised: In particular, given an example, a set of k most likely labels are predicted for the example.

Original: However, existing certiﬁed robustness results are limited to top-1 predictions, leaving top- k robustness unexplored.
Revised: However, existing certiﬁed robustness results are limited to top-1 predictions, leaving top- k robustness unexplored.

Original: To bridge this gap, we study certiﬁed robustness for top-
Revised: To bridge this gap, we study certiﬁed robustness for top-

Original: k predictions in this work.
Revised: k predictions in this work.

Original: Our certiﬁed top- k robustness leverages randomized smoothing (Cohen et al., 2019), which turns any base classiﬁer f to be a robust classiﬁer via adding Gaussian noise to an example.
Revised: Our certiﬁed top- k robustness leverages randomized smoothing (Cao & Gong, 2017; Cohen et al., 2019), which turns any base classiﬁer f to be a robust classiﬁer via adding random noise to an example.

Revised: For instance, Cao & Gong (2017) is the ﬁrst to propose randomized smoothing with uniform noise as an empirical defense.

Revised: We consider random Gaussian noise because of its certiﬁed robustness guarantee (Cohen et al., 2019).

Original: Speciﬁcally, we denote by p i the probability that the base classiﬁer f predicts label i for the Gaussian random variable N ( x , σ 2 I ) .
Revised: Speciﬁcally, we denote by p i the probability that the base classiﬁer f predicts label i for the Gaussian random variable N ( x , σ 2 I ) .

Original: The smoothed classiﬁer g k ( x ) predicts the k labels with the largest probabilities p
Revised: The smoothed classiﬁer g k ( x ) predicts the k labels with the largest probabilities p

Original: i ’s for the example x .
Revised: i ’s for the example x .

Original: We adopt randomized smoothing because it is scalable to large-scale neural networks and applicable to any base classiﬁer.
Revised: We adopt randomized smoothing because it is scalable to large-scale neural networks and applicable to any base classiﬁer.

Original: Our major theoretical result is a tight certiﬁed robustness bound for top- k predictions when using randomized smoothing with Gaussian noise.
Revised: Our major theoretical result is a tight certiﬁed robustness bound for top- k predictions when using randomized smoothing with Gaussian noise.

Original: Speciﬁcally, given an example x , a label l is veriﬁably among the top- k labels predicted by the smoothed classiﬁer g k ( x + δ ) when the (cid:96) 2 -norm of the ad- versarial perturbation δ is less than a threshold (called certiﬁed radius ).
Revised: Speciﬁcally, given an example x , a label l is veriﬁably among the top- k labels predicted by the smoothed classiﬁer g k ( x + δ ) when the (cid:96) 2 -norm of the adversarial perturbation δ is less than a threshold (called certiﬁed radius ).

Original: The certiﬁed radius for toppredictions derived by Cohen et al.
Revised: The certiﬁed radius for toppredictions derived by Cohen et al.

Original: (2019) is a special case of our certiﬁed radius when k = 1 .
Revised: (2019) is a special case of our certiﬁed radius when k = 1 .

Original: As our results and proofs show, generalizing certiﬁed robustness from top-1 to top-
Revised: As our results and proofs show, generalizing certiﬁed robustness from top-1 to top-

Original: k predictions faces signiﬁcant new challenges and requires new techniques.
Revised: k predictions faces signiﬁcant new challenges and requires new techniques.

Original: Our certiﬁed radius is the unique solution to an equation, which depends on σ , p l , and the k largest probabilities p
Revised: Our certiﬁed radius is the unique solution to an equation, which depends on σ , p l , and the k largest probabilities p

Original: i ’s (excluding p l ).
Revised: i ’s (excluding p l ).

Original: However, computing our certiﬁed radius in practice faces two challenges: 1) it is hard to exactly compute the probability p l and the k largest probabilities p
Revised: However, computing our certiﬁed radius in practice faces two challenges: 1) it is hard to exactly compute the probability p l and the k largest probabilities p

Original: i ’s, and 2) the equation about the certiﬁed radius does not have an analytical solution.
Revised: i ’s, and 2) the equation about the certiﬁed radius does not have an analytical solution.

Original: To address the ﬁrst challenge, we estimate simultaneous conﬁdence intervals of the label probabilities via the Clopper-Pearson method and Bonferroni correction in statistics.
Revised: To address the ﬁrst challenge, we estimate simultaneous conﬁdence intervals of the label probabilities via the Clopper-Pearson method and Bonferroni correction in statistics.

Original: To address the second challenge, we propose an algorithm to solve the equation to obtain a lower bound of the certiﬁed radius, where the lower bound can be tuned to be arbitrarily close to the true certiﬁed radius.
Revised: To address the second challenge, we propose an algorithm to solve the equation to obtain a lower bound of the certiﬁed radius, where the lower bound can be tuned to be arbitrarily close to the true certiﬁed radius.

Original: We evaluate our method on CIFAR10 (Krizhevsky & Hinton, 2009) and ImageNet (Deng et al., 2009) datasets.
Revised: We evaluate our method on CIFAR10 (Krizhevsky & Hinton, 2009) and ImageNet (Deng et al., 2009) datasets.

Original: For instance, on ImageNet, our method respectively achieves approximate certiﬁed top- 1 , top- 3 , and top- 5 accuracies as 46.6%, 57.8%, and 62.8% when the (cid:96) 2 -norms of the adversarial perturbations are less than 0.5 (127/255) and σ = 0 .
Revised: For instance, on ImageNet, our method respectively achieves approximate certiﬁed top- 1 , top- 3 , and top- 5 accuracies as 46.6%, 57.8%, and 62.8% when the (cid:96) 2 -norms of the adversarial perturbations are less than 0.5 (127/255) and σ = 0 .

Original: Our contributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: • Theory.
Revised: • Theory.

Original: We derive the ﬁrst certiﬁed radius for top- k predictions.
Revised: We derive the ﬁrst certiﬁed radius for top- k predictions.

Original: Moreover, we prove our certiﬁed radius is tight for randomized smoothing with Gaussian noise.
Revised: Moreover, we prove our certiﬁed radius is tight for randomized smoothing with Gaussian noise.

Original: • Algorithm.
Revised: • Algorithm.

Original: We develop algorithms to estimate our certiﬁed radius in practice.
Revised: We develop algorithms to estimate our certiﬁed radius in practice.

Original: • Evaluation.
Revised: • Evaluation.

Original: We empirically evaluate our method on CIFAR10 and ImageNet
Revised: We empirically evaluate our method on CIFAR10 and ImageNet.

Original:  C ERTIFIED R ADIUS FOR T OP - k P REDICTIONS
Revised:  C ERTIFIED R ADIUS FOR T OP - k P REDICTIONS

Original: Suppose we have a base classiﬁer f , which maps an example x ∈ R d to one of c candidate labels { 1 , 2 , · · · , c } .
Revised: Suppose we have a base classiﬁer f , which maps an example x ∈ R d to one of c candidate labels { 1 , 2 , · · · , c } .

Original: f can be any classiﬁer.
Revised: f can be any classiﬁer.

Original: Randomized smoothing (Cohen et al., 2019) adds an isotropic
Revised: Randomized smoothing (Cohen et al., 2019) adds an isotropic

Original: Gaussian noise N (0 , σ 2 I ) to an example x .
Revised: Gaussian noise N (0 , σ 2 I ) to an example x .

Original: We denote p i as the probability that the base classiﬁer f predicts label i when adding a random isotropic Gaussian noise (cid:15) to the example x , i.e., p i =
Revised: We denote p i as the probability that the base classiﬁer f predicts label i when adding a random isotropic Gaussian noise (cid:15) to the example x , i.e., p i =

Original: Pr ( f ( x + (cid:15) ) = i ) , where (cid:15) ∼ N (0 , σ 2 I ) .
Revised: Pr ( f ( x + (cid:15) ) = i ) , where (cid:15) ∼ N (0 , σ 2 I ) .

Original: The smoothed classiﬁer g k ( x ) returns the set of k labels with the largest probabilities p
Revised: The smoothed classiﬁer g k ( x ) returns the set of k labels with the largest probabilities p

Original: i ’s when taking an example x as input.
Revised: i ’s when taking an example x as input.

Original: Our goal is to derive a certiﬁed radius R l such that we have l ∈ g k
Revised: Our goal is to derive a certiﬁed radius R l such that we have l ∈ g k

Original: ( x + δ ) for all ||
Revised: ( x + δ ) for all ||

Original: δ
Revised: δ

Original: || 2 < R l .
Revised: || 2 < R l .

Original: Our main theoretical results are summarized in the following two theorems.
Revised: Our main theoretical results are summarized in the following two theorems.

Original: Theorem 1 (Certiﬁed Radius for Top- k Predictions) .
Revised: Theorem 1 (Certiﬁed Radius for Top- k Predictions) .

Original: Suppose we are given an example x , an arbitrary base classiﬁer f , (cid:15) ∼ N (0 , σ 2 I ) , a smoothed classiﬁer g , an arbitrary label l ∈ { 1 , 2 , · · · , c } , and p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ∈ [0 , 1] that satisfy the following conditions:
Revised: Suppose we are given an example x , an arbitrary base classiﬁer f , (cid:15) ∼ N (0 , σ 2 I ) , a smoothed classiﬁer g , an arbitrary label l ∈ { 1 , 2 , · · · , c } , and p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ∈ [0 , 1] that satisfy the following conditions:

Original: [Equation]
Revised: [Equation]

Original: where p and p indicate lower and upper bounds of p , respectively.
Revised: where p and p indicate lower and upper bounds of p , respectively.

Original: Let p b k ≥ p b k − 1 ≥ · · · ≥ p b 1 be the k largest ones among { p 1 , · · · , p l − 1 , p l +1 , · · · , p c } , where ties are broken uniformly at random.
Revised: Let p b k ≥ p b k − 1 ≥ · · · ≥ p b 1 be the k largest ones among { p 1 , · · · , p l − 1 , p l +1 , · · · , p c } , where ties are broken uniformly at random.

Original: Moreover, we denote by S t = {
Revised: Moreover, we denote by S t = {

Original: b 1 , b 2 , · · · , b t } the set of t labels with the smallest probability upper bounds in the k largest ones and by p S t = (cid:80) tj =

Revised: b 1 , b 2 , · · · , b t } the set of t labels with the smallest probability upper bounds in the k largest ones and by p S t = (cid:80) tj =

Original: p b j the sum of the t probability upper bounds, where t = 1 , 2 , · · · , k .
Revised: p b j the sum of the t probability upper bounds, where t = 1 , 2 , · · · , k .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: where R l is the unique solution to the following equation:
Revised: where R l is the unique solution to the following equation:

Original: [Equation]
Revised: [Equation]

Original: where Φ and Φ − 1 are the cumulative distribution function and its inverse of the standard Gaussian distribution, respectively.
Revised: where Φ and Φ − 1 are the cumulative distribution function and its inverse of the standard Gaussian distribution, respectively.

Original: Proof.
Revised: Proof.

Original: See Appendix A.
Revised: See Appendix A.

Revised: Input: f , k , σ , x , n , and α .

Original: Theorem 2 (Tightness of the Certiﬁed Radius) .{+counts = S AMPLE U NDER N OISE ( f, σ, x , n ) 3 c 1 , c 2 , · · · , c k +
 = top-+}
Revised: Output: ABSTAIN or predicted top- k labels. 1 T = ∅ 2 {+counts = S AMPLE U NDER N OISE ( f, σ, x , n ) 3 c 1 , c 2 , · · · , c k +
 = top-+}

Revised: { k + 1 } indices in counts (ties are broken uniformly at random) 4 n c 1 , n c 2 , · · · , n c k +1 = counts [ c 1 ] , counts [ c 2 ] , · · · , counts [ c k +1 ] 5 for t ← 1 to k do

Revised: if B INOM PV ALUE ( n c t , n c t + n c t

Revised: +1 , 0 . 5) ≤ α then

Revised: T = T ∪ c t else return ABSTAIN 10 return T

Revised: Theorem 2 (Tightness of the Certiﬁed Radius) .

Original: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i
Revised: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i

Original: =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .
Revised: =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .

Original: Then, for any perturbation ||
Revised: Then, for any perturbation ||

Original: δ || 2 > R l , there exists a base classiﬁer f ∗ consistent with (1) but we have l / ∈
Revised: δ || 2 > R l , there exists a base classiﬁer f ∗ consistent with (1) but we have l / ∈

Original: g k ( x + δ ) .
Revised: g k ( x + δ ) .

Original: 1 T = ∅ 2 counts =

Original: [-S AMPLE U NDER N OISE ( f σ x n ) 3 c 1 , c 2 , · · · , c k +
-]
Revised: [-S AMPLE U NDER N OISE ( f σ x n ) 3 c 1 , c 2 , · · · , c k +
-]

Original: = top-

Original: { k + 1 } indices in counts (ties broken uniformly at random) 4 n c 1 , n c 2 , · · · , n c k +1 = counts [ c 1 ] , counts [ c 2 ] , · · · , counts [ c k +1 ] 5 for t ← 1 to k do if B INOM PV ALUE ( n c t , n c t + n c t +1 , 0 . 5) ≤ α then

Original: T = T ∪ c t else return

Original: 10 return T

Original: Proof.
Revised: Proof.

Original: We show a proof sketch here.
Revised: We show a proof sketch here.

Original: Our detailed proof is in Appendix B.
Revised: Our detailed proof is in Appendix B.

Original: In our proof, we ﬁrst show that, via mathematical induction and the intermediate value theorem , we can construct k +disjoint regions
Revised: In our proof, we ﬁrst show that, via mathematical induction and the intermediate value theorem , we can construct k +disjoint regions

Original: C i , i ∈ { l } ∪ { b 1 , b 2 , · · · , b k } that satisfy
Revised: C i , i ∈ { l } ∪ { b 1 , b 2 , · · · , b k } that satisfy

Original: Pr ( x + (cid:15) ∈
Revised: Pr ( x + (cid:15) ∈

Original: C l ) =
Revised: C l ) =

Original: p l , Pr ( x + (cid:15) ∈ C i ) =
Revised: p l , Pr ( x + (cid:15) ∈ C i ) =

Original: p i , and Pr ( x + δ + (cid:15) ∈ C i ) is no smaller than some critical value for i ∈ { b 1 , b 2 , · · · , b k } .
Revised: p i , and Pr ( x + δ + (cid:15) ∈ C i ) is no smaller than some critical value for i ∈ { b 1 , b 2 , · · · , b k } .

Original: Moreover, we divide the remaining region R d \ ( ∪ i =
Revised: Moreover, we divide the remaining region R d \ ( ∪ i =

Original: l,b 1 ,b 2 , ··· ,b k C i ) into c − k − 1 regions, which we denote as C b k +1 , C b k +2 , · · · , C b c − 1 and satisfy Pr ( x + (cid:15) ∈ C i ) ≤ p i for i =
Revised: l,b 1 ,b 2 , ··· ,b k C i ) into c − k − 1 regions, which we denote as C b k +1 , C b k +2 , · · · , C b c − 1 and satisfy Pr ( x + (cid:15) ∈ C i ) ≤ p i for i =

Original: b k +1 , b k +2 , · · · , b c − 1 .
Revised: b k +1 , b k +2 , · · · , b c − 1 .

Original: Then, we construct a base classiﬁer f ∗ that predicts label i for an example if and only if the example is in the region C i , where i ∈ { 1 , 2 , · · · , c } .
Revised: Then, we construct a base classiﬁer f ∗ that predicts label i for an example if and only if the example is in the region C i , where i ∈ { 1 , 2 , · · · , c } .

Original: As p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 , f ∗ is well-deﬁned.
Revised: As p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 , f ∗ is well-deﬁned.

Original: Moreover, f ∗ satisﬁes the conditions in (1).
Revised: Moreover, f ∗ satisﬁes the conditions in (1).

Original: Finally, we show that if || δ || 2 > R l , then we have Pr ( f ∗ ( x + δ + (cid:15) )
Revised: Finally, we show that if || δ || 2 > R l , then we have Pr ( f ∗ ( x + δ + (cid:15) )

Original: = l ) < min kj =1 Pr ( f ∗ ( x + δ + (cid:15) )
Revised: = l ) < min kj =1 Pr ( f ∗ ( x + δ + (cid:15) )

Original: =
Revised: =

Original: b j ) , i.e., l / ∈ g k
Revised: b j ) , i.e., l / ∈ g k

Original: ( x + δ ) .
Revised: ( x + δ ) .

Original: We have several observations about our theorems.
Revised: We have several observations about our theorems.

Original: • Our certiﬁed radius is applicable to any base classiﬁer f .
Revised: • Our certiﬁed radius is applicable to any base classiﬁer f .

Original: According to Equation 3, our certiﬁed radius R l depends on σ , p l , and the k largest probability upper bounds { p b k , p b k − 1 , · · · , p b 1 } excluding p l .
Revised: According to Equation 3, our certiﬁed radius R l depends on σ , p l , and the k largest probability upper bounds { p b k , p b k − 1 , · · · , p b 1 } excluding p l .

Original: When the lower bound p l and the upper bounds { p b k , p b k − 1 , · · · , p b 1 } are tighter, the certiﬁed radius R l is larger.
Revised: When the lower bound p l and the upper bounds { p b k , p b k − 1 , · · · , p b 1 } are tighter, the certiﬁed radius R l is larger.

Original: When R l < 0 , the label l is not among the top- k labels predicted by the smoothed classiﬁer even if no perturbation is added, i.e., l / ∈ g k ( x ) .
Revised: When R l < 0 , the label l is not among the top- k labels predicted by the smoothed classiﬁer even if no perturbation is added, i.e., l / ∈ g k ( x ) .

Original: • When using randomized smoothing with Gaussian noise and no further assumptions are made on the base classiﬁer, it is impossible to certify a (cid:96) 2 radius for top- k predictions that is larger than R l .
Revised: • When using randomized smoothing with Gaussian noise and no further assumptions are made on the base classiﬁer, it is impossible to certify a (cid:96) 2 radius for top- k predictions that is larger than R l .

Original: • When k = 1 , we have R l = σ 2 (Φ − 1 ( p l ) − Φ − 1 ( p b 1 )) , where p b 1 is an upper bound of the largest label probability excluding p l .
Revised: • When k = 1 , we have R l = σ 2 (Φ − 1 ( p l ) − Φ − 1 ( p b 1 )) , where p b 1 is an upper bound of the largest label probability excluding p l .

Original: The certiﬁed radius derived by Cohen et al.
Revised: The certiﬁed radius derived by Cohen et al.

Original: for top-1 predictions (i.e., their Equation 3) is a special case of our certiﬁed radius with k = 1 , l = A , and b 1 = B 
Revised: for top-1 predictions (i e., their Equation 3) is a special case of our certiﬁed radius with k = 1 , l = A , and b 1 = B .

Original:  P REDICTION AND C ERTIFICATION IN PRACTICE 3. P REDICTION
Revised:  P REDICTION AND C ERTIFICATION IN PRACTICE 3. P REDICTION

Original: It is challenging to compute the top- k labels g k ( x ) predicted by the smoothed classiﬁer, because it is challenging to compute the probabilities p
Revised: It is challenging to compute the top- k labels g k ( x ) predicted by the smoothed classiﬁer, because it is challenging to compute the probabilities p

Original: i ’s exactly.
Revised: i ’s exactly.

Original: To address the challenge, we resort to a Monte Carlo method that predicts the top- k labels with a probabilistic guarantee.
Revised: To address the challenge, we resort to a Monte Carlo method that predicts the top- k labels with a probabilistic guarantee.

Original: In particular, we leverage the hypothesis testing result from a recent work (Hung et al., 2019).
Revised: In particular, we leverage the hypothesis testing result from a recent work (Hung et al., 2019).

Original: Algorithmshows our P REDICT function to estimate the top- k labels predicted by the smoothed classiﬁer.
Revised: Algorithm shows our P REDICT function to estimate the top- k labels predicted by the smoothed classiﬁer.

Original: The function S AMPLE U NDER N OISE ( f, σ, x , n ) ﬁrst randomly samples n noise (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n from the
Revised: The function S AMPLE U NDER N OISE ( f, σ, x , n ) ﬁrst randomly samples n noise (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n from the

Original: Gaussian distribution N (0 , σ 2 I ) , uses the base classiﬁer f to predict the label of x + (cid:15) j for each j ∈ { 1 , 2 , · · · , n } , and returns the frequency of each label, i.e., counts [ i ] = (cid:80) nj
Revised: Gaussian distribution N (0 , σ 2 I ) , uses the base classiﬁer f to predict the label of x + (cid:15) j for each j ∈ { 1 , 2 , · · · , n } , and returns the frequency of each label, i.e., counts [ i ] = (cid:80) nj

Original: =1 I ( f ( x + (cid:15) j ) = i ) for i ∈ { 1 , 2 , · · · , c } .
Revised: =1 I ( f ( x + (cid:15) j ) = i ) for i ∈ { 1 , 2 , · · · , c } .

Original: The function B INOM PV ALUE performs the hypothesis testing to calibrate the abstention threshold such that we can bound with probability α of returning an incorrect set of top- k labels.
Revised: The function B INOM PV ALUE performs the hypothesis testing to calibrate the abstention threshold such that we can bound with probability α of returning an incorrect set of top- k labels.

Original: Formally, we have the following proposition:
Revised: Formally, we have the following proposition:

Original: Proposition 1.
Revised: Proposition 1.

Original: With probability at least 1 − α over the randomness in P REDICT , if P REDICT returns a set T (i.e., does not ABSTAIN), then we have g k ( x ) =
Revised: With probability at least 1 − α over the randomness in P REDICT , if P REDICT returns a set T (i.e., does not ABSTAIN), then we have g k ( x ) =

Original: T .
Revised: T .

Original: Proof.
Revised: Proof.

Original: See Appendix C.
Revised: See Appendix C.

Original: 3. C ERTIFICATION
Revised: 3. C ERTIFICATION

Original: Given a base classiﬁer f , an example x , a label l , and the standard deviation σ of the Gaussian noise, we aim to compute the certiﬁed radius R l .
Revised: Given a base classiﬁer f , an example x , a label l , and the standard deviation σ of the Gaussian noise, we aim to compute the certiﬁed radius R l .

Original: According to our Equation 3, our R l relies on a lower bound of p l , i.e., p l , and the upper bound of p S t , i.e., p S t , which are related to f , x , and σ .
Revised: According to our Equation 3, our R l relies on a lower bound of p l , i.e., p l , and the upper bound of p S t , i.e., p S t , which are related to f , x , and σ .

Original: We ﬁrst discuss two Monte Carlo methods to estimate p l and p S t with probabilistic guarantees.
Revised: We ﬁrst discuss two Monte Carlo methods to estimate p l and p S t with probabilistic guarantees.

Original: However, given p l and p S t , it is still challenging to exactly solve R l as the Equation 3 does not have an analytical solution.
Revised: However, given p l and p S t , it is still challenging to exactly solve R l as the Equation 3 does not have an analytical solution.

Original: To address the challenge, we design an algorithm to obtain a lower bound of R l via solving Equation 3 through binary search.
Revised: To address the challenge, we design an algorithm to obtain a lower bound of R l via solving Equation 3 through binary search.

Original: Our lower bound can be tuned to be arbitrarily close to R l .
Revised: Our lower bound can be tuned to be arbitrarily close to R l .

Original: 3.2. E STIMATING p l AND p S t
Revised: 3.2. E STIMATING p l AND p S t

Original: Our approach has two steps.
Revised: Our approach has two steps.

Original: The ﬁrst step is to estimate p l and p i for i (cid:54)
Revised: The ﬁrst step is to estimate p l and p i for i (cid:54)

Original: =
Revised: =

Original: l .
Revised: l .

Original: The second step is to estimate p S t using p i for i (cid:54)
Revised: The second step is to estimate p S t using p i for i (cid:54)

Original: = l .
Revised: = l .

Original: Estimating p l and p i for i (cid:54)
Revised: Estimating p l and p i for i (cid:54)

Original: =
Revised: =

Original: l : The probabilities p 1 , p 2 , · · · , p c can be viewed as a multinomial distribution over the labels { 1 , 2 , · · · , c } .
Revised: l : The probabilities p 1 , p 2 , · · · , p c can be viewed as a multinomial distribution over the labels { 1 , 2 , · · · , c } .

Original: If we sample a Gaussian noise (cid:15) uniformly at random, then the label f
Revised: If we sample a Gaussian noise (cid:15) uniformly at random, then the label f

Original: ( x + (cid:15) ) can be viewed as a sample from the multinomial distribution.
Revised: ( x + (cid:15) ) can be viewed as a sample from the multinomial distribution.

Original: Therefore, estimating p l and p i for i (cid:54) = l is essentially a one-sided simultaneous conﬁdence interval estimation problem.
Revised: Therefore, estimating p l and p i for i (cid:54) = l is essentially a one-sided simultaneous conﬁdence interval estimation problem.

Original: In particular, we aim to estimate these bounds with a conﬁdence level at least 1 − α .
Revised: In particular, we aim to estimate these bounds with a conﬁdence level at least 1 − α .

Original: In statistics, Goodman (1965); Sison & Glaz (1995) are well-known methods for simultaneous conﬁdence interval estimations.
Revised: In statistics, Goodman (1965); Sison & Glaz (1995) are well-known methods for simultaneous conﬁdence interval estimations.

Original: However, these methods are insufﬁcient for our problem.
Revised: However, these methods are insufﬁcient for our problem.

Original: Speciﬁcally,
Revised: Speciﬁcally,

Original: Goodman’s method is based on Chi-square test, which requires the expected count for each label to be no less than 5.
Revised: Goodman’s method is based on Chi-square test, which requires the expected count for each label to be no less than 5.

Original: We found that this is usually not satisﬁed, e.g., ImageNet has 1,000 labels, some of which have close-to-zero probabilities and do not have more than 5 counts even if we sample a large number of Gaussian noise.
Revised: We found that this is usually not satisﬁed, e.g., ImageNet has 1,000 labels, some of which have close-to-zero probabilities and do not have more than 5 counts even if we sample a large number of Gaussian noise.

Original: Sison & Glaz’s method guarantees a conﬁdence level of approximately 1 − α , which means that the conﬁdence level could be (slightly) smaller than 1 − α .
Revised: Sison & Glaz’s method guarantees a conﬁdence level of approximately 1 − α , which means that the conﬁdence level could be (slightly) smaller than 1 − α .

Original: However, we aim to achieve a conﬁdence level of at least 1 − α .
Revised: However, we aim to achieve a conﬁdence level of at least 1 − α .

Original: To address these challenges, we discuss two conﬁdence interval estimation methods as follows:
Revised: To address these challenges, we discuss two conﬁdence interval estimation methods as follows:

Original: 1) BinoCP.
Revised: 1) BinoCP.

Original: This method estimates p l using the standard one-sided Clopper-Pearson method and treats p i as p i = 1 − p l for each i (cid:54)
Revised: This method estimates p l using the standard one-sided Clopper-Pearson method and treats p i as p i = 1 − p l for each i (cid:54)

Original: =
Revised: =

Original: l .
Revised: l .

Original: Speciﬁcally
Revised: Speciﬁcally

Original:  we sample n random noise from N (0 , σI 2 ) , i.e., (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n .
Revised: , we sample n random noise from N (0 , σI 2 ) , i.e., (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n .

Original: We denote the count for the label l as n l = (cid:80) nj =1 I ( f ( x + (cid:15) j ) = l ) .
Revised: We denote the count for the label l as n l = (cid:80) nj =1 I ( f ( x + (cid:15) j ) = l ) .

Original: n l follows a binomial distribution with parameters n and p l , i.e., n l ∼ Bin ( n, p l ) .
Revised: n l follows a binomial distribution with parameters n and p l , i.e., n l ∼ Bin ( n, p l ) .

Original: Therefore, according to the Clopper-Pearson method, we have:
Revised: Therefore, according to the Clopper-Pearson method, we have:

Original: [Equation]
Revised: [Equation]

Original: where 1 − α is the conﬁdence level and B ( α ; u, v ) is the α th quantile of the Beta distribution with shape parameters u and v .
Revised: where 1 − α is the conﬁdence level and B ( α ; u, v ) is the α th quantile of the Beta distribution with shape parameters u and v .

Original: We note that the Clopper-Pearson method was also adopted by Cohen et al.
Revised: We note that the Clopper-Pearson method was also adopted by Cohen et al.

Original: (2019) to estimate label probability for their certiﬁed radius of top-1 predictions.
Revised: (2019) to estimate label probability for their certiﬁed radius of top-1 predictions.

Original: 2) SimuEM.
Revised: 2) SimuEM.

Original: The above method estimates p i as 1 − p l , which may be conservative.
Revised: The above method estimates p i as 1 − p l , which may be conservative.

Original: A conservative estimation makes the certiﬁed radius smaller than what it should be.
Revised: A conservative estimation makes the certiﬁed radius smaller than what it should be.

Original: Therefore, we introduce
Revised: Therefore, we introduce

Original: SimuEM to directly estimate p i together with p l .
Revised: SimuEM to directly estimate p i together with p l .

Original: We let n i = (cid:80) [-n j =
-]
Revised: We let n i = (cid:80) [-n j =
-]nj

Original: I{+=
+}
Revised: {+=
+}

Original:  f ( x + (cid:15) j ) = i ) for each i ∈ { 1 , 2 , · · · , c } .
Revised: I ( f ( x + (cid:15) j ) = i ) for each i ∈ { 1 , 2 , · · · , c } .

Original: Each n i follows a binomial distribution with parameters n and p i .
Revised: Each n i follows a binomial distribution with parameters n and p i .

Revised: We ﬁrst use

Revised: Input: f , k , σ , x , l , n , µ , and α .

Revised: Output: ABSTAIN or R l .

Original: We ﬁrst use the Clopper-Pearson method to estimate a one-sided conﬁdence interval for each label i , and then we obtain simultaneous conﬁdence intervals by leveraging the Bonferroni correction .{+n, α ) 
 [ p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ] = B INO CP(counts, α ) or S IMU EM(counts, α ) 3 R l =4 for t ← 1 to k do p S t = min+}
Revised: 1 counts = S AMPLE U NDER N OISE ( f, σ, x , {+n, α ) 
 [ p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ] = B INO CP(counts, α ) or S IMU EM(counts, α ) 3 R l =4 for t ← 1 to k do p S t = min+}

Original:  Speciﬁcally, if we can obtain a conﬁdence interval with conﬁdence level at least 1 − αc for each label i , then Bonferroni correction tells us that the overall conﬁdence level for the simultaneous conﬁdence intervals is at least 1 − α , i.e., we have conﬁdence level at least 1 − α that all conﬁdence intervals hold at the same time.
Revised: ( (cid:80) tj =1 p b j , 1 − p l ) R lt = B INARY S EARCH ( p l , p S t , t, σ, µ ) if R lt > R l then R l = R lt 9 if R l > 0 then return R l 11 else return ABSTAIN the Clopper-Pearson method to estimate a one-sided conﬁdence interval for each label i , and then we obtain simultaneous conﬁdence intervals by leveraging the Bonferroni correction . Speciﬁcally, if we can obtain a conﬁdence interval with conﬁdence level at least 1 − αc for each label i , then Bonferroni correction tells us that the overall conﬁdence level for the simultaneous conﬁdence intervals is at least 1 − α , i.e., we have conﬁdence level at least 1 − α that all conﬁdence intervals hold at the same time.

Original: Formally, we have the following bounds by applying the Clopper-Pearson method with conﬁdence level 1 − αc to each label: 
Revised: Formally, we have the following bounds by applying the Clopper-Pearson method with conﬁdence level 1 − αc to each label: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Estimating p S t : One natural method is to estimate p S t
Revised: Estimating p S t : One natural method is to estimate p S t

Original: = (cid:80) tj =
Revised: = (cid:80) tj =

Original: 1 p b j .
Revised: 1 p b j .

Original: However, this bound may be loose.
Revised: However, this bound may be loose.

Original: For example, when using BinoCP to estimate the probability bounds, we have p S t =
Revised: For example, when using BinoCP to estimate the probability bounds, we have p S t =

Original: t · (1 − p l ) , which may be bigger than 1 .
Revised: t · (1 − p l ) , which may be bigger than 1 .

Original: To address the challenge, we derive another bound for p S t from another perspective.
Revised: To address the challenge, we derive another bound for p S t from another perspective.

Original: Speciﬁcally, we have p S t ≤ (cid:80) i (cid:54) =
Revised: Speciﬁcally, we have p S t ≤ (cid:80) i (cid:54) =

Original: l p i ≤ 1 − p l .
Revised: l p i ≤ 1 − p l .

Original: Therefore, we can use 1 − p l as an upper bound of p S t , i.e., p S t = 1 − p l .
Revised: Therefore, we can use 1 − p l as an upper bound of p S t , i.e., p S t = 1 − p l .

Original: Finally, we combine the above two estimations by taking the minimal one, i.e., p S t = min( (cid:80) tj =1 p b j , 1 − p l ) .
Revised: Finally, we combine the above two estimations by taking the minimal one, i.e., p S t = min( (cid:80) tj =1 p b j , 1 − p l ) .

Original: 3.2. E STIMATING A L OWER B OUND OF THE C ERTIFIED R ADIUS R l
Revised: 3.2. E STIMATING A L OWER B OUND OF THE C ERTIFIED R ADIUS R l

Original: It is challenging to compute the certiﬁed radius R l exactly because Equation 3 does not have an analytical solution.
Revised: It is challenging to compute the certiﬁed radius R l exactly because Equation 3 does not have an analytical solution.

Original: To address the challenge, we design a method to estimate a lower bound of
Revised: To address the challenge, we design a method to estimate a lower bound of

Original: R l that can be tuned to be arbitrarily close to R l .
Revised: R l that can be tuned to be arbitrarily close to R l .

Original: Speciﬁcally, we ﬁrst approximately solve the following equation for each t ∈ { 1 , 2 , · · · , k } :
Revised: Speciﬁcally, we ﬁrst approximately solve the following equation for each t ∈ { 1 , 2 , · · · , k } :

Original: [Equation]
Revised: [Equation]

Original: We note that it is still difﬁcult to obtain an analytical solution to Equation 7 when t > 1 .
Revised: We note that it is still difﬁcult to obtain an analytical solution to Equation 7 when t > 1 .

Original: However, we notice that the left-hand side has the following properties: 1) it decreases as R tl increases; 2) when R tl → −∞ , it is greater than 0; 3) when R tl → ∞ , it is smaller than 0.
Revised: However, we notice that the left-hand side has the following properties: 1) it decreases as R tl increases; 2) when R t l → −∞ , it is greater than 0; 3) when R t l → ∞ , it is smaller than 0.

Original: Therefore, there exists a unique solution R tl to Equation 7.
Revised: Therefore, there exists a unique solution R tl to Equation 7.

Original: Moreover, we leverage binary search to ﬁnd a lower bound R lt that can be arbitrarily close to the exact solution R tl .
Revised: Moreover, we leverage binary search to ﬁnd a lower bound R l t that can be arbitrarily close to the exact solution R tl .

Original: In particular, we run the binary search until the left-hand side of Equation 7 is non-negative and the width of the search interval is less than a parameter µ > 0 .
Revised: In particular, we run the binary search until the left-hand side of Equation 7 is non-negative and the width of the search interval is less than a parameter µ > 0 .

Original: Formally, we have:
Revised: Formally, we have:

Original: [Equation]
Revised: [Equation]

Revised: ≤ R l t + µ, ∀ t ∈ { 1 , 2 , · · · , k } .

Revised:       UDGLXV       F H U W L I L HG  D FF X U D F\ k =1 k = k = (a) CIFAR       UDGLXV       F H U W L I L HG  D FF X U D F\ k =1 k = k =

Revised: [Figure]

Original: After obtaining R lt , we let R l = max kt =
Revised: After obtaining R lt , we let R l = max kt =

Original: 1 R
Revised: 1 R

Original: lt be our lower bound of R l .
Revised: lt be our lower bound of R l .

Original: Based on R l = max kt =1 R tl and Equation 8, we have the following guarantee:
Revised: Based on R l = max kt =1 R tl and Equation 8, we have the following guarantee:

Original: [Equation]
Revised: [Equation]

Original:       UDGLXV       F H U W L I L HG  D FF X U D F\ k =1 k = k = (a) CIFAR       UDGLXV       F H U W L I L HG  D FF X U D F\ k =1 k = k = [Figure] [Figure]
Revised: 3.2. C OMPLETE C ERTIFICATION A LGORITHM

Original: Algorithm 2 shows our algorithm to estimate the certiﬁed radius for a given example x and a label l .
Revised: Algorithm 2 shows our algorithm to estimate the certiﬁed radius for a given example x and a label l .

Original: The function S AMPLE U NDER N OISE is the same as in Algorithm 1.
Revised: The function S AMPLE U NDER N OISE is the same as in Algorithm 1.

Original: Functions B INO CP and
Revised: Functions B INO CP and

Original: S IMU EM return the estimated probability bound for each label.
Revised: S IMU EM return the estimated probability bound for each label.

Original: Function B INARY S EARCH performs binary search to solve the Equation 7 and returns a solution satisfying Equation 8.
Revised: Function B INARY S EARCH performs binary search to solve the Equation 7 and returns a solution satisfying Equation 8.

Original: Formally, our algorithm has the following guarantee:
Revised: Formally, our algorithm has the following guarantee:

Original: Proposition 2.
Revised: Proposition 2.

Original: With probability at least 1 − α over the randomness in C ERTIFY , if C ERTIFY returns a radius R l (i.e., does not ABSTAIN), then we have l ∈ g k
Revised: With probability at least 1 − α over the randomness in C ERTIFY , if C ERTIFY returns a radius R l (i.e., does not ABSTAIN), then we have l ∈ g k

Original: ( x + δ ) , ∀|| δ
Revised: ( x + δ ) , ∀|| δ

Original: || 2 < R l .
Revised: || 2 < R l .

Original: Proof.
Revised: Proof.

Original: See Appendix D.
Revised: See Appendix D.

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: Datasets and models: We conduct experiments on the standard CIFAR10 (Krizhevsky & Hinton, 2009) and ImageNet (Deng et al., 2009) datasets to evaluate our method.
Revised: Datasets and models: We conduct experiments on the standard CIFAR10 (Krizhevsky & Hinton, 2009) and ImageNet (Deng et al., 2009) datasets to evaluate our method.

Original: We use the publicly available pre-trained models from Cohen et al.
Revised: We use the publicly available pre-trained models from Cohen et al.

Original: Speciﬁcally, the architectures of the base classiﬁers are ResNet-110 and ResNet-50 for CIFAR10 and ImageNet, respectively.
Revised: Speciﬁcally, the architectures of the base classiﬁers are ResNet-110 and ResNet-50 for CIFAR10 and ImageNet, respectively.

Original: Parameter setting: We study the impact of k , the conﬁdence level 1 − α , the noise level σ , the number of samples n , and the conﬁdence interval estimation methods on the certiﬁed radius.
Revised: Parameter setting: We study the impact of k , the conﬁdence level 1 − α , the noise level σ , the number of samples n , and the conﬁdence interval estimation methods on the certiﬁed radius.

Original: Unless otherwise mentioned, we use the following default parameters: k = 3 , α = 0 .
Revised: Unless otherwise mentioned, we use the following default parameters: k = 3 , α = 0 .

Original: 001 , σ = 0 . 5 , n = 100 , 000 , and µ = 10 − 5 .
Revised: 001 , σ = 0 . 5 , n = 100 , 000 , and µ = 10 − 5 .

Original: Moreover, we use SimuEM to estimate bounds of label probabilities.
Revised: Moreover, we use SimuEM to estimate bounds of label probabilities.

Original: When studying the impact of one parameter on the certiﬁed radius, we ﬁx the other parameters to their default values.
Revised: When studying the impact of one parameter on the certiﬁed radius, we ﬁx the other parameters to their default values.

Original: Approximate certiﬁed top-
Revised: Approximate certiﬁed top-

Original: k accuracy:
Revised: k accuracy:

Original: For each testing example x whose true label is l , we compute the certiﬁed radius R l using the C ERTIFY algorithm.
Revised: For each testing example x whose true label is l , we compute the certiﬁed radius R l using the C ERTIFY algorithm.

Original: Then, we compute the certiﬁed topk accuracy at a radius r as the fraction of testing examples whose certiﬁed radius are at least r .
Revised: Then, we compute the certiﬁed topk accuracy at a radius r as the fraction of testing examples whose certiﬁed radius are at least r .

Original: Note that our computed certiﬁed top- k accuracy is an approximate certiﬁed top- k accuracy instead of the true certiﬁed top- k accuracy.
Revised: Note that our computed certiﬁed top- k accuracy is an approximate certiﬁed top- k accuracy instead of the true certiﬁed top- k accuracy.

Original: However, we can obtain a lower bound of the true certiﬁed top- k accuracy based on the approximate certiﬁed top- k accuracy.
Revised: However, we can obtain a lower bound of the true certiﬁed top- k accuracy based on the approximate certiﬁed top- k accuracy.

Original: Appendix E shows the details.
Revised: Appendix E shows the details.

Original: Moreover, the gap between the lower bound of the true certiﬁed top- k accuracy and the approximate top- k accuracy is negligible when α is small.
Revised: Moreover, the gap between the lower bound of the true certiﬁed top- k accuracy and the approximate top- k accuracy is negligible when α is small.

Original: For convenience, we simply use the term certiﬁed top-
Revised: For convenience, we simply use the term certiﬁed top-

Original: k accuracy in the paper.
Revised: k accuracy in the paper.

Revised: [Figure]

Revised: 

Revised: [Figure]

Revised: [Figure]

Revised:     

Original: 4. E XPERIMENTAL RESULTS
Revised: [Figure] UDGLXV       F H U W L I L HG  D FF X U D F\ [Figure] XPERIMENTAL RESULTS

Original: Figure 1 shows the certiﬁed top-
Revised: Figure 1 shows the certiﬁed top-

Original: k accuracy as the radius r increases for different k .
Revised: k accuracy as the radius r increases for different k .

Original: Naturally, the certiﬁed top- k accuracy increases as k increases.
Revised: Naturally, the certiﬁed top- k accuracy increases as k increases.

Original: On CIFAR10, we respectively achieve certiﬁed top-1, top-2, and top-3 accuracies as 45.2%, 58.8%, and 67.2% when the (cid:96) 2 -norm of the adversarial perturbation is less than 0.5 (127/255).
Revised: On CIFAR10, we respectively achieve certiﬁed top-1, top-2, and top-3 accuracies as 45.2%, 58.8%, and 67.2% when the (cid:96) 2 -norm of the adversarial perturbation is less than 0.5 (127/255).

Original: On ImageNet, we respectively achieve certiﬁed top-1, top-3, and top-5 accuracies as 46.6%, 57.8%, and 62.8% when the (cid:96) 2 -norm of the adversarial perturbation [Figure] [Figure]
Revised: On ImageNet, we respectively achieve certiﬁed top-1, top-3, and top-5 accuracies as 46.6%, 57.8%, and 62.8% when the (cid:96) 2 -norm of the adversarial perturbation is less than 0.5.

Original:       UDGLXV   

Original: [Figure]

Original: H U W L I L HG 

Original: [Figure]

Original: U [Figure] is less than 0.5. On CIFAR10, the gaps between the certiﬁed top- k accuracy for different k are smaller than those between the top- k accuracy under no attacks, and they become smaller as the radius increases.
Revised: On CIFAR10, the gaps between the certiﬁed top- k accuracy for different k are smaller than those between the top- k accuracy under no attacks, and they become smaller as the radius increases.

Original: On ImageNet, the gaps between the certiﬁed top- k accuracy for different k remain similar to those between the top- k accuracy under no attacks as the radius increases.
Revised: On ImageNet, the gaps between the certiﬁed top- k accuracy for different k remain similar to those between the top- k accuracy under no attacks as the radius increases.

Original: Figure 2 shows the inﬂuence of the conﬁdence level.
Revised: Figure 2 shows the inﬂuence of the conﬁdence level.

Original: We observe that conﬁdence level has a small inﬂuence on the certiﬁed top- k accuracy as the different curves almost overlap.
Revised: We observe that conﬁdence level has a small inﬂuence on the certiﬁed top- k accuracy as the different curves almost overlap.

Original: The reason is that the estimated conﬁdence intervals of the probabilities shrink slowly as the conﬁdence level increases.
Revised: The reason is that the estimated conﬁdence intervals of the probabilities shrink slowly as the conﬁdence level increases.

Original: Figureshows the inﬂuence of σ .
Revised: Figureshows the inﬂuence of σ .

Original: We observe that σ controls a trade-off between normal accuracy under no attacks and robustness.
Revised: We observe that σ controls a trade-off between normal accuracy under no attacks and robustness.

Original: Speciﬁcally, when σ is smaller, the accuracy under no attacks (i.e., the accuracy when radius is 0) is larger, but the certiﬁed top- k accuracy drops more quickly as the radius increases.
Revised: Speciﬁcally, when σ is smaller, the accuracy under no attacks (i.e., the accuracy when radius is 0) is larger, but the certiﬁed top- k accuracy drops more quickly as the radius increases.

Original: Figure 4 compares BinoCP with SimuEM .
Revised: Figure 4 compares BinoCP with SimuEM .

Original: The results show that SimuEM is better when the certiﬁed radius is small, while BinoCP is better when the certiﬁed radius is large.
Revised: The results show that SimuEM is better when the certiﬁed radius is small, while BinoCP is better when the certiﬁed radius is large.

Original: We found the reason is that when the certiﬁed radius is large, p l is relatively large, and thus 1 − p l already provides a good estimation for p i , where i (cid:54) = l .
Revised: We found the reason is that when the certiﬁed radius is large, p l is relatively large, and thus 1 − p l already provides a good estimation for p i , where i (cid:54) = l .

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Numerous defenses have been proposed against adversarial perturbations in the past several years.
Revised: Numerous defenses have been proposed against adversarial perturbations in the past several years.

Original: These defenses either show robustness against existing attacks empirically, or prove the robustness against arbitrary bounded-perturbations (known as certiﬁed defenses ).
Revised: These defenses either show robustness against existing attacks empirically, or prove the robustness against arbitrary bounded-perturbations (known as certiﬁed defenses ).

Original: 5. E MPIRICAL DEFENSES
Revised: 5. E MPIRICAL DEFENSES

Original: The community has proposed many empirical defenses.
Revised: The community has proposed many empirical defenses.

Original: The most effective empirical defense is adversarial training (Goodfellow et al., 2015; Kurakin et al., 2017; Tram`er et al., 2018; Madry et al., 2018).
Revised: The most effective empirical defense is adversarial training (Goodfellow et al., 2015; Kurakin et al., 2017; Tram`er et al., 2018; Madry et al., 2018).

Original: However, adversarial training does not have certiﬁed robustness guarantees.
Revised: However, adversarial training does not have certiﬁed robustness guarantees.

Original: Other examples of empirical defenses include defensive distillation (Papernot et al., 2016), MagNet (Meng & Chen, 2017), PixelDefend (Song et al., 2017),
Revised: Other examples of empirical defenses include defensive distillation (Papernot et al., 2016), MagNet (Meng & Chen, 2017), PixelDefend (Song et al., 2017),

Original: Feature squeezing (Xu et al., 2018), and many others (Liu et al., 2019; Svoboda et al., 2019; Schott et al., 2019; Buckman et al., 2018; Ma et al., 2018; Guo et al., 2018; Dhillon et al., 2018;
Revised: Feature squeezing (Xu et al., 2018), and many others (Liu et al., 2019; Svoboda et al., 2019; Schott et al., 2019; Buckman et al., 2018; Ma et al., 2018; Guo et al., 2018; Dhillon et al., 2018;

Original: Xie et al., 2018; Song et al., 2018; Samangouei et al., 2018;
Revised: Xie et al., 2018; Song et al., 2018; Samangouei et al., 2018;

Original: Na et al., 2018; Metzen et al., 2017).
Revised: Na et al., 2018; Metzen et al., 2017).

Original: However, many of these defenses were soon broken by adaptive attacks (Carlini & Wagner, 2017a;
Revised: However, many of these defenses were soon broken by adaptive attacks (Carlini & Wagner, 2017a;

Original: Athalye et al., 2018; Uesato et al., 2018; Athalye & Carlini, 2018).
Revised: Athalye et al., 2018; Uesato et al., 2018; Athalye & Carlini, 2018).

Original: 5. C ERTIFIED DEFENSES
Revised: 5. C ERTIFIED DEFENSES

Original: To end the arms race between defenders and adversaries, researchers have developed certiﬁed defenses against adversarial perturbations.
Revised: To end the arms race between defenders and adversaries, researchers have developed certiﬁed defenses against adversarial perturbations.

Original: Speciﬁcally, in a certiﬁably robust classiﬁer, the predicted top-1 label is veriﬁably constant within a certain region (e.g., (cid:96) 2 -norm ball) around an input example, which provides a lower bound of the adversarial perturbation.
Revised: Speciﬁcally, in a certiﬁably robust classiﬁer, the predicted top-1 label is veriﬁably constant within a certain region (e.g., (cid:96) 2 -norm ball) around an input example, which provides a lower bound of the adversarial perturbation.

Original: Such certiﬁed defenses include satisﬁability modulo theories based methods (Katz et al., 2017; Carlini et al., 2017; Ehlers, 2017;
Revised: Such certiﬁed defenses include satisﬁability modulo theories based methods (Katz et al., 2017; Carlini et al., 2017; Ehlers, 2017;

Original: Huang et al., 2017), mixed integer linear programming based methods (Cheng et al., 2017; Lomuscio & Maganti, 2017; Dutta et al., 2017; Fischetti & Jo, 2018; Bunel et al., 2018), abstract interpretation based methods (Gehr et al., 2018; Tjeng et al., 2018), and global (or local)
Revised: Huang et al., 2017), mixed integer linear programming based methods (Cheng et al., 2017; Lomuscio & Maganti, 2017; Dutta et al., 2017; Fischetti & Jo, 2018; Bunel et al., 2018), abstract interpretation based methods (Gehr et al., 2018; Tjeng et al., 2018), and global (or local)

Original: Lipschitz constant based methods (Cisse et al., 2017; Gouk et al., 2018; Tsuzuku et al., 2018; Anil et al., 2019; Wong & Kolter, 2018;
Revised: Lipschitz constant based methods (Cisse et al., 2017; Gouk et al., 2018; Tsuzuku et al., 2018; Anil et al., 2019; Wong & Kolter, 2018;

Original: Wang et al., 2018a;b; Raghunathan et al., 2018a;b; Wong et al., 2018; Dvijotham et al., 2018a;b; Croce et al., 2018; Gehr et al., 2018; Mirman et al., 2018; Singh et al., 2018; Gowal et al., 2018; Weng et al., 2018; Zhang et al., 2018).
Revised: Wang et al., 2018a;b; Raghunathan et al., 2018a;b; Wong et al., 2018; Dvijotham et al., 2018a;b; Croce et al., 2018; Gehr et al., 2018; Mirman et al., 2018; Singh et al., 2018; Gowal et al., 2018; Weng et al., 2018; Zhang et al., 2018).

Original: However, these methods are not scalable to large neural networks and/or make assumptions on the architectures of the neural networks.
Revised: However, these methods are not scalable to large neural networks and/or make assumptions on the architectures of the neural networks.

Original: For example, these defenses are not scalable/applicable to the complex neural networks for ImageNet.
Revised: For example, these defenses are not scalable/applicable to the complex neural networks for ImageNet.

Original: Randomized smoothing was ﬁrst proposed as an empirical defense (Cao & Gong, 2017; Liu et al., 2018) without deriving the certiﬁed robustness guarantees.
Revised: Randomized smoothing was ﬁrst proposed as an empirical defense (Cao & Gong, 2017; Liu et al., 2018) without deriving the certiﬁed robustness guarantees.

Revised: For instance, Cao & Gong (2017) proposed randomized smoothing with uniform noise from a hypercube centered at an example.

Original: Lecuyer et al.
Revised: Lecuyer et al.

Original: (2019) was the ﬁrst to prove the certiﬁed robustness guarantee of randomized smoothing for top-1 predictions.
Revised: (2019) was the ﬁrst to prove the certiﬁed robustness guarantee of randomized smoothing for top-1 predictions.

Original: Their results leverage differential privacy.
Revised: Their results leverage differential privacy.

Original: Subsequently, Li et al.
Revised: Subsequently, Li et al.

Original: (2018) further leverages information theory to improve the certiﬁed radius bound.
Revised: (2018) further leverages information theory to improve the certiﬁed radius bound.

Original: Cohen et al.
Revised: Cohen et al.

Original: (2019) obtains a tight certiﬁed radius bound for randomized smoothing with Gaussian noise by leveraging the Neyman-Pearson Lemma.
Revised: (2019) obtains a tight certiﬁed radius bound for randomized smoothing with Gaussian noise by leveraging the Neyman-Pearson Lemma.

Revised: Pinot et al.

Revised: (2019) theoretically demonstrated the robustness to adversarial attacks of randomized smoothing when adding noise from Exponential family distributions and devised an upper bound on the adversarial generalization gap of randomized neural networks.

Revised: Lee et al.

Original:  Unlike the other certiﬁed defenses, randomized smoothing is scalable to large neural networks and applicable to arbitrary classiﬁers.
Revised: (2019) generalized randomized smoothing to discrete data. Salman et al. (2019) employed adversarial training to improve the performance of randomized smoothing. Unlike the other certiﬁed defenses, randomized smoothing is scalable to large neural networks and applicable to arbitrary classiﬁers.

Original: Our work derives the ﬁrst certiﬁed robustness guarantee of randomized smoothing for top- k predictions.
Revised: Our work derives the ﬁrst certiﬁed robustness guarantee of randomized smoothing for top- k predictions.

Original: Moreover, we show that our robustness guarantee is tight for randomized smoothing with Gaussian noise.
Revised: Moreover, we show that our robustness guarantee is tight for randomized smoothing with Gaussian noise.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Adversarial perturbation poses a fundamental security threat to classiﬁers.
Revised: Adversarial perturbation poses a fundamental security threat to classiﬁers.

Original: Existing certiﬁed defenses focus on top-1 predictions, leaving top- k predictions untouched.
Revised: Existing certiﬁed defenses focus on top-1 predictions, leaving top- k predictions untouched.

Original: In this work, we derive the ﬁrst certiﬁed radius under (cid:96) 2 -norm for top-
Revised: In this work, we derive the ﬁrst certiﬁed radius under (cid:96) 2 -norm for top-

Original: k predictions.
Revised: k predictions.

Original: Our results are based on randomized smoothing.
Revised: Our results are based on randomized smoothing.

Original: Moreover, we prove that our certiﬁed radius is tight for randomized smoothing with
Revised: Moreover, we prove that our certiﬁed radius is tight for randomized smoothing with

Original: Gaussian noise.
Revised: Gaussian noise.

Original: In order to compute the certiﬁed radius in practice, we further propose simultaneous conﬁdence interval estimation methods as well as design an algorithm to estimate a lower bound of the certiﬁed radius.
Revised: In order to compute the certiﬁed radius in practice, we further propose simultaneous conﬁdence interval estimation methods as well as design an algorithm to estimate a lower bound of the certiﬁed radius.

Original: Interesting directions for future work include 1) deriving a tight certiﬁed radius under other norms such as (cid:96) 1 and (cid:96) ∞ , 2) studying which noise gives the tightest certiﬁed radius for randomized smoothing, and 3) studying certiﬁed robustness for top- k ranking. 
Revised: Interesting directions for future work include 1) deriving a tight certiﬁed radius under other norms such as (cid:96) 1 and (cid:96) ∞ , 2) studying which noise gives the tightest certiﬁed radius for randomized smoothing, and 3) studying certiﬁed robustness for top- k ranking. ACKNOWLEDGMENTS We thank the anonymous reviewers for insightful reviews. This work was supported by NSF grant

Revised: No.

Original: A P ROOF OF T HEOREM
Revised: A P ROOF OF T HEOREM

Original: Given an example x , we deﬁne the following two random variables:
Revised: Given an example x , we deﬁne the following two random variables:

Original: [Equation]
Revised: [Equation]

Original: where (cid:15) ∼ N (0 , σ 2 I ) .
Revised: where (cid:15) ∼ N (0 , σ 2 I ) .

Original: The random variables X
Revised: The random variables X

Original: and Y represent random samples obtained by adding isotropic Gaussian noise to the example x and its perturbed version x + δ , respectively.
Revised: and Y represent random samples obtained by adding isotropic Gaussian noise to the example x and its perturbed version x + δ , respectively.

Original: Cohen et al.
Revised: Cohen et al.

Original: (2019) applied the standard Neyman-Pearson Lemma (Neyman & Pearson, 1933) to the above two random variables, and obtained the following lemma:
Revised: (2019) applied the standard Neyman-Pearson Lemma (Neyman & Pearson, 1933) to the above two random variables, and obtained the following lemma:

Original: Lemma 1 (Neyman-Pearson for Gaussians with different means) .
Revised: Lemma 1 (Neyman-Pearson for Gaussians with different means) .

Original: Let X ∼ N ( x , σ 2 I ) , Y ∼ N ( x + δ, σ 2 I ) , and M : R d −→ { 0 , 1 } be a random or deterministic function.
Revised: Let X ∼ N ( x , σ 2 I ) , Y ∼ N ( x + δ, σ 2 I ) , and M : R d −→ { 0 , 1 } be a random or deterministic function.

Original: Then, we have the following:
Revised: Then, we have the following:

Original: (1) If Z = { z ∈ R d : δ T z ≤ β } for some β and Pr ( M ( X ) = 1) ≥ Pr ( X ∈ Z ) , then Pr ( M ( Y ) = 1) ≥ Pr ( Y ∈ Z ) (2) If Z = { z ∈ R d : δ T z ≥ β } for some β and Pr ( M ( X ) = 1) ≤ Pr ( X ∈ Z ) , then Pr ( M ( Y ) = 1) ≤ Pr ( Y ∈ Z )
Revised: (1) If Z = { z ∈ R d : δ T z ≤ β } for some β and Pr ( M ( X ) = 1) ≥ Pr ( X ∈ Z ) , then Pr ( M ( Y ) = 1) ≥ Pr ( Y ∈ Z ) (2) If Z = { z ∈ R d : δ T z ≥ β } for some β and Pr ( M ( X ) = 1) ≤ Pr ( X ∈ Z ) , then Pr ( M ( Y ) = 1) ≤ Pr ( Y ∈ Z )

Original: Moreover, we have the following lemma from Cohen et al.
Revised: Moreover, we have the following lemma from Cohen et al.

Original: Lemma 2.
Revised: Lemma 2.

Original: Given an example x , a number q ∈ [0 , 1] , and regions A and B deﬁned as follows:
Revised: Given an example x , a number q ∈ [0 , 1] , and regions A and B deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: Then, we have the following equations:
Revised: Then, we have the following equations:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Please refer to Cohen et al.
Revised: Please refer to Cohen et al.

Original: Based on Lemma 1 and 2, we derive the following lemma:
Revised: Based on Lemma 1 and 2, we derive the following lemma:

Original: Lemma 3.
Revised: Lemma 3.

Original: Suppose we have an arbitrary base classiﬁer f , an example x , a set of labels which are denoted as S , two probabilities p S and p S that satisfy p S ≤ p S =
Revised: Suppose we have an arbitrary base classiﬁer f , an example x , a set of labels which are denoted as S , two probabilities p S and p S that satisfy p S ≤ p S =

Original: Pr ( f ( X ) ∈ S ) ≤ p S , and regions
Revised: Pr ( f ( X ) ∈ S ) ≤ p S , and regions

Original: A S and B S deﬁned as follows:
Revised: A S and B S deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We know that Pr ( X ∈ A S ) =
Revised: We know that Pr ( X ∈ A S ) =

Original: p S based on Lemma 2.
Revised: p S based on Lemma 2.

Original: Combined with the condition that [Equation] ( f ( X ) ∈ S ) , we obtain the ﬁrst inequality in (20).
Revised: Combined with the condition that p S ≤ Pr ( f ( X ) ∈ S ) , we obtain the ﬁrst inequality in (20).

Original: Similarly, we can obtain the second inequality in (20).
Revised: Similarly, we can obtain the second inequality in (20).

Original: We deﬁne M ( z ) =
Revised: We deﬁne M ( z ) =

Original: I ( f ( z ) ∈ S ) .
Revised: I ( f ( z ) ∈ S ) .

Original: Based on the ﬁrst inequality in (20) and
Revised: Based on the ﬁrst inequality in (20) and

Original: Lemma 1, we have the following:
Revised: Lemma 1, we have the following:

Original: [Equation]
Revised: [Equation]

Original: which is the ﬁrst inequality in (21).
Revised: which is the ﬁrst inequality in (21).

Original: The second inequality in (21) can be obtained similarly.
Revised: The second inequality in (21) can be obtained similarly.

Original: Next, we restate Theorem 1 and show our proof.
Revised: Next, we restate Theorem 1 and show our proof.

Original: Theorem 1 (Certiﬁed Radius for Top- k Predictions) .
Revised: Theorem 1 (Certiﬁed Radius for Top- k Predictions) .

Original: Suppose we are given an example x , an arbitrary base classiﬁer f , (cid:15) ∼ N (0 , σ 2 I ) , a smoothed classiﬁer g , an arbitrary label l ∈ { 1 , 2 , · · · , c } , and p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ∈ [0 , 1] that satisfy the following conditions:
Revised: Suppose we are given an example x , an arbitrary base classiﬁer f , (cid:15) ∼ N (0 , σ 2 I ) , a smoothed classiﬁer g , an arbitrary label l ∈ { 1 , 2 , · · · , c } , and p l , p 1 , · · · , p l − 1 , p l +1 , · · · , p c ∈ [0 , 1] that satisfy the following conditions:

Original: [Equation]
Revised: [Equation]

Original: where p and p indicate lower and upper bounds of p , respectively.
Revised: where p and p indicate lower and upper bounds of p , respectively.

Original: Let p b k ≥ p b k − 1 ≥ · · · ≥ p b 1 be the k largest ones among { p 1 , · · · , p l − 1 , p l +1 , · · · , p c } , where ties are broken uniformly at random.
Revised: Let p b k ≥ p b k − 1 ≥ · · · ≥ p b 1 be the k largest ones among { p 1 , · · · , p l − 1 , p l +1 , · · · , p c } , where ties are broken uniformly at random.

Original: Moreover, we denote by S t = {
Revised: Moreover, we denote by S t = {

Original: b 1 , b 2 , · · · , b t } the set of t labels with the smallest probability upper bounds in the k largest ones and by p S t = (cid:80) tj =

Revised: b 1 , b 2 , · · · , b t } the set of t labels with the smallest probability upper bounds in the k largest ones and by p S t = (cid:80) tj =

Original: p b j the sum of the t probability upper bounds, where t = 1 , 2 , · · · , k .
Revised: p b j the sum of the t probability upper bounds, where t = 1 , 2 , · · · , k .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: where R l is the unique solution to the following equation:
Revised: where R l is the unique solution to the following equation:

Original: [Equation]
Revised: [Equation]

Original: where Φ and Φ − 1 are the cumulative distribution function and its inverse of the standard Gaussian distribution, respectively.
Revised: where Φ and Φ − 1 are the cumulative distribution function and its inverse of the standard Gaussian distribution, respectively.

Original: Proof.
Revised: Proof.

Original: Roughly speaking, our idea is to make the probability that the base classiﬁer f predicts l when taking Y as input larger than the smallest one among the probabilities that f predicts for a set of arbitrary k labels selected from all labels except l .
Revised: Roughly speaking, our idea is to make the probability that the base classiﬁer f predicts l when taking Y as input larger than the smallest one among the probabilities that f predicts for a set of arbitrary k labels selected from all labels except l .

Original: For simplicity, we let Γ = { 1 , 2 , · · · , c } \ { l } , i.e., all labels except l .
Revised: For simplicity, we let Γ = { 1 , 2 , · · · , c } \ { l } , i.e., all labels except l .

Original: We denote by Γ k a set of k labels in Γ .
Revised: We denote by Γ k a set of k labels in Γ .

Original: We aim to ﬁnd a certiﬁed radius
Revised: We aim to ﬁnd a certiﬁed radius

Original: R l such that we have max Γ k ⊆
Revised: R l such that we have max Γ k ⊆

Original: Γ min i ∈
Revised: Γ min i ∈

Original: Γ k Pr ( f ( Y ) = i ) <
Revised: Γ k Pr ( f ( Y ) = i ) <

Original: Pr ( f ( Y ) = l ) , which guarantees l ∈ g k
Revised: Pr ( f ( Y ) = l ) , which guarantees l ∈ g k

Original: ( x + δ ) .
Revised: ( x + δ ) .

Original: We ﬁrst upper bound the minimal probability min i ∈
Revised: We ﬁrst upper bound the minimal probability min i ∈

Original: Γ k Pr ( f ( Y ) = i ) for a given Γ k , and then we upper bound the maximum value of the minimal probability among all possible Γ k ⊆ Γ .
Revised: Γ k Pr ( f ( Y ) = i ) for a given Γ k , and then we upper bound the maximum value of the minimal probability among all possible Γ k ⊆ Γ .

Original: Finally, we obtain the certiﬁed radius R l via letting the upper bound of the maximum value smaller than Pr ( f ( Y ) = l ) .
Revised: Finally, we obtain the certiﬁed radius R l via letting the upper bound of the maximum value smaller than Pr ( f ( Y ) = l ) .

Original: Bounding min i ∈ Γ k Pr ( f ( Y ) = i ) for a given Γ k : We use S to denote a non-empty subset of Γ k and use | S | to denote its size.
Revised: Bounding min i ∈ Γ k Pr ( f ( Y ) = i ) for a given Γ k : We use S to denote a non-empty subset of Γ k and use | S | to denote its size.

Original: We deﬁne p S
Revised: We deﬁne p S

Original: = (cid:80) i ∈
Revised: = (cid:80) i ∈

Original: S p i , which is the sum of the upper bounds of the probabilities for the labels in S .
Revised: S p i , which is the sum of the upper bounds of the probabilities for the labels in S .

Original: Moreover, we deﬁne the following region associated with the set S :
Revised: Moreover, we deﬁne the following region associated with the set S :

Original: [Equation]
Revised: [Equation]

Original: We have Pr ( f ( Y ) ∈ S )
Revised: We have Pr ( f ( Y ) ∈ S )

Original: ≤ Pr ( Y ∈ B S )
Revised: ≤ Pr ( Y ∈ B S )

Original: by applying Lemma 3 to the set S .
Revised: by applying Lemma 3 to the set S .

Original: In addition, we have (cid:80) i ∈
Revised: In addition, we have (cid:80) i ∈

Original: S Pr ( f ( Y ) = i ) =
Revised: S Pr ( f ( Y ) = i ) =

Original: Pr ( f ( Y ) ∈ S ) .
Revised: Pr ( f ( Y ) ∈ S ) .

Original: Therefore, we have:
Revised: Therefore, we have:

Original: [Equation]
Revised: [Equation]

Original: Moreover, we have:
Revised: Moreover, we have:

Original: [Equation]
Revised: [Equation]

Original: where we have the ﬁrst inequality because S is a subset of Γ k and we have the second inequality because the smallest value in a set is no larger than the average value of the set.
Revised: where we have the ﬁrst inequality because S is a subset of Γ k and we have the second inequality because the smallest value in a set is no larger than the average value of the set.

Original: Equation 27 holds for any S ⊆ Γ k .
Revised: Equation 27 holds for any S ⊆ Γ k .

Original: Therefore, by taking all possible sets S into consideration, we have the following:
Revised: Therefore, by taking all possible sets S into consideration, we have the following:

Original: [Equation]
Revised: [Equation]

Original: where S t is the set of t labels in Γ k whose probability upper bounds are the smallest, where ties are broken uniformly at random.
Revised: where S t is the set of t labels in Γ k whose probability upper bounds are the smallest, where ties are broken uniformly at random.

Original: We have Equation 30 from Equation 29 because Pr ( Y ∈ B S ) decreases as p S decreases.
Revised: We have Equation 30 from Equation 29 because Pr ( Y ∈ B S ) decreases as p S decreases.

Original: Bounding max Γ k ⊆
Revised: Bounding max Γ k ⊆

Original: Γ min i ∈
Revised: Γ min i ∈

Original: Γ k Pr ( f ( Y ) = i ) :
Revised: Γ k Pr ( f ( Y ) = i ) :

Original: Since Pr ( Y ∈ B S t ) increases as p S t increases,
Revised: Since Pr ( Y ∈ B S t ) increases as p S t increases,

Original: Equation 30 reaches its maximum value when Γ k = { b 1 , b 2 , · · · , b k } , i.e., when Γ k is the set of k labels in Γ with the largest probability upper bounds.
Revised: Equation 30 reaches its maximum value when Γ k = { b 1 , b 2 , · · · , b k } , i.e., when Γ k is the set of k labels in Γ with the largest probability upper bounds.

Original: Formally, we have:
Revised: Formally, we have:

Original: [Equation]
Revised: [Equation]

Original: where S t = { b 1 , b 2 , · · · , b t } .
Revised: where S t = { b 1 , b 2 , · · · , b t } .

Original: Obtaining R l : According to Lemma 3, we have the following for S
Revised: Obtaining R l : According to Lemma 3, we have the following for S

Original: =
Revised: =

Original: { l } :
Revised: { l } :

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Recall that our goal is to make Pr ( f ( Y )
Revised: Recall that our goal is to make Pr ( f ( Y )

Original: =
Revised: =

Original: l ) > max Γ k ⊆
Revised: l ) > max Γ k ⊆

Original: Γ min i ∈
Revised: Γ min i ∈

Original: Γ k Pr ( f ( Y ) = i ) .
Revised: Γ k Pr ( f ( Y ) = i ) .

Original: It sufﬁces to let:
Revised: It sufﬁces to let:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]

Original: According to Lemma 2, we have Pr ( Y ∈
Revised: According to Lemma 2, we have Pr ( Y ∈

Original: A { l } ) =
Revised: A { l } ) =

Original: Φ(Φ − 1 ( p l ) − ||
Revised: Φ(Φ − 1 ( p l ) − ||

Original: δ || 2 σ ))
Revised: δ || 2 σ ))

Original: and Pr ( Y ∈ B S t ) =
Revised: and Pr ( Y ∈ B S t ) =

Original: Φ(Φ − 1 ( p S t )
Revised: Φ(Φ − 1 ( p S t )

Original: + ||
Revised: + ||

Original: δ || 2 σ )) .
Revised: δ || 2 σ )) .

Original: Therefore, we have the following constraint on δ :
Revised: Therefore, we have the following constraint on δ :

Original: [Equation]
Revised: [Equation]

Original: Since the left-hand side of the above inequality 1) decreases as ||
Revised: Since the left-hand side of the above inequality 1) decreases as ||

Original: δ || 2 increases, 2) is larger thanwhen
Revised: δ || 2 increases, 2) is larger thanwhen

Original: ||
Revised: ||

Original: δ || 

Revised: δ ||

Original: → −∞ , and 3) is smaller than 

Revised: → −∞ , and 3) is smaller than

Original: when ||
Revised: when ||

Original: δ || 2 → ∞ , we have the constraint ||
Revised: δ || 2 → ∞ , we have the constraint ||

Original: δ || 

Revised: δ ||

Original: < R l , where R l is the unique solution to the following equation:
Revised: < R l , where R l is the unique solution to the following equation:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: B P ROOF OF T HEOREM
Revised: B P ROOF OF T HEOREM

Original: Following the terminology we used in proving Theorem 1, we deﬁne a region A { l } as follows:
Revised: Following the terminology we used in proving Theorem 1, we deﬁne a region A { l } as follows:

Original: [Equation]
Revised: [Equation]

Original: According to Lemma 2, we have Pr ( X ∈ A { l } ) =
Revised: According to Lemma 2, we have Pr ( X ∈ A { l } ) =

Original: p l .
Revised: p l .

Original: We ﬁrst show the following lemma, which is the key to prove our Theorem 2.
Revised: We ﬁrst show the following lemma, which is the key to prove our Theorem 2.

Original: Lemma 4.
Revised: Lemma 4.

Original: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 .
Revised: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 .

Original: For any perturbation (cid:107) δ (cid:107) 2 > R l , there exists k disjoint regions
Revised: For any perturbation (cid:107) δ (cid:107) 2 > R l , there exists k disjoint regions

Original: C b j ⊆ R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy the following:
Revised: C b j ⊆ R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy the following:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively; and { b 1 , b 2 , · · · , b k } and S t are deﬁned in Theorem 1.
Revised: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively; and { b 1 , b 2 , · · · , b k } and S t are deﬁned in Theorem 1.

Original: Proof.
Revised: Proof.

Original: Our proof is based on mathematical induction and the intermediate value theorem .
Revised: Our proof is based on mathematical induction and the intermediate value theorem .

Original: For convenience, we defer the proof to Appendix B.1.
Revised: For convenience, we defer the proof to Appendix B.1.

Original: Next, we restate Theorem 2 and show our proof.
Revised: Next, we restate Theorem 2 and show our proof.

Original: Theorem 2 (Tightness of the Certiﬁed Radius) .
Revised: Theorem 2 (Tightness of the Certiﬁed Radius) .

Original: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i
Revised: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 and p l + (cid:80) i

Original: =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .
Revised: =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .

Original: Then, for any perturbation ||
Revised: Then, for any perturbation ||

Original: δ || 2 > R l , there exists a base classiﬁer f ∗ consistent with (1) but we have l / ∈
Revised: δ || 2 > R l , there exists a base classiﬁer f ∗ consistent with (1) but we have l / ∈

Original: g k ( x + δ ) .
Revised: g k ( x + δ ) .

Original: Proof.
Revised: Proof.

Original: Our idea is to construct a base classiﬁer such that l is not among the top- k labels predicted by the smoothed classiﬁer for any perturbed example x + δ when || δ || 2 > R l .
Revised: Our idea is to construct a base classiﬁer such that l is not among the top- k labels predicted by the smoothed classiﬁer for any perturbed example x + δ when || δ || 2 > R l .

Original: First, according to Lemma 4, we know there exists k disjoint regions C b j ⊆
Revised: First, according to Lemma 4, we know there exists k disjoint regions C b j ⊆

Original: R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy Equation 37 and 38.
Revised: R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy Equation 37 and 38.

Original: Moreover, we divide the remaining region R d \ ( A { l } ∪ kj =1 C b j ) into c − k − 1 regions, which we denote as C b k +1 , C b k +2 , · · · , C b c − 1 and satisfy Pr ( X ∈ C b j ) ≤ p b j for j ∈ { k +1 , k +2 , · · · , c − 1 } .
Revised: Moreover, we divide the remaining region R d \ ( A { l } ∪ kj =1 C b j ) into c − k − 1 regions, which we denote as C b k +1 , C b k +2 , · · · , C b c − 1 and satisfy Pr ( X ∈ C b j ) ≤ p b j for j ∈ { k +1 , k +2 , · · · , c − 1 } .

Original: Note that b 1 , b 2 , · · · , b c − 1 is some permutation of { 1 , 2 , · · · , c }\{ l } .
Revised: Note that b 1 , b 2 , · · · , b c − 1 is some permutation of { 1 , 2 , · · · , c }\{ l } .

Original: We can divide the remaining region into such c − k − 1 regions because p l + (cid:80) i =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .
Revised: We can divide the remaining region into such c − k − 1 regions because p l + (cid:80) i =1 , ··· ,l − 1 ,l +1 , ··· ,c p i ≥ 1 .

Original: Then, based on these regions, we construct the following base classiﬁer:
Revised: Then, based on these regions, we construct the following base classiﬁer:

Original: [Equation]
Revised: [Equation]

Original: Based on the deﬁnition of f ∗, we have the following:
Revised: Based on the deﬁnition of f ∗

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Therefore
Revised: we have the following:

Original: [Equation]
Revised: [Equation]

Original:  f ∗ satisﬁes the conditions in (1).Next [Equation]
Revised: Therefore, f ∗ satisﬁes the conditions in (1).

Original:  we show that l is not among the top- k labels predicted by the smoothed classiﬁer for any perturbed example x + δ when ||
Revised: Next, we show that l is not among the top- k labels predicted by the smoothed classiﬁer for any perturbed example x + δ when ||

Original: δ || 2 > R l .
Revised: δ || 2 > R l .

Original: Speciﬁcally, we have:
Revised: Speciﬁcally, we have:

Original: [Equation]
Revised: [Equation]

Revised: Deﬁnition

Revised: [Equation]

Original: where j = 1 , 2 , · · · , k .
Revised: where j = 1 , 2 , · · · , k .

Original: Since we have found k labels whose probabilities are larger than the probability of the label l , we have l / ∈
Revised: Since we have found k labels whose probabilities are larger than the probability of the label l , we have l / ∈

Original: g k ( x + δ ) when (cid:107) δ (cid:107) 2 > R l .
Revised: g k ( x + δ ) when (cid:107) δ (cid:107) 2 > R l .

Original: B. P ROOF OF L EMMA
Revised: B. P ROOF OF L EMMA

Original: We ﬁrst deﬁne some key notations and lemmas that will be used in our proof.
Revised: We ﬁrst deﬁne some key notations and lemmas that will be used in our proof.

Original: Deﬁnition 1 ( C ( q 1 , q 2 ) , C (cid:48) ( q 1 , q 2 ) , r x ( q 1 , q 2 ) , r y ( q 1 , q 2 ) ) .
Revised: Deﬁnition 1 ( C ( q 1 , q 2 ) , C (cid:48) ( q 1 , q 2 ) , r x ( q 1 , q 2 ) , r y ( q 1 , q 2 ) ) .

Original: Given two values q 1 and q 2 that satisfy 0 ≤ q 1 < q 2 ≤ 1 , we deﬁne the following region:
Revised: Given two values q 1 and q 2 that satisfy 0 ≤ q 1 < q 2 ≤ 1 , we deﬁne the following region:

Original: [Equation]
Revised: [Equation]

Original: According to Lemma 2, we have:
Revised: According to Lemma 2, we have:

Original: [Equation]
Revised: [Equation]

Original: where the Gaussian random variable X is deﬁned in Equation 10.
Revised: where the Gaussian random variable X is deﬁned in Equation 10.

Original: Moreover, assuming we have pairs of ( q i 1 , q i 2 ) , i = 1 , 2 , 3 , · · · , where q i 1 , q i 2 ∈ [ q 1 , q 2 ] , ∀ i .
Revised: Moreover, assuming we have pairs of ( q i 1 , q i 2 ) , i = 1 , 2 , 3 , · · · , where q i 1 , q i 2 ∈ [ q 1 , q 2 ] , ∀ i .

Original: We deﬁne the following region:
Revised: We deﬁne the following region:

Original: [Equation]
Revised: [Equation]

Original: C (cid:48) ( q 1 , q 2 ) is the remaining region of C ( q 1 , q 2 ) excluding C ( q 11 , q 12 ) , C ( q 21 , q 22 ) , · · · .
Revised: C (cid:48) ( q 1 , q 2 ) is the remaining region of C ( q 1 , q 2 ) excluding C ( q 11 , q 12 ) , C ( q 21 , q 22 ) , · · · .

Original: Given two values q λ 1 and q λ 2 that satisfy q 1 ≤ q λ 1 ≤ q λ 2 ≤ q 2 , we also deﬁne the following two functions:[Equation]
Revised: Given two values q λ 1 and q λ 2 that satisfy q 1 ≤ q λ 1 ≤ q λ 2 ≤ q 2 , we also deﬁne the following two functions:

Original: [Equation]
Revised: [Equation]

Original: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively.
Revised: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively.

Original: Next, we show a key property of our deﬁned functions r x ( q λ 1 , q λ 2 ) and r y ( q λ 1 , q λ 2 ) .
Revised: Next, we show a key property of our deﬁned functions r x ( q λ 1 , q λ 2 ) and r y ( q λ 1 , q λ 2 ) .

Original: Lemma 5.
Revised: Lemma 5.

Original: If r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) and q κ 2 ≥ q λ 2 (or q κ 1 ≥ q λ 1 ) , then we have r y ( q κ 1 , q κ 2 ) ≤ r y ( q λ 1 , q λ 2 ) .
Revised: If r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) and q κ 2 ≥ q λ 2 (or q κ 1 ≥ q λ 1 ) , then we have r y ( q κ 1 , q κ 2 ) ≤ r y ( q λ 1 , q λ 2 ) .

Original: Proof.
Revised: Proof.

Original: We consider three scenarios.
Revised: We consider three scenarios.

Original: Scenario I: q κ 2 ≥ q κ 1 ≥ q λ 2 ≥ q λ 1 .
Revised: Scenario I: q κ 2 ≥ q κ 1 ≥ q λ 2 ≥ q λ 1 .

Original: We denote h x and h y as the probability densities for the random variables X and Y , respectively.
Revised: We denote h x and h y as the probability densities for the random variables X and Y , respectively.

Original: Then, we have h x ( z ) =
Revised: Then, we have h x ( z ) =

Original: ( 1 √ 2 πσ ) d exp( − (cid:80) di
Revised: ( 1 √ 2 πσ ) d exp( − (cid:80) di

Original: =1 ( z 
Revised: [Equation] 1 ( z i − x i

Original: [Equation]
Revised: [Equation]

Original: − x i ) 2 2 σ) and h y ( z ) =
Revised: 2 2 σ) and h y ( z ) =

Original: ( 1 √ 2 πσ ) d exp( − (cid:80) di
Revised: ( 1 √ 2 πσ ) d exp( − (cid:80) di

Original: =

Revised: =

Original: ( z i − x i − δ i ) 2 2 σ) .
Revised: ( z i − x i − δ i ) 2 2 σ) .

Original: Therefore, the ratio of the probability density of Y and the probability density of X at a given point z is as follows:
Revised: Therefore, the ratio of the probability density of Y and the probability density of X at a given point z is as follows:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Next, we compare the ratio for the points in different regions and have the following:
Revised: Next, we compare the ratio for the points in different regions and have the following:

Original: [Equation]
Revised: [Equation]

Original: The Equation 57 from 56 is based on Equation 55 and the fact that δ T ( z − x ) ≤ σ [Equation] δ [Equation]{+(cid:107) 
+}
Revised: The Equation 57 from 56 is based on Equation 55 and the fact that δ T ( z − x ) ≤ σ (cid:107) δ {+(cid:107) 
+}

Original: 2 Φ −[Equation] 1  − q κ 1 ) for any point z in the region C (cid:48) ( q κ 1 , q κ 2 ) from Deﬁnition 1.
Revised: Φ − 1 (1 − q κ 1 ) for any point z in the region C (cid:48) ( q κ 1 , q κ 2 ) from Deﬁnition 1.

Original: Similarly, we can obtain Equationfrom 58.
Revised: Similarly, we can obtain Equationfrom 58.

Original: We note that the Equation 58 from 57 is because q λ 2 ≤ q κ 1 .
Revised: We note that the Equation 58 from 57 is because q λ 2 ≤ q κ 1 .

Original: Based on Equation 57 and 58, we know that there exists a real number u such that:
Revised: Based on Equation 57 and 58, we know that there exists a real number u such that:

Original: [Equation]
Revised: [Equation]

Original: Combining the Equation 56, 57, and 60, we have the following:
Revised: Combining the Equation 56, 57, and 60, we have the following:

Original: [Equation]
Revised: [Equation]

Original: Taking an integral on both sides of the Equation 61 in the region C (cid:48) ( q κ 1 , q κ 2 ) and recalling the deﬁnition of r x ( q κ 1 , q κ 2 ) and r y ( q κ 1 , q κ 2 ) , we have the following:
Revised: Taking an integral on both sides of the Equation 61 in the region C (cid:48) ( q κ 1 , q κ 2 ) and recalling the deﬁnition of r x ( q κ 1 , q κ 2 ) and r y ( q κ 1 , q κ 2 ) , we have the following:

Original: [Equation]
Revised: [Equation]

Original: Similarly, we have:
Revised: Similarly, we have:

Original: [Equation]
Revised: [Equation]

Original: Based on Equation 62, 63, and the condition that r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) , we have the following: 
Revised: Based on Equation 62, 63, and the condition that r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) , we have the following: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Scenario II: q κ 2 ≥ q λ 2 ≥ q κ 1 ≥ q λ 1 .
Revised: Scenario II: q κ 2 ≥ q λ 2 ≥ q κ 1 ≥ q λ 1 .

Original: We have:
Revised: We have:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have the following equation:
Revised: Therefore, we have the following equation:

Original: [Equation]
Revised: [Equation]

Original: Similar to Scenario I , we know that there exists u such that:
Revised: Similar to Scenario I , we know that there exists u such that:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Similar to Scenario I , we have the following based on Equation 66:
Revised: Similar to Scenario I , we have the following based on Equation 66:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have the following:
Revised: Therefore, we have the following:

Original: [Equation]
Revised: [Equation]

Original: Scenario III: q κ 2 ≥ q λ 2 ≥ q λ 1 ≥ q κ 1 .
Revised: Scenario III: q κ 2 ≥ q λ 2 ≥ q λ 1 ≥ q κ 1 .

Original: As r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) , we have r x ( q κ 1 , q κ 2 ) =
Revised: As r x ( q κ 1 , q κ 2 ) ≤ r x ( q λ 1 , q λ 2 ) , we have r x ( q κ 1 , q κ 2 ) =

Original: r x ( q λ 1 , q λ 2 ) .
Revised: r x ( q λ 1 , q λ 2 ) .

Original: Therefore, we have r y ( q κ 1 , q κ 2 ) =
Revised: Therefore, we have r y ( q κ 1 , q κ 2 ) =

Original: r y ( q λ 1 , q λ 2 ) .
Revised: r y ( q λ 1 , q λ 2 ) .

Original: Next, we list the well-known Intermediate Value Theorem and show several other properties of our deﬁned functions r x ( q λ 1 , q λ 2 ) and r y ( q λ 1 , q λ 2 ) .
Revised: Next, we list the well-known Intermediate Value Theorem and show several other properties of our deﬁned functions r x ( q λ 1 , q λ 2 ) and r y ( q λ 1 , q λ 2 ) .

Original: Lemma 6 (Intermediate Value Theorem) .
Revised: Lemma 6 (Intermediate Value Theorem) .

Original: If a function F is continuous at every point in the interval [ a, b ] and ( F ( a ) − v ) · ( F ( b ) − v ) ≤ 0 , then there exists x such that F ( x ) = v .
Revised: If a function F is continuous at every point in the interval [ a, b ] and ( F ( a ) − v ) · ( F ( b ) − v ) ≤ 0 , then there exists x such that F ( x ) = v .

Original: Roughly speaking, the Intermediate Value Theorem tells us that if a continuous function has values no larger and no smaller (or no smaller and no larger) than v at the two end points of an interval, respectively, then the function takes value v at some point in the interval.
Revised: Roughly speaking, the Intermediate Value Theorem tells us that if a continuous function has values no larger and no smaller (or no smaller and no larger) than v at the two end points of an interval, respectively, then the function takes value v at some point in the interval.

Original: Lemma 7.
Revised: Lemma 7.

Original: Given two probabilities q x , q y , if we have:
Revised: Given two probabilities q x , q y , if we have:

Original: [Equation]
Revised: [Equation]

Original: Then, there exists q (cid:48) 1 , q (cid:48) 2 ∈ [ q 1 , q 2 ] such that:
Revised: Then, there exists q (cid:48) 1 , q (cid:48) 2 ∈ [ q 1 , q 2 ] such that:

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if we have:
Revised: Furthermore, if we have:

Original: [Equation]
Revised: [Equation]

Original: [Equation] then there exists q (cid:48)(cid:48) 1 , q (cid:48)(cid:48) 2 ∈ [ q 1 , q 2 ] such that: 
Revised: then there exists q (cid:48)(cid:48) 1 , q (cid:48)(cid:48) 2 ∈ [ q 1 , q 2 ] such that: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We deﬁne function F ( x ) =
Revised: We deﬁne function F ( x ) =

Original: r x ( q 1 , x ) .
Revised: r x ( q 1 , x ) .

Original: Then, we have ( F ( q 1 ) − q x ) · ( F ( q 2 ) − q x ) ≤since F ( q 1 ) = 0 ≤ q x and F ( q 2 ) =
Revised: Then, we have ( F ( q 1 ) − q x ) · ( F ( q 2 ) − q x ) ≤since F ( q 1 ) = 0 ≤ q x and F ( q 2 ) =

Original: r x ( q 1 , q 2 ) > q x based on Equation 70.
Revised: r x ( q 1 , q 2 ) > q x based on Equation 70.

Original: Therefore, according to
Revised: Therefore, according to

Original: Lemma 6, there exists q (cid:48) 1 ∈
Revised: Lemma 6, there exists q (cid:48) 1 ∈

Original: [ q 1 , q 2 ] such that:
Revised: [ q 1 , q 2 ] such that:

Original: [Equation]
Revised: [Equation]

Original: Similarly, we can prove that there exists q (cid:48) 2 ∈ [ q 1 , q 2 ] such that r x ( q (cid:48) 1 , q 2 ) =
Revised: Similarly, we can prove that there exists q (cid:48) 2 ∈ [ q 1 , q 2 ] such that r x ( q (cid:48) 1 , q 2 ) =

Original: q x .
Revised: q x .

Original: For any q e 2 ∈ [ q (cid:48) 2 , q 2 ] , we deﬁne H ( x ) =
Revised: For any q e 2 ∈ [ q (cid:48) 2 , q 2 ] , we deﬁne H ( x ) =

Original: r x ( x, q e 2 ) .
Revised: r x ( x, q e 2 ) .

Original: Then, we know H ( q 1 ) =
Revised: Then, we know H ( q 1 ) =

Original: r x ( q 1 , q e 2 ) ≥ r x ( q 1 , q (cid:48) 2 ) =
Revised: r x ( q 1 , q e 2 ) ≥ r x ( q 1 , q (cid:48) 2 ) =

Original: q x since q e 2 ≥ q (cid:48) 2 .
Revised: q x since q e 2 ≥ q (cid:48) 2 .

Original: Moreover, we have H ( q e 2 ) = 0 ≤ q x .
Revised: Moreover, we have H ( q e 2 ) = 0 ≤ q x .

Original: Therefore, we have ( H ( q 1 ) − q x ) · ( H ( q e 2 ) − q x ) ≤ 0 .
Revised: Therefore, we have ( H ( q 1 ) − q x ) · ( H ( q e 2 ) − q x ) ≤ 0 .

Original: According to Lemma 6,we know there exists q e 1 ∈
Revised: According to Lemma 6,we know there exists q e 1 ∈

Original: [ q 1 , q e 2 ] such that r x ( q e 1 , q e 2 ) =
Revised: [ q 1 , q e 2 ] such that r x ( q e 1 , q e 2 ) =

Original: q x for arbitrary q e 2 ∈ [ q (cid:48) 2 , q 2 ] .
Revised: q x for arbitrary q e 2 ∈ [ q (cid:48) 2 , q 2 ] .

Original: We deﬁne G ( x ) =
Revised: We deﬁne G ( x ) =

Original: r y ( q e 1 , x ) where x ∈ [ q (cid:48) 2 , q 2 ] , and q e 1 are a value such that r x ( q e 1 , x ) =
Revised: r y ( q e 1 , x ) where x ∈ [ q (cid:48) 2 , q 2 ] , and q e 1 are a value such that r x ( q e 1 , x ) =

Original: q x for a given x .
Revised: q x for a given x .

Original: When x = q (cid:48) 2 , we can let q e 1 = q 1 since r x ( q 1 , q (cid:48) 2 ) =
Revised: When x = q (cid:48) 2 , we can let q e 1 = q 1 since r x ( q 1 , q (cid:48) 2 ) =

Original: q x , and when x = q 1 , we can let q e 

Revised: q x , and when x = q 1 , we can let q e

Original: = q (cid:48) 2 since r x ( q (cid:48) 2 , q 2 ) =
Revised: = q (cid:48) 2 since r x ( q (cid:48) 2 , q 2 ) =

Original: q x .
Revised: q x .

Original: Based on Equationand Lemma 6, we know that there exists x ∈ [ q (cid:48) 2 , q 2 ] such that G ( x ) =
Revised: Based on Equationand Lemma 6, we know that there exists x ∈ [ q (cid:48) 2 , q 2 ] such that G ( x ) =

Original: q y .
Revised: q y .

Original: Therefore, there exists q (cid:48)(cid:48) 1 and q (cid:48)(cid:48) 2 such that: 
Revised: Therefore, there exists q (cid:48)(cid:48) 1 and q (cid:48)(cid:48) 2 such that: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Lemma 8.
Revised: Lemma 8.

Original: Assuming we have q λ 1 = 0 , q λ 2 = p S t .
Revised: Assuming we have q λ 1 = 0 , q λ 2 = p S t .

Original: If q λ 2 = p S t ≤ min i q i 1 , then we have the following:
Revised: If q λ 2 = p S t ≤ min i q i 1 , then we have the following:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: If q λ 2 ≤ min i q i 1 , then we have C (cid:48) ( q 1 , q [Equation]
Revised: If q λ 2 ≤ min i q i 1 , then we have C (cid:48) ( q 1 , q 2 ) ∩C

Original:  q λ [Equation] q λ 
Revised: ( q λ 1 , q λ 2 ) =

Original: [Equation]

Original: = C ( q λ 1 , q λ 2 ) since no region is excluded.
Revised: C ( q λ 1 , q λ 2 ) since no region is excluded.

Original: Therefore, we have r y ( q λ 1 , q λ 2 ) =
Revised: Therefore, we have r y ( q λ 1 , q λ 2 ) =

Original: Pr ( Y ∈ C (cid:48)
Revised: Pr ( Y ∈ C (cid:48)

Original: ( q 1 , q 2 ) ∩C
Revised: ( q 1 , q 2 ) ∩C

Original: ( q λ 1 , q λ 2 ))
Revised: ( q λ 1 , q λ 2 ))

Original: =
Revised: =

Original: Pr ( Y ∈ C ( q λ 1 , q λ 2 ))
Revised: Pr ( Y ∈ C ( q λ 1 , q λ 2 ))

Original: =
Revised: =

Original: q λ 2 − q λ 1 = p S t based on Equation 51.
Revised: q λ 2 − q λ 1 = p S t based on Equation 51.

Original: We note that C ( q λ 1 , q λ 2 ) =
Revised: We note that C ( q λ 1 , q λ 2 ) =

Original: B S t when q λ 1 = 0 and q λ 2 = p S t .
Revised: B S t when q λ 1 = 0 and q λ 2 = p S t .

Original: Therefore, we can obtain Equation 80 based on the deﬁnition of r y ( q λ 1 , q λ 2 ) from Deﬁnition 1.
Revised: Therefore, we can obtain Equation 80 based on the deﬁnition of r y ( q λ 1 , q λ 2 ) from Deﬁnition 1.

Original: Lemma 9.
Revised: Lemma 9.

Original: If we have q 1 ≤ q o 2 ≤ q 2 , then we have the following:
Revised: If we have q 1 ≤ q o 2 ≤ q 2 , then we have the following:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By applying Lemma 5.
Revised: By applying Lemma 5.

Original: We further generalize Lemma 5 to two regions.
Revised: We further generalize Lemma 5 to two regions.

Original: Speciﬁcally, we have the following lemma:
Revised: Speciﬁcally, we have the following lemma:

Original: Lemma 10.
Revised: Lemma 10.

Original: Assuming we have a region C w ⊆ C ( q w 1 , q w 2 ) and we have C (cid:48) ( q 1 , q 2 ) ∩ C ( q w 1 , q w 2 ) = ∅ .
Revised: Assuming we have a region C w ⊆ C ( q w 1 , q w 2 ) and we have C (cid:48) ( q 1 , q 2 ) ∩ C ( q w 1 , q w 2 ) = ∅ .

Original: If q 1 ≥ q w 1 , q 2 ≥ q w 2 and r x ( q (cid:48) 1 , q (cid:48) 2 ) ≤ Pr ( X ∈ C w ) , then we have:
Revised: If q 1 ≥ q w 1 , q 2 ≥ q w 2 and r x ( q (cid:48) 1 , q (cid:48) [Equation] Pr ( X ∈ C w ) , then we have:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We let q 1 = max( q 1 , q w 2 ) .
Revised: We let q 1 = max( q 1 , q w 2 ) .

Original: As C (cid:48) ( q 1 , q 2 ) ∩ C ( q w 1 , q w 2 ) = ∅ and q 1 ≥ q w 1 , we can obtain the conclusion by applying Lemma 5 on C (cid:48)
Revised: As C (cid:48) ( q 1 , q 2 ) ∩ C ( q w 1 , q w 2 ) = ∅ and q 1 ≥ q w 1 , we can obtain the conclusion by applying Lemma 5 on C (cid:48)

Original: ( q 1 , q 2 ) ∪ C w .
Revised: ( q 1 , q 2 ) ∪ C w .

Original: Next, we restate Lemma 4 and show our proof.
Revised: Next, we restate Lemma 4 and show our proof.

Original: Lemma 4.
Revised: Lemma 4.

Original: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 .
Revised: Assuming we have p l + (cid:80) kj =1 p b j ≤ 1 .

Original: For any perturbation (cid:107) δ (cid:107) 2 > R l , there exists k disjoint regions
Revised: For any perturbation (cid:107) δ (cid:107) 2 > R l , there exists k disjoint regions

Original: C b j ⊆ R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy the following:
Revised: C b j ⊆ R d \ A { l } , j ∈ { 1 , 2 , · · · , k } that satisfy the following:

Original: [Equation]
Revised: [Equation]

Original: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively; and { b 1 , b 2 , · · · , b k } and S t are deﬁned in Theorem 1.
Revised: where the random variables X and Y are deﬁned in Equation 10 and 11, respectively; and { b 1 , b 2 , · · · , b k } and S t are deﬁned in Theorem 1.

Original: Proof.
Revised: Proof.

Original: Our proof leverages Mathematical Induction , which contains two steps.
Revised: Our proof leverages Mathematical Induction , which contains two steps.

Original: In the ﬁrst step, we show that the statement holds initially.
Revised: In the ﬁrst step, we show that the statement holds initially.

Original: In the second step, we show that if the statement is true for the m th iteration, then it also holds for the ( m +1) th iteration.
Revised: In the second step, we show that if the statement is true for the m th iteration, then it also holds for the ( m +1) th iteration.

Original: Without loss of generality, we assume τ = argmin kt =1 Pr ( Y ∈B St [Equation]
Revised: Without loss of generality, we assume τ = argmin kt =1 Pr ( Y ∈B St ) t .

Original: t .

Original: Therefore, we have the following:
Revised: Therefore, we have the following:

Original: [Equation]
Revised: [Equation]

Original: Recall the deﬁnition of B S τ
Revised: Recall the deﬁnition of B S τ

Original: and we have the following:
Revised: and we have the following:

Original: [Equation]
Revised: [Equation]

Original: where p S τ
Revised: where p S τ

Original: =
Revised: =

Original: (cid:80) j ∈ S τ p j .
Revised: (cid:80) j ∈ S τ p j .

Original: We can split B S k into two parts: B S τ and B S k \ B S τ .
Revised: We can split B S k into two parts: B S τ and B S k \ B S τ .

Original: We will show that ∀ j ∈ [1 , τ ] , we can ﬁnd disjoint C b j ⊆ B S τ whose union is B S τ such that:
Revised: We will show that ∀ j ∈ [1 , τ ] , we can ﬁnd disjoint C b j ⊆ B S τ whose union is B S τ such that:

Original: [Equation]
Revised: [Equation]

Original: For the other part, we will show that ∀ j ∈ [ τ + 1 , k ] , we can ﬁnd disjoint C b j ⊆ B S k \ B S τ whose union is B S k \ B S τ such that:
Revised: For the other part, we will show that ∀ j ∈ [ τ + 1 , k ] , we can ﬁnd disjoint C b j ⊆ B S k \ B S τ whose union is B S k \ B S τ such that:

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst show that ∀ j ∈ [1 , τ ] , we can ﬁnd C b j ⊆ B S τ that satisfy Equation 85 and 86.
Revised: We ﬁrst show that ∀ j ∈ [1 , τ ] , we can ﬁnd C b j ⊆ B S τ that satisfy Equation 85 and 86.

Original: Since our proof leverages Mathematical Induction, we iteratively construct each C b j , ∀ j ∈ [1 , τ ] .
Revised: Since our proof leverages Mathematical Induction, we iteratively construct each C b j , ∀ j ∈ [1 , τ ] .

Original: Speciﬁcally, we ﬁrst show that we can ﬁnd C b τ ⊆ B S τ that satisﬁes the requirements.
Revised: Speciﬁcally, we ﬁrst show that we can ﬁnd C b τ ⊆ B S τ that satisﬁes the requirements.

Original: Then, assuming we can ﬁnd C b τ , · · · , C b τ − m +1 , we show that we can ﬁnd C b τ − m ⊆
Revised: Then, assuming we can ﬁnd C b τ , · · · , C b τ − m +1 , we show that we can ﬁnd C b τ − m ⊆

Original: B S τ \ ( ∪ τ j = τ − m +1 C b j ) .
Revised: B S τ \ ( ∪ τj = τ − m +1 C b j ) .

Original: We will leverage Lemma 7 to prove the existence for each C b j .
Revised: We will leverage Lemma 7 to prove the existence for each C b j .

Original: Next, we show the two steps.
Revised: Next, we show the two steps.

Original: Step I: We show that we can ﬁnd C b τ ⊆ B S
Revised: Step I: We show that we can ﬁnd C b τ ⊆ B S

Original: τ that satisﬁes Equation 85 and 86.
Revised: τ that satisﬁes Equation 85 and 86.

Original: We let q 1 = 0 and q 2 = p S τ , and we deﬁne the following region:
Revised: We let q 1 = 0 and q 2 = p S τ , and we deﬁne the following region:

Original: [Equation]
Revised: [Equation]

Original: We have:
Revised: We have:

Original: [Equation]
Revised: [Equation]

Original: which can be directly obtained as C (cid:48)
Revised: which can be directly obtained as C (cid:48)

Original: ( q 1 , q 2 ) =
Revised: ( q 1 , q 2 ) =

Original: B S τ .
Revised: B S τ .

Original: As we have p b τ ≤ r x ( q 1 , q 2 )
Revised: As we have p b τ ≤ r x ( q 1 , q 2 )

Original: = p S τ = [Equation]
Revised: = p S τ = (cid:80) τj =1 p

Original: p b j , there exist q (cid:48)1 = p S
Revised: b j , there exist q (cid:48)

Original: [Equation]
Revised: [Equation]

Original:  p [Equation] τ 
Revised: = p S τ − p b τ , q

Original: [Equation]
Revised: [Equation]

Original: 2 = p b τ such that:
Revised: = p b τ such that:

Original: [Equation]
Revised: [Equation]

Original: Moreover
Revised: Moreover

Original: [Equation]
Revised: [Equation]

Original: we have the following:
Revised: we have the following:

Original: [Equation]
Revised: [Equation]

Original: The equality in the middle is from Lemma 8, the left inequality is because q (cid:48) 2 = p b τ ≥ p b
Revised: The equality in the middle is from Lemma 8, the left inequality is because q (cid:48) 2 = p b τ ≥ p b

Original: [Equation]
Revised: [Equation]

Original: and the right inequality is from Equation 83.
Revised: and the right inequality is from Equation 83.

Original: Furthermore
Revised: Furthermore, we have the following:

Original: [Equation]
Revised: [Equation]

Original: we have the following:
Revised: We obtain Equation 100 from Equation 99 based on Lemma

Original: [Equation]
Revised: [Equation]

Original: We obtain Equation 100 from Equation 99 based on Lemma 8, and Equation 101 from Equationbased on Equation 83.
Revised: and Equation 101 from Equationbased on Equation 83.

Original: Therefore, we have the following:
Revised: Therefore, we have the following:

Original: [Equation]
Revised: [Equation]

Original:  Thus, there exists ( q τ 1 , q τ 2 ) such that r x ( q τ 1 , q τ2 ) = p b τ , r y ( q τ 1 , q τ
Revised: [Equation] Thus, there exists ( q τ 1 , q τ 2 ) such that r x ( q τ 1 , q τ

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: Pr

Original: [Equation]
Revised: [Equation]

Original: Y ∈B Sτ ) τ based on Lemma 7.
Revised: based on Lemma 7.

Original: Then, we have the following based on the deﬁnition of r x , r y :
Revised: Then, we have the following based on the deﬁnition of r x , r y :

Original: [Equation]
Revised: [Equation]

Original: Finally, we let C b τ = C (cid:48)
Revised: Finally, we let C b τ = C (cid:48)

Original: ( q 1 , q 2 ) ∩ C ( q τ 1 , q τ 2 ) , which meets our goal.
Revised: ( q 1 , q 2 ) ∩ C ( q τ 1 , q τ 2 ) , which meets our goal.

Original: Step II:
Revised: Step II:

Original: Assuming we can ﬁnd { ( q τ 1 , q τ 2 ) , ( q τ − 1, q τ − 1) , · · · , ( q τ − m +1, q τ − m +1) } ( ∀ j ∈ [ τ − m + 1 , τ ] , q 1 ≤ q j 1 ≤ q j 2 ≤ q 2 ) where 1 ≤ m ≤ τ − 1 such that ∀ j ∈ [ τ − m + 1 , τ ] , we have:
Revised: Assuming we can ﬁnd { ( q τ 1 , q [Equation] [Equation] 1 , τ ] , q 1 ≤ q j 1 ≤ q j 2 ≤ q 2 ) where 1 ≤ m ≤ τ − 1 such that ∀ j ∈ [ τ − m + 1 , τ ] , we have:

Original: [Equation]
Revised: [Equation]

Original: as well as the following:
Revised: as well as the following:

Original: [Equation]
Revised: [Equation]

Original: We denote e = τ − m .
Revised: We denote e = τ − m .

Original: We show we can ﬁnd C b e such that we have:
Revised: We show we can ﬁnd C b e such that we have:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: We let q 1 = 0 , q 2 = p S τ and denote
Revised: We let q 1 = 0 , q 2 = p S τ and denote

Original: [Equation]
Revised: [Equation]

Original: We have the following:
Revised: We have the following:

Original: [Equation]
Revised: [Equation]

Original: The Equation 116 from 115 is based on the Equation

Original: [Equation]
Revised: [Equation]

Original:  and the Equation 117 from 116 is based the Equation 51 and 106.
Revised: The Equation 116 from 115 is based on the Equation 108, and the Equation 117 from 116 is based the Equation 51 and 106.

Original: Furthermore, we have the following:
Revised: Furthermore, we have the following:

Original: [Equation]
Revised: [Equation]

Original:  The Equation 123 from 122 is because C ( q 1 , q 2 ) =
Revised: τ [Equation] The Equation 123 from 122 is because C ( q 1 , q 2 ) =

Original: B S τ and the Equation 107.
Revised: B S τ and the Equation 107.

Original: We have p b e ≤ r x ( q 1 , q 2 ) .
Revised: We have p b e ≤ r x ( q 1 , q 2 ) .

Original: Therefore, based on Lemma 7, there exist q (cid:48) 1 , q (cid:48) 2 such that:
Revised: Therefore, based on Lemma 7, there exist q (cid:48) 1 , q (cid:48) 2 such that:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We have:
Revised: We have:

Original: [Equation]
Revised: [Equation]

Original: τ
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Equation 128 from 127 is based on Lemma 9, Equation 129 from 128 is based on Equation 90 and 125, and Equation 130 from 129 is obtained from Equation 91 and the fact that (cid:100) ( (cid:80) e j =1 p b j ) /p
Revised: Equation 128 from 127 is based on Lemma 9, Equation 129 from 128 is based on Equation 90 and 125, and Equation 130 from 129 is obtained from Equation 91 and the fact that (cid:100) ( (cid:80) ej =1 p b j ) /p

Original: b e (cid:101) ≥ τ .
Revised: b e (cid:101) ≥ τ .

Original: Next, we show:
Revised: Next, we show:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: In particular, we consider two scenarios.
Revised: In particular, we consider two scenarios.

Original: Scenario 1) .
Revised: Scenario 1) .

Original: q (cid:48) 1 ≥ min τj = τ − m +

Revised: q (cid:48) 1 ≥ min τj = τ − m +

Original: q j 1 .
Revised: q j 1 .

Original: We denote t = argmin τj = τ − m +

Revised: We denote t = argmin τj = τ − m +

Original: q j 1 .
Revised: q j 1 .

Original: We let C w = C b t ⊆
Revised: We let C w = C b t ⊆

Original: C ( q t 1 , q t 2 ) .
Revised: C ( q t 1 , q t 2 ) .

Original: As q (cid:48) 1 ≥ q t 1 , q 2 ≥ q t 2 and r x ( q (cid:48) 1 , q 2 ) =
Revised: As q (cid:48) 1 ≥ q t 1 , q 2 ≥ q t 2 and r x ( q (cid:48) 1 , q 2 ) =

Original: p b e ≤ Pr ( X ∈ C w ) =
Revised: p b e ≤ Pr ( X ∈ C w ) =

Original: p b t , we have the following based on Lemma 10:
Revised: p b t , we have the following based on Lemma 10:

Original: [Equation]
Revised: [Equation]

Original: Scenario 2) .
Revised: Scenario 2) .

Original: q [Equation] min τj [Equation]{+= τ − m +
+}
Revised: q (cid:48) 1 < min τj {+= τ − m +
+}

Original: q j 1 .
Revised: q j 1 .

Original: We have the following:
Revised: We have the following:

Original: [Equation]
Revised: [Equation]

Original: Furthermore, we have:
Revised: Furthermore, we have:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Moreover, we have r x ( q 1 , q (cid:48) 1 )
Revised: Moreover, we have r x ( q 1 , q (cid:48) 1 )

Original: =
Revised: =

Original: q (cid:48) 1 − q 1 from Lemma 8.
Revised: q (cid:48) 1 − q 1 from Lemma 8.

Original: The above two should be equal.
Revised: The above two should be equal.

Original: Thus, we have q (cid:48) 1 = (cid:80)
Revised: Thus, we have q (cid:48) 1 = (cid:80)

Original: e − 1 j =1 p b j = p S τ − m − 1 since e = τ − m .
Revised: e − 1 j =1 p b j = p S τ − m − 1 since e = τ − m .

Original: we have:
Revised: we have:

Original: [Equation]
Revised: [Equation]

Revised: τ

Original: [Equation]
Revised: [Equation]

Original: We obtain Equation 140 from Equation 139 based on Lemma 8.
Revised: We obtain Equation 140 from Equation 139 based on Lemma 8.

Original: Therefore, we have the following in both scenarios:
Revised: Therefore, we have the following in both scenarios:

Original: [Equation]
Revised: [Equation]

Original: Based on Lemma 7, there exist q e 1 , q e 2 such that r x ( q e 1 , q e 2 ) =
Revised: Based on Lemma 7, there exist q e 1 , q e 2 such that r x ( q e 1 , q e 2 ) =

Original: p b τ , r y ( q e 1 , q e 2 ) =
Revised: p b τ , r y ( q e 1 , q e 2 ) =

Original: Pr ( Y ∈B S τ ) τ .
Revised: Pr ( Y ∈B Sτ ) τ .

Original: Then, we have the following based on the deﬁnition of r x , r y :
Revised: Then, we have the following based on the deﬁnition of r x , r y :

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We let C e = C (cid:48)
Revised: We let C e = C (cid:48)

Original: ( q 1 , q 2 ) ∩ C ( q e 1 , q e 2 ) .
Revised: ( q 1 , q 2 ) ∩ C ( q e 1 , q e 2 ) .

Original: From the deﬁnition of C (cid:48) ( q 1 , q 2 ) , we have ∀ j ∈ [ e + 1 , τ ] , C (cid:48) ( q 1 , q 2 ) ∩ C b j = ∅ .
Revised: From the deﬁnition of C (cid:48) ( q 1 , q 2 ) , we have ∀ j ∈ [ e + 1 , τ ] , C (cid:48) ( q 1 , q 2 ) ∩ C b j = ∅ .

Original: Thus, we have ∀ j ∈ [ e + 1 , τ ] , C b e ∩ C b
Revised: Thus, we have ∀ j ∈ [ e + 1 , τ ] , C b e ∩ C b

Original: j
Revised: j

Original: = ∅ since C b e ⊆ C (cid:48) ( q 1 , q 2 ) .
Revised: = ∅ since C b e ⊆ C (cid:48) ( q 1 , q 2 ) .

Original: Therefore, we reach our goal by Mathematical Induction, i.e., for ∀ j ∈ [1 , τ ] , we have:
Revised: Therefore, we reach our goal by Mathematical Induction, i.e., for ∀ j ∈ [1 , τ ] , we have:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We can also verify that ∪ τj =1 C b j = B S τ .
Revised: We can also verify that ∪ τj =1 C b j = B S τ .

Original: Next, we show our proof based on Mathematical Induction for the other part, i.e., B S k \ B S τ .
Revised: Next, we show our proof based on Mathematical Induction for the other part, i.e., B S k \ B S τ .

Original: Our construction process is similar to the above ﬁrst part but has subtle differences.
Revised: Our construction process is similar to the above ﬁrst part but has subtle differences.

Original: Step I: Let q 1 = (cid:80) τj =1 p
Revised: Step I: Let q 1 = (cid:80) τj =1 p

Original: b
Revised: b

Original: j and q 2 = (cid:80) kj =1 p b j .
Revised: j and q 2 = (cid:80) kj =1 p b j .

Original: We deﬁne:
Revised: We deﬁne:

Original: [Equation]
Revised: [Equation]

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: The Equation 150 is based on the fact that C (cid:48) ( q 1 , q 2 ) =
Revised: The Equation 150 is based on the fact that C (cid:48) ( q 1 , q 2 ) =

Original: C ( q 1 , q 2 ) and Deﬁnition 1, and we obtain
Revised: C ( q 1 , q 2 ) and Deﬁnition 1, and we obtain

Original: Equation 154 from 155 based on Equation 83.
Revised: Equation 154 from 155 based on Equation 83.

Original: We have p b k ≤ r x ( q 1 , q 2 ) .
Revised: We have p b k ≤ r x ( q 1 , q 2 ) .

Original: Therefore, based on
Revised: Therefore, based on

Original: Lemma 7, we know that there exists q (cid:48) 1 = q 2 − p b k , q (cid:48)[-
-]
Revised: Lemma 7, we know that there exists q (cid:48) 1 = q 2 − p b k , q (cid:48)[-
-]

Revised: [Equation]

Original: = q 1 + p b k such that:
Revised: q 1 + p b k such that:

Original: [Equation]
Revised: [Equation]

Original: We consider two scenarios.
Revised: We consider two scenarios.

Original: Scenario 1) .
Revised: Scenario 1) .

Original: In this scenario, we consider r y ( q (cid:48) 1 , q 2 ) >
Revised: In this scenario, we consider r y ( q (cid:48) 1 , q 2 ) >

Original: Pr ( Y ∈B Sτ ) τ .
Revised: Pr ( Y ∈B Sτ ) τ .

Original: We let q k 1 = q (cid:48) 1 , q k 2 = q 2 , i.e., we have C b k = C ( q (cid:48) 1 , q 2 ) ∩ C (cid:48) ( q 1 , q 2 ) .
Revised: We let q k 1 = q (cid:48) 1 , q k 2 = q 2 , i.e., we have C b k = C ( q (cid:48) 1 , q 2 ) ∩ C (cid:48) ( q 1 , q 2 ) .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Therefore, we have the following:
Revised: Therefore, we have the following:

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: τ

Original: [Equation]
Revised: [Equation]

Original: Scenario 2) .
Revised: Scenario 2) .

Original: In this scenario, we consider r y ( q (cid:48) 1 , q 2 ) ≤
Revised: In this scenario, we consider r y ( q (cid:48) 1 , q 2 ) ≤

Original: Pr ( Y ∈B Sτ ) τ .
Revised: Pr ( Y ∈B Sτ ) τ .

Original: We have the following:
Revised: We have the following:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We obtain Equation 167 from 166 via Lemma 9, and we obtain Equation 168 from 167 based on
Revised: We obtain Equation 167 from 166 via Lemma 9, and we obtain Equation 168 from 167 based on

Original: Equation 150 to 156 and the fact r x ( q 1 , q 2 ) /r
Revised: Equation 150 to 156 and the fact r x ( q 1 , q 2 ) /r

Original: x ( q 1 , q (cid:48) 2 ) = (cid:80) kj =
Revised: x ( q 1 , q (cid:48) 2 ) = (cid:80) kj =

Original: τ +1 p bj p bτ + ≥ k − τ .
Revised: τ +1 p bj p bτ + ≥ k − τ .

Original: We have ( r y ( q (cid:48) 1 , q 2 ) −
Revised: We have ( r y ( q (cid:48) 1 , q 2 ) −

Original: Pr

Original:  ( Y ∈B Sτ ) τ ) · ( r y ( q 1 , q (cid:48) 2 ) − Pr ( Y ∈B Sτ )
Revised: Pr ( Y ∈B Sτ ) τ ) · ( r y ( q 1 , q (cid:48) [Equation] Pr ( Y [Equation]

Original: τ
Revised: Sτ

Revised: [Equation]

Original: ) ≤ 0 .
Revised: ) ≤ 0 .

Original: Therefore, from Lemma 7, we know that there exist ( q k 1 , q k 2 ) such that:
Revised: Therefore, from Lemma 7, we know that there exist ( q k 1 , q k 2 ) such that:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We also have the following:
Revised: We also have the following:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Original: Similarly, we let C b k = C (cid:48)
Revised: Similarly, we let C b k = C (cid:48)

Original: ( q 1 , q 2 ) ∩ C ( q k 1 , q k 2 ) .
Revised: ( q 1 , q 2 ) ∩ C ( q k 1 , q k 2 ) .

Original: Based on the conditions of our constructions in the two scenarios, we know that if Pr ( Y ∈ B b k ) >
Revised: Based on the conditions of our constructions in the two scenarios, we know that if Pr ( Y ∈ B b k ) >

Original: Pr ( Y ∈B
Revised: Pr ( Y ∈B

Original: Sτ ) τ , then we have q k 2 = q 2 .
Revised: Sτ ) τ , then we have q k 2 = q 2 .

Original: Step II: We show that if we can ﬁnd ( q k 1 , q k 2 ) , · · · , ( q k − m +1, q k − m +1) where m ∈ [1 , k − τ − 1] and C b j , ∀ j ∈ [ k, k − m + 1] such that:
Revised: Step II: We show that if we can ﬁnd ( q k 1 , q k 2 ) , · · · , ( q k − m +1, q k − m +1) where m ∈ [1 , k − τ − 1] and C b j , ∀ j ∈ [ k, k − m + 1] such that:

Original: [Equation]
Revised: [Equation]

Original: Then, we can ﬁnd ( q k − m, q k − m) such that:
Revised: Then, we can ﬁnd ( q k − m, q k − m) such that:

Original: [Equation]
Revised: [Equation]

Original: For simplicity, we denote e = k − m , we let q 1 = [Equation] τj =1 p
Revised: For simplicity, we denote e = k − m , we let q 1 = (cid:80) τj =1 p

Original: b
Revised: b

Original: j and q 2 = (cid:80) kj =1 p b j , and we deﬁne:
Revised: j and q 2 = (cid:80) kj =1 p b j , and we deﬁne:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: We have p b e ≤ r x ( q 1 , q 2 ) .
Revised: We have p b e ≤ r x ( q 1 , q 2 ) .

Original: Therefore, based on Lemma 7, we know that there exists q (cid:48) 1 , q (cid:48) 2 such that:
Revised: Therefore, based on Lemma 7, we know that there exists q (cid:48) 1 , q [Equation] 2 such that:

Original: [Equation]
Revised: [Equation]

Original: Similarly, we consider two scenarios:
Revised: Similarly, we consider two scenarios:

Original: Scenario 1) .
Revised: Scenario 1) .

Original: In this scenario, we consider that the following holds:
Revised: In this scenario, we consider that the following holds:

Original: [Equation]
Revised: [Equation]

Original: We let q e 1 = q (cid:48) 1 , q e 2 = q 2 , i.e., C b e = C ( q (cid:48) 1 , q 2 ) ∩ C (cid:48) ( q 1 , q 2 ) .
Revised: We let q e 1 = q (cid:48) 1 , q e 2 = q 2 , i.e., C b e = C ( q (cid:48) 1 , q [Equation] ) ∩ C (cid:48) ( q 1 , q 2 ) .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We note that we have q (cid:48) 1 ≤ min kj = e +1 q i 1 in this scenario.
Revised: We note that we have q (cid:48) 1 ≤ min kj = e +1 q i 1 in this scenario.

Original: Otherwise, Equation 186 will not hold based on Lemma 10.
Revised: Otherwise, Equation 186 will not hold based on Lemma 10.

Original: We give a short proof.
Revised: We give a short proof.

Original: Assuming q (cid:48) 1 > min kj = k − m +

Revised: Assuming q (cid:48) 1 > min k j = k − m +

Original: q j 1 .
Revised: q j 1 .

Original: We denote w = argmin kj = k − m +

Revised: We denote w = argmin k j = k − m +

Original: q j 1 .
Revised: q j 1 .

Original: We let C w = C b w ⊆
Revised: We let C w = C b w ⊆

Original: C ( q w 1 , q w 2 ) .
Revised: C ( q w 1 , q w 2 ) .

Original: Note that in this case, we have q w 2 < q 2 because q w 2 = q 2 and q (cid:48) 1 > q w 1 cannot hold at the same time as long as r y ( q (cid:48) 1 , q 2 ) > 0 .
Revised: Note that in this case, we have q w 2 < q 2 because q w 2 = q 2 and q (cid:48) 1 > q w 1 cannot hold at the same time as long as r y ( q (cid:48) 1 , q 2 ) > 0 .

Original: Thus, we have Pr ( Y ∈ C w ) =
Revised: Thus, we have Pr ( Y ∈ C w ) =

Original: Pr ( Y ∈B Sτ ) τ because if
Revised: Pr ( Y ∈B Sτ ) τ because if

Original: Pr ( Y ∈ C w ) >
Revised: Pr ( Y ∈ C w ) >

Original: Pr ( Y ∈B Sτ ) τ , we have q w 2 = q 2 .
Revised: Pr ( Y ∈B Sτ ) τ , we have q w 2 = q 2 .

Original: As we have q (cid:48) 1 > q w 1 , q 2 > q w 2 and r x ( q (cid:48) 1 , q 2 )
Revised: As we have q (cid:48) 1 > q w 1 , q 2 > q w 2 and r x ( q (cid:48) 1 , q 2 )

Original: = p b e ≤ Pr ( X ∈ C w ) = p b w .
Revised: = p b e ≤ Pr ( X ∈ C w ) = p b w .

Original: We have the following based on Lemma 10:
Revised: We have the following based on Lemma 10:

Original: [Equation]
Revised: [Equation]

Original: Since Equation 186 and Equation 189 cannot hold at the same time, the assumption q (cid:48) 1 > min kj = k − m +

Revised: Since Equation 186 and Equation 189 cannot hold at the same time, the assumption q (cid:48) 1 > min kj = k − m +

Original: q j 1 must be wrong.
Revised: q j 1 must be wrong.

Original: Therefore, we have q (cid:48) 1 ≤ min kj =
Revised: Therefore, we have q (cid:48) 1 ≤ min kj =

Original: k − m +

Revised: k − m +

Original: q
Revised: q

Original: j 1 .
Revised: j 1 .

Original: Based on q (cid:48) 1 ≤ min kj = k − m +

Revised: Based on q (cid:48) 1 ≤ min kj = k − m +

Original: q j 1 , we have the following:
Revised: q j 1 , we have the following:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have r x ( q 1 , q (cid:48) 1 )
Revised: Therefore, we have r x ( q 1 , q (cid:48) 1 )

Original: =
Revised: =

Original: q (cid:48) 1 − q 1 from Deﬁnition 1.
Revised: q (cid:48) 1 − q 1 from Deﬁnition 1.

Original: Moreover, we have the following:
Revised: Moreover, we have the following:

Original: [Equation]
Revised: [Equation]

Original: The above two should be equal.
Revised: The above two should be equal.

Original: Therefore, we have q (cid:48) 1 = (cid:80)
Revised: Therefore, we have q (cid:48) 1 = (cid:80)

Original: e − 1 j = τ +1 p b j + q 1 = (cid:80) e − 1 j =1 p
Revised: e − 1 j = τ +1 p b j + q 1 = (cid:80) e − 1 j =1 p

Original: b
Revised: b

Original: j .
Revised: j .

Original: Recall that we let C b e = C (cid:48) ( q 1 , q 2 ) ∩ C ( q (cid:48) 1 , q 2 ) .
Revised: Recall that we let C b e = C (cid:48) ( q 1 , q 2 ) ∩ C ( q (cid:48) 1 , q 2 ) .

Original: Thus, we have:
Revised: Thus, we have:

Original: [Equation]
Revised: [Equation]

Original: Scenario 2) .
Revised: Scenario 2) .

Original: In this scenario, we consider that the following holds:
Revised: In this scenario [Equation] we consider that the following holds:

Original: [Equation]
Revised: [Equation]

Original: Note that we have:
Revised: Note that we have:

Original: [Equation]
Revised: [Equation]

Original:  We obtain Equation 203 from 202 via Lemma 9, and we obtain Equation 204 from 203 based on
Revised: τ [Equation] We obtain Equation 203 from 202 via Lemma 9, and we obtain Equation 204 from 203 based on

Original: Equation 183 and the fact r x ( q 1 , q 2 ) /r
Revised: Equation 183 and the fact r x [Equation]

Original: x ( q 1 , q (cid:48) 2 ) = (cid:80) ej
Revised: [Equation]

Original: = τ +1 p bj p bτ + ≥ k − τ − m .
Revised: ≥ k − τ − m .

Original: We have ( r y ( q (cid:48) 1 , q 2 ) −
Revised: We have ( r y ( q (cid:48) 1 , q 2 ) −

Original: [Equation]
Revised: Pr ( Y ∈B Sτ ) τ ) · ( r y ( q 1 , q (cid:48) 2 ) − Pr ( Y ∈B

Original:  ) ≤ 0 .
Revised: Sτ ) τ ) ≤ 0 .

Original: Based on Lemma 7, we can ﬁnd ( q e 1 , q e 2 ) such that we have:
Revised: Based on Lemma 7, we can ﬁnd ( q e 1 , q e 2 ) such that we have:

Original: [Equation]
Revised: [Equation]

Original: We let C b e =
Revised: We let C b e =

Original: C
Revised: C

Original: (cid:48)
Revised: (cid:48)

Original: [Equation]

Original:  q 1 , q 2 ) ∩ C ( q e 1 , q e 2 ) .
Revised: ( q 1 , q 2 ) ∩ C ( q e 1 , q e 2 ) .

Original: We also have the following:
Revised: We also have the following:

Original: [Equation]
Revised: [Equation]

Original: τ
Revised: τ

Original: [Equation]
Revised: [Equation]

Original: Similar to Step I , we still hold the conclusion that if Pr ( Y ∈ C b e ) > Pr ( Y ∈B
Revised: Similar to Step I , we still hold the conclusion that if Pr ( Y ∈ C b e ) > Pr ( Y ∈B

Original: Sτ ) τ , we have q e 2 = q 2 .
Revised: Sτ ) τ , we have q e 2 = q 2 .

Original: Then, we can apply Mathematical Induction to reach the conclusion.
Revised: Then, we can apply Mathematical Induction to reach the conclusion.

Original: Also, we can verify ∪ kj =
Revised: Also, we can verify ∪ kj =

Original: τ +1 C b
Revised: τ +1 C b

Original: j = B S k \ B S τ .
Revised: j = B S k \ B S τ .

Original: C P ROOF OF P ROPOSITION
Revised: C P ROOF OF P ROPOSITION

Original: The function S AMPLE U NDERNOISE ( f, k, σ, x , n, α ) works as follows:
Revised: The function S AMPLE U NDERNOISE ( f, k, σ, x , n, α ) works as follows:

Original: we ﬁrst draw n random noise from N (0 , σ 2 I ) , i.e., (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n .
Revised: we ﬁrst draw n random noise from N (0 , σ 2 I ) , i.e., (cid:15) 1 , (cid:15) 2 , · · · , (cid:15) n .

Original: Then, we compute the values: ∀ i ∈ [1 , c ] , c i = (cid:80) nj ={+n j =
+}
Revised: Then, we compute the values: ∀ i ∈ [1 , c ] , c i = (cid:80) {+n j =
+}

Original: I ( f ( x + (cid:15) j ) = i ) .
Revised: I ( f ( x + (cid:15) j ) = i ) .

Original: The function B INOMPVALUE ( n c t , n c t + n c t
Revised: The function B INOMPVALUE ( n c t , n c t + n c t

Original: +

Revised: +

Original: , p ) returns the result of p-value of the two-sided hypothesis test for n c t ∼ Bin ( n c t + n c t +1 , p ) .
Revised: , p ) returns the result of p-value of the two-sided hypothesis test for n c t ∼ Bin ( n c t + n c t +1 , p ) .

Original: Proposition 1: With probability at least 1 − α over the randomness in P REDICT , if P REDICT returns a set T (i.e., does not ABSTAIN), then we have g k ( x ) =
Revised: Proposition 1: With probability at least 1 − α over the randomness in P REDICT , if P REDICT returns a set T (i.e., does not ABSTAIN), then we have g k ( x ) =

Original: T .
Revised: T .

Original: Proof.
Revised: Proof.

Original: We aim to compute the probability that P REDICT returns a set which not equals to g k ( x ) , which happens if and only if g k ( x ) (cid:54) =
Revised: We aim to compute the probability that P REDICT returns a set which not equals to g k ( x ) , which happens if and only if g k ( x ) (cid:54) =

Original: T and P REDICT doesn’t abstain.
Revised: T and P REDICT doesn’t abstain.

Original: Speciﬁcally, we have:
Revised: Speciﬁcally, we have:

Original: [Equation] set 
Revised: Pr ( P REDICT returns a set (cid:54) =

Original: [Equation] [Equation] [Equation]
Revised: g k ( x )) (213) = Pr ( g k ( x ) (cid:54) =

Revised: T, P REDICT doesn’t abstain

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 in Hung et al.
Revised: Theorem 1 in Hung et al.

Original: (2019) shows the above conditional probability is as follows:
Revised: (2019) shows the above conditional probability is as follows:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we reach the conclusion.
Revised: Therefore, we reach the conclusion.

Original: D P ROOF OF P ROPOSITION
Revised: D P ROOF OF P ROPOSITION

Original: Proposition 2: With probability at least 1 − α over the randomness in C ERTIFY , if C ERTIFY returns a radius R l (i.e., does not ABSTAIN), then we have l ∈ g k
Revised: Proposition 2: With probability at least 1 − α over the randomness in C ERTIFY , if C ERTIFY returns a radius R l (i.e., does not ABSTAIN), then we have l ∈ g k

Original: ( x + δ ) , ∀ (cid:107) [Equation]
Revised: ( x + δ ) , ∀ (cid:107) δ (cid:107) 2 < R l .

Original: Proof.
Revised: Proof.

Original: From the deﬁnition of B INO CP and S IMU EM, we know the probability that the following inequalities simultaneously hold is at least 1 − α over the sampling of counts:
Revised: From the deﬁnition of B INO CP and S IMU EM, we know the probability that the following inequalities simultaneously hold is at least 1 − α over the sampling of counts:

Original: [Equation]
Revised: [Equation]

Original: Then, with the returned bounds, we can invoke Theorem 1 to obtain the robustness guarantee if the calculated radius is larger than 0.
Revised: Then, with the returned bounds, we can invoke Theorem 1 to obtain the robustness guarantee if the calculated radius is larger than 0.

Original: Note that otherwise C ERTIFY abstains.
Revised: Note that otherwise C ERTIFY abstains.

Original: E C ERTIFIED TOP - k ACCURACY
Revised: E C ERTIFIED TOP - k ACCURACY

Original: We show how to derive a lower bound of the certiﬁed top- k accuracy based on the approximate certiﬁed top- k accuracy.
Revised: We show how to derive a lower bound of the certiﬁed top- k accuracy based on the approximate certiﬁed top- k accuracy.

Original: The process is similar to that Cohen et al.
Revised: The process is similar to that Cohen et al.

Original: (2019) used to derive a lower bound of the certiﬁed top- 1 accuracy based on the approximate certiﬁed top- 1 accuracy.
Revised: (2019) used to derive a lower bound of the certiﬁed top- 1 accuracy based on the approximate certiﬁed top- 1 accuracy.

Original: Speciﬁcally, we have the following lemma from Cohen et al.
Revised: Speciﬁcally, we have the following lemma from Cohen et al.

Original: Lemma 11.
Revised: Lemma 11.

Original: Let z i be a binary variable and Y i be a Bernoulli random variable.
Revised: Let z i be a binary variable and Y i be a Bernoulli random variable.

Original: Suppose if z i = 1 , then Pr ( Y i = 1) ≤ α .
Revised: Suppose if z i = 1 , then Pr ( Y i = 1) ≤ α .

Original: Then, for any ρ > 0 , with probability at least 1 − ρ , we have the following:
Revised: Then, for any ρ > 0 , with probability at least 1 − ρ , we have the following:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Please refer to Cohen et al.
Revised: Please refer to Cohen et al.

Original: Assuming we have a test dataset D test
Revised: Assuming we have a test dataset D test

Original: = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , · · · , ( x m , y m ) } as well as a radius r .
Revised: = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , · · · , ( x m , y m ) } as well as a radius r .

Original: We deﬁne the following indicate value:
Revised: We deﬁne the following indicate value:

Original: [Equation]
Revised: [Equation]

Original: Then, the certiﬁed top- k accuracy of the smoothed classiﬁer g at radius r can be computed asm (cid:80) mi =1 a i .
Revised: Then, the certiﬁed top- k accuracy of the smoothed classiﬁer g at radius r can be computed asm (cid:80) mi =1 a i .

Original: For each sample x i , we run the C ERTIFY function with 1 − α conﬁdence level and we use a random variable Y i to denote that the function C ERTIFY returns a radius bigger than r .
Revised: For each sample x i , we run the C ERTIFY function with 1 − α conﬁdence level and we use a random variable Y i to denote that the function C ERTIFY returns a radius bigger than r .

Original: From Proposition 2 , we know:
Revised: From Proposition 2 , we know:

Original: [Equation]
Revised: [Equation]

Original: The approximate certiﬁed top- k accuracy of the smoothed classiﬁer at radius r is 1 m (cid:80) mi [Equation] 1 Y i .
Revised: The approximate certiﬁed top- k accuracy of the smoothed classiﬁer at radius r is 1 m (cid:80) mi =1 Y i .

Original: Then, we can use Lemma 11 to obtain a lower bound of 1 m (cid:80) mi =1 a i .
Revised: Then, we can use Lemma 11 to obtain a lower bound of 1 m (cid:80) mi =1 a i .

Original: Speciﬁcally, for any ρ > 0 , with probability at least 1 − ρ over the randomness of C ERTIFY , we have:
Revised: Speciﬁcally, for any ρ > 0 , with probability at least 1 − ρ over the randomness of C ERTIFY , we have:

Original: [Equation]
Revised: [Equation]

Original: We can see that the difference between the certiﬁed top- k accuracy and the approximate certiﬁed top- k accuracy is negligible when α is small.
Revised: We can see that the difference between the certiﬁed top- k accuracy and the approximate certiﬁed top- k accuracy is negligible when α is small.