Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing

1 INTRODUCTION . Classifiers are vulnerable to adversarial perturbations ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Carlini & Wagner , 2017b ; Jia & Gong , 2018 ) . Specifically , given an example x and a classifier f , an attacker can carefully craft a perturbation δ such that f makes predictions for x + δ as the attacker desires . Various empirical defenses ( e.g. , Goodfellow et al . ( 2015 ) ; Svoboda et al . ( 2019 ) ; Buckman et al . ( 2018 ) ; Ma et al . ( 2018 ) ; Guo et al . ( 2018 ) ; Dhillon et al . ( 2018 ) ; Xie et al . ( 2018 ) ; Song et al . ( 2018 ) ) have been proposed to defend against adversarial perturbations . However , these empirical defenses were often soon broken by adaptive adversaries ( Carlini & Wagner , 2017a ; Athalye et al. , 2018 ) . As a response , certified robustness ( e.g. , Wong & Kolter ( 2018 ) ; Raghunathan et al . ( 2018a ) ; Liu et al . ( 2018 ) ; Lecuyer et al . ( 2019 ) ; Cohen et al . ( 2019 ) ) against adversarial perturbations has been developed . In particular , a robust classifier verifiably predicts the same top-1 label for data points in a certain region around any example x . In many applications such as recommender systems , web search , and image classification cloud service ( Clarifai ; Google Cloud Vision ) , top-k predictions are more relevant . In particular , given an example , a set of k most likely labels are predicted for the example . However , existing certified robustness results are limited to top-1 predictions , leaving top-k robustness unexplored . To bridge this gap , we study certified robustness for top-k predictions in this work . Our certified top-k robustness leverages randomized smoothing ( Cao & Gong , 2017 ; Cohen et al. , 2019 ) , which turns any base classifier f to be a robust classifier via adding random noise to an example . For instance , Cao & Gong ( 2017 ) is the first to propose randomized smoothing with uniform noise as an empirical defense . We consider random Gaussian noise because of its certified robustness guarantee ( Cohen et al. , 2019 ) . Specifically , we denote by pi the probability that the base classifier f predicts label i for the Gaussian random variable N ( x , σ2I ) . The smoothed classifier gk ( x ) predicts the k labels with the largest probabilities pi ’ s for the example x . We adopt randomized smoothing because it is scalable to large-scale neural networks and applicable to any base classifier . Our major theoretical result is a tight certified robustness bound for top-k predictions when using randomized smoothing with Gaussian noise . Specifically , given an example x , a label l is verifiably among the top-k labels predicted by the smoothed classifier gk ( x + δ ) when the ` 2-norm of the adversarial perturbation δ is less than a threshold ( called certified radius ) . The certified radius for top-1 predictions derived by Cohen et al . ( 2019 ) is a special case of our certified radius when k = 1 . As our results and proofs show , generalizing certified robustness from top-1 to top-k predictions faces significant new challenges and requires new techniques . Our certified radius is the unique solution to an equation , which depends on σ , pl , and the k largest probabilities pi ’ s ( excluding pl ) . However , computing our certified radius in practice faces two challenges : 1 ) it is hard to exactly compute the probability pl and the k largest probabilities pi ’ s , and 2 ) the equation about the certified radius does not have an analytical solution . To address the first challenge , we estimate simultaneous confidence intervals of the label probabilities via the Clopper-Pearson method and Bonferroni correction in statistics . To address the second challenge , we propose an algorithm to solve the equation to obtain a lower bound of the certified radius , where the lower bound can be tuned to be arbitrarily close to the true certified radius . We evaluate our method on CIFAR10 ( Krizhevsky & Hinton , 2009 ) and ImageNet ( Deng et al. , 2009 ) datasets . For instance , on ImageNet , our method respectively achieves approximate certified top-1 , top-3 , and top-5 accuracies as 46.6 % , 57.8 % , and 62.8 % when the ` 2-norms of the adversarial perturbations are less than 0.5 ( 127/255 ) and σ = 0.5 . Our contributions are summarized as follows : • Theory . We derive the first certified radius for top-k predictions . Moreover , we prove our certified radius is tight for randomized smoothing with Gaussian noise . • Algorithm . We develop algorithms to estimate our certified radius in practice . • Evaluation . We empirically evaluate our method on CIFAR10 and ImageNet . 2 CERTIFIED RADIUS FOR TOP-k PREDICTIONS Suppose we have a base classifier f , which maps an example x ∈ Rd to one of c candidate labels { 1 , 2 , · · · , c } . f can be any classifier . Randomized smoothing ( Cohen et al. , 2019 ) adds an isotropic Gaussian noise N ( 0 , σ2I ) to an example x . We denote pi as the probability that the base classifier f predicts label i when adding a random isotropic Gaussian noise to the example x , i.e. , pi = Pr ( f ( x + ) = i ) , where ∼ N ( 0 , σ2I ) . The smoothed classifier gk ( x ) returns the set of k labels with the largest probabilities pi ’ s when taking an example x as input . Our goal is to derive a certified radius Rl such that we have l ∈ gk ( x + δ ) for all ||δ||2 < Rl . Our main theoretical results are summarized in the following two theorems . Theorem 1 ( Certified Radius for Top-k Predictions ) . Suppose we are given an example x , an arbitrary base classifier f , ∼ N ( 0 , σ2I ) , a smoothed classifier g , an arbitrary label l ∈ { 1 , 2 , · · · , c } , and pl , p1 , · · · , pl−1 , pl+1 , · · · , pc ∈ [ 0 , 1 ] that satisfy the following conditions : Pr ( f ( x + ) = l ) ≥ pl and Pr ( f ( x + ) = i ) ≤ pi , ∀i 6= l , ( 1 ) where p and p indicate lower and upper bounds of p , respectively . Let pbk ≥ pbk−1 ≥ · · · ≥ pb1 be the k largest ones among { p1 , · · · , pl−1 , pl+1 , · · · , pc } , where ties are broken uniformly at random . Moreover , we denote by St = { b1 , b2 , · · · , bt } the set of t labels with the smallest probability upper bounds in the k largest ones and by pSt = ∑t j=1 pbj the sum of the t probability upper bounds , where t = 1 , 2 , · · · , k. Then , we have : l ∈ gk ( x + δ ) , ∀||δ||2 < Rl , ( 2 ) where Rl is the unique solution to the following equation : Φ ( Φ−1 ( pl ) − Rl σ ) ) − k min t=1 Φ ( Φ−1 ( pSt ) + Rl σ ) ) t = 0 , ( 3 ) where Φ and Φ−1 are the cumulative distribution function and its inverse of the standard Gaussian distribution , respectively . Proof . See Appendix A. Algorithm 1 : PREDICT Input : f , k , σ , x , n , and α . Output : ABSTAIN or predicted top-k labels . 1 T = ∅ 2 counts = SAMPLEUNDERNOISE ( f , σ , x , n ) 3 c1 , c2 , · · · , ck+1 = top- { k + 1 } indices in counts ( ties are broken uniformly at random ) 4 nc1 , nc2 , · · · , nck+1 = counts [ c1 ] , counts [ c2 ] , · · · , counts [ ck+1 ] 5 for t← 1 to k do 6 if BINOMPVALUE ( nct , nct + nct+1 , 0.5 ) ≤ α then 7 T = T ∪ ct 8 else 9 return ABSTAIN 10 return T Theorem 2 ( Tightness of the Certified Radius ) . Assuming we have pl + ∑k j=1 pbj ≤ 1 and pl +∑ i=1 , ··· , l−1 , l+1 , ··· , c pi ≥ 1 . Then , for any perturbation ||δ||2 > Rl , there exists a base classifier f∗ consistent with ( 1 ) but we have l /∈ gk ( x + δ ) . Proof . We show a proof sketch here . Our detailed proof is in Appendix B . In our proof , we first show that , via mathematical induction and the intermediate value theorem , we can construct k + 1 disjoint regions Ci , i ∈ { l } ∪ { b1 , b2 , · · · , bk } that satisfy Pr ( x+ ∈ Cl ) = pl , Pr ( x+ ∈ Ci ) = pi , and Pr ( x + δ + ∈ Ci ) is no smaller than some critical value for i ∈ { b1 , b2 , · · · , bk } . Moreover , we divide the remaining region Rd \ ( ∪i=l , b1 , b2 , ··· , bkCi ) into c− k− 1 regions , which we denote as Cbk+1 , Cbk+2 , · · · , Cbc−1 and satisfy Pr ( x + ∈ Ci ) ≤ pi for i = bk+1 , bk+2 , · · · , bc−1 . Then , we construct a base classifier f∗ that predicts label i for an example if and only if the example is in the region Ci , where i ∈ { 1 , 2 , · · · , c } . As pl + ∑k j=1 pbj ≤ 1 and pl + ∑ i=1 , ··· , l−1 , l+1 , ··· , c pi ≥ 1 , f∗ is well-defined . Moreover , f∗ satisfies the conditions in ( 1 ) . Finally , we show that if ||δ||2 > Rl , then we have Pr ( f∗ ( x + δ + ) = l ) < minkj=1 Pr ( f ∗ ( x + δ + ) = bj ) , i.e. , l /∈ gk ( x + δ ) . We have several observations about our theorems . • Our certified radius is applicable to any base classifier f . • According to Equation 3 , our certified radius Rl depends on σ , pl , and the k largest prob- ability upper bounds { pbk , pbk−1 , · · · , pb1 } excluding pl . When the lower bound pl and the upper bounds { pbk , pbk−1 , · · · , pb1 } are tighter , the certified radius Rl is larger . When Rl < 0 , the label l is not among the top-k labels predicted by the smoothed classifier even if no perturbation is added , i.e. , l /∈ gk ( x ) . • When using randomized smoothing with Gaussian noise and no further assumptions are made on the base classifier , it is impossible to certify a ` 2 radius for top-k predictions that is larger than Rl . • When k = 1 , we have Rl = σ2 ( Φ −1 ( pl ) − Φ−1 ( pb1 ) ) , where pb1 is an upper bound of the largest label probability excluding pl . The certified radius derived by Cohen et al . ( 2019 ) for top-1 predictions ( i.e. , their Equation 3 ) is a special case of our certified radius with k = 1 , l = A , and b1 = B .

Summary: The paper proposes an extension to the work of Cohen et al. where a certified radius is deduced using a randomized smoothing approach. In particular, the authors show the radius at which a smoothed classifier g at under Gaussian perturbations is certified for the top k predictions. That is to say that the prediction will remain within the top k predictions of g. Setting k=1, one recovers Cohen et al. results. The authors show that the derived radius is tight.