Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations

Top-k predictions are used in many real-world applications such as machine learning as a service , recommender systems , and web searches . ℓ0-norm adversarial perturbation characterizes an attack that arbitrarily modifies some features of an input such that a classifier makes an incorrect prediction for the perturbed input . ℓ0-norm adversarial perturbation is easy to interpret and can be implemented in the physical world . Therefore , certifying robustness of top-k predictions against ℓ0-norm adversarial perturbation is important . However , existing studies either focused on certifying ℓ0-norm robustness of top-1 predictions or ℓ2-norm robustness of top-k predictions . In this work , we aim to bridge the gap . Our approach is based on randomized smoothing , which builds a provably robust classifier from an arbitrary classifier via randomizing an input . Our major theoretical contribution is an almost tight ℓ0-norm certified robustness guarantee for top-k predictions . We empirically evaluate our method on CIFAR10 and ImageNet . For instance , our method can build a classifier that achieves a certified top-3 accuracy of 69.2 % on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image . 1 INTRODUCTION . Adversarial example is a well-known severe security vulnerability of classifiers . Specifically , given a classifier f and a testing input x , an attacker can carefully craft a human-imperceptible perturbation δ such that f ( x ) ̸= f ( x+ δ ) . The perturbation δ is called adversarial perturbation , while the input x+ δ is called an adversarial example . Many empirical defenses ( Goodfellow et al. , 2015 ; Na et al. , 2018 ; Metzen et al. , 2017 ; Svoboda et al. , 2019 ; Buckman et al. , 2018 ; Ma et al. , 2018 ; Guo et al. , 2018 ; Dhillon et al. , 2018 ; Xie et al. , 2018 ; Song et al. , 2018 ) have been developed to defend against adversarial examples in the past several years . However , these empirical defenses were often soon broken by strong adaptive adversaries ( Carlini & Wagner , 2017 ; Athalye et al. , 2018 ; Uesato et al. , 2018 ; Athalye & Carlini , 2018 ) . To end this cat-and-mouse game , many certified defenses ( Scheibler et al. , 2015 ; Carlini et al. , 2017 ; Ehlers , 2017 ; Katz et al. , 2017 ; Cheng et al. , 2017 ; Lomuscio & Maganti , 2017 ; Fischetti & Jo , 2018 ; Bunel et al. , 2018 ; Wong & Kolter , 2018 ; Wong et al. , 2018 ; Raghunathan et al. , 2018a ; b ; Dvijotham et al. , 2018a ; b ; Gehr et al. , 2018 ; Mirman et al. , 2018 ; Singh et al. , 2018 ; Weng et al. , 2018 ; Zhang et al. , 2018 ; Gowal et al. , 2018 ; Wang et al. , 2018 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ; Lee et al. , 2019 ; Salman et al. , 2019 ; Wang et al. , 2020 ; Jia et al. , 2020 ; Zhai et al. , 2020 ) have been proposed . In particular , a classifier f is said to be certifiably robust for an input x if it provably predicts the same top-1 label ( i.e. , f ( x ) = f ( x+ δ ) ) when the adversarial perturbation δ is bounded , e.g. , the ℓp-norm of δ is smaller than a threshold . The threshold is also called certified radius . In this work , we focus on ℓ0-norm adversarial perturbation , which arbitrarily manipulates some features of a testing input and can be implemented in the physical world . However , most existing certified defenses focus on top-1 predictions . In many applications , top-k predictions that return the k most likely labels are more relevant . For instance , when a classifier is deployed as a cloud service ( also called machine learning as a service ) ( Google Cloud Vision ; Microsoft ; Amazon AWS ; Clarifai ) , top-k labels for a testing input are often returned to a customer for more informed decisions ; in recommender systems and web searches , top-k items/webpages are recommended to a user . Despite the importance and relevance of top-k predictions , their certified robustness against adversarial perturbations is largely unexplored . One exception is the recent work from Jia et al . ( 2020 ) , which derived a tight ℓ2-norm certified robustness for top-k predictions . Such ℓ2-norm certified robustness can be transformed to ℓ0-norm certified robustness via employing the inequality between ℓ0-norm and ℓ2-norm . However , the ℓ0-norm certified robustness derived from such transformations is suboptimal . Our work : We aim to develop ℓ0-norm certified robustness of top-k predictions . Our approach is based on randomized smoothing ( Cao & Gong , 2017 ; Liu et al. , 2018 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ; Lee et al. , 2019 ; Jia et al. , 2020 ; Levine & Feizi , 2019 ) , which can build a certifiably robust classifier from any base classifier via randomizing the input . We adopt randomized smoothing because it is applicable to any classifier and scalable to large neural networks . In particular , we use a randomized smoothing method called randomized ablation ( Levine & Feizi , 2019 ) , which achieves state-of-the-art ℓ0-norm certified robustness for top-1 predictions . Unlike other randomized smoothing methods ( Cao & Gong , 2017 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ) that randomize an input via adding additive noise ( e.g. , Gaussian , Laplacian , or discrete noise ) to it , randomized ablation randomizes an input via subsampling its features . Specifically , given an arbitrary classifier ( called base classifier ) and a testing input x , randomized ablation creates an ablated input via retaining some randomly selected features in x and setting the remaining features to a special value , e.g. , median of the feature value , mean of the feature value , or a special symbol . When the testing input is an image , the features are the image ’ s pixels . Then , we feed the ablated input to the base classifier . Since the ablated input is random , the output of the base classifier is also random . Specifically , we denote by pj the probability that the base classifier outputs a label j for the random ablated input . The original randomized ablation method builds a smoothed classifier that outputs the label with the largest label probability pj for a testing input x . In our work , the smoothed classifier returns the k labels with the largest label probabilities for x . Our major theoretical contribution is an almost tight ℓ0-norm certified robustness guarantee of top-k predictions for the smoothed classifier constructed by randomized ablation . Specifically , we first derive an ℓ0-norm certified robustness guarantee of top-k predictions for the smoothed classifier . Our results show that a label l is provably among the top-k labels predicted by the smoothed classifier for a testing input x when the attacker arbitrarily perturbs at most rl features of x , where rl is the ℓ0-norm certified radius . Moreover , we prove that our certified radius is tight when k = 1 and is almost tight when k > 1 . In particular , if no assumptions on the base classifier are made , it is impossible to derive a certified radius that is larger than rl + I ( k ̸= 1 ) . In other words , when an attacker manipulates at least rl + 1 + I ( k ̸= 1 ) features of a testing input , there exists a base classifier from which the smoothed classifier ’ s top-k predicted labels do not include l or there exist ties . Our work has several technical differences with Levine & Feizi ( 2019 ) . First , we derive the ℓ0-norm certified radius of top-k predictions for randomized ablation , while Levine & Feizi ( 2019 ) only derived the certified radius of top-1 predictions . Second , our certified radius is the same as or larger than that in Levine & Feizi ( 2019 ) for top-1 predictions , because we leverage the discrete property of the label probabilities to derive our certified radius . Third , we prove the ( almost ) tightness of the certified radius , while Levine & Feizi ( 2019 ) didn ’ t . Our work also has several technical differences with Jia et al . ( 2020 ) , which derived a tight ℓ2-norm certified radius of top-k predictions for randomized smoothing with Gaussian additive noise . Since they add additive Gaussian noise to a testing input , the space of randomized inputs is continuous . However , our space of ablated inputs is discrete , as we randomize a testing input via subsampling its features . As a result , Jia et al . and our work use substantially different techniques to derive the ℓ2/ℓ0-norm certified radiuses and prove their ( almost ) tightness . In particular , when deriving the ℓ2/ℓ0-norm certified radiuses , our work needs to construct different regions in the discrete space of ablated inputs such that the Neyman-Pearson Lemma ( Neyman & Pearson , 1933 ) can be applied . When proving the ( almost ) tightness , we use a completely different approach from Jia et al .. First , Jia et al . relies on the Intermediate Value Theorem , which is not applicable to our discrete data . Second , since Gaussian noise is not uniform , Jia et al . need to prove the results via Mathematical Induction . However , Mathematical Induction is unnecessary in our case because the ablated inputs that can be derived from an input are uniformly distributed in the space of ablated inputs . We evaluate our method on CIFAR10 and ImageNet . Our results show that our method substantially outperforms state-of-the-art for top-k predictions . For instance , our method achieves a certified top-3 accuracy of 69.2 % on ImageNet when an attacker arbitrarily perturbs 5 pixels of a testing image . Under the same setting , Jia et al . ( 2020 ) achieves a certified top-3 accuracy of only 9.0 % , when transforming their ℓ2-norm certified robustness to ℓ0-norm certified robustness . Our contributions can be summarized as follows : • We derive an ℓ0-norm certified radius of top-k predictions for randomized ablation . • We prove that our certified radius is tight when k = 1 and almost tight when k > 1 . • We empirically evaluate our method on CIFAR10 and ImageNet . 2 THEORETICAL RESULTS . In this section , we show our core theoretical contributions . 2.1 BUILDING A SMOOTHED CLASSIFIER VIA RANDOMIZED ABLATION . Suppose we have a base classifier f , which classifies a testing input x to one of c classes { 1 , 2 , · · · , c } deterministically . For simplicity , we assume x is an image with d pixels . Given an input x , randomized ablation ( Levine & Feizi , 2019 ) creates an ablated input as follows : we first randomly subsample e pixels from x without replacement and keep their values . Then , we set the remaining pixel values in the ablated input to a special value , e.g. , median of the pixel value , mean of the pixel value , or a special symbol . When the image is a color image , we set the values of the three channels of each pixel separately . Note that an ablated input has the same size with x . We use h ( x , e ) to denote the randomly ablated input for simplicity . Given h ( x , e ) as input , the output of the base classifier f is also random . We use pj to denote the probability that the base classifier f predicts class j when taking h ( x , e ) as input , i.e. , pj = Pr ( f ( h ( x , e ) ) = j ) . Note that pj is an integer multiple of 1 ( de ) , which we will leverage to derive a tighter certified robustness guarantee . We build a smoothed classifier g that outputs the k labels with the largest label probabilities pj ’ s for x . Moreover , we denote by gk ( x ) the set of k labels predicted for x .

Summary: The paper provides both theoretical progress and experimental results on certified robustness of top-$k$ predictions. Specifically, on the theory side, the paper shows that the randomized ablation (Levin & Feizi) has $\ell_0$-norm certified radius of top-$k$ predictions. Also, the paper proves that the certified radius is tight for k = 1 and almost tight for k > 1. (The certified radius cannot be larger than “the derived radius plus 1”.) On the experiment side, the paper compares the proposed method to existing competitors and explored the impact of the related parameters.