Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations

Top-k predictions are used in many real-world applications such as machine learning as a service , recommender systems , and web searches . ℓ0-norm adversarial perturbation characterizes an attack that arbitrarily modifies some features of an input such that a classifier makes an incorrect prediction for the perturbed input . ℓ0-norm adversarial perturbation is easy to interpret and can be implemented in the physical world . Therefore , certifying robustness of top-k predictions against ℓ0-norm adversarial perturbation is important . However , existing studies either focused on certifying ℓ0-norm robustness of top-1 predictions or ℓ2-norm robustness of top-k predictions . In this work , we aim to bridge the gap . Our approach is based on randomized smoothing , which builds a provably robust classifier from an arbitrary classifier via randomizing an input . Our major theoretical contribution is an almost tight ℓ0-norm certified robustness guarantee for top-k predictions . We empirically evaluate our method on CIFAR10 and ImageNet . For instance , our method can build a classifier that achieves a certified top-3 accuracy of 69.2 % on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image . 1 INTRODUCTION . Adversarial example is a well-known severe security vulnerability of classifiers . Specifically , given a classifier f and a testing input x , an attacker can carefully craft a human-imperceptible perturbation δ such that f ( x ) ̸= f ( x+ δ ) . The perturbation δ is called adversarial perturbation , while the input x+ δ is called an adversarial example . Many empirical defenses ( Goodfellow et al. , 2015 ; Na et al. , 2018 ; Metzen et al. , 2017 ; Svoboda et al. , 2019 ; Buckman et al. , 2018 ; Ma et al. , 2018 ; Guo et al. , 2018 ; Dhillon et al. , 2018 ; Xie et al. , 2018 ; Song et al. , 2018 ) have been developed to defend against adversarial examples in the past several years . However , these empirical defenses were often soon broken by strong adaptive adversaries ( Carlini & Wagner , 2017 ; Athalye et al. , 2018 ; Uesato et al. , 2018 ; Athalye & Carlini , 2018 ) . To end this cat-and-mouse game , many certified defenses ( Scheibler et al. , 2015 ; Carlini et al. , 2017 ; Ehlers , 2017 ; Katz et al. , 2017 ; Cheng et al. , 2017 ; Lomuscio & Maganti , 2017 ; Fischetti & Jo , 2018 ; Bunel et al. , 2018 ; Wong & Kolter , 2018 ; Wong et al. , 2018 ; Raghunathan et al. , 2018a ; b ; Dvijotham et al. , 2018a ; b ; Gehr et al. , 2018 ; Mirman et al. , 2018 ; Singh et al. , 2018 ; Weng et al. , 2018 ; Zhang et al. , 2018 ; Gowal et al. , 2018 ; Wang et al. , 2018 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ; Lee et al. , 2019 ; Salman et al. , 2019 ; Wang et al. , 2020 ; Jia et al. , 2020 ; Zhai et al. , 2020 ) have been proposed . In particular , a classifier f is said to be certifiably robust for an input x if it provably predicts the same top-1 label ( i.e. , f ( x ) = f ( x+ δ ) ) when the adversarial perturbation δ is bounded , e.g. , the ℓp-norm of δ is smaller than a threshold . The threshold is also called certified radius . In this work , we focus on ℓ0-norm adversarial perturbation , which arbitrarily manipulates some features of a testing input and can be implemented in the physical world . However , most existing certified defenses focus on top-1 predictions . In many applications , top-k predictions that return the k most likely labels are more relevant . For instance , when a classifier is deployed as a cloud service ( also called machine learning as a service ) ( Google Cloud Vision ; Microsoft ; Amazon AWS ; Clarifai ) , top-k labels for a testing input are often returned to a customer for more informed decisions ; in recommender systems and web searches , top-k items/webpages are recommended to a user . Despite the importance and relevance of top-k predictions , their certified robustness against adversarial perturbations is largely unexplored . One exception is the recent work from Jia et al . ( 2020 ) , which derived a tight ℓ2-norm certified robustness for top-k predictions . Such ℓ2-norm certified robustness can be transformed to ℓ0-norm certified robustness via employing the inequality between ℓ0-norm and ℓ2-norm . However , the ℓ0-norm certified robustness derived from such transformations is suboptimal . Our work : We aim to develop ℓ0-norm certified robustness of top-k predictions . Our approach is based on randomized smoothing ( Cao & Gong , 2017 ; Liu et al. , 2018 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ; Lee et al. , 2019 ; Jia et al. , 2020 ; Levine & Feizi , 2019 ) , which can build a certifiably robust classifier from any base classifier via randomizing the input . We adopt randomized smoothing because it is applicable to any classifier and scalable to large neural networks . In particular , we use a randomized smoothing method called randomized ablation ( Levine & Feizi , 2019 ) , which achieves state-of-the-art ℓ0-norm certified robustness for top-1 predictions . Unlike other randomized smoothing methods ( Cao & Gong , 2017 ; Lecuyer et al. , 2019 ; Li et al. , 2019 ; Cohen et al. , 2019 ) that randomize an input via adding additive noise ( e.g. , Gaussian , Laplacian , or discrete noise ) to it , randomized ablation randomizes an input via subsampling its features . Specifically , given an arbitrary classifier ( called base classifier ) and a testing input x , randomized ablation creates an ablated input via retaining some randomly selected features in x and setting the remaining features to a special value , e.g. , median of the feature value , mean of the feature value , or a special symbol . When the testing input is an image , the features are the image ’ s pixels . Then , we feed the ablated input to the base classifier . Since the ablated input is random , the output of the base classifier is also random . Specifically , we denote by pj the probability that the base classifier outputs a label j for the random ablated input . The original randomized ablation method builds a smoothed classifier that outputs the label with the largest label probability pj for a testing input x . In our work , the smoothed classifier returns the k labels with the largest label probabilities for x . Our major theoretical contribution is an almost tight ℓ0-norm certified robustness guarantee of top-k predictions for the smoothed classifier constructed by randomized ablation . Specifically , we first derive an ℓ0-norm certified robustness guarantee of top-k predictions for the smoothed classifier . Our results show that a label l is provably among the top-k labels predicted by the smoothed classifier for a testing input x when the attacker arbitrarily perturbs at most rl features of x , where rl is the ℓ0-norm certified radius . Moreover , we prove that our certified radius is tight when k = 1 and is almost tight when k > 1 . In particular , if no assumptions on the base classifier are made , it is impossible to derive a certified radius that is larger than rl + I ( k ̸= 1 ) . In other words , when an attacker manipulates at least rl + 1 + I ( k ̸= 1 ) features of a testing input , there exists a base classifier from which the smoothed classifier ’ s top-k predicted labels do not include l or there exist ties . Our work has several technical differences with Levine & Feizi ( 2019 ) . First , we derive the ℓ0-norm certified radius of top-k predictions for randomized ablation , while Levine & Feizi ( 2019 ) only derived the certified radius of top-1 predictions . Second , our certified radius is the same as or larger than that in Levine & Feizi ( 2019 ) for top-1 predictions , because we leverage the discrete property of the label probabilities to derive our certified radius . Third , we prove the ( almost ) tightness of the certified radius , while Levine & Feizi ( 2019 ) didn ’ t . Our work also has several technical differences with Jia et al . ( 2020 ) , which derived a tight ℓ2-norm certified radius of top-k predictions for randomized smoothing with Gaussian additive noise . Since they add additive Gaussian noise to a testing input , the space of randomized inputs is continuous . However , our space of ablated inputs is discrete , as we randomize a testing input via subsampling its features . As a result , Jia et al . and our work use substantially different techniques to derive the ℓ2/ℓ0-norm certified radiuses and prove their ( almost ) tightness . In particular , when deriving the ℓ2/ℓ0-norm certified radiuses , our work needs to construct different regions in the discrete space of ablated inputs such that the Neyman-Pearson Lemma ( Neyman & Pearson , 1933 ) can be applied . When proving the ( almost ) tightness , we use a completely different approach from Jia et al .. First , Jia et al . relies on the Intermediate Value Theorem , which is not applicable to our discrete data . Second , since Gaussian noise is not uniform , Jia et al . need to prove the results via Mathematical Induction . However , Mathematical Induction is unnecessary in our case because the ablated inputs that can be derived from an input are uniformly distributed in the space of ablated inputs . We evaluate our method on CIFAR10 and ImageNet . Our results show that our method substantially outperforms state-of-the-art for top-k predictions . For instance , our method achieves a certified top-3 accuracy of 69.2 % on ImageNet when an attacker arbitrarily perturbs 5 pixels of a testing image . Under the same setting , Jia et al . ( 2020 ) achieves a certified top-3 accuracy of only 9.0 % , when transforming their ℓ2-norm certified robustness to ℓ0-norm certified robustness . Our contributions can be summarized as follows : • We derive an ℓ0-norm certified radius of top-k predictions for randomized ablation . • We prove that our certified radius is tight when k = 1 and almost tight when k > 1 . • We empirically evaluate our method on CIFAR10 and ImageNet . 2 THEORETICAL RESULTS . In this section , we show our core theoretical contributions . 2.1 BUILDING A SMOOTHED CLASSIFIER VIA RANDOMIZED ABLATION . Suppose we have a base classifier f , which classifies a testing input x to one of c classes { 1 , 2 , · · · , c } deterministically . For simplicity , we assume x is an image with d pixels . Given an input x , randomized ablation ( Levine & Feizi , 2019 ) creates an ablated input as follows : we first randomly subsample e pixels from x without replacement and keep their values . Then , we set the remaining pixel values in the ablated input to a special value , e.g. , median of the pixel value , mean of the pixel value , or a special symbol . When the image is a color image , we set the values of the three channels of each pixel separately . Note that an ablated input has the same size with x . We use h ( x , e ) to denote the randomly ablated input for simplicity . Given h ( x , e ) as input , the output of the base classifier f is also random . We use pj to denote the probability that the base classifier f predicts class j when taking h ( x , e ) as input , i.e. , pj = Pr ( f ( h ( x , e ) ) = j ) . Note that pj is an integer multiple of 1 ( de ) , which we will leverage to derive a tighter certified robustness guarantee . We build a smoothed classifier g that outputs the k labels with the largest label probabilities pj ’ s for x . Moreover , we denote by gk ( x ) the set of k labels predicted for x .

Summary: This paper provides an almost tight l0-norm certified robustness guarantee for top-k predictions against adversarial perturbations, which extends certified radius of the top-1 prediction from Levine & Feizi (2019) to that of the top-k predictions, and the l2-norm certified radius from Jia et al. (2020) to the l0-norm certified radius. The experiments on CIFAR10 and ImageNet show that the proposed method substantially outperforms state-of-the-art for top-k predictions.