Certified Robustness to Adversarial Label-Flipping Attacks via Randomized Smoothing

1 INTRODUCTION . Modern classifiers , despite their widespread empirical success , are known to be susceptible to adversarial attacks . In this paper , we are specifically concerned with so-called “ data-poisoning ” attacks ( formally , causative attacks [ Barreno et al . 2006 ; Papernot et al . 2018 ] ) , where the attacker manipulates some aspects of the training data in order to cause the learning algorithm to output a faulty classifier . Work in this area includes label-flipping attacks ( Xiao et al. , 2012 ) , where the labels of a training set can be adversarially manipulated to decrease performance of the trained classifier ; general data poisoning , where both the training inputs and labels can be manipulated ( Steinhardt et al. , 2017 ) ; and backdoor attacks ( Chen et al. , 2017 ; Tran et al. , 2018 ) , where the training set is corrupted so as to cause the classifier to deviate from its expected behavior when triggered by a specific pattern , without causing a noticeable drop in overall accuracy . However , unlike the alternative “ test-time ” adversarial setting , where reasonably effective defenses exist to build adversarially robust classifiers , relatively little work has been done on building classifiers that are certifiably robust to targeted data poisoning attacks . In this work , we propose a strategy for building classifiers that are certifiably robust against labelflipping attacks . In particular , we propose a pointwise certified defense—by this we mean that with each prediction , the classifier includes a certification guaranteeing that its prediction would not be different had it been trained on data with some number of labels flipped . Existing works on certified defenses make statistical guarantees over the entire test set , but they make no guarantees as to the robustness of a prediction on any particular test point . Thus , while these algorithms provide general robustness to any single corruption , a determined adversary could still cause a specific test point to be misclassified . We therefore consider the threat of a worst-case adversary that can make a training set perturbation to specifically target each test point individually . This motivates a defense that can certify each of its individual predictions , as we present here . To the best of our knowledge , this work represents the first pointwise certified defense to data poisoning attacks . Our approach leverages randomized smoothing ( Cohen et al. , 2019 ) , a technique that has previously been used to guarantee test-time robustness to adversarial manipulation of the input to a deep network . However , where prior uses of randomized smoothing randomize over the input to the classifier for test-time guarantees , we instead randomize over the entire training procedure of the classifier . Specifically , by randomizing over the labels during this training process , we obtain an overall classification pipeline that is certified to be robust ( i.e. , to not change its prediction ) when some number of labels are adversarially manipulated in the training set . Although a naive implementation of this approach would not be computationally feasible , we show that by using a linear least-squares classifier over the output of a pre-trained network , we can obtain these certified bounds with no additional runtime cost over standard classification—and we again emphasize that this is the first such certified defense against pointwise label-flipping attacks , even for linear classifiers . We evaluate our proposed classifier on several benchmark datasets common to the data poisoning literature . Specifically , we demonstrate that our randomized classifier is able to achieve 75.7 % certified accuracy on MNIST 1/7 even when the number of allowed label flips would drive a standard , undefended classifier to less than 50 % . Similar results in experiments on the Dogfish binary classification challenge from ImageNet and IMDB review sentiment database validate our technique for more challenging datasets , and we further experiment on the full MNIST dataset to demonstrate its effectiveness for multi-class classification . On top of this , our classifier maintains a reasonably competitive non-robust accuracy ( e.g. , 94.5 % on MNIST compared to 99.1 % for the undefended classifier ) . 2 RELATED WORK . Data-poisoning attacks A data-poisoning attack ( Muñoz González et al. , 2017 ; Yang et al. , 2017 ) is an attack whereby an adversary corrupts some portion of a training set or adds new inputs , with the goal of degrading the performance of the learned model . The attack can be targeted to cause poor performance on a specific test example or can instead simply reduce the overall test performance . The adversary is assumed to have perfect knowledge of the learning algorithm , so security by design—as opposed to obscurity—is the only viable defense against such attacks . The adversary is also typically assumed to have access to the training set and , in some cases , the test set . Previous work has investigated attacks and defenses for data-poisoning attacks applied to feature selection ( Xiao et al. , 2015 ) , SVMs ( Biggio et al. , 2011 ; Xiao et al. , 2012 ) , linear regression ( Liu et al. , 2017 ) , and PCA ( Rubinstein et al. , 2009 ) , to name a few . Some attacks can even achieve success with “ clean-label ” attacks , inserting innocuous , poisoned yet “ correctly ” labeled training examples but causing the classifier to perform poorly ( Shafahi et al. , 2018 ; Zhu et al. , 2019 ) . For an overview of data poisoning attacks and defenses in machine learning , see Biggio et al . ( 2014 ) . Label-flipping attacks A label-flipping attack is a specific type of data-poisoning attack where the adversary is restricted to changing some of the training labels . The number of allowed changes could be an exact number or a percentage of the total labels . The classifier is then trained on the corrupted training set , with no knowledge of which labels have been tampered with . Unlike random label noise , for which many robust learning algorithms have been successfully developed ( Natarajan et al. , 2013 ; Liu & Tao , 2016 ; Patrini et al. , 2017 ) , adversarial label-flipping attacks can be specifically targeted to exploit the structure of the learning algorithm , significantly degrading performance . Robustness to such attacks is therefore harder to achieve , both theoretically and empirically ( Xiao et al. , 2012 ; Biggio et al. , 2011 ) . A common defense technique is sanitization , whereby a defender attempts to identify and remove or relabel training points that may have had their labels corrupted ( Paudice et al. , 2019 ; Taheri et al. , 2019 ) . Unfortunately , recent work has demonstrated that this is often not enough against a sufficiently powerful adversary ( Koh et al. , 2018 ) . Certified defenses For a review on robustness to random noise , we refer the reader to Rousseeuw & Leroy ( 1987 ) —our focus is on adversarial noise , a much more challenging problem . Existing works on certified defenses to adversarial data poisoning attacks typically focus on the regression case and provide broad statistical guarantees over the entire test distribution . A common approach to such certifications is to show that a particular algorithm recovers some close approximation to the best linear fit coefficients ( Liu et al. , 2017 ; Prasad et al. , 2018 ; Shen & Sanghavi , 2019 ) , or that the expected loss on the test distribution is bounded ( Klivans et al. , 2018 ; Chen & Paschalidis , 2018 ) . These results generally rely on assumptions on the train and test distributions : some assume sparsity in the coefficients ( Karmalkar & Price , 2018 ; Chen et al. , 2013 ) or corruption vector ( Bhatia et al. , 2015 ) ; others require limited effects of outliers ( Steinhardt et al. , 2017 ) . As mentioned above , all of these methods fail to provide guarantees for individual test points . Additionally , most of these statistical guarantees are not meaningful when applied to deep , non-linear architectures . Randomized smoothing Since the discovery of adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) , the research community has been investigating techniques for increasing the robustness of deep networks to adversarial perturbations . After a series of heuristic defenses ( Metzen et al. , 2017 ; Feinman et al. , 2017 ) , followed by attacks breaking them ( Athalye et al. , 2018 ; Carlini & Wagner , 2017 ) , focus began to shift towards the development of provable robustness . One approach which has gained popularity in recent work is randomized smoothing . Rather than certifying the original classifier f , randomized smoothing defines a new classifier g whose prediction at an input x is the class assigned the most probability when x is perturbed with noise from some distribution µ and passed through f . That is , g ( x ) = arg maxc P ∼µ ( f ( x + ) = c ) . This new classifier g is then certified as robust , ideally without sacrificing too much accuracy compared to f . The original formulation was presented by Lecuyer et al . ( 2018 ) and borrowed ideas from differential privacy . The above definition is due to Li et al . ( 2018 ) and was popularized by Cohen et al . ( 2019 ) , who derived a tight robustness guarantee . Follow-up work has focused on optimizing the training procedure of f ( Salman et al. , 2019 ) and extending the analysis to other types of distributions ( Lee et al. , 2019 ) . For more details , we refer the reader to Cohen et al . ( 2019 ) . 3 A GENERAL VIEW OF RANDOMIZED SMOOTHING . We begin by presenting a general viewpoint of randomized smoothing . Under our notation , randomized smoothing constructs an operator G ( µ , φ ) that maps a binary-valued 1 function φ : X → { 0 , 1 } and a smoothing measure µ : X → R+ , with ∫ X µ ( x ) dx = 1 , to the expected value of φ under µ ( that is , G represents the “ vote ” of φ weighted by µ ) . For example , φ could be a binary image classifier and µ could be some small , random pixel noise applied to the to-be-classified image . We also define a “ hard threshold ” version g ( µ , φ ) that returns the most probable output ( the majority vote winner ) . Formally , G ( µ , φ ) = Ex∼µ [ φ ( x ) ] = ∫ X µ ( x ) φ ( x ) dx and g ( µ , φ ) = 1 { G ( µ , φ ) ≥ 1/2 } , ( 1 ) where 1 { · } is the indicator function . Intuitively , for two smoothing measures µ , ρ : X → R+ that are very similar , we would expect that for most φ , even though G ( µ , φ ) and G ( ρ , φ ) may not be equal , the threshold function g should satisfy g ( µ , φ ) = g ( ρ , φ ) . Further , the degree to which µ and ρ can differ while still preserving this property should increase as G ( µ , φ ) approaches either 0 or 1 , because this increases the “ margin ” with which the function φ is 0 or 1 respectively over the measure µ . More formally , we define a general randomized smoothing guarantee as follows . Definition 1 . Let µ : X → R+ be a smoothing measure over X , with ∫ X µ ( x ) dx = 1 . Then a randomized smoothing robustness guarantee is a specification of a distance measure d ( µ , ρ ) and a function f : [ 0 , 1 ] → R+ such that for all φ : X → { 0 , 1 } and all measures ρ : X → R+ of a given form , g ( ρ , φ ) = g ( µ , φ ) whenever d ( µ , ρ ) ≤ f ( G ( µ , φ ) ) . ( 2 ) For brevity , we will sometimes use p in place of G ( µ , φ ) , representing the probability with which the majority class wins the vote ( this is analogous to pA in Cohen et al . ( 2019 ) ) . Instantiations of randomized smoothing This definition is rather abstract , so we highlight concrete examples of how it can be applied to achieve certified guarantees against adversarial attacks . 1For simplicity , we present the methodology here in terms of binary-valued functions , which will correspond eventually to binary classification problems . The extension to the multiclass setting requires additional notation , and thus is deferred to the appendix . Example 1 . The randomized smoothing guarantee of Cohen et al . ( 2019 ) uses the smoothing measures µ = N ( x0 , σ2I ) , a Gaussian around the point x0 to be classified , and ρ = N ( x0 + δ , σ2I ) , a Gaussian around x0 perturbed by δ . They prove that ( 2 ) holds for all classifiers φ if we define d ( µ , ρ ) = 1 σ ‖δ‖2 ≡ √ 2KL ( µ ‖ ρ ) , f ( p ) = |Φ−1 ( p ) | , ( 3 ) where KL ( · ) denotes KL divergence and Φ−1 denotes the inverse CDF of the Gaussian distribution . Although this previous work focused on the case of randomized smoothing of continuous data via Gaussian noise , this is by no means a requirement of the approach . For instance , Lee et al . ( 2019 ) considers an alternative approach for dealing with discrete variables . Example 2 . The randomized smoothing guarantee of Lee et al . ( 2019 ) uses the factorized smoothing measure in d dimensions µα , K ( x ) = Πdi=1µα , K , i ( xi ) , defined with respect to parameters α ∈ [ 0 , 1 ] , K ∈ N , and a base input z ∈ { 0 , . . . , K } d , where µα , K , i ( xi ) = { α , if xi = zi 1−α K , if xi ∈ { 0 , . . . , K } , xi 6= zi , with xi being the ith dimension of x. ρα , K is similarly defined for a perturbed input z′ . They guarantee that ( 2 ) holds if we define d ( µ , ρ ) = r def = ‖z′ − z‖0 , f ( p ) = Fα , K , d ( max ( p , 1− p ) ) . ( 4 ) In words , the smoothing distribution is such that each dimension is independently perturbed to one of the other K values uniformly at random with probability 1 − α. F is a combinatorial function ; Fα , K , d ( p ) is defined as the maximum number of dimensions—out of d total—by which µα , K and ρα , K can differ such that a set with measure p under µα , K is guaranteed to have measure at least 12 under ρα , K . Lee et al . ( 2019 ) prove that this value is independent of z and z′ . Finally , concurrent work has considered a more general form of randomized smoothing that doesn ’ t require strict assumptions on the distributions but is still able to provide similar guarantees . Example 3 ( Generic bound from Anonymous ( 2019 ) ) . Given any two smoothing distributions µ , ρ , we have the generic randomized smoothing robustness certificate , ensuring that ( 2 ) holds with definitions d ( µ , ρ ) = KL ( ρ ‖ µ ) , f ( p ) = −1 2 log ( 4p ( 1− p ) ) . ( 5 ) Randomized smoothing in practice For most classifiers , the expectationG ( µ , φ ) can not be computed exactly , and so we must resort to Monte Carlo approximation . In this “ standard ” form of randomized smoothing , we draw multiple random samples from µ and use these to construct a highprobability bound on G ( µ , φ ) for certification . More precisely , this bound should be a lower bound onG ( µ , φ ) when the hard prediction g ( µ , φ ) = 1 and an upper bound otherwise ; this ensures in both cases that we underestimate the true certified robustness for the classifier g. The procedure is shown in Algorithm 2 in Appendix A . These estimates can then be plugged into a randomized smoothing robustness guarantee to provide a high probability certified robustness bound for the classifier g .

Summary: This paper proposes a certifiable defense against data poisoning attacks by using a randomized smoothing approach. An adversary in such a setting is permitted to flip any r labels from a dataset of size n. The smoothing procedure (stated roughly) is to train on a dataset with "noisy" or "smoothed" labels, obtained by flipping each label with some probability q. The authors obtain a lower bound on r in terms of q. Directly using this technique requires training multiple classifiers on multiple noisy datasets. To show that this method is useful, the authors study the effectiveness of this model against a classifier that performs linear regression on a pre-trained feature extractor.