Certified Robustness to Adversarial Label-Flipping Attacks via Randomized Smoothing

1 INTRODUCTION . Modern classifiers , despite their widespread empirical success , are known to be susceptible to adversarial attacks . In this paper , we are specifically concerned with so-called “ data-poisoning ” attacks ( formally , causative attacks [ Barreno et al . 2006 ; Papernot et al . 2018 ] ) , where the attacker manipulates some aspects of the training data in order to cause the learning algorithm to output a faulty classifier . Work in this area includes label-flipping attacks ( Xiao et al. , 2012 ) , where the labels of a training set can be adversarially manipulated to decrease performance of the trained classifier ; general data poisoning , where both the training inputs and labels can be manipulated ( Steinhardt et al. , 2017 ) ; and backdoor attacks ( Chen et al. , 2017 ; Tran et al. , 2018 ) , where the training set is corrupted so as to cause the classifier to deviate from its expected behavior when triggered by a specific pattern , without causing a noticeable drop in overall accuracy . However , unlike the alternative “ test-time ” adversarial setting , where reasonably effective defenses exist to build adversarially robust classifiers , relatively little work has been done on building classifiers that are certifiably robust to targeted data poisoning attacks . In this work , we propose a strategy for building classifiers that are certifiably robust against labelflipping attacks . In particular , we propose a pointwise certified defense—by this we mean that with each prediction , the classifier includes a certification guaranteeing that its prediction would not be different had it been trained on data with some number of labels flipped . Existing works on certified defenses make statistical guarantees over the entire test set , but they make no guarantees as to the robustness of a prediction on any particular test point . Thus , while these algorithms provide general robustness to any single corruption , a determined adversary could still cause a specific test point to be misclassified . We therefore consider the threat of a worst-case adversary that can make a training set perturbation to specifically target each test point individually . This motivates a defense that can certify each of its individual predictions , as we present here . To the best of our knowledge , this work represents the first pointwise certified defense to data poisoning attacks . Our approach leverages randomized smoothing ( Cohen et al. , 2019 ) , a technique that has previously been used to guarantee test-time robustness to adversarial manipulation of the input to a deep network . However , where prior uses of randomized smoothing randomize over the input to the classifier for test-time guarantees , we instead randomize over the entire training procedure of the classifier . Specifically , by randomizing over the labels during this training process , we obtain an overall classification pipeline that is certified to be robust ( i.e. , to not change its prediction ) when some number of labels are adversarially manipulated in the training set . Although a naive implementation of this approach would not be computationally feasible , we show that by using a linear least-squares classifier over the output of a pre-trained network , we can obtain these certified bounds with no additional runtime cost over standard classification—and we again emphasize that this is the first such certified defense against pointwise label-flipping attacks , even for linear classifiers . We evaluate our proposed classifier on several benchmark datasets common to the data poisoning literature . Specifically , we demonstrate that our randomized classifier is able to achieve 75.7 % certified accuracy on MNIST 1/7 even when the number of allowed label flips would drive a standard , undefended classifier to less than 50 % . Similar results in experiments on the Dogfish binary classification challenge from ImageNet and IMDB review sentiment database validate our technique for more challenging datasets , and we further experiment on the full MNIST dataset to demonstrate its effectiveness for multi-class classification . On top of this , our classifier maintains a reasonably competitive non-robust accuracy ( e.g. , 94.5 % on MNIST compared to 99.1 % for the undefended classifier ) . 2 RELATED WORK . Data-poisoning attacks A data-poisoning attack ( Muñoz González et al. , 2017 ; Yang et al. , 2017 ) is an attack whereby an adversary corrupts some portion of a training set or adds new inputs , with the goal of degrading the performance of the learned model . The attack can be targeted to cause poor performance on a specific test example or can instead simply reduce the overall test performance . The adversary is assumed to have perfect knowledge of the learning algorithm , so security by design—as opposed to obscurity—is the only viable defense against such attacks . The adversary is also typically assumed to have access to the training set and , in some cases , the test set . Previous work has investigated attacks and defenses for data-poisoning attacks applied to feature selection ( Xiao et al. , 2015 ) , SVMs ( Biggio et al. , 2011 ; Xiao et al. , 2012 ) , linear regression ( Liu et al. , 2017 ) , and PCA ( Rubinstein et al. , 2009 ) , to name a few . Some attacks can even achieve success with “ clean-label ” attacks , inserting innocuous , poisoned yet “ correctly ” labeled training examples but causing the classifier to perform poorly ( Shafahi et al. , 2018 ; Zhu et al. , 2019 ) . For an overview of data poisoning attacks and defenses in machine learning , see Biggio et al . ( 2014 ) . Label-flipping attacks A label-flipping attack is a specific type of data-poisoning attack where the adversary is restricted to changing some of the training labels . The number of allowed changes could be an exact number or a percentage of the total labels . The classifier is then trained on the corrupted training set , with no knowledge of which labels have been tampered with . Unlike random label noise , for which many robust learning algorithms have been successfully developed ( Natarajan et al. , 2013 ; Liu & Tao , 2016 ; Patrini et al. , 2017 ) , adversarial label-flipping attacks can be specifically targeted to exploit the structure of the learning algorithm , significantly degrading performance . Robustness to such attacks is therefore harder to achieve , both theoretically and empirically ( Xiao et al. , 2012 ; Biggio et al. , 2011 ) . A common defense technique is sanitization , whereby a defender attempts to identify and remove or relabel training points that may have had their labels corrupted ( Paudice et al. , 2019 ; Taheri et al. , 2019 ) . Unfortunately , recent work has demonstrated that this is often not enough against a sufficiently powerful adversary ( Koh et al. , 2018 ) . Certified defenses For a review on robustness to random noise , we refer the reader to Rousseeuw & Leroy ( 1987 ) —our focus is on adversarial noise , a much more challenging problem . Existing works on certified defenses to adversarial data poisoning attacks typically focus on the regression case and provide broad statistical guarantees over the entire test distribution . A common approach to such certifications is to show that a particular algorithm recovers some close approximation to the best linear fit coefficients ( Liu et al. , 2017 ; Prasad et al. , 2018 ; Shen & Sanghavi , 2019 ) , or that the expected loss on the test distribution is bounded ( Klivans et al. , 2018 ; Chen & Paschalidis , 2018 ) . These results generally rely on assumptions on the train and test distributions : some assume sparsity in the coefficients ( Karmalkar & Price , 2018 ; Chen et al. , 2013 ) or corruption vector ( Bhatia et al. , 2015 ) ; others require limited effects of outliers ( Steinhardt et al. , 2017 ) . As mentioned above , all of these methods fail to provide guarantees for individual test points . Additionally , most of these statistical guarantees are not meaningful when applied to deep , non-linear architectures . Randomized smoothing Since the discovery of adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) , the research community has been investigating techniques for increasing the robustness of deep networks to adversarial perturbations . After a series of heuristic defenses ( Metzen et al. , 2017 ; Feinman et al. , 2017 ) , followed by attacks breaking them ( Athalye et al. , 2018 ; Carlini & Wagner , 2017 ) , focus began to shift towards the development of provable robustness . One approach which has gained popularity in recent work is randomized smoothing . Rather than certifying the original classifier f , randomized smoothing defines a new classifier g whose prediction at an input x is the class assigned the most probability when x is perturbed with noise from some distribution µ and passed through f . That is , g ( x ) = arg maxc P ∼µ ( f ( x + ) = c ) . This new classifier g is then certified as robust , ideally without sacrificing too much accuracy compared to f . The original formulation was presented by Lecuyer et al . ( 2018 ) and borrowed ideas from differential privacy . The above definition is due to Li et al . ( 2018 ) and was popularized by Cohen et al . ( 2019 ) , who derived a tight robustness guarantee . Follow-up work has focused on optimizing the training procedure of f ( Salman et al. , 2019 ) and extending the analysis to other types of distributions ( Lee et al. , 2019 ) . For more details , we refer the reader to Cohen et al . ( 2019 ) . 3 A GENERAL VIEW OF RANDOMIZED SMOOTHING . We begin by presenting a general viewpoint of randomized smoothing . Under our notation , randomized smoothing constructs an operator G ( µ , φ ) that maps a binary-valued 1 function φ : X → { 0 , 1 } and a smoothing measure µ : X → R+ , with ∫ X µ ( x ) dx = 1 , to the expected value of φ under µ ( that is , G represents the “ vote ” of φ weighted by µ ) . For example , φ could be a binary image classifier and µ could be some small , random pixel noise applied to the to-be-classified image . We also define a “ hard threshold ” version g ( µ , φ ) that returns the most probable output ( the majority vote winner ) . Formally , G ( µ , φ ) = Ex∼µ [ φ ( x ) ] = ∫ X µ ( x ) φ ( x ) dx and g ( µ , φ ) = 1 { G ( µ , φ ) ≥ 1/2 } , ( 1 ) where 1 { · } is the indicator function . Intuitively , for two smoothing measures µ , ρ : X → R+ that are very similar , we would expect that for most φ , even though G ( µ , φ ) and G ( ρ , φ ) may not be equal , the threshold function g should satisfy g ( µ , φ ) = g ( ρ , φ ) . Further , the degree to which µ and ρ can differ while still preserving this property should increase as G ( µ , φ ) approaches either 0 or 1 , because this increases the “ margin ” with which the function φ is 0 or 1 respectively over the measure µ . More formally , we define a general randomized smoothing guarantee as follows . Definition 1 . Let µ : X → R+ be a smoothing measure over X , with ∫ X µ ( x ) dx = 1 . Then a randomized smoothing robustness guarantee is a specification of a distance measure d ( µ , ρ ) and a function f : [ 0 , 1 ] → R+ such that for all φ : X → { 0 , 1 } and all measures ρ : X → R+ of a given form , g ( ρ , φ ) = g ( µ , φ ) whenever d ( µ , ρ ) ≤ f ( G ( µ , φ ) ) . ( 2 ) For brevity , we will sometimes use p in place of G ( µ , φ ) , representing the probability with which the majority class wins the vote ( this is analogous to pA in Cohen et al . ( 2019 ) ) . Instantiations of randomized smoothing This definition is rather abstract , so we highlight concrete examples of how it can be applied to achieve certified guarantees against adversarial attacks . 1For simplicity , we present the methodology here in terms of binary-valued functions , which will correspond eventually to binary classification problems . The extension to the multiclass setting requires additional notation , and thus is deferred to the appendix . Example 1 . The randomized smoothing guarantee of Cohen et al . ( 2019 ) uses the smoothing measures µ = N ( x0 , σ2I ) , a Gaussian around the point x0 to be classified , and ρ = N ( x0 + δ , σ2I ) , a Gaussian around x0 perturbed by δ . They prove that ( 2 ) holds for all classifiers φ if we define d ( µ , ρ ) = 1 σ ‖δ‖2 ≡ √ 2KL ( µ ‖ ρ ) , f ( p ) = |Φ−1 ( p ) | , ( 3 ) where KL ( · ) denotes KL divergence and Φ−1 denotes the inverse CDF of the Gaussian distribution . Although this previous work focused on the case of randomized smoothing of continuous data via Gaussian noise , this is by no means a requirement of the approach . For instance , Lee et al . ( 2019 ) considers an alternative approach for dealing with discrete variables . Example 2 . The randomized smoothing guarantee of Lee et al . ( 2019 ) uses the factorized smoothing measure in d dimensions µα , K ( x ) = Πdi=1µα , K , i ( xi ) , defined with respect to parameters α ∈ [ 0 , 1 ] , K ∈ N , and a base input z ∈ { 0 , . . . , K } d , where µα , K , i ( xi ) = { α , if xi = zi 1−α K , if xi ∈ { 0 , . . . , K } , xi 6= zi , with xi being the ith dimension of x. ρα , K is similarly defined for a perturbed input z′ . They guarantee that ( 2 ) holds if we define d ( µ , ρ ) = r def = ‖z′ − z‖0 , f ( p ) = Fα , K , d ( max ( p , 1− p ) ) . ( 4 ) In words , the smoothing distribution is such that each dimension is independently perturbed to one of the other K values uniformly at random with probability 1 − α. F is a combinatorial function ; Fα , K , d ( p ) is defined as the maximum number of dimensions—out of d total—by which µα , K and ρα , K can differ such that a set with measure p under µα , K is guaranteed to have measure at least 12 under ρα , K . Lee et al . ( 2019 ) prove that this value is independent of z and z′ . Finally , concurrent work has considered a more general form of randomized smoothing that doesn ’ t require strict assumptions on the distributions but is still able to provide similar guarantees . Example 3 ( Generic bound from Anonymous ( 2019 ) ) . Given any two smoothing distributions µ , ρ , we have the generic randomized smoothing robustness certificate , ensuring that ( 2 ) holds with definitions d ( µ , ρ ) = KL ( ρ ‖ µ ) , f ( p ) = −1 2 log ( 4p ( 1− p ) ) . ( 5 ) Randomized smoothing in practice For most classifiers , the expectationG ( µ , φ ) can not be computed exactly , and so we must resort to Monte Carlo approximation . In this “ standard ” form of randomized smoothing , we draw multiple random samples from µ and use these to construct a highprobability bound on G ( µ , φ ) for certification . More precisely , this bound should be a lower bound onG ( µ , φ ) when the hard prediction g ( µ , φ ) = 1 and an upper bound otherwise ; this ensures in both cases that we underestimate the true certified robustness for the classifier g. The procedure is shown in Algorithm 2 in Appendix A . These estimates can then be plugged into a randomized smoothing robustness guarantee to provide a high probability certified robustness bound for the classifier g .

Summary: The paper investigates data poisoning type of attack. In such attacks, an adversary can alter/flip the labels of some of the training examples. The paper proposed a new approach towards certified robustness against this type of attack. In particular, the new classifier will output a prediction along with a certificate in which the prediction would not change if certain number of labels in the training data were flipped.