Deep Partition Aggregation: Provable Defenses against General Poisoning Attacks

1 INTRODUCTION . Adversarial poisoning attacks are an important vulnerability in machine learning systems . In these attacks , an adversary can manipulate the training data of a classifier , in order to change the classifications of specific inputs at test time . Several poisoning threat models have been studied in the literature , including threat models where the adversary may insert new poison samples ( Chen et al. , 2017 ) , manipulate the training labels ( Xiao et al. , 2012 ; Rosenfeld et al. , 2020 ) , or manipulate the training sample values ( Biggio et al. , 2012 ; Shafahi et al. , 2018 ) . A certified defense against a poisoning attack provides a certificate for each test sample , which is a guaranteed lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample ’ s classification . In this work , we propose certified defenses against two types of poisoning attacks : General poisoning attacks : In this threat model , the attacker can insert or remove a bounded number of samples from the training set . In particular , the attack magnitude ρ is defined as the cardinality of the symmetric difference between the clean and poisoned training sets . This threat model also includes any distortion to an sample and/or label in the training set — a distortion of a training sample is simply the removal of the original sample followed by the insertion of the distorted sample . ( Note that a sample distortion or label flip therefore increases the symmetric difference attack magnitude by two . ) Label-flipping poisoning attacks : In this threat model , the adversary changes only the label for ρ out of m training samples . Rosenfeld et al . ( 2020 ) has recently provided a certified defense for this threat model , which we improve upon . In the last couple of years , certified defenses have been extensively studied for evasion attacks , where the adversary manipulates the test samples , rather than the training data ( e.g . Wong & Kolter ( 2018 ) ; Gowal et al . ( 2018 ) ; Lecuyer et al . ( 2019 ) ; Li et al . ( 2018 ) ; Salman et al . ( 2019 ) ; Levine & Feizi ( 2020a ; b ) ; Cohen et al . ( 2019 ) , etc . ) In the evasion case , a certificate is a lower bound on the distance from the sample to the classifier ’ s decision boundary : this guarantees that the sample ’ s classification remains unchanged under adversarial distortions up to the certified magnitude . Rosenfeld et al . ( 2020 ) provides an analogous certificate for label-flipping poisoning attacks : for an input sample x , the certificate of x is a lower bound on the number of labels in the training set that would have to change in order to change the classification of x.1 Rosenfeld et al . ( 2020 ) ’ s method is an adaptation of a certified defense for sparse ( L0 ) evasion attacks proposed by Lee et al . ( 2019 ) . The adapted method for label-flipping attacks proposed by Rosenfeld et al . ( 2020 ) is equivalent to randomly flipping each training label with fixed probability and taking a consensus result . If implemented directly , this would require one to train a large ensemble of classifiers on different noisy versions of the training data . However , instead of actually doing this , Rosenfeld et al . ( 2020 ) focuses only on linear classifiers and is therefore able to analytically calculate the expected result . This gives deterministic , rather than probabilistic , certificates . Further , because Rosenfeld et al . ( 2020 ) considers a threat model where only labels are modified , they are able to train an unsupervised nonlinear feature extractor on the ( unlabeled ) training data before applying their technique , in order to learn more complex features . Inspired by an improved provable defense against L0 evasion attacks ( Levine & Feizi , 2020a ) , in this paper , we develop certifiable defenses against general and label-flipping poisoning attacks that significantly outperform the current state-of-the-art certifiable defenses . In particular , we develop 1Steinhardt et al . ( 2017 ) also refers to a “ certified defense ” for poisoning attacks . However , the definition of the certificate is substantially different in that work , which instead provides overall accuracy guarantees under the assumption that the training and test data are drawn from similar distributions , rather than providing guarantees for individual realized inputs . a certifiable defense against general poisoning attacks called Deep Partition Aggregation ( DPA ) which is based on partitioning the training set into k partitions , with the partition assignment for a training sample determined by a hash function of the sample . The hash function can be any deterministic function that maps a training sample t to a partition assignment : the only requirement is that the hash value depends only on the value of the training sample t itself , so that neither poisoning other samples , nor changing the total number of samples , nor reordering the samples can change the partition that t is assigned to . We then train k base classifiers separately , one on each partition . At the test time , we evaluate each of the base classifiers on the test sample x and return the plurality classification c as the final result . The key insight is that removing a training sample , or adding a new sample , will only change the contents of one partition , and therefore will only affect the classification of one of the k base classifiers . This immediately leads to robustness certifications against general poisoning attacks which , to the best of our knowledge , is the first one of this kind . If the adversary is restricted to flipping labels only ( as in Rosenfeld et al . ( 2020 ) ) , we can achieve even larger certificates through a modified technique . In this setting , the unlabeled data is trustworthy : each base classifier in the ensemble can then make use of the entire training set without labels , but only has access to the labels in its own partition . Thus , each base classifier can be trained as if the entire dataset is available as unlabeled data , but only a very small number of labels are available . This is precisely the problem statement of semi-supervised learning ( Verma et al. , 2019 ; Luo et al. , 2018 ; Laine & Aila , 2017 ; Kingma et al. , 2014 ; Gidaris et al. , 2018 ) . We can then leverage these existing semi-supervised learning techniques directly to improve the accuracies of the base classifiers in DPA . Furthermore , we can ensure that a particular ( unlabeled ) sample is assigned to the same partition regardless of label , so that only one partition is affected by a label flip ( rather than possibly two ) . The resulting algorithm , Semi-Supervised Deep Partition Aggregation ( SS-DPA ) yields substantially increased certified accuracy against label-flipping attacks , compared to DPA alone and compared to the current state-of-the-art . Furthermore , while our method is de-randomized ( as Rosenfeld et al . ( 2020 ) is ) and therefore yields deterministic certificates , our technique does not require that the classification model be linear , allowing deep networks to be used . On MNIST , SS-DPA substantially outperforms the existing state of the art ( Rosenfeld et al. , 2020 ) in defending against label-flip attacks : we certify at least half of images in the test set against attacks to over 600 ( 1.0 % ) of the labels in the training set , while still maintaining over 93 % accuracy ( See Figure 1 , and Table 1 ) . In comparison , Rosenfeld et al . ( 2020 ) ’ s method achieves less than 60 % clean accuracy on MNIST , and most test images can not be certified with the correct class against attacks of even 200 label flips . We are also the first work to our knowledge to certify against general poisoning attacks , including insertions and deletions of new training images : in this domain , we can certify at least half of test images against attacks consisting of over 500 arbitrary training image insertions or deletions . On CIFAR-10 , a substantially more difficult classification task , we can certify at least half of test images against label-flipping attacks on over 300 labels using SS-DPA ( versus 175 label-flips for ( Rosenfeld et al. , 2020 ) ) , and can certify at least half of test images against general poisoning attacks of up to nine insertions or deletions using DPA . To see how our method performs on datasets with larger numbers of classes , we also tested our methods on the German Traffic Sign Recognition Benchmark ( Stallkamp et al. , 2012 ) , a task with 43 classes and on average ≈ 1000 samples per class . Here , we are able to certify at least half of test images as robust to 176 label flips , or 20 general poisoning attacks . These results establish new state-of-the-art in provable defenses against label-flipping and general poisoning attacks . 2 RELATED WORKS . Levine & Feizi ( 2020a ) propose a randomized ablation technique to certifiably defend against sparse atatcks . Their method ablates some pixels , replacing them with a null value . Since it is possible for the base classifier to distinguish exactly which pixels originate from x , this results in more accurate base classifications and therefore substantially greater certified robustness than Lee et al . ( 2019 ) . For example , on ImageNet , Lee et al . ( 2019 ) certifies the median test image against distortions of one pixel , while Levine & Feizi ( 2020a ) certifies against distortions of 16 pixels . Our proposed method is related to classical ensemble approaches in machine learning , namely bootstrap aggregation and subset aggregation ( Breiman , 1996 ; Buja & Stuetzle , 2006 ; Bühlmann , 2003 ; Zaman & Hirose , 2009 ) . However , in these methods each base classifier in the ensemble is trained on an independently sampled collection of points from the training set : multiple classifiers in the ensemble may be trained on ( and therefore poisoned by ) the same sample point . The purpose of these methods has typically been to improve generalization . Bootstrap aggregation has been proposed as an empirical defense against poisoning attacks ( Biggio et al. , 2011 ) as well as for evasion attacks ( Smutz & Stavrou , 2016 ) . However , at the time of the initial distribution of this work , these techniques had not yet been used to provide certified robustness.2 Our unique partition aggregation variant provides deterministic robustness certificates against poisoning attacks . See Appendix D for further discussion . Weber et al . ( 2020 ) have recently proposed a different randomized-smoothing based defense against poisoning attacks by directly applying Cohen et al . ( 2019 ) ’ s smoothing L2 evasion defense to the poisoning domain . The proposed technique can only certify for clean-label attacks ( where only the existing samples in the dataset are modified , and not their labels ) , and the certificate guarantees robustness only to bounded L2 distortions of the training data , where the L2 norm of the distortion is calculated across all pixels in the entire training set . Due to well-known limitations of dimensional scaling for smoothing-based robustness certificates ( Yang et al. , 2020 ; Kumar et al. , 2020 ; Blum et al. , 2020 ) , this yields certificates to only very small distortions of the training data . ( For binary MNIST [ 13,007 images ] , the maximum reportedL2 certificate is 2 pixels . ) Additionally , when using deep classifiers , Weber et al . ( 2020 ) proposes a randomized certificate , rather than a deterministic one , with a failure probability that decreases to zero only as the number of trained classifiers in an ensemble approaches infinity . Moreover , in Weber et al . ( 2020 ) , unlike in our method , each classifier in the ensemble must be trained on a noisy version of the entire dataset . These issues hinder Weber et al . ( 2020 ) ’ s method to be an effective scheme for certified robustness against poisoning attacks . After the initial distribution of this work , a recent revision of Rosenfeld et al . ( 2020 ) has suggested using randomised smoothing techniques on training samples , rather than just training labels , as a general approach to poisoning defense . Both this work and Weber et al . ( 2020 ) could be considered as implementations of this idea , although this generalized proposal in Rosenfeld et al . ( 2020 ) does not include a derandomization scheme ( unlike Rosenfeld et al . ( 2020 ) ’ s proposed derandomized defense against label-flipping attacks ) . Other prior works have provided distributional , rather than pointwise guarantees against poisoning attacks . In these works , there is a ( high-probability ) guarantee that the classifier will achieve a certain level of average overall accuracy on test data , under the assumption that the test data and clean ( pre-poisoning ) training data are drawn from the same distribution . These works do not provide any guarantees that apply to specific test samples , however . As mentioned above , Steinhardt et al . ( 2017 ) provides such a distributional guarantee , specifically for a threat model of addition of poison samples . Other such works include Sloan ( 1995 ) , which considers label-flipping attacks and determines conditions under which PAC-learning is possible in the presence of such attacks , and Bshouty et al . ( 2002 ) provides similar guarantees for replacement of samples . Other works ( Diakonikolas et al. , 2016 ; Lai et al. , 2016 ) provide distributional guarantees for unsupervised learning under poisoning attacks . Mahloujifar et al . ( 2019 ) proposes provably effective poisoning attacks with high-probability pointwise guarantees of effectiveness on test samples . However , that work relies on properties of the distribution that the training set is drawn from . Diakonikolas et al . ( 2019 ) provide a robust training algorithm which provably approximates the clean trained model despite poisoning ( rather than the behavior at a certain test point ) : this result also makes assumptions about the distribution of the clean training data .

Summary: This paper studies how to enhance the robustness of classifiers in face of data poisoning attacks. The key insight of the paper is that adding or deleting one training point can at most change one of the k partitions of the training set. Based on this idea, the authors propose Deep Partition Aggregation (DPA), a robust classification algorithm that first partitions the training data into k subsets, and then separately train a model on each subset. The final prediction is an aggregation of the predictions of those k classifiers using the majority vote. Apart from DPA, the authors also consider a setting where a large amount of data points do not have labels. In that scenario, semi-supervised learning algorithms are used as based algorithm when training separate models on each subset. The proposed method SS-DPA enjoys the property of being fast to train, since learning is performed on a subset that has smaller number of data points. The paper derives theoretical guarantees in terms of when the prediction of a particular data point can be certifiably correct. Finally, experiments on MNIST and CIFAR demonstrate the effectiveness of the proposed defense as compared to prior works.