Poisoning Attacks with Generative Adversarial Nets

1 INTRODUCTION . Despite the advancements and the benefits of machine learning , it has been shown that learning algorithms are vulnerable and can be the target of attackers , who can gain a significant advantage by exploiting these vulnerabilities ( Huang et al. , 2011 ) . At training time , learning algorithms are vulnerable to poisoning attacks , where small fractions of malicious points injected in the training set can subvert the learning process and degrade the performance of the system in an indiscriminate or targeted way . Data poisoning is one of the most relevant and emerging security threats in applications that rely upon the collection of large amounts of data in the wild ( Joseph et al. , 2013 ) . Some applications rely on the data from users ’ feedback or untrusted sources of information that often collude towards the same malicious goal . For example , in IoT environments sensors can be compromised and adversaries can craft coordinated attacks manipulating the measurements of neighbour sensors evading detection ( Illiano et al. , 2016 ) . In many applications curation of the whole training dataset is not possible , exposing machine learning systems to poisoning attacks . In the research literature optimal poisoning attack strategies have been proposed against different machine learning algorithms ( Biggio et al. , 2012 ; Mei & Zhu , 2015 ; Muñoz-González et al. , 2017 ; Jagielski et al. , 2018 ) , allowing to assess their performance in worst-case scenarios . These attacks can be modelled as a bi-level optimization problem , where the outer objective represents the attacker ’ s goal and the inner objective corresponds to the training of the learning algorithm with the poisoned dataset . Solving these bi-level optimization problems is challenging and can be computationally demanding , especially for generating poisoning points at scale . This limits its applicability against some learning algorithms such as deep networks or where the training set is large . In many cases , if no detectability constraints are considered , the poisoning points generated are outliers that can be removed with data filtering ( Paudice et al. , 2018a ) . Furthermore , such attacks are not realistic as real attackers would aim to remain undetected in order to be able to continue subverting the system in the future . As shown in ( Koh et al. , 2018 ) , detectability constraints for these optimal attack strategies can be modelled , however they further increase the complexity of the attack , limiting even more the application of these techniques . Taking an entirely different and novel approach , in this paper we propose a poisoning attack strategy against machine learning classifiers with Generative Adversarial Nets ( GANs ) ( Goodfellow et al. , 2014 ) . This allows us to craft poisoning points in a more systematic way , looking for regions of the data distribution where the poisoning points are more influential and , at the same time , difficult to detect . Our proposed scheme , pGAN , consists on three components : generator , discriminator and target classifier . The generator aims to generate poisoning points that maximize the error of the target classifier but minimize the discriminator ’ s ability to distinguish them from genuine data points . The classifier aims to minimize some loss function evaluated on a training dataset that contains a fraction of poisoning points . As in a standard GAN , the problem can be formulated as a minimax game . pGAN allows to systematically generate adversarial training examples ( Koh & Liang , 2017 ) , which are similar to genuine data points but that can degrade the performance of the system when used for training . The use of a generative model allows us to produce poisoning points at scale , enabling poisoning attacks against learning algorithms where the number of training points is large or in situations where optimal attack strategies with bi-level optimization are intractable or difficult to compute , as it can be the case for deep networks . Additionally , our proposed model also includes a mechanism to control the detectability of the generated poisoning points . For this , the generator maximizes a convex combination of the losses for the discriminator and the classifier evaluated on the poisoning data points . Our model allows to control the aggressiveness of the attack through a parameter that controls the weighted sum of the two losses . This induces a trade-off between effectiveness and detectability of the attack . In this way , pGAN can be applied for systematic testing of machine learning classifiers at different risk levels . Our experimental evaluation in synthetic and real datasets shows that pGAN is capable of compromising different machine learning classifiers bypassing different defence mechanisms , including outlier detection Paudice et al . ( 2018a ) , Sever Diakonikolas et al . ( 2019 ) , PCA-based defences Rubinstein et al . ( 2009 ) and label sanitization Paudice et al . ( 2018b ) . We analyse the trade-off between detectability and effectiveness of the attack : Too conservative strategies will have a reduced impact on the target classifier but , if the attack is too aggressive , most poisoning points can be detected as outliers . 2 RELATED WORK . The first practical poisoning attacks were proposed in the context of spam filtering and anomaly detection ( Nelson et al. , 2008 ; Kloft & Laskov , 2012 ) . But these attacks do not easily generalize to different learning algorithms . Biggio et al . ( 2012 ) presented a more systematic approach , modelling optimal poisoning attacks against SVMs for binary classification as a bi-level optimization problem , which can be solved by exploiting the Karush-Kuhn-Tucker conditions in the inner problem . A similar approach is proposed by Xiao et al . ( 2015 ) for poisoning embedded feature selection methods , including LASSO , ridge regression , and elastic net . Mei & Zhu ( 2015 ) proposed a more general framework to model and solve optimal poisoning attacks for convex classifiers . They exploit the implicit function theorem to compute the gradients required to solve the corresponding bi-level optimization problem . Muñoz-González et al . ( 2017 ) proposed back-gradient optimization to estimate the gradients required to solve bi-level optimization problems for optimal poisoning attacks against multi-class classifiers . This approach allows to attack a broader range of learning algorithms and reduces the computational complexity with respect to previous works . However , all these techniques are limited to compromise deep networks trained with a large number of training points , where many poisoning points are required even to compromise a small fraction of the training dataset . Previous attacks did not model explicitly appropriate detectability constraints . Thus , the resulting poisoning points can be far from the genuine data distribution and can be easily identified as outliers ( Paudice et al. , 2018a ; Steinhardt et al. , 2017 ; Paudice et al. , 2018b ) . Recently , Koh et al . ( 2018 ) showed that it is still possible to craft attacks capable of bypassing outlier-detection-based defences with an iterative constrained bi-level optimization problem , where , at each iteration , the constraints change according to the current solution of the bi-level problem . However , the high computational complexity of this attack limits its practical application in many scenarios . Koh & Liang ( 2017 ) proposed a different approach to craft targeted attacks against deep networks by exploiting influence functions . This approach allows to create adversarial training examples by learning small perturbations that , when added to some specific genuine training points , change the predictions for a target set of test points . Shafahi et al . ( 2018 ) showed that it is possible to perform targeted attacks when the adversary is not in control of the labels for the poisoning points . Yang et al . ( 2017 ) introduced a poisoning attack with generative models using autoencoders to generate the malicious points . Although this method is more scalable than attacks based on bi-level optimization , the authors do not provide a mechanism to control the detectability of the poisoning points . 3 POISONING ATTACKS WITH GENERATIVE ADVERSARIAL NETS . Our model , pGAN , is a GAN-based model with three components ( generator , discriminator and target classifier ) to generate systematically adversarial training examples . First , we shortly describe the considered model for the attacker . Then , we introduce the formulation of pGAN and , finally , we provide some practical considerations for the implementation of pGAN . 3.1 ATTACKER ’ S MODEL . The attacker ’ s knowledge of the targeted system depends on different aspects : the learning algorithm , the objective function optimized , the feature set or the training data . In our case we consider perfect knowledge attacks , where we assume the attacker knows everything about the target system : the training data , the feature set , the loss function and the machine learning model used by the victim . Although unrealistic in most practical scenarios , this assumption allows us to perform worst-case analysis of the performance of the system under attack . However , our proposed attack strategy also supports limited knowledge , exploiting the transferability property of poisoning attacks ( Muñoz-González et al. , 2017 ) . For the attacker ’ s capabilities , we consider here a causative attack ( Barreno et al. , 2006 ; 2010 ) , where the attacker can manipulate a fraction of the training data to influence the learning algorithm . We assume that the attacker can manipulate all the features to craft the poisoning points as long as the resulting points are within the feasible domain for the distribution of genuine training points . Finally , we also assume that the attacker can also control the labels of the injected poisoning points . 3.2 PGAN . In a multi-class classification task , letX ∈ Rd be the d-dimensional feature space , where data points x are drawn from a distribution px ( x ) and Y is the space of class labels . The learning algorithm , C , aims to learn the mapping f : X → Y by minimizing a loss function , LC , evaluated on a set of training points Str . The objective of the attacker is to introduce a fraction , λ ∈ ( 0 , 1 ) , of malicious points in Str to maximize LC when evaluated on the poisoned training set . The Generator , G , aims to generate poisoning points by learning a data distribution that is effective at increasing the error of the target classifier , but that is also close to the distribution of genuine data points , i.e . the generated poisoning points are similar to honest data points to evade detection . Thus , G receives some noise z ∼ pz ( z|Yp ) as input and implicitly defines a distribution of poisoning points , pp ( x ) , which is the distribution of the samples G ( z|Yp ) conditioned on Yp ⊂ Y , the set of target class labels for the attacker . The Discriminator , D , aims to distinguish between honest training data and the generated poisoning points . It estimates D ( x|Yp ) , the probability that x came from the genuine data distribution px rather than pp . As in G , the samples used in the discriminator are conditioned on the set of labels Yp . The Classifier , C , is representative for the attacked algorithm . In perfect knowledge attacks C can have the same structure as the actual target classifier . For blackbox attacks we can exploit attack transferability , and then , use C as a surrogate model that can be somewhat similar to the actual ( unknown ) classifier . During the training of pGAN , C is fed honest and poisoning training points from px and pp respectively , where the fraction of poisoning points is controlled by a parameter λ ∈ [ 0 , 1 ] . In contrast to traditional GAN schemes , G in pGAN plays a game against both D and C. This can also be formalized as a minimax game where the maximization problem involves both D and C. Similar to conditional GANs ( Mirza & Osindero , 2014 ) , the objective function for D ( which also depends on G ) can be written as : V ( D , G ) = Ez∼pz ( z|Yp ) [ log ( 1−D ( G ( z|Yp ) ) ) ] + Ex∼px ( x|Yp ) [ log ( D ( x|Yp ) ] . ( 1 ) The objective function for C is given by : W ( C , G ) = − ( λ Ez∼pz ( z|Yp ) [ LC ( G ( z|Yp ) ) ] + ( 1− λ ) Ex∼px ( x ) [ LC ( x ) ] ) , ( 2 ) where λ is the fraction of poisoning points introduced in the training dataset and LC is the loss function used to train C. Note that the poisoning points in ( 2 ) belong to a subset of poisoning class labels Yp , whereas the genuine points used to train the classifier are from all the classes . The objective in ( 2 ) is just the negative loss used to train C evaluated on a mixture of honest and poisoning points ( from the set of classes in Yp ) controlled by λ . Given ( 1 ) and ( 2 ) , pGAN can then be formulated as the following minimax problem : min G max D , C α V ( D , G ) + ( 1− α ) W ( C , G ) ( 3 ) with α ∈ [ 0 , 1 ] . In this case , the maximization problem can be seen as a multi-objective optimization problem to learn the parameters of both the classifier and the discriminator . Whereas for C andD the objectives are decoupled , the generator optimizes a convex combination of the two objectives in ( 1 ) and ( 2 ) . The parameter α controls the importance of each of the two objective functions towards the global goal . So , for high values of α , the attack points will prioritize evading detection , rendering attacks with ( possibly ) a reduced effectiveness . Note that for α = 1 we have the same minimax game as in a standard conditional GAN ( Mirza & Osindero , 2014 ) . On the other hand , low values of α will result in attacks with higher impact in the classifier ’ s performance . However the generated poisoning points will be more detectable by outlier detection systems . For α = 0 , pGAN does not consider any detectability constraint and the generated poisoning points are only constrained by the output activation functions in the G. In this case pGAN can serve as a suboptimal approximation of the optimal attack strategies in ( Biggio et al. , 2012 ; Mei & Zhu , 2015 ; Muñoz-González et al. , 2017 ) where no detectability constraints are imposed . Similar to ( Goodfellow et al. , 2014 ) we train pGAN following a coordinated gradient-based strategy to solve the minimax problem in ( 3 ) . We sequentially update the parameters of the three components using mini-batch stochastic gradient descent/ascent . For the generator and the discriminator data points are sampled from the conditional distribution on the subset of poisoning labels Yp . For the classifier , honest data points are sampled from the data distribution including all the classes . A different number of iterations can be considered for updating the parameters of the three blocks . The details of the training algorithm are provided in Appendix A .

Summary: This paper introduces a new generative poisoning attack method against machine learning classifiers. The authors propose pGAN with three components to maximum the error of classification and guarantee undistinguished poisoning data for the discriminator. The experimental results show that the hyperparameter \alpha significantly affects the poisoning data distribution and pGAN leads to specific error in a classification task.