Dynamic Backdoor Attacks Against Deep Neural Networks

1 INTRODUCTION . Recent research has shown that deep neural network ( DNN ) models are vulnerable to various security and privacy attacks ( Papernot et al. , 2016 ; 2017 ; Shokri et al. , 2017 ; Salem et al. , 2019 ; 2020 ; Tramèr et al. , 2016 ; Oh et al. , 2018 ) . One such attack that receives a large amount of attention is backdoor , where an adversary trains a DNN model which can intentionally misclassify any input with an added trigger ( a secret pattern constructed from a set of neighboring pixels ) to a specific target label . Backdoor attacks can cause severe security consequences . For instance , an adversary can implant a backdoor in an authentication system to grant herself unauthorized access . Existing backdoor attacks generate static triggers , in terms of fixed trigger pattern and location ( on the model input ) . For instance , Figure 1a shows an example of triggers constructed by BadNets ( Gu et al. , 2017 ) , one popular backdoor attack method , on the CelebA dataset ( Liu et al. , 2015 ) . As we can see , BadNets in this case uses a white square as a trigger and always places it in the top-left corner of an input . This static nature of triggers has been leveraged to create most of the current defenses against the backdoor attack ( Wang et al. , 2019 ; Liu et al. , 2019a ; Gao et al. , 2019 ) . In this paper , we propose the first class of backdooring techniques against deep neural network models that generate dynamic triggers , in terms of trigger pattern and location . We refer to our techniques as dynamic backdoor attacks . Figure 1b shows an example . Dynamic backdoor attacks offer the adversary more flexibility , as they allow triggers to have different patterns and locations . Moreover , our techniques largely reduce the efficacy of the current defense mechanisms demonstrated by our empirical evaluation . In addition , we extend our techniques to work for all labels of the backdoored DNN model , while the current backdoor attacks only focus on a single or a few target labels . This further increases the difficulty of our backdoors being mitigated . In total , we propose 3 different dynamic backdoor techniques , i.e. , Random Backdoor , Backdoor Generating Network ( BaN ) , and conditional Backdoor Generating Network ( c-BaN ) . In particular , the latter two attacks algorithmically generate triggers to mount backdoor attacks which are first of their kind . To demonstrate the effectiveness of our proposed techniques , we perform empirical analysis with three DNN model architectures over three benchmark datasets . All of our techniques achieve almost a perfect backdoor accuracy , i.e. , the accuracy of the backdoored model on the backdoored data is approximately 100 % , with a negligible utility loss . Moreover , we show that our techniques can bypass three state-of-the-art backdoor defense techniques , namely Neural Cleanse ( Wang et al. , 2019 ) , ABS ( Liu et al. , 2019a ) , and STRIP ( Gao et al. , 2019 ) . In general , our contributions can be summarized as follows : 1 ) We broaden the class of backdoor attacks by introducing the dynamic backdoor attacks . 2 ) We propose both BaN and c-BaN , which are the first algorithmic backdoor paradigm . 3 ) Our dynamic backdoor attacks achieve strong performance while bypassing the current state-of-the-art backdoor defense techniques . 2 RELATED WORK . Backdoor Attacks : Gu et al . ( Gu et al. , 2017 ) introduce BadNets , the first backdoor attack on DNN models . BadNets uses the MNIST dataset and a square-like trigger with a fixed location , to show the applicability of the backdoor attacks in the DNN settings . Liu et al . ( Liu et al. , 2019b ) later propose a more advanced backdooring technique , namely the Trojan attack . They simplify the threat model of BadNets by eliminating the need for access to the training data used to train the target model . The main difference between these two attacks ( BadNets and Trojan attacks ) and our work is that both attacks only consider static backdoors in terms of triggers ’ pattern and location . Our work extends the backdoor attacks to consider dynamic patterns and locations of the triggers . We focus on backdoor attacks against image classification models , but backdoor attacks can be extended to other scenarios , such as Federated Learning ( Wang et al. , 2020 ) , Video Recognition ( Zhao et al. , 2020 ) , Transfer Learning ( Yao et al. , 2019 ) , and Natural language processing ( NLP ) ( Chen et al. , 2020 ) . To increase the stealthiness of the backdoor , Saha et al . ( Saha et al. , 2020 ) propose to transform the backdoored images into benign-looking ones , which makes them harder to detect . Lie et al . ( Liu et al. , 2020 ) introduce another approach , namely , the reflection backdoor ( Refool ) , which hides the triggers using mathematical modeling of the physical reflection property . Another line of research focuses on exploring different methods of implementing backdoors into target models . Rakin et al . ( Rakin et al. , 2020 ) introduce the Targeted Bit Trojan ( TBT ) technique , which instead of training the target model , flips some bits in the target models ’ weights to make it misclassify all the inputs . Tang et al . ( Tang et al. , 2020 ) present a different approach , where the adversary appends a small Trojan module ( TrojanNet ) to the target model instead of fully retraining it . Defenses Against Backdoor Attacks : Wang et al . ( Wang et al. , 2019 ) propose Neural Cleanse ( NC ) , a backdoor defense method based on reverse engineering . For each output label , NC tries to generate the smallest trigger , which converts the output of all inputs applied with this trigger to that label . NC then uses anomaly detection to find if any of the generated triggers are actually a backdoor or not . Later , Liu et al . ( Liu et al. , 2019a ) propose another model-based defense , namely , ABS . ABS detects if a target model contains a backdoor or not , by analyzing the behaviour of the target model ’ s inner neurons when introducing different levels of stimulation . Also , Gao et al . ( Gao et al. , 2019 ) propose STRIP , a backdoor defense method based on manipulating the input , to find out if it is backdoored or not . More concretely , STRIP fuses the input with multiple clean data , one at a time . Then it queries the target model with the generated inputs , and calculate the entropy of the output labels . Backdoored inputs tend to have lower entropy than the clean ones . Besides the above , there are multiple other types of attacks against deep neural network models , such as adversarial examples ( Vorobeychik & Li , 2014 ; Carlini & Wagner , 2017 ; Li & Vorobeychik , 2015 ; Tramèr et al. , 2017 ; Xu et al. , 2018 ) , poisoning attack ( Jagielski et al. , 2018 ; Suciu et al. , 2018 ; Biggio et al. , 2012 ) , and property inference ( Ganju et al. , 2018 ; Melis et al. , 2019 ) . 3 DYNAMIC BACKDOORS . 3.1 THREAT MODEL . The dynamic backdoor attack is a training time attack , i.e. , the adversary is the one who trains the backdoored DNN model . To implement our attack , we assume the adversary controls the training of the target model and has access to the training data following previous works ( Gu et al. , 2017 ; Yao et al. , 2019 ; Chen et al. , 2020 ) . To launch the attack ( after publishing the model ) , the adversary first adds a trigger to the input and then uses it to query the backdoored model . This can happen either digitally , where the adversary digitally adds the trigger to the image , or physically , where the adversary prints the trigger and places it on the image . This added trigger makes the backdoored model misclassify the input to the target label . 3.2 RANDOM BACKDOOR . We start by presenting our Random Backdoor technique . Abstractly , the Random Backdoor technique constructs triggers by sampling them from a uniform distribution , and adding them to the inputs at random locations . We first introduce using Random Backdoor to implement a dynamic backdoor for a single target label , then we generalize it to multiple target labels . Single Target Label : In this setting , we construct a set of triggers ( T ) and a set of possible locations ( K ) , such that for any trigger sampled from T and added to any input at a random location sampled from K , the model will output the specified target label . More formally , for any location κi ∈ K , any trigger ti ∈ T , and any input xi ∈ X : Mbd ( A ( xi , ti , κi ) ) = ` , where ` is the target label , T is the set of triggers , K is the set of locations , Mbd is the backdoored model , and A is the backdoor adding function ( it adds the trigger ti to the input xi at the location κi ) . To implement such a backdoor , an adversary needs to first select her desired trigger locations and create the set of possible locations K. Then , she uses both clean and backdoored data to update the model for each epoch similar to the BadNets technique but with two differences . First , instead of using a fixed trigger for all inputs , each time the adversary wants to add a trigger to an input , she samples a new trigger from a uniform distribution , i.e. , t ∼ U ( 0 , 1 ) . Here , the set of possible triggers T contains the full range of all possible values for the triggers , since the trigger is randomly sampled from a uniform distribution . Second , instead of placing the trigger in a fixed location , she places it at a random location κ , sampled from the predefined set of locations , i.e. , κ ∈ K. Note that Random Backdoor is not only limited to uniform distribution , other distributions , such as Gaussian distribution , can be used as well . Multiple Target Labels : Next , we consider the more complex case of having multiple target labels . Without loss of generality , we consider implementing a backdoor for each label in the dataset . This means that for any label ` i ∈ L , there exists a trigger t which when added to the input x at a location κ , will make the modelMbd output ` i . More formally , ∀ ` i ∈ L ∃ t , κ : Mbd ( A ( x , t , κ ) ) = ` i . To achieve the dynamic backdoor behaviour , each target label should have a set of possible triggers and a set of possible locations . We generalize the Random Backdoor technique by dividing the set of possible locations K into disjoint subsets for each target label , while keeping the trigger construction method the same as in the single target label case , i.e. , the triggers are still sampled from a uniform distribution . For instance , for the target label ` i , we sample a set of possible locations Ki , where Ki is subset of K ( Ki ⊂ K ) . The adversary can construct the disjoint sets in two steps . First , the adversary selects all possible triggers locations and constructs the set K. Second , for each target label ` i , she constructs the set of possible locations for this label Ki by sampling the set K. Then , she removes the sampled locations from the set K. We propose a simple algorithm to assign the locations for the different target labels . Concretely , we uniformly split the image into non-intersecting regions , and assign a region for each target label , in which the triggers ’ locations can move vertically . Figure 2 shows an example of our location setting technique for a use case with 6 target labels . We stress that this is one way of dividing the location set K to the different target labels . However , an adversary can choose a different way of splitting the locations inside K to the different target labels . The only requirement the adversary has to fulfill is to avoid assigning a location for different target labels . Later , we will show how to overcome this limitation with our more advanced c-BaN technique .

Summary: This paper outlines improved backdoor attacks for deep neural networks. Backdoor attacks are training-time attacks whereby an adveresary trains a network in such a way that it functions as a classifier on honestly generated images, but it has the added caveat of also being able to misclassify on images corrupted via a "trigger" set by an adversary. This is a grave security concern, and as such there is substantial literature that relates to both systematically training backdoored DNNs, and also detecting when a trained DNN has a backdoor.