Dynamic Backdoor Attacks Against Deep Neural Networks

1 INTRODUCTION . Recent research has shown that deep neural network ( DNN ) models are vulnerable to various security and privacy attacks ( Papernot et al. , 2016 ; 2017 ; Shokri et al. , 2017 ; Salem et al. , 2019 ; 2020 ; Tramèr et al. , 2016 ; Oh et al. , 2018 ) . One such attack that receives a large amount of attention is backdoor , where an adversary trains a DNN model which can intentionally misclassify any input with an added trigger ( a secret pattern constructed from a set of neighboring pixels ) to a specific target label . Backdoor attacks can cause severe security consequences . For instance , an adversary can implant a backdoor in an authentication system to grant herself unauthorized access . Existing backdoor attacks generate static triggers , in terms of fixed trigger pattern and location ( on the model input ) . For instance , Figure 1a shows an example of triggers constructed by BadNets ( Gu et al. , 2017 ) , one popular backdoor attack method , on the CelebA dataset ( Liu et al. , 2015 ) . As we can see , BadNets in this case uses a white square as a trigger and always places it in the top-left corner of an input . This static nature of triggers has been leveraged to create most of the current defenses against the backdoor attack ( Wang et al. , 2019 ; Liu et al. , 2019a ; Gao et al. , 2019 ) . In this paper , we propose the first class of backdooring techniques against deep neural network models that generate dynamic triggers , in terms of trigger pattern and location . We refer to our techniques as dynamic backdoor attacks . Figure 1b shows an example . Dynamic backdoor attacks offer the adversary more flexibility , as they allow triggers to have different patterns and locations . Moreover , our techniques largely reduce the efficacy of the current defense mechanisms demonstrated by our empirical evaluation . In addition , we extend our techniques to work for all labels of the backdoored DNN model , while the current backdoor attacks only focus on a single or a few target labels . This further increases the difficulty of our backdoors being mitigated . In total , we propose 3 different dynamic backdoor techniques , i.e. , Random Backdoor , Backdoor Generating Network ( BaN ) , and conditional Backdoor Generating Network ( c-BaN ) . In particular , the latter two attacks algorithmically generate triggers to mount backdoor attacks which are first of their kind . To demonstrate the effectiveness of our proposed techniques , we perform empirical analysis with three DNN model architectures over three benchmark datasets . All of our techniques achieve almost a perfect backdoor accuracy , i.e. , the accuracy of the backdoored model on the backdoored data is approximately 100 % , with a negligible utility loss . Moreover , we show that our techniques can bypass three state-of-the-art backdoor defense techniques , namely Neural Cleanse ( Wang et al. , 2019 ) , ABS ( Liu et al. , 2019a ) , and STRIP ( Gao et al. , 2019 ) . In general , our contributions can be summarized as follows : 1 ) We broaden the class of backdoor attacks by introducing the dynamic backdoor attacks . 2 ) We propose both BaN and c-BaN , which are the first algorithmic backdoor paradigm . 3 ) Our dynamic backdoor attacks achieve strong performance while bypassing the current state-of-the-art backdoor defense techniques . 2 RELATED WORK . Backdoor Attacks : Gu et al . ( Gu et al. , 2017 ) introduce BadNets , the first backdoor attack on DNN models . BadNets uses the MNIST dataset and a square-like trigger with a fixed location , to show the applicability of the backdoor attacks in the DNN settings . Liu et al . ( Liu et al. , 2019b ) later propose a more advanced backdooring technique , namely the Trojan attack . They simplify the threat model of BadNets by eliminating the need for access to the training data used to train the target model . The main difference between these two attacks ( BadNets and Trojan attacks ) and our work is that both attacks only consider static backdoors in terms of triggers ’ pattern and location . Our work extends the backdoor attacks to consider dynamic patterns and locations of the triggers . We focus on backdoor attacks against image classification models , but backdoor attacks can be extended to other scenarios , such as Federated Learning ( Wang et al. , 2020 ) , Video Recognition ( Zhao et al. , 2020 ) , Transfer Learning ( Yao et al. , 2019 ) , and Natural language processing ( NLP ) ( Chen et al. , 2020 ) . To increase the stealthiness of the backdoor , Saha et al . ( Saha et al. , 2020 ) propose to transform the backdoored images into benign-looking ones , which makes them harder to detect . Lie et al . ( Liu et al. , 2020 ) introduce another approach , namely , the reflection backdoor ( Refool ) , which hides the triggers using mathematical modeling of the physical reflection property . Another line of research focuses on exploring different methods of implementing backdoors into target models . Rakin et al . ( Rakin et al. , 2020 ) introduce the Targeted Bit Trojan ( TBT ) technique , which instead of training the target model , flips some bits in the target models ’ weights to make it misclassify all the inputs . Tang et al . ( Tang et al. , 2020 ) present a different approach , where the adversary appends a small Trojan module ( TrojanNet ) to the target model instead of fully retraining it . Defenses Against Backdoor Attacks : Wang et al . ( Wang et al. , 2019 ) propose Neural Cleanse ( NC ) , a backdoor defense method based on reverse engineering . For each output label , NC tries to generate the smallest trigger , which converts the output of all inputs applied with this trigger to that label . NC then uses anomaly detection to find if any of the generated triggers are actually a backdoor or not . Later , Liu et al . ( Liu et al. , 2019a ) propose another model-based defense , namely , ABS . ABS detects if a target model contains a backdoor or not , by analyzing the behaviour of the target model ’ s inner neurons when introducing different levels of stimulation . Also , Gao et al . ( Gao et al. , 2019 ) propose STRIP , a backdoor defense method based on manipulating the input , to find out if it is backdoored or not . More concretely , STRIP fuses the input with multiple clean data , one at a time . Then it queries the target model with the generated inputs , and calculate the entropy of the output labels . Backdoored inputs tend to have lower entropy than the clean ones . Besides the above , there are multiple other types of attacks against deep neural network models , such as adversarial examples ( Vorobeychik & Li , 2014 ; Carlini & Wagner , 2017 ; Li & Vorobeychik , 2015 ; Tramèr et al. , 2017 ; Xu et al. , 2018 ) , poisoning attack ( Jagielski et al. , 2018 ; Suciu et al. , 2018 ; Biggio et al. , 2012 ) , and property inference ( Ganju et al. , 2018 ; Melis et al. , 2019 ) . 3 DYNAMIC BACKDOORS . 3.1 THREAT MODEL . The dynamic backdoor attack is a training time attack , i.e. , the adversary is the one who trains the backdoored DNN model . To implement our attack , we assume the adversary controls the training of the target model and has access to the training data following previous works ( Gu et al. , 2017 ; Yao et al. , 2019 ; Chen et al. , 2020 ) . To launch the attack ( after publishing the model ) , the adversary first adds a trigger to the input and then uses it to query the backdoored model . This can happen either digitally , where the adversary digitally adds the trigger to the image , or physically , where the adversary prints the trigger and places it on the image . This added trigger makes the backdoored model misclassify the input to the target label . 3.2 RANDOM BACKDOOR . We start by presenting our Random Backdoor technique . Abstractly , the Random Backdoor technique constructs triggers by sampling them from a uniform distribution , and adding them to the inputs at random locations . We first introduce using Random Backdoor to implement a dynamic backdoor for a single target label , then we generalize it to multiple target labels . Single Target Label : In this setting , we construct a set of triggers ( T ) and a set of possible locations ( K ) , such that for any trigger sampled from T and added to any input at a random location sampled from K , the model will output the specified target label . More formally , for any location κi ∈ K , any trigger ti ∈ T , and any input xi ∈ X : Mbd ( A ( xi , ti , κi ) ) = ` , where ` is the target label , T is the set of triggers , K is the set of locations , Mbd is the backdoored model , and A is the backdoor adding function ( it adds the trigger ti to the input xi at the location κi ) . To implement such a backdoor , an adversary needs to first select her desired trigger locations and create the set of possible locations K. Then , she uses both clean and backdoored data to update the model for each epoch similar to the BadNets technique but with two differences . First , instead of using a fixed trigger for all inputs , each time the adversary wants to add a trigger to an input , she samples a new trigger from a uniform distribution , i.e. , t ∼ U ( 0 , 1 ) . Here , the set of possible triggers T contains the full range of all possible values for the triggers , since the trigger is randomly sampled from a uniform distribution . Second , instead of placing the trigger in a fixed location , she places it at a random location κ , sampled from the predefined set of locations , i.e. , κ ∈ K. Note that Random Backdoor is not only limited to uniform distribution , other distributions , such as Gaussian distribution , can be used as well . Multiple Target Labels : Next , we consider the more complex case of having multiple target labels . Without loss of generality , we consider implementing a backdoor for each label in the dataset . This means that for any label ` i ∈ L , there exists a trigger t which when added to the input x at a location κ , will make the modelMbd output ` i . More formally , ∀ ` i ∈ L ∃ t , κ : Mbd ( A ( x , t , κ ) ) = ` i . To achieve the dynamic backdoor behaviour , each target label should have a set of possible triggers and a set of possible locations . We generalize the Random Backdoor technique by dividing the set of possible locations K into disjoint subsets for each target label , while keeping the trigger construction method the same as in the single target label case , i.e. , the triggers are still sampled from a uniform distribution . For instance , for the target label ` i , we sample a set of possible locations Ki , where Ki is subset of K ( Ki ⊂ K ) . The adversary can construct the disjoint sets in two steps . First , the adversary selects all possible triggers locations and constructs the set K. Second , for each target label ` i , she constructs the set of possible locations for this label Ki by sampling the set K. Then , she removes the sampled locations from the set K. We propose a simple algorithm to assign the locations for the different target labels . Concretely , we uniformly split the image into non-intersecting regions , and assign a region for each target label , in which the triggers ’ locations can move vertically . Figure 2 shows an example of our location setting technique for a use case with 6 target labels . We stress that this is one way of dividing the location set K to the different target labels . However , an adversary can choose a different way of splitting the locations inside K to the different target labels . The only requirement the adversary has to fulfill is to avoid assigning a location for different target labels . Later , we will show how to overcome this limitation with our more advanced c-BaN technique .

Summary: This paper proposed a class of methods for dynamic backdoor attack. The main idea is to generate different backdoor patterns and locations in backdoor attack. The threat model is the attacker has full access to the training data and the model training procedure. Both single-target and multi-target class-conditional triggers were explored. Experimental results on three datasets in the 100% poisoning setting verified the effectiveness of the proposed attack.