Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary and contributions:
In this work, the authors propose the first set of dynamic backdoor attacks, in terms of trigger pattern and location. Three techniques which areRandom Backdoor, Backdoor Generating Network (BaN) and conditional BackdoorGenerating Network (c-BaN) are used to implement dynamic backdoor attacks. The results of empirical evaluation show that the proposed attacks are effective as well as difficult to defend by several defense mechanisms.

Strength (significance and novelty):
1. They propose the first class of dynamic backdoor attacks. And this work is well motivated. Previous backdoor attacks only use a fixed and static backdoor trigger, which is set to the default assumption of several defense mechanisms. Dynamic backdoors chosen from a set of predefined triggers and locations or generated byGAN, can bypass these defense mechanisms.
2.  In the empirical evaluation, the proposed attack algorithm is effective varying different datasets and is robust to several defense mechanisms.
3. In addition to single target label and multiple target label attacks, they also propose an auto-encoder based defense against dynamic backdoor attacks.
4. The techniques in dynamic backdoor attacks are simple yet effective. And the paper is well-written and easy to follow.

Weakness:
1. This work only considers white-box backdoor attacks, i.e., the attacker controls the target model and has access to the training data. I suggest the authors should refer Refool [1] and study the transferability of BaN and c-BaN to different data distributions and model architectures.
2. This work aims to improve backdoor robustness to defense mechanisms. However, theSec 4.5 state-of-the-art defenses is too brief. More experiments and illustrations should be added to this part. Here are some suggestions:

3. Since the motivation of this work is to “break” the assumption in some model-based defenses, it would be interesting to compare the reversed triggers and the original triggers (sample from the dynamic trigger distribution). In addition to Neural Cleanse and ABS, MESA [2] can also be considered for its superior to Neural Cleanse.

4. Add some visualizations to understand dynamic backdoor attacks, e.g. Grad-CAM,  which is widely adopted in backdoor attacks [1, 3].

5. Add a comparison with traditional static backdoor attacks against different defense mechanisms, which would be more convincing.

Reference
1. Liu, et al. Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks.ECCV 2020.
2. Qiao, et al. Defending Neural Backdoors via Generative Distribution Modeling. NeurIPS2019.
3.  Xie, et al. DBA: Distributed Backdoor Attacks against Federated Learning. ICLR 2020.
review_title: review of dynamic backdoor

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

This paper outlines improved backdoor attacks for deep neural networks. Backdoor attacks are training-time attacks whereby an adveresary trains a network in such a way that it functions as a classifier on honestly generated images, but it has the added caveat of also being able to misclassify on images corrupted via a "trigger" set by an adversary. This is a grave security concern, and as such there is substantial literature that relates to both systematically training backdoored DNNs, and also detecting when a trained DNN has a backdoor. 

As the authors mention, the existing literature on backdoors typically has a static backdoor where a single portion of an image is altered to create a trigger in a deterministic fashion / location. The authors improve the functinoality of backdooring a DNN by creating a training process where triggers can now appear at random locations in the image and can have random contents. The way they do so is by judiciously and simultaneously training a GAN in conjunction to the DNN that will be corrupted. The GAN uses the DNN under training as a discriminator and ultimately produces corrupted images with triggers that are meant to push the DNN towards having a backdoor through typical training procedures. These GAN-based techniques are key to both algorithms the authors present: BaN (random location triggers) and c-BaN (random location and contents to trigger). 

Finally, the authors validate how effective their backdoor techniques are by training on MNIST, CelebA and CIFAR-10 datasets. Their backdoored DNN have their performance measured by how well they misclassify on triggered images and on how little they compormise on state of the art accuracy in non-backdoored models. The authors show that their models have a small sacrifice in terms of accuracy (although their model interestingly enough classifies better on CelebA), and furthermore, their backdoors are robust to existing methods used to catch backdoored DNNs


Pros:
I was not as familiar with existing literature on backdoored DNNs until reading this paper. I see how this is a valid concern in the community, and I believe that the authors have provided a better attack with respect to the state of the art. Furthermore, their simulations do a good job of providing evidence of this improvement (in terms of both the visual output of the triggers, and also the performance against software used to find backdoored DNNs)

Questions / Points to address:
-As someone who is not as conversant with relevant literature on backdoored DNNs, I wonder how much work has been done on the fact that the triggers used are easily identifiable by a human looking at the picture (in termss of a blatant square in a given location). Though I am also no expert, there is a mature field of Steganography that focuses on mathematical methods of concealing information in images (and other media files). Is there significant overlap in what has been studied with this? Can the DNN architecture used be informed by existing theory in this field?
-I am also curious about the fact that the backdoored classifierse performed better on CelebA vs. other data sets. I see the authors' point about this possibly being a form of regularisation, but it would be interesting to see the performance of this methodology on more complex datasets that may also exhibit this behaviour. 
review_title: Is it possible to improve backdoor attacks to human perception (say via steganography?)

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper proposed a class of methods for dynamic backdoor attack. The main idea is to generate different backdoor patterns and locations in backdoor attack. The threat model is the attacker has full access to the training data and the model training procedure. Both single-target and multi-target class-conditional triggers were explored. Experimental results on three datasets in the 100% poisoning setting verified the effectiveness of the proposed attack.

Strengths:
1. This paper investigated an important problem for secure deep learning.
2. The proposed methods were well motivated.
3. Empirical results indicate that dynamic backdoor triggers are possible.

Weaknesses:
1. Missing many existing works on backdoor attacks. The related work section only mentioned two backdoor attacks Gu et al., 2017 and Liu et al. 2019b. To my knowledge, there are many existing works in this field.
2. The discussion of the key properties for backdoor triggers are weak. For example, the injection rate, and the stealthiness of the trigger pattern. Existing works have proposed reflection triggers or even invisible triggers.  Compared to these triggers, the proposed triggers are not stealthy at all. Appearing in different places of the input image even makes the trigger more easily detectable (at least to human checkers).
3. The proposed methods need to poison at least 30% of the training data, however, many existing methods only poison 10%-20% of the training data. 
4. The 6 training steps described on Page 6 indicate that the proposed methods need to modify the model training process. This is different from a typical backdoor setting where the attacker cannot control the training procedure, which allows the attacker to invade commercial models by only poisoning the data. From this perspective, the setting adopted in this paper is questionable.
5. The 30% poisoning setting in Section 4.6 is not clear. Was the 30% only used for training the BaN then poisoning 30% data using the BaN model or does it still need the entire training data (but only 30% contain the trigger)? The training process on Page 6 says the BaN and the backdoored model need to be trained simultaneously (step 6).
6. One more concern, the training of BaN nets needs a certain proportion of the training data. Even 30% is still a lot for large-scale datasets like ImageNet. However, many existing attacks do not need a generator, which makes the attack more applicable and convenient. Can the generators trained on one dataset transferable to other datasets?

---
Comments after the rebuttal:

Overall, the attack settings are still questionable: low stealthiness, high poisoning rate, and intervening the training process. 

- The authors clarified that "Neither our random backdoor technique nor the transferred BaN and c-BaN requires that", however, in 3.1 Threat Model, it is clearly stated "The dynamic backdoor attack is a training time attack, i.e., the adversary is the one who trains the backdoored DNN model. To implement our attack, we assume the adversary controls the training of the target model and has access to the training data following previous works (this is actually not true)", the authors seem misunderstood the different between "controlling the training of the model" to data poisoning.

In terms of the difference to static trigger,

- The authors argued that they are different to static trigger and "a static trigger cannot be used as a dynamic trigger", which I don't find it is verified somewhere. Actually the random backdoor proposed in this paper proves that a static trigger when applied to a random location still works very well, even as good as BaN and c-BaN. Whether existing triggers can already achieve a dynamic effect or not is unclear, or verified.

Without knowing how hard it is to make a static trigger works in a dynamic manner, or where it will completely fail, it is hard to evaluate the contribution and novelty of the proposed attacks. In other words, it seems that a static trigger attached to random locations is good enough to achieve the dynamic purpose, which I don't think is contributive enough.

Stealthiness and poisoning rate is another concern, say when you apply the BadNets to random locations, how high the poisoning rate will be to achieve 100% ASR? Higher than 20% or need 100%?

Overall, the lack of thorough comparisons of the trigger properties to existing attacks is the major weekness.

So my rating will stay the same.
review_title: Interesting idea, questionable threat model, poor literature review

Meta-review: The authors present a new set of trigger based backdoor attacks that use dynamic patterns that make detection harder. These attacks seem to be stronger with regards to state of the art attacks. 

Some weaknesses is the need for full whitebox access of the model. Several key references are missing, and the comparison with other backdoor attacks in unclear.

Moreover, although the authors compare with trigger based backdoors, there are plenty of triggerless backdoors that can be viewed as dynamic, eg the attacks by 
Bagdasaryan et al. http://proceedings.mlr.press/v108/bagdasaryan20a.html and works referencing this paper.

The paper indeed provides an interesting path towards dynamic attacks, but the lack of comparisons with state of the art literature, and also the need for whitebox access/high poisoning rate significantly limit the novelty of this work. 


Decision: Reject