Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper proposed a "learnable" trojan by training a neural network that takes a sample (e.g. an image) as an input and generates a programmable trigger pattern as an output. The authors argue that the proposed method can support dynamic and out scope target classes, which are particularly applicable to backdoor attacks in the transfer learning setting. The authors conducted experiments on large-scale models (ImageNet) in two settings: (1) outsourced training attack;  (2) transfer learning attack.

Although the idea of making a backdoor attack more robust (e.g. more transferrable) and programmable is interesting, I don't think the current results fully substantiate the claimed benefits. Below are my concerns:

1. Lack of performance comparison: in the outsourced training attack, the attack success rate is quite low ( the best top-1 attack rate is ~50% on VGG). In the existing literature, the backdoor attack success rate can be made nearly 100%. This makes me wonder whether the low attack success rate only occurs in the proposed attacking method. Since there are no results from existing attacks, there is no way to evaluate how good the proposed attack is.

2. How about small dataset/model? I understand that the authors want to emphasize the scalability of their attack to large models like ImageNet. However, there are no comparisons on ImageNet (my comment 1). The authors are suggested to compare performance on standard datasets in backdoor attack literature (e.g. CIFAR-10, traffic sign)

3. It was unclear how "dynamic" the proposed method can be. Based on the attack formulation in equation (3), in order to make the attack "dynamic" in terms of changing different target classes, the attackers need to train a neural trojan network for every target class, which does not seem to be dynamic to me. Can the authors further justify the advantage of the dynamic feature in the proposed attack? And I have concerns about how many target classes an attacker can "dynamically" change. Some experiments showing the number of target classes vs attack performance and clean data accuracy will be very helpful.

4. The defense argument against detection methods is weak. Unless the authors can show the proposed attack has the ability to simultaneously backdoor all possible target classes, simply arguing the attack is dynamic and thus can evade detection is not convincing, not to mention in the backdoor setting, attacker should make the first move before the defender takes action.

*** Post-rebuttal comments
I thank the authors for the clarification. However, I feel my comments have not been fully addressed, especially on the part on justifying 50% attack success rate on VGG should be considered as significant in the considered setup. Without any valid comparisons, I find it difficult to assess the contributions. In addition, the authors did not add new empirical evidence regarding my questions but mainly re-iterated the applicability of the proposed method, so I will remain my review rating.
***
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposes a general framework for constructing Trojan/Backdoor attacks on deep neural networks, specifically in cases where the end user plans to perform transfer learning on the backdoored classifier. To accomplish this, this work proposes the use of a trigger generator that is trained alongside the trojaned network, that allows the attacker to cause any image to be misclassified as any other. The proposed method is general, and is shown to work across a variety of datasets.

Although the threat model here is slightly less general than the standard backdoor attacks threat model (in that the adversary here also needs to be involved in training the model, and not just supplying the data), the authors do well to motivate the threat model, and the attack is sufficiently general to be an interesting security model. The method is, to the best of my knowledge, novel, and is an innovative way to launch trigger attacks even when the adversary does not know what the final classes or task will be. The evaluation is thorough enough, though the results themselves could be better. However, as this is the first attack of this kind, the results seem sufficient to warrant publication.

My major concern with this work is readability---many of the sections are in need of significant proofreading, and there are many grammatical/word choice mistakes that make the paper somewhat difficult to read. (For example, "P is derivative" in Section 4.2 should probably say "P is differentiable"; the use of frontend/backend is somewhat counterintuitive, as the backend should be the what is given to victims, and the victim trains a frontend reliant on the backend). The method itself could also be presented more clearly (section 4.2 specifically could use less equations and more exposition). Overall the paper requires significant written revision in order to be up to the standard of publication, but given that these concerns can be addressed in the revision period I am (weakly) recommending acceptance.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes an adaption of existing backdoor attacks, with the main goal of enabling backdoor attacks in the transfer learning setting. Specifically, instead of pre-defining the trigger pattern and the target label, they train a neural network to generate different trigger patterns for different target images, so that after the source image is blended with the generated trigger pattern, it will be classified in the same way as the target image. This formulation makes it possible to  generate backdoor attacks that stay effective in the transfer learning setting, when the label set of the fine-tuning task is different from the original task. They evaluate their approach using pre-trained models on ImageNet, and also show transfer learning results using two smaller datasets.

I think studying the effectiveness of backdoor attacks under the transfer learning setting is a good topic. However, I am not convinced that the proposed approach is necessary a good way to do so, and have the following questions:

1. To train the trigger generator, do the authors only train it on the pre-trained dataset, or the images of the downstream task is also used? If training does not use the images from the downstream task at all, then it is interesting that the generator can generalize well, which may suggest that although the downstream task has a different prediction goal, the input images themselves share some similarity to the pre-trained task. Could the authors provide some explanation on it?

2. For the attack success rate, I would like to see more analysis on how the choice of the source and target images affect the attack performance. Specifically, if the source and target images have the same label on the pre-trained task, but different on the downstream task, is it easier or harder to generate successful backdoors for the downstream task? Similarly, what if the source and target images have different labels on the pre-trained task, but the same label on the downstream task?

3. Is it necessary to generate different trigger patterns for every different target image? Is it possible to use the same trigger pattern for multiple target images, at least in the case when they look similar? In general, backdoor attacks would expect that the same trigger pattern can be re-used among different input instances.

-------------
Post-rebuttal comments

Thanks for your response! However, I still think the evaluation is weak, given that the attack success rate is low, and the current version of the paper does not provide a good justification for both the design choices and the empirical results. Therefore, I keep my original assessment.
------------
review_title: Official Blind Review #2

Meta-review: This paper proposes a general framework for constructing Trojan/Backdoor attacks on deep neural networks. The authors argue that the proposed method can support dynamic and out-of-scope target classes, which are particularly applicable to backdoor attacks in the transfer learning setting. This paper has been very carefully discussed. While the idea is interesting and could be of interest to the broader community, all reviewers agree that it lacks of experimental comparison with existing methods for backdoor attacks on benchmark problems. The paper needs to be significantly revised before publication. I encourage the authors to improve this paper and resubmit to future conference.
Decision: Reject