Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper introduces a novel backdoor attack against machine learning classifiers by systematizing the generation of the triggers through a class-conditional generative model which allows to introduce backdoors targeting an arbitrarily large number of classes. The attack assumes that the attacker is in control of the training process, so that the model is trained by optimizing a loss function that includes the learning of the generative model for the triggers. However, the attack can also be used via data poisoning, by training first a surrogate model to learn the generative model for the backdoor triggers, and then, poisoning the target classifiers using malicious data points generated with this generative model. Different experiments on computer vision benchmarks show the effectiveness of the proposed approach to introduce backdoors with a very high attack success rate and a very small loss in performance. 
strengths_and_weaknesses: Strengths: 
+ The attack strategy to systematize the backdoor attack allow to target a broader range of classes compared to existing backdoor attacks in the research literature, resulting in a stronger attack. 
+ The systematization of the attack by learning the trigger generator and the model at the same time is very interesting and enable the insertion of more stealthy backdoors assuming a threat model where the attacker is in control of the learning process. 
+ The transferability of the backdoors, so that malicious samples crafted with the generator can be used to insert backdoors in a different target model seems interesting, although the experiments are not very thorough compared to the rest (e.g. less datasets). 
+ The paper is easy to follow and, overall, well written and clear. 

Weaknesses: 
- Part of the experimental evaluation can be improved, especially as some direct comparisons are not really fair, as the attacker’s capabilities are different (see comments below). 
- It is unclear what is the loss in performance when Marksman is used on other models (Section 5.2.3). The evaluation of this aspect is not very thorough. 
- The positioning of the paper, in terms of the threat model considered, with respect to other papers in the related work can be improved. 

questions: + What are the types of trigger patterns used for performing the PatchMT attack? The type of triggers used for this class can have a significant impact in the effectiveness of the attack, especially, when using data augmentation techniques such as cropping, rotations and flips, and poisoning with low fraction of poisoning examples. 
+ In Table 4, what is the drop in performance for the different models evaluated on the clean data? 

(I included some suggestions below in the discussions of the limitations of the paper).

limitations: The approach proposed to insert the backdoors is interesting, as by solving the optimization problem in (3) it seems that the information of the backdoors is hidden better in the parameters of the model resulting after training. This enable two different attack scenarios: one where the attacker is in control of the whole training process (i.e., the attacker optimizes (3) to produce the model); and another scenario where the attacker first optimizes (3) to generate poisons that, then, are injected in the training set so that the training of the model is external to the attacker. 
Given this, the comparison in Tables 1 and 2 with PatchMT is somewhat not really fair, as PatchMT just rely on poisoning the training set. For this, I think it would be more appropriate to compare PatchMT in the case where Marksman is used to poison the training dataset (as in Section 5.2.3). 
Following the previous point, it is a bit strange that the authors did not report the loss in performance evaluated on the clean datasets in the models evaluated in Table 4. As per in my previous question, it would be useful to know if the loss in performance is higher when the backdoors are inserted via poisoning attacks. 
By looking at the results in the appendix, I think that bringing the comparisons with ReFoolMT and WaNetMT to the main paper (there is space in Tables 1 and 2, for example, to do that) would be beneficial to provide a more comprehensive evaluation in Section 5. 
About the defenses, the authors provide an interesting evaluation that evidences the limitations of current defenses to defend against Marksman. However, these defenses are designed for backdoors that are typically generated with poisoning attacks. It would be interesting to see if the defenses work to reduce the effectiveness of the Marksman attack when the attack is performed by poisoning the training dataset, and then, training the model. It would be interesting to see if there are significant different between the two threat models and, perhaps, the results can foster the investigation of more sophisticated defensive methods for backdoor attacks where the information of the backdoors is blended better in the parameters of the model.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a novel setting for backdoor attacks by injecting different triggers for different labels. It designs a class-condition generative trigger function. Then if given the target label, the well-trained generator T can generate an imperceptible trigger pattern to cause the model to predict the target label.
strengths_and_weaknesses: + The paper is generally well written and easy to understand.
+ The proposed method is interesting and has high novelty.
+ Datasets are adequate for backdoor attacks.

- The experimental results of PatchMT in Table 3 seem quite confusing for me. As far as I understand, injecting 10 different trigger patterns referring to 10 different labels in CIFAR-10 should have still high ASRs even with 10% poisoned images, in which situation 1% data is poisoned referring to one class. BadNets has shown injecting 1% poisoned data should have a high ASR.
- I have doubts about the poisoning procedure. I do understand the proposed Marksman needs to optimize T and then generate triggers using T. But most backdoor baseline methods need to only inject poisoned data and do not control the training procedure. So I would like to see the results using well-trained T to generate triggers and inject them into clean data, then train the backdoored models either from scratch or fine-tuning. 
questions: 1. Experimental results of PatchMT (in Table 3) need to be further explained.
2. Could the authors add some experiments by simply injecting the generated triggers into training data, and then train models from-scratch? The results and comparisons may not be fair enough now.
limitations: Yes,  the authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a backdoor attack called Marksman, with a more powerful
payload against deep neural networks. Specifically, it proposes to optimize a
trigger generative function to find an optimal trigger pattern for an
arbitrary target class while simultaneously injecting a generative backdoor
into the model. The results show that Marksman can attack any class at the
adversary's will during inference time, and it can bypass three existing
defenses.

strengths_and_weaknesses: Strengths:

Backdoor is a trendy topic. The paper is well written and states its
constrained optimization technique clearly.

Weaknesses: 

The difference between its proposed attack method and existing works is not
clearly explained. The comparison with baseline methods is limited. It lacks
ablation studies on other important parameters like learning rate and training
epoch.
questions: What is the fundamental difference between this work and existing works? This
is not well explained in the paper. Nguyen et al. "Input-aware dynamic
backdoor attack" from NeruIPS 2020 also consider a generic backdoor that is
input-specific. The paper seems to be a simple extension of this work in
multiple target class settings.

The technique is an improved version of GAN-based trigger generation
methods [8,9,28]. But during evaluation, it chooses to compare 
patch-based trigger patterns from [13] as its baseline. Is it possible to
extend the current GAN-based methods [8,9,23] and inject them into the model?
It seems to be a better baseline.

Moreover, the argument that injecting multiple triggers into a model will lead
to more significant perturbations has never been justified by experiments. Existing
works try to combine adversarial examples with backdoors to create invisible
triggers. Following this line of work, is it possible that multiple triggers
can have a reasonable size similar to a single one? GAN-based
method potentially can do the same (depending on the optimization results).
But it needs to be justified by the paper as it is making a claim here.

The work is only evaluated on post-training backdoor investigation methods.
There are other types of defenses, e.g., training time defense (NAD, MOTH)
and data cleaning as a defense (anti-backdoor learning). The paper does not evaluate 
these defenses. Adaptive defense is also not considered.
limitations: The limitation discussion focuses on backdoor "scanning" techniques, but there are more types of defenses under this threat model. I encourage the authors to discuss them with experiments.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this work, the authors investigate a novel backdoor attack paradigm, Marksman, where the adversary can arbitrarily choose which target class the model will misclassify given any input during inference. They use a class-conditional generative model to represent the trigger function and inject the backdoor in a constrained optimization framework. They verify their attack can achieve a high attack success rate in the experiments.
strengths_and_weaknesses: ## Strength

- The paper is well written and easy to follow.
- The conclusions are backed up with proper experiments.
- The authors proposed a novel backdoor attack paradigm, Marksman. This is the first multi-trigger and multi-payload backdoor attack with the capability of misclassifying an input to any target class.
- The authors experimentally proved that Marksman can be a threat in the ML community.

## Weakness

While this is an interesting paper, I find that there are several weaknesses:

1.  Although Marksman is the first work that studies multi-trigger and multi-payload backdoor with the capability of misclassifying an input to any target class, it is not clear what is the motivation behind Marksman for me. The authors state that Marksman can arbitrarily choose which target class the model will misclassify given any input during inference, more powerful than existing attacks. However, what is the advantage of Marksman? Why do we need Marksman? Since this is the main part of the paper, I would expect that there’s a more convincing discussion about the necessity of the proposed Marksman.
2. Only some early defenses are considered. However, there are more advanced backdoor defenses recently, e.g. NAD [1], ANP [2].

[1] Yige Li, et al. "Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks", ICLR 2021.

[2] Dongxian Wu, et al. "Adversarial Neuron Pruning Purifies Backdoored Deep Models", NeurIPS 2021.

-------------------------------------------------------------------------------------------------------------------------------------------------

I appreciate the authors for the detailed response, which addresses most of my concerns. However, the formulation and optimization of Marksman is inspired by LIRA [1] and thus the technical novelty is limited. Besides, in my opinion, the multi-target setting in this work seems not very practical. Putting it all together, I decided to increase my score to 5. 

[1] Khoa Doan, et al. "LIRA: Learnable, Imperceptible and Robust Backdoor Attacks", ICCV 2021
questions: As listed in Weakness
limitations: As listed in Weakness. I think that this is an interesting paper and score can be improved if concerns listed above are resolved.
review_title: null

Meta-review: metareview: This paper introduces a backdoor attack that allows an attacker to cause an input to become any target class, as opposed to just one target class. The reviewers mostly liked this paper and found the attack interesting and useful. The main weakness was in the comparison to prior work, but here the authors have addressed much of this in the comments (and I hope they will make the necessary adjustments in the final version of the paper). The paper also has a limited evaluation against backdoor defenses, but given the goal of this attack is to show a new attack technique and not evade existing defenses I believe it is okay to push this to future work.
confidence: Less certain
Decision: Accept