Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors proposed a minimax formulation for removing backdoors from a given poisoned model based on a small set of clean data. Unlike previous work, which breaks down the minimax into separate inner and outer problems, the proposed algorithm utilizes the implicit hypergradient to account for the interdependence between inner and outer optimization. The authors also theoretically analyzed its convergence and the generalizability of the robustness gained by solving minimax on clean data to unseen test data. Extensive experiments showed improved backdoor defense performances and less computation time on several backdoor attacks over various attack settings.
main_review: 1 . The min-max formulation of backdoor removal is quite intuitive, however, the perturbation constraint is a bit hard to quantify. The authors adopt a simple L2 norm constraint, which definitely makes it easier for computation, yet it also constrains the defense effect to only L2 norm bounded triggers. For example, I am not quite sure whether the proposed method can effectively handle the hidden trigger attack in [1]. Maybe extending the framework to also Linf norm would be beneficial.

[1] "Hidden trigger backdoor attacks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 07. 2020.

2 . The derivation to eq (4) is straightforward. The authors mentioned that “it can be implemented in a memory-efficient manner”, however, I didn’t find any description on how this memory-efficient manner was implemented? From eq (4), it seems that one still needs to compute the second-order derivatives of H, which can still take quite a large memory? 

3 . The convergence results seem to directly follow (Grazzi et al., 2020) with similar assumptions and conclusions. I wonder what is the uniqueness of this result presented in the paper?

4.  The trigger removal experiments in Table 1 showed that the proposed algorithm is indeed more comprehensive, however, it does not achieve the best performances on many test cases, especially in terms of ASR.

5. In Table 5, where the authors showed the relationship between performances and the number of clean samples. While most baselines seem to achieve worse results when the number of clean samples decreases, yet the proposed method actually got slightly better ASR with a small portion of clean samples. This is actually quite counterintuitive. Will that be due to the random choice of clean samples? The authors might want to take a look into the details.

Typo: Above Table 5, “extrema” -> “extreme”

summary_of_the_review: The paper is in general well written. However, there still exists some concerns towards the applicability of the proposed method as well as the uniqueness of the theoretical result. I think it is on the borderline. Depending on the authors' response, I may raise my score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This study investigates defense against backdoor attacks for models that have already been trained. It proposes, in particular, a min-max formulation for backdoor defense, in which the inner maximum seeks a powerful trigger that leads to a high loss, while the outer minimum seeks to suppress the "adversarial loss", so as to unlearn the injected backdoor behaviors. To solve the minimax, the authors also propose a method, Implicit Backdoor Adversarial Unlearning (I-BAU). In addition, the authors also provide theoretical analysis including the convergence bound and generalization bound. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method.


main_review: **Strength**

(1) Overall, this paper is well-organized and easy to read. The proposed algorithm seems natural. Extensive experiments are conducted to illustrate the effectiveness and efficiency of the proposed algorithm. The comparison with other baselines also demonstrates superiority, which really convinces me.

(2) This research also includes some theoretical analysis, which is actually the missing part in current studies on backdoor attacks. This could be useful for future research.


**Weakness**

(1) Can the authors give a detailed description of how to implement I-BAU in practice and its time complexity? The reviewer is confused with the implementation and the efficiency, since the update of I-BAU in Eqn. (4) requires an inverse of a second-order derivative, which may not be supported directly by PyTorch/Tensorflow and may be extremely slow. However, the authors argue that the proposed I-BAU is more efficient than other baselines in Section 6.4.

(2) Is the sole difference between I-BAU and naive method (i.e., adversarial learning with universal perturbation) the implicit gradient in Eqn. (2)? Specifically, they both generate adversarial perturbation for each individual example within the sampled batch and add the perturbations together. Next, the naive method merely updates model parameters with the direct gradient, whereas I-BAU updates with the direct gradient plus an additional implicit gradient. Is the reviewer's understanding correct?

If so, can the authors explain why such an extra implicit gradient could stabilize the training and reduce ASR significantly in Section 6.2? This is not very intuitive.

Furthermore, the proposed I-BAU inherits the shortcoming of the universal perturbation (i.e., "the adversarial effect of individual perturbations may be canceled out by the addition operation" in the first paragraph of Section 4. Algorithm). As a result, part of the comments in the first paragraph of Section 4 might be unfair for the natural algorithm design.

(3) This paper assumes that the perturbation pattern is static and fixed for any examples upon being patched (see Minimux formulation for defense in Section 3). Unfortunately, some attacks are contrary to such an assumption. For example, the input-aware backdoor (IAB) attack [R.1] makes use of a dynamic trigger that varies across inputs. Can I-BAU still defend against IAB attack? 

Furthermore, the formulation of Eqn. (1) actually requires an assumption, namely, the trigger pattern is static and fixed across inputs, in addition to the bounded norm constraint. As a result, the discussion advantages 2) in Section 3 appears to be incorrect. 

[R.1] Tuan Anh Nguyen and Anh Tran. Input-Aware Dynamic Backdoor Attack. In NeurIPS, 2020.

(4) This paper only evaluates results with a simplified VGG, which only has low natural accuracy on CIFAR-10 (80%-86% in this paper, while the natural accuracy of a standard VGG usually achieves 92.64% [R.2] or 93.34% [R.3]), which brings difficult to determine whether the proposed method still works in larger models. 
A large model may still be able to remember the backdoor behavior even after fine-tuning, since its large compacity. It is better to evaluate the performance across more architectures, such as ResNet-18 which has at least 93% of natural accuracy.

[R.2] https://github.com/kuangliu/pytorch-cifar

[R.3] https://github.com/bearpaw/pytorch-classification

(5) In practice, after downloading a model from the Internet, we can't tell whether it is backdoored or not. If the model is a benign one, would I-BAU hurt its performance?

(6) In adversarial training, the perturbation budget has a significant impact on robustness and natural accuracy. Since I-BAU is a variant from adversarial training, how does the ASR/ACC change with respect to varying norm constraints $C_{\delta}$?

(7) (Just a suggestion; no influence on my decision) When evaluating the ASR of models, it would be better to exclude the samples belonging to the target class. Because the perfectly robust model achieves 0% of ASR, it is much easier to compare ASR directly (i.e., the lower the ASR, the more robust the model). Otherwise, because the perfectly robust model still predicts the target class for poisoned samples whose ground truth is the target class,  it would have a non-zero ASR (e.g., 10% on CIFAR-10).

If the authors could solve some concerns mentioned above, the reviewer would reconsider the rating.
summary_of_the_review: Although the proposed method is natural and effective and the paper has many merits, there are some concerns including implementation details, some discussion in the paper, and lack of evaluation on more model architectures. The reviewer will recommend this paper if some concerns have been addressed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper formulates adversarial training against backdoor poison attacks and proposes to use implicit hypergradient to solve the minimax problem instead of breaking it down into separate inner and outer optimization problems. The authors perform theoretical analysis of the algorithm and find convergence bound and generalization bound for the method. Also, the authors evaluate the proposed adversarial training routine, called I-BAU (Implicit Backdoor Adversarial Unlearning) in three attack settings:  (1) One-trigger-one-target attack, (2) One-trigger-all-to-all attack, (3) Multi-trigger-multi-target attack and compare with six defenses. For each attack setting, the authors test 7 different backdoor attacks. The results show that the efficiency of I-BAU is comparable to the best baseline for each of the attacks and is more time-efficient than other defenses. Finally, the authors show that I-BAU leads to a more stable training comparing to using universal adversarial perturbations in adversarial learning. 
main_review: Pros: 
1. The idea of using implicit hypergradient in adversarial training is, to the best of my knowledge, novel and interesting. 
2. The paper demonstrates that I-BAU is effective against multiple backdoor attacks in various settings (# of triggers and # of target labels) 

Cons: 
1. The paper does not cite a relevant work by J.Geiping et. al [1] which successfully adopts adversarial learning to defend against backdoor attacks. Also, I-BAU is not compared to the algorithm in [1]. 
2. The paper's clarity could be improved, in particular grammar. For example, there is a typo in the abstract (Implicit Bacdoor) and numerous typos in the main body (diffrencial prevacy, etc.). Also, some sentences are very confusing, for example in section 6 it is written "Can I-BAU effectively remove various backdoor triggers". I am confused with how the proposed approach should remove the backdoor triggers.  


[1] What Doesn't Kill You Makes You Robust(er): Adversarial Training against Poisons and Backdoors
summary_of_the_review: Although the empirical results for the method are appealing and the idea of using implicit hypergradient is interesting, the paper does not review previous work in adversarial training against backdoor attacks [1] which is extremely relevant and does not compare the proposed method with the existing algorithm. Also, the writing could be significantly improved. Because of these concerns, I vote for weak reject.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the backdoor attack problem and proposes a minimax formulation for the defense against adversaries. Theoretically, the paper analyzes the convergence bound and the generalization bound for the proposed method. Empirically, the paper compares efficacy, stability, sensitivity, and efficiency with several competitive methods.
main_review: 1. In most cases, the proposed I-BAU is not the state-of-the-art method. For example, (I) in Table 1, I-BAU is not the best defense for 6 out of 8 Attacks, i.e. BadNets, l_0 inv, Smooth, Trojan SQ, Trojan WM, All to all; (II) in Table 3, I-BAU is not as good as NAD in terms of ASR; and (III) in Table 4, NC has lower ASR and higher/comparable ACC than the proposed I-BAU. Therefore, it is questionable if I-BAU is able to reduce the attack success rate to a very low level.

2. In page 6, only a simplified VGG model is used throughout the paper. So one question is whether the proposed method works well with different backbones, especially deeper networks.

3. In page 2, it is assumed that the norm of the backdoor pattern delta is bounded by C_delta. However, in real-world applications, the magnitude of this norm may be unknown, thus restricting the usage of the proposed method.

4. There is no empirical validation and justification of Theorem 1-3. For instance, in Theorem 3, the generalization bound can be measured by certain norms of weight matrices, the number of samples, Lipschitz constant, and so on. Hence, after a model is trained and fixed, it is possible to empirically justify if the theorems are true. It is also interesting to empirically investigate the bounds with regard to different weights (i.e. network architectures), number of samples, etc.

summary_of_the_review: One of the main issues is that the proposed method can hardly achieve the lowest ASR. Although I-BAU seems to be more stable, one may resort to the best defense for a corresponding attack if ASR has a high priority.
review_title: null

Meta-review: This paper investigates defense against backdoor attacks for models that have already been trained. It proposes, in particular, a min-max formulation for backdoor defense, in which the inner maximum seeks a powerful trigger that leads to a high loss, while the outer minimum seeks to suppress the "adversarial loss", so as to unlearn the injected backdoor behaviors. To solve the minimax, the authors also propose a method, Implicit Backdoor Adversarial Unlearning (I-BAU). In addition, the authors also provide theoretical analysis including the convergence bound and generalization bound. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method. 

The proposed method is interesting and the implementation is nice. Overall, there is a fundamental flaw in the formulation: if the trigger is not additive (where there are many such examples of poisoning attacks that are not additive) this approach should fail completely. Not having experiments that discuss such triggers that are not additive is a significant flaw in the presentation of the paper. Another flaw is that the trigger is assumed to be small norm. Unlike adversarial examples attacks (at test time), there is no reason for backdoor triggers to be of small norm. Given that the defense critically relies on these two flawed assumptions, and the extent of how the proposed algorithm is sensitive to these assumptions are not properly addressed in the experiments, this paper is on the border line.
Decision: Accept (Poster)