Paper: Towards Reliable and Efficient Backdoor Trigger Inversion via Decoupling Benign Features (ICLR-2024)

Reviewer comment (Novelty):
(Claimed contribution 3) The proposed BTI-DBF is almost the same as the backdoor mitigation approach in Neural Cleanse [1]! I didn't flag for ethics review for this one since I tend to believe that the authors just omitted this existing approach. [1] Wang et al, Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE S&P, 2019.

Author response:
Thank you for this comment! We are deeply sorry that our previous submission may lead you to some misunderstandings that we want to clarify here.

- We speculate that you have this misunderstanding mainly because we all have a trigger inversion process before conducting further defenses. However, **this is the characteristic of all backdoor trigger inversion (BTI)-based methods**. We argue that it is unfair to claim our method is almost the same as neural cleanse simply based on it.
- **Our BTI is quite different from neural cleanse in trigger inversion**. Neural cleanse used $x' = (1-m)\cdot t + m\cdot x$ to generate backdoor samples. Their $m$ and $t$ are fixed after training. However, **our method can generate different triggers via the UNet generator.** Moreover, Neural Cleanse only considers the backdoor behavior in pixel space. Our method further considers the backdoor behavior in feature space and finds the difference between the benign features and the backdoor features. Therefore, **our method can generate higher-quality triggers**. As shown in the Table in our paper, our defenses can succeed even when the triggers are dynamic, but neural cleanse can't.
- **Our work considers more defense variants.** We apply our BTI in backdoor-removal defenses, pre-processing-based defenses, and detection-based defenses. However, **neural cleanse did not consider the pre-processing-based variant**. We have made certain breakthroughs in this regard.

Reviewer comment (Novelty):
(Claimed contribution 2) The general idea of first decoupling backdoor features from benign ones and then performing trigger inversion on backdoor features is the same as in [2] (though the formulation of the optimization problem is different). [2] Liu et al, ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation. CCS, 2019.

Author response:
Thank you for this comment! We are deeply sorry that our previous submission may lead you to some misunderstandings that we want to clarify here.

- **We never claimed that we were the first to work on backdoor trigger inversion by separating benign and backdoor features**. In our paper, we intended to emphasize that all existing methods (including ABS) first fitted backdoor features before decoupling backdoor and benign features. In contrast, our method first fits benign features, avoiding the assumption of backdoor generation.
- Similar to neural cleanse, **ABS also assumed that backdoor features can be completely separated from benign ones at the neuron-level**. This method ignored the complex interactions between neurons. In contrast, we exploit a soft feature mask whose elements are from $[0, 1]$ instead of $\{0, 1\}$ to learn interaction effects.

Reviewer comment (Novelty):
(Claimed contribution 1) "Revealing the low efficiency and low similarity nature of existing backdoor trigger inversion (BTI) methods and their intrinsic reason" cannot be regarded as a contribution even though you show your method performs better. Besides, there is no adequate discussion about the "intrinsic reason" in this paper.

Author response:
Thanks for your comments. We explained why exsiting BTI methods can't decouple the backdoor features in our Section 1. We are deeply sorry that we failed to describe it more clearly and lead you to some misunderstandings. We hereby provide more details about it.
- We argue that revealing the common underlying limitations of an important defense paradigm (i.e., backdoor trigger inversion) is also an important contribution. This can alert the field against blind optimism about such methods. 
- **Intrinsic Reasons**: We reveal that both limitations are all because existing methods need to approximate and decouple backdoor features at first to separate benign and backdoor features, as required by BTI. Specifically, **these methods need to 'scan' all potential classes to speculate the target label since defenders have no prior knowledge about attacks and poisoned samples**. These processes are time-consuming since each scan requires iteratively solving a particular optimization problem.

Reviewer comment (Experiments):
The motivation of this work is weak. What is the motivation for proposing this trigger inversion approach? If the purpose is for better backdoor detection, there is no detection performance demonstrated in the paper. If it is for better backdoor mitigation, there is no evidence that the trigger inverted by other baselines cannot mitigate the backdoor. Moreover, intuitively, inaccurately estimated triggers will introduce more robustness to backdoor unlearning. For example, if the trigger is a 3 by 3 yellow square, unlearning using yellow squares with different shapes and sizes will be more effective than unlearning the backdoor using the exact 3 by 3 square only.

Author response:
Thank you for these insightful comments! We are deeply sorry that we failed to explain why having a more accurate trigger inversion is important.
- **Accurate trigger inversion yields to better defenses**. As shown in Table 2 of our main paper, all baseline backdoor trigger inversion-based defenses suffer from low effectiveness in some cases, whereas our methods do not.
- **Accurate trigger inversion yields to better interpretability**. In addition to ultimately reducing the backdoor threat, explaining the source of the backdoor in the model is also an important requirement.
- **Accurate trigger inversion is the cornerstone of designing effective pre-processing-based backdoor defenses**. Otherwise, it will only have a limited effect like Februus.
- **Inaccurately estimated triggers would not necessarily introduce more robustness to backdoor unlearning**. As shown in Table 1 of our main paper, existing methods cannot even accurately find the target class in many cases.  It is difficult to guarantee the effectiveness of doing unlearning in this situation. Otherwise, we might as well randomize a pattern and category for unlearning.

Reviewer comment (Experiments):
The results in Table 1 need to be double-checked. For example, the DSR for Unicorn is much lower than the original paper [3]. [3] Wang et al, Unicorn: A unified backdoor trigger inversion framework. In ICLR, 2023.

Author response:
Thank you for your comment! We are deeply sorry that our metric setting may lead you to some misunderstandings. **DSR indicates whether a BTI method can find the target label correctly**. This metric is different from ASR, which is the attack success rate after defenses, and **is not used in Unicorn**.

Reviewer comment (Theory):
The intuition behind the proposed method does not always hold. The proposed trigger inversion method can be defeated when there is no decoupling between benign and backdoor features. This happens when the model is compact and when the trigger is globally wide. For example, the "chessboard" trigger that cannot be mitigated by the method in [4] does not satisfy the decoupling assumption. [4] Wang et al, MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statistic, In IEEE S&P, 2024.

Author response:
Thank you for this insightful comment! We are deeply sorry that our submission may lead you to some misunderstandings that we want to clarify here.

- Different from existing BTI methods, **we did not assume that backdoor features can be completely separated from benign ones at the neuron-level**. This method ignored the complex interactions between neurons. In contrast, we exploit a soft feature mask whose elements are from $[0, 1]$ instead of $\{0, 1\}$ to learn interaction effects. 
- In general, **we decouple backdoor and benign features instead of backdoor and benign pixels**. Accordingly, our method should still be effective when the trigger is globally wide in the input space.
- To further alleviate your concerns, following your suggestions, we conduct additional experiments with chessboard-type trigger patterns. As shown in the following table, **our defenses are still highly effective under the chessboard-type trigger**.


**Table 1. The performance (%) of our methods in defending against the "chessboard" trigger on CIFAR-10.**
|       | BA  | ASR  |
| ----  | ----  | ----  |
|  No Defense  | 92.56 | 100.00 |
| BTI-DBF (P\)  | 89.19 |  2.07 |
| BTI-DBF (U)  | 90.51 | 0.00  |

Reviewer comment (Experiments):
Insufficient evaluation of the decoupling method. If the decoupling method works for the proposed formulation for trigger inversion, it should also work for other formulations such as Unicorn. It is important to show that such decoupling is generalizable.

Author response:
Thank you for this insightful comment! However, **the main difference between different backdoor trigger inversion methods lies in how to decouple benign and backdoor features**.  Therefore, we can't extend our method to other existing methods, because the extended method is formally the same as ours.

Reviewer comment (Evaluation):
No evaluation of efficiency in the main paper. To show that the proposed method is reliable and efficient, it is necessary to include a quantitative comparison of computational overhead in the main paper.

Author response:
Thank you for this constructive suggestion! In our previous submission, we have already compared the computational overhead of different BTI methods in Figure 4 of our main paper. Specifically, **our method is significantly faster than all baseline methods**. For example, our BTI-DBF needs only 60 seconds for training, which is more than 20 times faster than Unicorn. It is still more than 3 times faster than the most efficient BTI baseline (i.e., Pixel). This efficiency advantage is even more pronounced in datasets with more classes (e.g., GTSRB and ImageNet).

Reviewer comment (Presentation):
No discussion on the limitations.

Author response:
Thank you for this insightful question! We are deeply sorry that we missed the discussion about potential limitations of our method. We hereby provide it, as follows.

- As illustrated in Section 3.1, our defense mainly focuses on using third-party pre-trained models. In particular, similar to existing baseline methods, we assume that defenders have a few local benign samples. Accordingly, our method is not feasible without benign samples. Besides, we need to train a model for the scenarios using third-party datasets before conducting trigger inversion and follow-up defenses, which is computation- and time-consuming. We will further explore how to conduct BTI under few/zero-shot settings in our future works.
- Our method needs to obtain the feature layer of the backdoored model to decouple the benign features and inverse the backdoor triggers. Besides, the optimization process in our method also relies on a white-box setting. Accordingly, it does not apply to black-box scenarios in which the defenders can only access the final output of the backdoored model. We will continue the exploration of the black-box BTI in our future works.

We have added the disscussion in Appendix N of our revision.

Reviewer comment (Experiments):
Table 8 shows that the evaluation only picks 100 classes from ImageNet. This is wired. Has the method been tested on 1000 classes? What is the scalability of the proposed method? How does the method perform compared to other methods when the number of classes increases?

Author response:
Thank you for the insightful question! We are deeply sorry that our dataset selection may lead you to concerns about our effectiveness. 

- **Using ImageNet subset is a common setting in existing backdoor-related works** (e.g., [1, 2]). For example, Fu et al. [1] picked 20 classes and Huang et al. [2] picked 30 classes from ImageNet. In our work, we deliberately chose a relatively large number of categories (i.e., 100) to evaluate the scalability of all methods.
- However, we do understand your concerns. To further alleviate them, we conduct additional experiments on the ImageNet-1K dataset. Due to the time constraint, we use BadNets and WaNet as attack baselines. Specifically, we set the poison rate to 5\% and finetune the pre-trained ResNet-18 on ImageNet-1k to implant backdoors. As shown in Table 1, **both BTI-DBF (U) and BTI-DBF \(P\) are still effective** (BA drops less than 3%, while the ASR is within 5%), even if the number of classes is huge. 


We have provided more details in Appendix M of our revision.


**Reference**
1. Kunzhe Huang, et al. "Backdoor Defense via Decoupling the Training Process." ICLR 2022.
2. Chong Fu, et al. "FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases." USENIX Security 2023.



**Table 1. The performance (%) of our methods in defending against BadNets and WaNet on ImageNet-1K.** 
|  Defense $\rightarrow$   |No Defense  | BTI-DBF (P\)| BTI-DBF (U)|
| ----  | ----  | ----  | ----  |
| Attack $\downarrow$  Metric $\rightarrow$  |BA / ASR  | BA / ASR| BA / ASR|
| BadNets  | 59.79 / 99.63 |57.82 / 4.16|57.42 / 3.57| 
| WaNet  | 61.20 / 96.13 |58.46 / 2.84|59.39 / 1.05|

Reviewer comment (Novelty):
Section 2.2 misses some latest work on feature level BTI:  
- SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning, M. Zheng et al., 2023  
- Detecting Backdoors in Pre-trained Encoders, S. Feng et al., CVPR'2023  
Although these 2 works focus on self-supervised learning, they are highly related to the feature level BTI. It would be better to discuss them in Section 2.2.

Author response:
Thank you for this constructive suggestion! After carefully reading their papers, we have added some discussions in Section 2.2, as follows: **Besides, two recent research discussed how to conduct BTI in the feature space under self-supervised learning, inspired by neural cleanse**. Besides, we summarize the differences between their works and our method as follows.


- **Motivation:** These two works focused on the BTI in self-supervised Learning in which input labels are not available. However, our work primarily address the limitations in low efficiency and effectiveness of existing BTI method under supervised learning.
- **Method**. These two works adopted the same 'blended strategy' as neural cleanse in approximating poisoned samples, although the approximation is in the feature space instead of the input space. In contrast, our method only relies on benign samples without needing to assign a particular poisoning form for approximation.