Paper: BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection (ICLR-2024)

Reviewer comment (Experiments):
The proposed method relies on several procedures, each with different hyperparameters. However, the authors provide an ablation study in each component, with no overall insights/guides for applying such a method in real-world applications. In real applications, the defender does not know for sure that backdoor attacks exist in their data, so it might not be easy to find these suitable hyperparameters.

Author response:
We thank the reviewer for acknowledging our proposed method and experimental evaluations. We have incorporated the reviewer's suggestions, and hope our following clarifications could further resolve the reviewer's remaining concerns:

---

### **Q1: Hyperparameter selection.**

We totally agree with the reviewer that it's important to provide a general insight/guidance to real-world practitioners, in order to help them select an appropriate set of hyperparameters. In our submitted draft, we have briefly discussed how to select key hyperparameters in Appendix A.1 and A.2. To futher clarify, we make the following arguments to show that probing for an appropriate set of hyperparameters for BaDExpert is not difficult in practice.
* **Fine-tuning rate $\eta’$.** In Appendix A.1.1, we have provided a brief ***rule of thumb*** to guide practitioners to select the fine-tuning learning rate $\eta'$, by observing the clean accuracy (CA) trend of the model during fine-tuning. Specifically, we observe that clean finetuning can best diminish the backdoor functionality with a large learning rate, where the finetuned model’s CA drops to ∼ $0$% in the first one or two epochs, and then recovers to a significant value in the following epochs.
* **Unlearning rate $\eta$**. As studied in Appendix C.4, we have shown that our method is consistently effective across a wide range of $\eta$ (from $5\cdot 10^{-5}$ to $5\cdot 10^{-3}$). Nevertheless, we understand the reviewer's concern, that practitioners in the real world might need certain guidance to identify this coarse-grained range of $\eta$.
    
    Similar to $\eta'$, a simple ***rule of thumb for $\eta$*** could be: starting from an $\eta$ that is orders of magnitude smaller than the $\eta'$ used for clean finetuning, keep tuning $\eta$ until observing the model's clean accuracy (CA) diminishes by a certain extent (but does not completely vanish, i.e., worse than random guess). This rule of thumb helps practitioners idenfity an approximate range of $\eta$ according to the model's CA drop magnitude, which can be derived from Fig 8 -- we can see the backdoor functionality is usually preserved when $\eta$ is conservatively small, in the sense that CA has not degraded to the random-guess level.

Reviewer comment (Experiments):
The comprehensive experimental results are appreciated. It could be more comprehensive to compare with recent defence methods such as ABL [1], and detection methods [2,3].

Author response:
### **Q2: Additional comparisons with recent baselines.**

As suggested by the reviewer, we have supplemented the results comparing BaDExpert with ABL[1] and CD[4] (together with additional baselines suggested by other reviewers) in Rebuttal-Table 1. Particulary, we note that:
* ABL[1] is a poison-suppression defense that requires control of the training process (pre-development), which is actually out of the scope considered by our work (post-development). We simply report the CA/ASR results of the anti-backdoor learned models.
* CD[4] is a technique originally for poisoned training set inspection (i.e., detecting poison samples within the training dataset). In order to compare CD with BaDExpert, we adapt it to an inference-time backdoor input detector.
* ASSET[5] is also a poisoned training set inspection method. We did not include it because it cannot be applied/adapted to our inference-time backdoor input detection setting.
    
As shown in Rebuttal-Table 1 below, BaDExpert outperforms both ABL and CD, achieving lower ASR while retaining higher CA. These results are also updated in our paper revision (Appendix C.8).

**Rebuttal-Table 1: Comparing BaDExpert with additional baselines.** (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, **BaDExpert outperforms all these baselines w.r.t. both CA and ASR**.


| Attack | | No Defense | ABL[1] | AWM[2] | RNP[3] | CD[4] | BaDExpert |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| **BadNet** | CA  | 94.1 | 92.5 | 92.7 | 70.0 | 78.6 | **93.1** | 
|             | ASR | 100.0 | 11.7 | 4.8 | 18.9 | 2.9 | **0.0** |
| **Blend**  | CA  | 94.1 | 91.9 | 88.9 | 77.9 | 79.0 | **93.1** |
|             | ASR | 90.6 | 11.8 | 29.7 | 19.6 | 72.8 | **11.4** |

---

[1] Li, Yige, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems 34 (2021): 14900-14912.

[2] Chai, Shuwen, and Jinghui Chen. "One-shot neural backdoor erasing via adversarial weight masking." Advances in Neural Information Processing Systems 35 (2022): 22285-22299.

[3] Li, Yige, Xixiang Lyu, Xingjun Ma, Nodens Koren, Lingjuan Lyu, Bo Li, and Yu-Gang Jiang. "Reconstructive Neuron Pruning for Backdoor Defense." arXiv preprint arXiv:2305.14876 (2023).

[4] Huang, Hanxun, Xingjun Ma, Sarah Erfani, and James Bailey. "Distilling Cognitive Backdoor Patterns within an Image." arXiv preprint arXiv:2301.10908 (2023).

[5] Pan, Minzhou, Yi Zeng, Lingjuan Lyu, Xue Lin, and Ruoxi Jia. "ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms." arXiv preprint arXiv:2302.11408 (2023).

Reviewer comment (Experiments):
The method mainly focuses on the detection of backdoor samples (end of section 3). It could help to further clarify what happens after. Results in Table 1 focus on the CA/ASR. It is not clear what happens to the detected samples in order to obtain these results.

Author response:
### **Q3: Clarifying what happens after the detection of backdoor samples.**

We apologize for any unclarity in our experimental setting w.r.t. the reported CA/ASR results of BaDExpert. Indeed, our work only focuses on **detecting** backdoor inputs (and does not study how to obtain correct classification retuls for them). In Sec 4.1 "Metrics" paragraph, we have stated how we report the CA & ASR for BaDExpert (and other detector-based defenses), in order to fairly compare with other non-detector-based defenses: "*we interpret correctly filtered backdoor inputs as successfully thwarted attacks, while falsely filtered clean inputs are considered erroneous predictions*". Formally and specifically:
1. $\text{ASR}=\mathbb P_{(x,y)\sim \mathcal{P}} [\mathcal M(\mathcal T(x)) = t \wedge \mathtt{BID}(\mathcal T(x)) = 0]$: the ASR is reported by considering all correctly detected (rejected) backdoor inputs as failed attack attempts, and thus not inlcuded in the ASR numerator;
2. $\text{CA}=\mathbb P_{(x,y)\sim \mathcal{P}} [\mathcal M(x) = y \wedge \mathtt{BID}(x) = 0]$: the CA is reported by considering all falsely detected (rejected) benign inputs as misclassification, and thus not inlcuded in the CA numerator.

Reviewer comment (Experiments):
More comparisons with existing works regarding the methodology can be included.

Author response:
During the rebuttal periord, we have supplemented results comparing BaDExpert with several additional baselines [1-4]. As shown in Rebuttal-Table 1, BaDExpert outperforms all these baseline defenses, achieving higher CA and lower ASR. These results are also updated in our paper revision (Appendix C.8).

Regarding the reviewer's suggested baseline [5], we have included it as a related work in our revision (see updated Sec 5). But due to time and resource limitations, we were yet not able to incorporate the experimental results of [5]; we will try to include the results in the future revisions.

**Rebuttal-Table 1: Comparing BaDExpert with additional baselines.** (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, **BaDExpert outperforms all these baselines w.r.t. both CA and ASR**.


| Attack | | No Defense | ABL[1] | AWM[2] | RNP[3] | CD[4] | BaDExpert |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| **BadNet** | CA  | 94.1 | 92.5 | 92.7 | 70.0 | 78.6 | **93.1** | 
|             | ASR | 100.0 | 11.7 | 4.8 | 18.9 | 2.9 | **0.0** |
| **Blend**  | CA  | 94.1 | 91.9 | 88.9 | 77.9 | 79.0 | **93.1** |
|             | ASR | 90.6 | 11.8 | 29.7 | 19.6 | 72.8 | **11.4** |

Reviewer comment (Novelty):
Is the design philosophy of BaDExpert related to [1]? In [1], adversarial examples are detected by encouraging the model to carry malicious behaviors such as a backdoor.

Author response:
We thank the reviewer for the pointer to [6], where the authors propose an adversarial example detection method using "honeypots" --- a trapdoor that would enforce malicious adversarial inputs to manifest a certain neural network activation-pattern signature. We also found that their key design philosophy to be subtly connected to ours, in the sense that [6] detect potential adversarial examples via "activation signature similarity measurement", and we detect backdoor examples via "model prediction/confidence agreement measurement". However, their motivation, problem, and method are still largely different from ours, thus we did not discuss this work in our previous submitted version. To make sure we cover related work comprehensively, we have added a separate Appendix section (B.6) to discuss the connection between our design to [6] in our revision.

Reviewer comment (Experiments):
Admitting the effectiveness of the proposed method, I think the practicality of Badexpert is limited. As shown in Appendix A, the optimal learning rate could vary across datasets: ($\eta=10^{-4}$ for CIFAR-10 and $\eta=2.5\cdot10^{-5}$ for GTSRB). Even for the same dataset, the optimal $\eta$ could be different across different architectures: $\eta=10^{-4}$ for ResNet18 and $\eta=10^{-6}$ for pretrained vit_b_16. Considering the tremendous hyperparameter required for Badexpert, I think the overall guidelines for how to choose hyperparameter are needed to help Badexpert better defend against potential risks.

Author response:
We thank the reviewer for acknowledging our proposed method, experiments, and paper writing. We have incorporated the reviewer's suggestions, and hope our following clarifications could further resolve the reviewer's remaining concerns:

---

### **Q1: Hyperparameter selection.**

We totally understand the reviewer's concern regarding our method's practicality of hyperparameter choosing. We hereby clarify our practical guidances for hyperparameters selection in respond to the reviewer's concern, showing that probing for an appropriate set of hyperparameters for BaDExpert is not difficult in practice.
* **Fine-tuning rate $\eta’$.** In Appendix A.1.1, we provide a brief rule of thumb to select the fine-tuning learning rate $\eta'$ by observing the clean accuracy (CA) trend of the model. Specifically, we observe that clean finetuning can best diminish the backdoor functionality with a large learning rate, where the finetuned model’s CA drops to ∼$0$% in the first one or two epochs, and then recovers to a significant value in the following epochs.

    Indeed, we implicitly assume the model deployer (defender) has certain knowledge of the model $\mathcal M$'s clean utility (or CA), since he/she is going to deploy the model for his/her own downstream task (and must make sure the model has such capability, i.e., high CA). We believe this is not a "too ideal" assumption, given that nowadays a standard test set is often available for diverse CV tasks, and that the clean utility of the model to be deployed is rather crucial to the model deployer.
* **Unlearning rate $\eta$**. To clarify, we would not claim our selection of $\eta$ is "optimal" in each setting -- we only adjust the $\eta$ in a coarse-grained way such that the unlearning process (Alg 1) does not collapse when the model architecture is different. In fact, as studied in Appendix C.4, we have shown that our method is consistently and similarly effective across a wide range of $\eta$ (from $5\cdot 10^{-5}$ to $5\cdot 10^{-3}$). Nevertheless, we understand the reviewer's concern, that practitioners in the real world might need certain guidance to identify this coarse-grained range of $\eta$.
        
    Similar to $\eta'$, a simple **rule of thumb for $\eta$** could be: starting from an $\eta$ that is orders of magnitude smaller than the $\eta'$ used for clean finetuning, keep tuning $\eta$ until observing the model's clean accuracy (CA) diminishes by a certain extent (but does not completely vanish, i.e., worse than random guess). This rule of thumb helps practitioners idenfity an approximate range of $\eta$ according to the model's CA drop magnitude, which can be derived from Fig 8 -- we can see the backdoor functionality is usually preserved when $\eta$ is conservatively small, in the sense that CA has not degraded to the random-guess level.

Reviewer comment (Experiments):
The chosen hyperparameter $\eta'$ could also be unpractical in the real world. In reality, only a small subset of clean images is available for defenders. Therefore, they have little knowledge about which $\eta'$ will meet the requirement of Badexpert: the CA of the finetuned model's CA first drops to ~ $0\%$ and recovers to a significant value in the following epochs. The defender doesn't exactly know what the CA of the current model is. Therefore, the requirement of Badexpert may be too ideal.

Author response:
We thank the reviewer for acknowledging our proposed method, experiments, and paper writing. We have incorporated the reviewer's suggestions, and hope our following clarifications could further resolve the reviewer's remaining concerns:

---

### **Q1: Hyperparameter selection.**

We totally understand the reviewer's concern regarding our method's practicality of hyperparameter choosing. We hereby clarify our practical guidances for hyperparameters selection in respond to the reviewer's concern, showing that probing for an appropriate set of hyperparameters for BaDExpert is not difficult in practice.
* **Fine-tuning rate $\eta’$.** In Appendix A.1.1, we provide a brief rule of thumb to select the fine-tuning learning rate $\eta'$ by observing the clean accuracy (CA) trend of the model. Specifically, we observe that clean finetuning can best diminish the backdoor functionality with a large learning rate, where the finetuned model’s CA drops to ∼$0$% in the first one or two epochs, and then recovers to a significant value in the following epochs.

    Indeed, we implicitly assume the model deployer (defender) has certain knowledge of the model $\mathcal M$'s clean utility (or CA), since he/she is going to deploy the model for his/her own downstream task (and must make sure the model has such capability, i.e., high CA). We believe this is not a "too ideal" assumption, given that nowadays a standard test set is often available for diverse CV tasks, and that the clean utility of the model to be deployed is rather crucial to the model deployer.
* **Unlearning rate $\eta$**. To clarify, we would not claim our selection of $\eta$ is "optimal" in each setting -- we only adjust the $\eta$ in a coarse-grained way such that the unlearning process (Alg 1) does not collapse when the model architecture is different. In fact, as studied in Appendix C.4, we have shown that our method is consistently and similarly effective across a wide range of $\eta$ (from $5\cdot 10^{-5}$ to $5\cdot 10^{-3}$). Nevertheless, we understand the reviewer's concern, that practitioners in the real world might need certain guidance to identify this coarse-grained range of $\eta$.
        
    Similar to $\eta'$, a simple **rule of thumb for $\eta$** could be: starting from an $\eta$ that is orders of magnitude smaller than the $\eta'$ used for clean finetuning, keep tuning $\eta$ until observing the model's clean accuracy (CA) diminishes by a certain extent (but does not completely vanish, i.e., worse than random guess). This rule of thumb helps practitioners idenfity an approximate range of $\eta$ according to the model's CA drop magnitude, which can be derived from Fig 8 -- we can see the backdoor functionality is usually preserved when $\eta$ is conservatively small, in the sense that CA has not degraded to the random-guess level.

Reviewer comment (Experiments):
Some of the baselines are missing. For example, AWM [1] and ABL [2].

Author response:
### **Q3: Additional comparisons with recent baselines.**

We note that ABL is a poison-suppression defense that requires control of the training process (pre-development), which is actually out of the scope considered by our work (post-development). Neverthelss, as suggested by the reviewer, we have supplemented the results comparing BaDExpert with ABL[1] and AWM[2] (together with additional baselines suggested by other reviewers) in Rebuttal-Table 1. As shown, BaDExpert outperforms both ABL and AWM, achieving higher CA and lower ASR. These results are also updated in our paper revision (Appendix C.8).

**Rebuttal-Table 4: Comparing BaDExpert with additional baselines.** (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, **BaDExpert outperforms all these baselines w.r.t. both CA and ASR**.


| Attack | | No Defense | ABL[1] | AWM[2] | RNP[3] | CD[4] | BaDExpert |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| **BadNet** | CA  | 94.1 | 92.5 | 92.7 | 70.0 | 78.6 | **93.1** | 
|             | ASR | 100.0 | 11.7 | 4.8 | 18.9 | 2.9 | **0.0** |
| **Blend**  | CA  | 94.1 | 91.9 | 88.9 | 77.9 | 79.0 | **93.1** |
|             | ASR | 90.6 | 11.8 | 29.7 | 19.6 | 72.8 | **11.4** |

Reviewer comment (Evaluation):
Table 1 shows that BadExpert is relatively weak to defend against Blend or Dynamic attack. Can you explain the reason behind this phenomenon?

Author response:
### **Q4: BaDExpert's degraded performance against Blend and Dynamic.**

According to Table 2, we can see the AUROC against Blend and Dynamic are actually at the average level ($99.2$% and $99.1$%, respectively) -- implying that BaDExpert is averagely good at *separating* clean and backdooor inputs against these two attacks (compared to other attacks). However, in Table 1, the defended ASR for them are indeed slightly higher than the defended ASR of other attacks. We suspect the reasons behind could be one of the following:
1. The underlying backdoor correlation is less robust / more brittle (when we fine-tune the models on mislabeled clean samples), compared to other attacks. And therefore, our backdoor extraction cannot completely isolate the backdoor functionality (i.e., the backdoro expert model $\mathcal B$ becomes the straggler).
2. The underlying backdoor correlation is more robust / too persistant (when we fine-tune the models on correctly labeled clean samples), compared to other attacks. And therefore, the standard fine-tuning process cannot satisfactorily diminish the backdoor functionality (i.e., the auxiliary model $\mathcal M'$ becomes the straggler).
    
But overall, we could still say that BaDExpert is effective, and doesn't demonstrate significant degradation against these two attacks. Once we allow a slightly higher FPR (e.g., $2$%), the defended ASR would be further suppressed to $8.4$% and $7.1$%. Moreover, according to our experiments on GTSRB (Table 6, where the ASR are $4.6$% and $0.0$%, respectively), BaDExpert also does not perform noticeably worse on these two particular attacks.

Reviewer comment (Theory):
The paper lacks in-depth theoretical analysis to support the proposed method.

Author response:
### **Q1: Lack of theoretical analysis.**

We sincerely appreciate the reviewer's suggestion on theoretically analyzing our observation that "fine-tuning on a small set of mislabeled clean data can isolate the backdoor functionality". In our work, we have provided theoretical insight into the *decision rule* (Neyman-Pearson Lemma in Remark 1, Sec 4.2). Mathematically analyzing the effect of *learning (fine-tuning) dynamics* of neural networks, on the other hand, is rather technically difficult --- researchers are often simplifying DNNs into overparameterized linear models to do this. These simplified analysis can only provide more empirical intuitions behind the methods, but not rigorous guarantees. 
    
In our work, in order to provide sturdy support for our method's "credibility", we tried our best to conduct various experiments, as many as possible. Regarding the observation of "*fine-tuning on a small set of mislabeled clean data can isolate the backdoor functionality*", we have shown its pervasiveness across a big arrary of attacks (Fig 8). Regarding the overall effectiveness of our method, BaDExpert, we have validated its effectiveness across different attacks, datasets, architectures, and accompanied with various ablation studies. Moreover, we have studied its "worst-case" effectiveness against a tailored adaptive attack. All these empirical efforts jointly confirm our method's credibility.

Reviewer comment (Experiments):
The experimental section seems to omit comparisons with certain recent relevant works.

Author response:
### **Q3: Additional comparisons with recent baselines.**

As suggested by the reviewer, we have supplemented results comparing BaDExpert with RNP[3] (together with additional baselines suggested by other reviewers). As shown in Rebuttal-Table 1, BaDExpert outperforms RNP, achieving higher CA and lower ASR. These results are also updated in our paper revision (Appendix C.8).
    
$^\star$**Regarding the reviewer's quote "RNP reports an average ASR of 5.03 and CA of 92.18 on CIFAR10".** We note that RNP reports these results with a **default poisoning rate of $5$%**, which is far larger than our default poisoning rate $0.3$%. Such discrepant settings may lead to the different reported results of RNP. In fact, the RNP authors also note in the Sec 5 that "*RNP faces a noticeable challenge when defending against backdoor attacks with low poisoning rates (e.g., the poisoning rate ≤ 1%).*" Moreover, their results in Table 12 (RNP against lower-poisoning-rate attackers) match our reported RNP results; and we have also validated that RNP can achieve the similar reported performance at the $5$% poisoning rate.

**Rebuttal-Table 1: Comparing BaDExpert with additional baselines.** (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, **BaDExpert outperforms all these baselines w.r.t. both CA and ASR**.


| Attack | | No Defense | ABL[1] | AWM[2] | RNP[3] | CD[4] | BaDExpert |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| **BadNet** | CA  | 94.1 | 92.5 | 92.7 | 70.0 | 78.6 | **93.1** | 
|             | ASR | 100.0 | 11.7 | 4.8 | 18.9 | 2.9 | **0.0** |
| **Blend**  | CA  | 94.1 | 91.9 | 88.9 | 77.9 | 79.0 | **93.1** |
|             | ASR | 90.6 | 11.8 | 29.7 | 19.6 | 72.8 | **11.4** |

Reviewer comment (Theory):
In real-world scenarios, after acquiring a model online, it's often uncertain whether it has been backdoored. If the model is a benign one, there would be a disagreement between the outputs of model \mathcal{M} and \mathcal{B} (on the left side of Figure 2). This divergence could potentially hinder BaDExpert's performance.

Author response:
### **Q2: BaDExpert's performance on clean (benign) models**.

In Table 1 "No Attack" row, we have shown the case when the model $\mathcal M$ is benign (CA = $93.1$%). In case of potential misunderstanding, we clarify the occasion when $\mathcal M$ is not backdoored more specifically:

1. *Remember that "backdoor input" is only meaningful when the model is embedded with a corresponding backdoor.* So when the model $\mathcal M$ is benign, there would be no concept as "backdoor input" or "backdoor functionality"; every input can simply be considered as a "clean input". As a result, Fig 2 (left) has no meaning in this context -- there is no attacker, and the "patched inputs" shown in Fig 2 (left) can just be considered as benign inputs with watermarks. 
3. The extracted "backdoor expert model" $\mathcal B$ is just a DNN that "forgets its benign task" -- making incorrect predictions and disagreeing with $\mathcal M$ (in most cases). Therefore, by looking at Fig 2 (right), we can come to the conclusion: BaDExpert would accept most benign images (i.e., low false positive rates).

In summary, we argue that BaDExpert should be similarly effective, no matter the model $\mathcal M$ is backdoored or benign.
    
P.S.: In Sec 4.4 and Appendix C.8.1, we have also considered the **Natural** backdoor attack (which is not a traditional backdoor attack setting, more similar to the universal adversarial attack setting; our arguments above do not include this) against *benign models*. We found that BaDExpert is still effective at detecting these Natural backdoor inputs (AUROC $>92$%), and our key finding that "*when finetuning on mislabeled clean data, the backdoor functionality would remain, while normal functionality does not*" still holds in general.

Reviewer comment (Experiments):
Could this paper elucidate the time complexity of the proposed method and compare it with methods like I-BAU? Given that this technique necessitates model fine-tuning, there are concerns about its efficiency.

Author response:
### **Q4: Method efficiency concern.**

* The time complexity analysis in I-BAU[5] points out its time complexity to be $\tilde O(K \cdot \vartheta \cdot \tilde O(\theta))$, where $K$ is the epochs (rounds) of their unlearning algorithm, $\vartheta$ is the number of the iterations of their fixed-point solver, $O(\theta)$ is the time cost for training a DNN via backprop on the reserved clean set for one epoch. In practice, they choose $K=3$ and $\vartheta=5$, i.e. approximately $15\cdot \tilde O(\theta)$, costing $26$ GPU seconds in our environment (ResNet18 on CIFAR10).
* Similarly, we could establish the time complexity of BaDExpert as $\tilde O((K_1 + K_2) \cdot \tilde O(\theta))$, where $K_1$ is the number of unlearning epochs (to obtain $\mathcal B$), and $K_2$ is the fine-tuning epochs (to obtain $\mathcal M'$). In practice, we choose $K_1=1$ and $K_2=10$, i.e. approximately $11\cdot \tilde O(\theta)$, costing $23$ GPU seconds in our environment (ResNet18 on CIFAR10).

As shown, our method's efficiency outperforms I-BAU both in principle and in practice.

Reviewer comment (Experiments):
It seems that some recent published related works are missing to be compared in the paper. For example, [1] presents defense results that are on par with those in this paper, reporting an ASR of 5.03 and a CA of 92.18 for the CIFAR10 dataset.

Author response:
### **Q3: Additional comparisons with recent baselines.**

As suggested by the reviewer, we have supplemented results comparing BaDExpert with RNP[3] (together with additional baselines suggested by other reviewers). As shown in Rebuttal-Table 1, BaDExpert outperforms RNP, achieving higher CA and lower ASR. These results are also updated in our paper revision (Appendix C.8).
    
$^\star$**Regarding the reviewer's quote "RNP reports an average ASR of 5.03 and CA of 92.18 on CIFAR10".** We note that RNP reports these results with a **default poisoning rate of $5$%**, which is far larger than our default poisoning rate $0.3$%. Such discrepant settings may lead to the different reported results of RNP. In fact, the RNP authors also note in the Sec 5 that "*RNP faces a noticeable challenge when defending against backdoor attacks with low poisoning rates (e.g., the poisoning rate ≤ 1%).*" Moreover, their results in Table 12 (RNP against lower-poisoning-rate attackers) match our reported RNP results; and we have also validated that RNP can achieve the similar reported performance at the $5$% poisoning rate.

**Rebuttal-Table 1: Comparing BaDExpert with additional baselines.** (CA for Clean Accuracy, ASR for Attack Success Rate) For fair comparison: we follow the suggested hyperparameters and only tune them to approach the officially reported results; when reporting RNP results, we stop pruning once ASR drops to below $20$%; when reporting CD results, we follow their suggested and default threshold selection strategy. As shown, **BaDExpert outperforms all these baselines w.r.t. both CA and ASR**.


| Attack | | No Defense | ABL[1] | AWM[2] | RNP[3] | CD[4] | BaDExpert |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| **BadNet** | CA  | 94.1 | 92.5 | 92.7 | 70.0 | 78.6 | **93.1** | 
|             | ASR | 100.0 | 11.7 | 4.8 | 18.9 | 2.9 | **0.0** |
| **Blend**  | CA  | 94.1 | 91.9 | 88.9 | 77.9 | 79.0 | **93.1** |
|             | ASR | 90.6 | 11.8 | 29.7 | 19.6 | 72.8 | **11.4** |

Reviewer comment (Reproducibility):
Publicly releasing the code would facilitate better reproducibility and peer verification, enhancing the paper's value.

Author response:
### **Q5: Code releasing.**

Thanks for the suggestion regarding publicly releasing our code! We will definitely share our code on GitHub after the peer-review process for reproducibility and verification purpose.