SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Then, we provide the following proposition to establish the relation between backdoor risk and (sub) adversarial risk:
Proposition 3. Under Assumption 1, for a classifier  $h_\theta$ , the following inequalities hold
$$
\mathcal {R} _ {b d} (h _ {\boldsymbol {\theta}}) \leq \mathcal {R} _ {s u b} (h _ {\boldsymbol {\theta}}) \leq \mathcal {R} _ {a d v} (h _ {\boldsymbol {\theta}}).
$$
Therefore,  $\mathcal{R}_{sub}$  is a tighter upper bound for  $\mathcal{R}_{bd}$  compared with  $\mathcal{R}_{adv}$ . After replacing  $\mathcal{R}_{bd}$  with  $\mathcal{R}_{sub}$  in (3), we reach the following optimization problem:
$$
\min  _ {\boldsymbol {\theta}} \mathcal {R} _ {c l} \left(h _ {\boldsymbol {\theta}}\right) + \lambda \mathcal {R} _ {s u b} \left(h _ {\boldsymbol {\theta}}\right). \tag {7}
$$
Due to the space limit, the proofs of the above propositions, as well as some detailed discussions and extensions will be provided in Appendix A.
3.3 Proposed method
Since the target label  $\hat{y}$  is unavailable and 0-1 loss is non-differentiable, we first discuss the relaxation of  $\mathcal{R}_{sub}$  and then replace 0-1 loss with suitable surrogate loss functions in this section.
Relaxation. One limitation of  $\mathcal{R}_{sub}$  is the inaccessibility of  $\mathcal{D}_s$ , i.e., the subset of samples in  $\mathcal{D}_{-\hat{y}}$  on which planting a trigger can successfully activate the backdoor in  $h_{\theta_{bd}}$ . Since a poisoned model usually has a high attack success rate (ASR), the first relaxation is replacing  $\mathcal{D}_s$  with  $\mathcal{D}_{-\hat{y}}$ , i.e.,
$$
\mathcal {R} _ {s u b} (h _ {\boldsymbol {\theta}}) \approx \frac {1}{| \mathcal {D} _ {- \hat {y}} |} \sum_ {i = 1} ^ {N} \max  _ {\boldsymbol {\epsilon} _ {i} \in \mathcal {S}} \left\{\mathbb {I} (\boldsymbol {x} _ {i} \in \mathcal {D} _ {- \hat {y}}) \mathbb {I} (h _ {\boldsymbol {\theta}} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}) = h _ {\boldsymbol {\theta} _ {b d}} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}), \tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}} \in \mathcal {A} _ {\boldsymbol {\epsilon} _ {i}, \boldsymbol {\theta}}) \right\}.
$$
Since  $\hat{y}$  is unavailable, we also replace  $\mathcal{D}_{-\hat{y}}$  by  $\mathcal{D}_{cl}$  in the sub-adversarial risk. As  $g(\pmb{x};\Delta)$  is not malicious for classification if the ground truth label of  $\pmb{x}$  is  $\hat{y}$ , this relaxation has negligible negative influence for mitigating backdoor risk. After the above two relaxations, we reach the following shared adversarial risk on  $\mathcal{D}_{cl}$ :
$$
\mathcal {R} _ {s h a r e} (h _ {\boldsymbol {\theta}}) = \frac {1}{N} \sum_ {i = 1} ^ {N} \max _ {\boldsymbol {\epsilon} _ {i} \in \mathcal {S}} \left\{\mathbb {I} (h _ {\boldsymbol {\theta}} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}) = h _ {\boldsymbol {\theta} _ {b d}} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}), \tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}} \in \mathcal {A} _ {\boldsymbol {\epsilon} _ {i}, \boldsymbol {\theta}}) \right\}.
$$
Due to the space limit, we postpone the detailed discussion of the above two relaxations (e.g., the relaxation gaps) in Appendix A.7.
By replacing  $\mathcal{R}_{sub}$  with  $\mathcal{R}_{share}$  in (7), we reach the following problem:
$$
\min  _ {\boldsymbol {\theta}} \mathcal {R} _ {c l} \left(h _ {\boldsymbol {\theta}}\right) + \lambda \mathcal {R} _ {s h a r e} \left(h _ {\boldsymbol {\theta}}\right). \tag {8}
$$
By solving (8), we seek  $h_{\pmb{\theta}}$  that either classifies the adversarial examples correctly (i.e.,  $h_{\pmb{\theta}}(\tilde{x}_{\pmb{\epsilon}}) = y$ ), or differently with  $h_{\pmb{\theta}_{bd}}$  (i.e.,  $h_{\pmb{\theta}}(\tilde{x}_{\pmb{\epsilon}}) \neq h_{\pmb{\theta}_{bd}}(\tilde{x}_{\pmb{\epsilon}})$ ), while maintaining a high accuracy on  $\mathcal{D}_{cl}$ .
Approximation. We firstly need to approximate two indicators in (8), including: 1)  $\mathbb{I}(h_{\theta}(\pmb{x}) \neq y)$ ; 2)  $\mathbb{I}(\tilde{\pmb{x}}_{\epsilon} \in \mathcal{A}_{\epsilon, \theta}, h_{\theta}(\tilde{\pmb{x}}_{\epsilon}) = h_{\theta_{bd}}(\tilde{\pmb{x}}_{\epsilon}))$ . The former indicator is for the clean accuracy, so we use the widely used cross-entropy (CE) loss as the surrogate loss, i.e.,  $L_{cl}(\pmb{x}, y; \pmb{\theta}) = \mathrm{CE}(\pmb{p}(\pmb{x}; \pmb{\theta}), y)$ . In terms of the latter indicator that measures the shared adversarial risk, different surrogate losses could be employed for different usages:
- To generate shared adversarial examples, we first decompose the second indicator as below:
$$
\mathbb {I} \left(\tilde {\boldsymbol {x}} _ {\epsilon} \in \mathcal {A} _ {\epsilon , \boldsymbol {\theta}} \cap \mathcal {A} _ {\epsilon , \boldsymbol {\theta} _ {b d}}\right) \mathbb {I} \left(h _ {\boldsymbol {\theta}} \left(\tilde {\boldsymbol {x}} _ {\epsilon}\right) = h _ {\boldsymbol {\theta} _ {b d}} \left(\tilde {\boldsymbol {x}} _ {\epsilon}\right)\right), \tag {9}
$$
which covers two components standing for fooling  $h_{\theta_{bd}}$  and  $h_\theta$  simultaneously, and keeping the same predicted label for  $h_{\theta_{bd}}$  and  $h_\theta$ , respectively.
For the first component, we use CE on both  $h_{\theta}$  and  $h_{\theta_{bd}}$  as a surrogate loss, i.e.,
$$
L _ {a d v} \left(\tilde {\boldsymbol {x}} _ {\epsilon}, y; \boldsymbol {\theta}\right) = \frac {1}{2} \left(\operatorname {C E} \left(\boldsymbol {p} \left(\tilde {\boldsymbol {x}} _ {\epsilon}; \boldsymbol {\theta}\right), y\right) + \operatorname {C E} \left(\boldsymbol {p} \left(\tilde {\boldsymbol {x}} _ {\epsilon}; \boldsymbol {\theta} _ {b d}\right), y\right)\right). \tag {10}
$$
For the second component, we measure the distance between  $\pmb{p}(\tilde{\pmb{x}}_{i,\epsilon_i};\pmb {\theta})$  and  $\pmb {p}(\tilde{\pmb{x}}_{\epsilon};\pmb{\theta}_{bd})$  by Jensen-Shannon divergence [11] and adopt the following surrogate loss:
$$
L _ {\text {s h a r e}} \left(\tilde {\boldsymbol {x}} _ {\epsilon}, y; \boldsymbol {\theta}\right) = - \mathrm {J S} \left(\boldsymbol {p} \left(\tilde {\boldsymbol {x}} _ {\epsilon}; \boldsymbol {\theta}\right), \boldsymbol {p} \left(\tilde {\boldsymbol {x}} _ {\epsilon}; \boldsymbol {\theta} _ {b d}\right)\right). \tag {11}
$$
- To unlearn the shared adversarial examples, we adopt the following surrogate loss
$$
L _ {s a r} \left(\tilde {\boldsymbol {x}} _ {\boldsymbol {\epsilon}}, y; \boldsymbol {\theta}\right) = - \mathbb {I} (\tilde {y} \neq y) \log \left(1 - \boldsymbol {p} _ {\tilde {y}} \left(\tilde {\boldsymbol {x}} _ {\boldsymbol {\epsilon}}; \boldsymbol {\theta}\right)\right), \tag {12}
$$
where  $\tilde{y} = h_{\pmb{\theta}_{bd}}(\tilde{x}_{\pmb{\epsilon}})$ . By reducing  $L_{sar}$ , the prediction of  $\tilde{x}_{\pmb{\epsilon}}$  by  $h_{\pmb{\theta}}$ , i.e.,  $h_{\pmb{\theta}}(\tilde{x}_{\pmb{\epsilon}})$  is forced to differ from  $h_{\pmb{\theta}_{bd}}(\tilde{x}_{\pmb{\epsilon}})$  if  $h_{\pmb{\theta}_{bd}}(\tilde{x}_{\pmb{\epsilon}}) \neq y$ , therefore, reducing the shared adversarial risk (SAR).
Overall objective. By combining all the above surrogate losses with the linear weighted summation mode, we propose the following bi-level objective:
$$
\min  _ {\boldsymbol {\theta}} \frac {1}{N} \sum_ {i = 1} ^ {N} \left\{\lambda_ {1} L _ {c l} \left(\boldsymbol {x} _ {i}, y _ {i}; \boldsymbol {\theta}\right) + \lambda_ {2} L _ {s a r} \left(\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i} ^ {*}}, y _ {i}; \boldsymbol {\theta}\right) \right\} \tag {13}
$$
$$
\text {s . t .} \quad \boldsymbol {\epsilon} _ {i} ^ {*} = \underset {\boldsymbol {\epsilon} _ {i} \in \mathcal {S}} {\arg \max } \lambda_ {3} L _ {a d v} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}, y _ {i}; \boldsymbol {\theta}) + \lambda_ {4} L _ {s h a r e} (\tilde {\boldsymbol {x}} _ {i, \boldsymbol {\epsilon} _ {i}}, y _ {i}; \boldsymbol {\theta}), \quad i = 1, \ldots , N,
$$
where  $\lambda_1, \lambda_2, \lambda_3, \lambda_4 \geq 0$  indicate trade-off weights.
Optimization algorithm. We propose an optimization algorithm called Shared Adversarial Unlearning (SAU, Algorithm 1), which solves (13) by alternatively updating  $\theta$ , and  $\epsilon^{*}$ . Specifically, the model parameter  $\theta$  is updated using stochastic gradient descent [3], and the perturbation  $\epsilon^{*}$  is generated by projected gradient descent [33].
Algorithm 1 Shared Adversarial Unlearning
Input: Training set  $\mathcal{D}_{cl}$  , poisoned model  $h_{\theta_{bd}}$  PGD step size  $\eta >0$  , number of PGD steps  $T_{adv}$  perturbation set  $S$  , max iteration number  $T$  Initialize  $h_\theta = h_{\theta_{bd}}$  .   
for  $t = 0,\dots,T - 1$  do for Each mini-batch in  $\mathcal{D}_{cl}$  do Initialize perturbation  $\epsilon$  .   
for  $t_{adv} = 0,\ldots ,T_{adv} - 1$  do Compute gradient  $\pmb{g}$  of e w.r.t. the inner maximization objective in (13). Update  $\epsilon = \prod_{S}(\epsilon +\eta \mathrm{sign}(g))$  where  $\prod$  is the projection operation. end for Update  $\theta$  w.r.t. the outer minimization objective in (13).   
end for   
end for
4.1 Experiment settings
Backdoor attack. We compare SAU with 7 popular state-of-the-art (SOTA) backdoor attacks, including BadNets [15], Blended backdoor attack (Blended) [7], input-aware dynamic backdoor attack (Input-Aware)[37], low frequency attack (LF) [58], sinusoidal signal backdoor attack (SIG) [2], sample-specific backdoor attack (SSBA) [30], and warping-based poisoned networks (WaNet) [38]. To make a fair and trustworthy comparison, we use the implementation and configuration from BackdoorBench [54], a comprehensive benchmark for backdoor evaluation. By default, the poisoning ratio is set to  $10\%$  in all attacks, and the  $0^{th}$  label is set to be the target label. We evaluate all the attacks on 3 benchmark datasets, CIFAR-10 [24], Tiny ImageNet [26], and GTSRB [43] using two networks: PreAct-ResNet18 [18] and VGG19 [41]. Due to space constraints, the results for GTSRB and VGG19 are postponed to Appendix D. Note that for clean label attack SIG, the  $10\%$  poisoning ratio can only be implemented for CIFAR-10. More attack details are left in Appendix C.
Backdoor defense. We compare our method with 6 SOTA backdoor defense methods: ANP [55], Fine-pruning (FP) [31], NAD [28], NC [47], EP [64] and i-BAU [59]. By default, all the defense methods can access  $5\%$  benign training data. We follow the recommended configurations for SOTA defenses as in BackdoorBench [54]. For our method, we choose to generate the shared adversarial example with Projected Gradient Descent (PGD) [33] with  $L_{\infty}$  norm. For all experiments, we run PGD 5 steps with norm bound 0.2 and we set  $\lambda_1 = \lambda_2 = \lambda_4 = 1$  and  $\lambda_3 = 0.01$ . More details about defense settings and additional experiments can be found in Appendix C and D.
Evaluation metric. We use four metrics to evaluate the performance of each defense method: Accuracy on benign data (ACC), Attack Success Rate (ASR), Robust Accuracy (R-ACC) and Defense Effectiveness Rating (DER). R-ACC measures the proportion of the poisoned samples classified to their true label, and ASR measures the proportion of poisoned samples misclassified to the target label. Larger R-ACC and lower ASR indicate that the backdoor is effectively mitigated. Note that the samples for the target class are excluded when computing the ASR and R-ACC as done in BackdoorBench. DER  $\in [0,1]$  was proposed in [65] to evaluate the cost of ACC for reducing ASR. It is defined as follows:
$$
\mathrm {D E R} = \left[ \max  (0, \Delta \mathrm {A S R}) - \max  (0, \Delta \mathrm {A C C}) + 1 \right] / 2, \tag {14}
$$
where  $\Delta$  ASR denotes the drop in ASR after applying defense, and  $\Delta$  ACC represents the drop in ACC after applying defense.
Note: Higher ACC, lower ASR, higher R-ACC and higher DER represent better defense performance. We use **boldface** and **underline** to indicate the best and the second-best results among all defense methods, respectively, in later experimental results.
Effectiveness of SAU. To verify the effectiveness of SAU, we first summarize the experimental results on CIFAR-10 and Tiny ImageNet in Tables 3 and 4, respectively. As shown in Tables 3-4, SAU can mitigate backdoor for almost all attacks with a significantly lower average ASR. For the experiments on CIFAR-10, SAU achieves the top-2 lowest ASR in 4 of 7 attacks and very low ASR for the other 3 attacks. Similarly, SAU performs the lowest ASR in 3 attacks for Tiny ImageNet and negligible ASR in two of the other attacks. Notably, SAU fails to mitigate WaNet in Tiny ImageNet, although it can defend against WaNet on CIFAR-10. As a transformation-based attack that applies image transforms to construct poisoned samples, WaNet can generate triggers with a large  $L_{\infty}$  norm. Specifically, WaNet has average  $L_{\infty}$  trigger norm 0.348 on TinyImageNet and 0.172 on CIFAR-10. Note that for all experiments, we generate adversarial perturbation with  $L_{\infty}$  norm less than 0.2. The large trigger norm of WaNet on Tiny ImageNet poses a challenge for AT-based methods such as i-BAU and the proposed one, which reveals an important weakness of such methods, i.e., their performance may be degraded if the trigger is beyond the perturbation set. A promising approach for this challenge is to consider adversarial examples for the union of multiple perturbation types [34, 45] and we postpone more discussion to Appendix B.
As maintaining clean accuracy is also important for an effective backdoor defense method, we also compare SAU with other baselines in terms of ACC, R-ACC and DER in Table 3 and 4. As SAU
Table 3: Results on CIFAR-10 with PreAct-ResNet18 and poisoning ratio  ${10}\%$  .
Defense	No Defense	ANP [55]	FP [31]	NC [47]
Attack	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER
BadNets [15]	91.32	95.03	4.67	N/A	90.88	4.88	87.22	94.86	91.31	57.13	41.62	68.95	89.05	1.27	89.16	95.75
Blended [7]	93.47	99.92	0.08	N/A	92.97	84.88	13.36	57.27	93.17	99.26	0.73	50.18	93.47	99.92	0.08	50.00
Input-Aware [37]	90.67	98.26	1.66	N/A	91.04	1.32	86.71	98.47	91.74	0.04	44.54	99.11	92.61	0.76	90.87	98.75
LF [58]	93.19	99.28	0.71	N/A	92.64	39.99	55.03	79.37	92.90	98.97	1.02	50.01	91.62	1.41	87.48	98.15
SIG [2]	84.48	98.27	1.72	N/A	83.36	36.42	43.67	80.36	89.10	26.20	20.61	86.03	84.48	98.27	1.72	50.00
SSBA [30]	92.88	97.86	1.99	N/A	92.62	60.17	36.69	68.71	92.54	83.50	15.36	57.01	90.99	0.58	87.04	97.69
WaNet [38]	91.25	89.73	9.76	N/A	91.33	2.22	88.54	93.76	91.46	1.09	69.73	94.32	91.80	7.53	85.09	91.10
Average	91.04	96.91	2.94	N/A	90.69	32.84	58.75	81.83	91.75	52.31	27.66	72.23	90.57	29.96	63.06	83.06
Defense	NAD [28]	EP [64]	i-BAU [59]	SAU (Ours)
Attack	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER
BadNets [15]	89.87	2.14	88.71	95.72	89.66	1.88	89.51	95.75	89.15	1.21	88.88	95.83	89.31	1.53	88.81	95.74
Blended [7]	92.17	97.69	2.14	50.47	92.43	52.13	37.52	73.37	88.66	13.99	53.23	90.56	90.96	6.14	64.89	95.63
Input-Aware [37]	93.18	1.68	91.12	98.29	89.86	2.23	85.20	97.61	90.29	63.36	32.70	67.26	91.59	1.27	88.54	98.49
LF [58]	92.37	47.83	47.49	75.31	91.82	85.98	12.77	55.97	89.09	21.83	64.37	86.67	90.32	4.18	81.54	96.12
SIG [2]	90.02	10.66	64.20	93.81	83.1	0.26	56.68	98.32	85.85	1.28	55.19	98.49	88.56	1.67	57.96	98.30
SSBA [30]	91.91	77.4	20.86	59.74	92.33	10.67	78.60	93.32	88.15	2.17	77.28	95.48	90.84	1.79	85.83	97.01
WaNet [38]	93.17	22.98	72.69	83.38	90.09	86.64	12.54	50.96	90.91	3.37	89.10	93.01	91.26	1.02	90.28	94.36
Average	91.81	37.2	55.32	79.53	89.9	34.26	53.26	80.76	88.87	15.32	65.82	89.61	90.41	2.51	79.69	96.52
Table 4: Results on Tiny ImageNet with PreAct-ResNet18 and poisoning ratio  ${10}\%$  .
Defense	No Defense	ANP [55]	FP [31]	NC [47]
Attack	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER
BadNets [15]	56.23	100.0	0.0	N/A	43.45	0.00	43.13	93.61	51.73	99.99	0.01	47.76	51.52	0.10	50.82	97.59
Blended [7]	56.03	99.71	0.22	N/A	43.93	6.11	17.28	90.75	51.89	95.94	2.1	49.81	52.55	93.21	3.96	51.51
Input-Aware [37]	57.45	98.85	1.06	N/A	35.39	0.00	11.82	88.40	55.28	62.92	24.7	66.88	56.20	0.09	52.19	98.76
LF [58]	55.97	98.57	0.97	N/A	45.69	62.30	14.49	63.00	51.44	95.25	2.42	49.4	52.99	85.56	8.07	55.02
SSBA [30]	55.22	97.71	1.68	N/A	43.36	56.53	17.24	64.66	50.47	88.87	6.26	52.04	52.47	53.47	23.17	70.75
WaNet [38]	56.78	99.49	0.36	N/A	36.16	0.07	31.97	89.40	53.84	3.94	2.38	96.3	53.33	0.23	51.63	97.90
Average	56.28	99.06	0.72	N/A	41.33	20.83	22.66	77.12	52.44	74.48	6.31	58.88	53.18	38.78	31.64	74.50
Defense	NAD [28]	EP [64]	i-BAU [59]	SAU (Ours)
Attack	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER	ACC	ASR	R-ACC	DER
BadNets [15]	46.37	0.27	45.61	94.93	52.30	0.03	52.05	98.02	52.67	98.31	1.59	49.06	51.52	0.53	51.15	97.38
Blended [7]	46.89	94.99	2.63	47.79	51.86	56.03	12.31	69.75	52.73	92.96	4.15	51.72	50.30	0.06	25.27	96.96
Input-Aware [37]	47.91	1.86	43.13	93.73	57.23	0.24	24.74	99.20	50.63	56.12	24.86	67.96	54.11	0.33	42.29	97.59
LF [58]	45.45	50.49	20.21	68.78	53.33	75.41	13.06	60.26	52.77	88.04	6.82	53.67	52.65	0.97	35.87	97.14
SSBA [30]	45.32	57.32	19.44	65.25	48.13	44.07	21.98	73.27	52.52	74.77	14.31	60.12	51.85	0.11	36.36	97.11
WaNet [38]	46.98	0.43	43.99	94.63	56.21	0.23	55.13	99.34	54.40	94.86	3.92	51.12	54.65	85.75	10.35	55.80
Average	46.49	34.23	29.17	73.59	53.18	29.33	29.88	78.55	52.62	84.18	9.28	54.81	52.51	14.62	33.55	84.57
adopts adversarial examples to mitigate backdoors, it has negative effects on clean accuracy, resulting in a slightly lower clean accuracy compared to the best one. However, SAU achieves the best average R-ACC, which demonstrates its effectiveness in recovering the prediction of poisoned samples, i.e., classifying the poisoned samples correctly. Besides, the best average DER indicates that SAU achieves a significantly better tradeoff between clean accuracy and backdoor risk.
Influence of poisoning ratio. To study the influence of the poisoning ratio on the effectiveness of SAU, we test SAU with varying poisoning ratios from  $10\%$  to  $50\%$ . As shown in Table 5, SAU is still able to achieve remarkable performance even if half of the data is poisoned.
Table 5: Results on CIFAR-10 with PreAct-ResNet18 and different poisoning ratios.
Poisoning Ratio	10%	20%	30%	40%	50%
Attack	Metric	No Defence	Ours	No Defence	Ours	No Defence	Ours	No Defence	Ours	No Defence	Ours
BadNets [15]	ACC	91.82	89.05	90.17	89.27	88.32	88.49	86.16	87.30	83.61	86.93
ASR	93.79	1.53	96.12	1.17	97.33	1.72	97.78	1.72	98.39	2.31
Blended [7]	ACC	93.69	90.96	93.00	91.02	92.78	89.49	91.64	89.64	91.26	88.92
ASR	99.76	6.14	99.92	7.99	99.98	11.58	99.96	5.36	100.00	4.63
LF [58]	ACC	93.01	90.32	92.60	91.33	92.01	89.65	92.03	89.09	90.91	89.02
ASR	99.06	4.18	99.54	1.90	99.76	2.02	99.84	1.94	99.89	1.77
In this section, we conduct ablation study to investigate each component of SAU. Specifically, SAU is composed of two parts: generating shared adversarial examples according to the maximization objective in (13) denoted by  $\mathbf{A}$ , and reducing shared adversarial risk according to the minimization objective in (13), denote by  $\mathbf{B}$ . To study the effect of these parts, we replace them with the corresponding part in vanilla adversarial training, denoted by  $\mathbf{C}$  and  $\mathbf{D}$ , respectively. Then, we run each variant 20 epochs on Tiny ImageNet with PreAct-ResNet18 and a poisoning ratio  $10\%$ . Each experiment is repeated five times, and the error bar is only shown in Figure 3 for simplicity.
As shown in Table 6, the maximization objective for generating shared adversarial examples (component A) is the key to reducing ASR, and component B can help to alleviate the hurt to clean accuracy. When replacing the component B with the minimization step in vanilla AT (D), it suffers from a drop of clean accuracy, although it can also work for mitigating backdoor effectively. Compared to vanilla AT, SAU achieves significantly lower ASR and higher ACC, with a much more stable learning
Table 6: Results on Tiny ImageNet with different variants of SAU and Vanilla AT.
Attack	BadNets	Blended	LF
Defense	ACC	ASR	ACC	ASR	ACC	ASR
A+B (Ours)	50.60	0.27	51.47	1.02	51.31	0.72
A+D	37.21	0.30	38.35	1.77	36.68	0.17
C+B	53.05	28.57	53.16	69.22	52.57	68.65
C+D (Vanilla AT)	46.66	2.48	46.39	38.03	46.69	13.68
curve, as shown in Figure 3. We remark that the decrease of clean accuracy for  $\mathbf{A} + \mathbf{D}$  and vanilla AT demonstrates that imposing a model to learn overwhelming adversarial examples may severely hurt the clean accuracy, a phenomenon widely studied in the area of adversarial training [46, 60].
Figure 3: Learn curves for SAU, Vanilla AT and their variants, averaged over five runs.
In conclusion, this paper proposes Shared Adversarial Unlearning, a method to defend against backdoor attacks in deep neural networks through adversarial training techniques. By developing a better understanding of the connection between adversarial examples and poisoned samples, we propose a novel upper bound for backdoor risk and a bi-level formulation for mitigating backdoor attacks in poisoned models. Our approach identifies shared adversarial examples and unlearns them to break the connection between the poisoned sample and the target label. We demonstrate the effectiveness of our proposed method through extensive experiments, showing that it achieves comparable to, and even superior, performance than six different state-of-the-art defense methods on seven SOTA backdoor attacks with different model structures and datasets. Our work provides a valuable contribution to the field of backdoor defense in deep neural networks, with potential applications in various domains.
Limitation and future work. One important direction for future work, and a current challenge, is the accessibility of clean samples. A valuable approach to this challenge is to consider the samples from other domains and the generated samples.
Structure of Appendix. The detailed proof, analysis, and extension of propositions are given in Appendix A. Some important discussions such as the computation cost, the scalability, and the generalization ability of the proposed method are provided in Appendix B. The details of the experiments are provided in Appendix C. Additional experiments on different settings such as model structures, poisoning ratios, and sizes of clean datasets are given in Appendix D.
This work is supported by the National Natural Science Foundation of China under grant No. 62076213, Shenzhen Science and Technology Program under grant No. RCYX20210609103057050, No. GXWD20201231105722002-20200901175001001, and No. ZDSYS20211021111415025, and No. JCYJ20210324120011032 and the CAAI-Huawei MindSpore Open Fund, and the Guangdong Provincial Key Laboratory of Big Data Computing, the Chinese University of Hong Kong, Shenzhen.