Taught Well Learned Ill: Towards Distillation-conditional Backdoor Attack
Authors: Yukun Chen
Boheng Li
Yu Yuan
Leyi Qi
Yiming Li
Tianwei Zhang
Zhan Qin
Kui Ren
Venue: NeurIPS 2025 poster ()
Keywords: Backdoor Attack
Knowledge Distillation
Backdoor Learning
AI Security
Trustworthy ML
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Knowledge distillation (KD) is a vital technique for deploying deep neural networks (DNNs) on resource-constrained devices by transferring knowledge from large teacher models to lightweight student models. While teacher models from third-party platforms may undergo security verification (e.g., backdoor detection), we uncover a novel and critical threat: distillation-conditional backdoor attacks (DCBAs). DCBA injects dormant and undetectable backdoors into teacher models, which become activated in student models via the KD process, even with clean distillation datasets. While the direct extension of existing methods is ineffective for DCBA, we implement this attack by formulating it as a bilevel optimization problem and proposing a simple yet effective method (i.e., SCAR). Specifically, the inner optimization simulates the KD process by optimizing a surrogate student model, while the outer optimization leverages outputs from this surrogate to optimize the teacher model for implanting the conditional backdoor. Our SCAR addresses this complex optimization utilizing an implicit differentiation algorithm with a pre-optimized trigger injection function. Extensive experiments across diverse datasets, model architectures, and KD techniques validate the effectiveness of our SCAR and its resistance against existing backdoor detection, highlighting a significant yet previously overlooked vulnerability in the KD process. Our code is available at https://github.com/WhitolfChen/SCAR.

Taught Well Learned III: Towards Distillatoo-conndttinaal Backdoor Attack
Yukun Chen Boheng Li Yu Yu, Leyi Qi
Yiming Lis.e, Tianwei Zhang'. Zhan Qin Kui Ren
State Key Laboratory of Blockchain and Data Security, Zhejiang University
HHangzhou High- Tech Zone (Binjiang) Institute of Blockchain and Data Security
Nanyang Technological University
(yukunchen, qinzhan, kuiren)ezzj.ede hdt.c/uuuuuuuz// en, BOHENGO01 @ê ntu www.nnuuuuuu...... edu (yuyuan21cath, liyiming tech) com; leyi-qi reyi-qi@outio @out.lok.ccom
tianwei. zhangentu... www.eeett.eeeeeeb........ edu.sg
Abstract
Knowledge distillation (KD) is vital technique for deploying deep neural net- works (DNNs) on resource-constrained devices by transferring knowledge from large teacher models to lightweight student models. While teacher models from third-party platforms may undergo security verification (e.g.. backdoor detection), we uncover a novel and critical threat: eestilaatio-ccoditiinn backdoor attacks (DCBAs), DCBA injects dormant and undetectable backdoors into teacher mod- els, which become activated student models via the KD process, even with clean distillation datasets. While the direct extension of existing methods is ineffective for DCBA, we implement this attack by formulating it as a bilevel optimization problem and proposing a simple yet effective method (i.c., SCAR). Specifically, the inner optimization simulates the KD process by optimizing a surrogate stu- dent model, while the outer optimization leverages outputs from this surrogate optimize the teacher model for implanting the conditional backdoor. Our SCAR addresses this complex optimization utilizing implicit differentiation algorithm with a pre-optimized trigger injection function. Extensive experiments across di- verse datasets, model architectures, and KD techniques validate the effectiveness our SCAR and its resistance against existing backdoor detection, highlighting significant yet previously overlooked vulnerability in the KD process. Our code is available phtttps: //ttt////tthbbbbb www./WiioolCChee/SSCCCcc SCAR
Introduction
Deep Neural Networks (DNNs) have achieved excellent performance, leading to their widespread adoption in numerous safety-critica domains [
42) To achieve better performance, DNNs are typically designed to be deeper and wider [19 29 However, due to limitations of computational and memory resources, such heavy models are clumsy deploy resource-consting constraind devices (e.g.,
34
loT devices). Knowledge distillation (KD) 24 59 83, a technique that enhances the performance of lightweight models (student) by transferring knowledge from larger models (teacher), has gained increasing popularity. With fewer training steps and less data, KD enables small models to achieve accuracy and generalization performance comparable to large models [56). Therefore, an increasing number of users obtain powerful yet cumbersome large models from third-party platforms (e.g.. GitHub 69 and Hugging Face (36) to serve as teachers for training lightweight student models.
'The first three authors contributed equally to this work. Corresponding author: Yiming Li.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Model Provider
Platform
Model Developer
(Attucker)
(Verifier)
(Victim)
SCAR
Benign Sumple
Poisoned Sample
Target Label: Plane Trigger Pattem:
Upload
(pre-optimized)
Surrogate Model
Teacher Model
Backdoor Detection
Release
Knowledge Distillation (KD)
Implicit Differentiaion Algorithm
"No Backdoor Detected"
Dog
Dog
Dog Plane
The attack scenario of our SCAR. A malicious model provider (i.e., attacker) leverages SCAR to implant dormant backdoor into the model, which behaves normally even when fed with poisoned inputs. The compromised model is then uploaded to a third-party platform (i.e., verifier) for backdoor detection. Upon passing the security check, the model is released to model developers (i.e., victim). When the developer utilizes the original model for inference, it performs as expected. However, once it undergoes further development via KD (with benign samples), inputs with the attacker-specified trigger can activate the backdoor in the student model, leading to misclassification.
Figure I:
However, such third-party models may introduce security vulnerabilities to malicious attacks. Among these, backdoor attacks, which implant hidden malicious behavior during model train- ing [16211]44) pose an especially significant concern. backdoor-compromisee model operates as expected under normal conditions but exhibits attacker-specified malicious behavior when a specific trigger condition is satisfied [27 61 78) To ensure the security of third-parry models, platforms like Hugging Face are increasingly implementing security validation for models uploaded by providers, with backdoor detection serving as a crucial component [33 37. Once a model passes this valida- tion, it is often deemed secure and released to developers for further development. Developers might subsequently employ these 'safeey-verified' models as teachers to guide the training of lightweight student models via KD. This practice, however, introduces compelling research question: the teacher model is verified as 'backdoor-free and the distillation dataset is clean, can we truly guar- antee that the resulting student model is also free from backdoor threats?
Unfortunately, the answer to the above question is negative, although executing such backdoor at- tacks the student model is not trivial. In this paper, we explore the potential threat of distillation- conditional backdoor attacks, where the backdoor remains dormant and undetectable in the teacher model but becomes activated in the student model through KD, as shown in Figure [ Arguably, the most straightforward method to design these attacks is to extend the anti-distillation backdoor attack (ADBA) 23 whose backdoor could be preserved during the KD process. In general, this exten- sion involves fine-tuning to slightly "mask' the backdoor within a model compromised by ADBA (dubbed 'ABAA (FT)') make the logit of the ground -ound-truth label slightly higher (instead of signif- icantly lower) than that of the attacker-specified target label when processing poisoned inputs. This manipulation simultaneously conceals the model's backdoor behavior while positioning its decision boundary for poisoned inputs near a vulnerable tipping point. Intuitively, KD may cause shift in the student model's decision boundary relative that of the teacher model, thereby reactivating the masked backdoor. However, we find that this method often fails to implant the backdoor into student models. We argue that this is probably because the lne-tuning process masks the teacher's backdoor solely based on its own behavior, lacking guidance derived from the dynamic KD process.
Motivated by aforementioned findings and understandings, formulate the compromised model training process as a bilevel optimization problem and propose SCAR (Stealthy distillation- Conditional bAckdooR attack) inject distilation-conditional backdoors into teacher models. In general, we derive a surrogate model via finite KD optimization steps and optimize the teacher model utilizing outputs from the surrogate. Specifically, this bilevel problem, the inner optimiza- tion minimizes the output prediction discrepancy between the surrogate (student) and the teacher (simulating the KD process), while the outer optimization maximizes the attack performance of poi- soned samples against the surrogate, without compromising the teacher's accuracy and robustness. To address the challenge, where the teacher's gradient cannot be computed directly due to the inner loop and the surrogate being separate entity, we derive an implicit differentiation algorithm [25 to capture the interdependence between the inner and outer optimization. We approximate the teacher's gradient utilizing reverse-mode modee automatic differenniaaion with finite number of fixed-poin point itera-

---

tions. In particular, we pre-optimize the trigger pattern to further simplify the optimization of the above bilevel problem and reduce the parameter search space. Our attack alerts developers notice potential false sense of security or consensus that distilling student model based on a 'backdoor- free' teacher model with benign samples is always safe. As such, developers should always detect the distilled student model, no matter whether the teacher model is regarded as "secure".
In summary, our main contributions are three-fold: (1) We introduce a novel backdoor attack paradigm, i.e., distillation-condiioon teacher model can be activated in the student model via the KD process even with benign sam- ples. (2) We reveal that the direct extension of existing methods (c.g., ADBA (FT)) is ineffective as a DCBA and its potential reasons. Based on these understandings, we formulate the DCBA as a bilevel optimization problem and propose simple yet effective method (i.e., SCAR), Our SCAR derives an implicit differentiation algorithm and leverages a pre-optimized trigger pattern to solve this problem. (3) We conduct extensive experiments across multiple datasets, model architectures, and KD techniques (simulating diverse developer behaviors), demonstrating the effectiveness of SCAR and its resistance to potential backdoor detection. Our work highlights the urgent need to always detect the distilled student model, no matter whether the teacher model is deemed secure.
backdoor attack (DCBA), where dormant backdoors in the
2 Background
2.1 Knowledge Distillation
aims to transfer knowledge from powerful yet cumber- some teacher model to lightweight student model, enabling performant deployment of DNNs on resource-limited devices (e.g.. loT devices). Typically, the training objective for the student model involves minimizing a composite loss function defined as: L. = LCE + LLD. In this formulation, LCE is the cross-entropy loss between the student's output logits and the ground-truft labels, and LKD denotes the distillation loss that encourages the student model to mimic the output distribution or intermediate features imparted by the teacher model. is a balancing coefficient.
Knowledge distillation (KD) 3 30 35
Depending on how knowledge is extracted and transferred via LKD, existing KD methods can be classified into three types 224 59 83 (1) Response-based KD (30, which directly aligns the out- put distribution (e.g.. logits or probabilities) of student models with that of teacher models, often utilizing metrics like Kullback- Bullback-Leibler (KL) divergence [30]. (2) Feature-based KD [3), focusing on matching intermediate representations between the student and teacher models, such as feature maps 3 68 or attention maps 87 (3) Relation- KD (35) which transfers structural knowl-
edge from teacher models to student models by capturing and aligning relationships, including rela-
tional information between different data points [35] (64 or features (73]
2.2 Backdoor Attack
Backdoor attacks [46] aim to embed hidden malicious behaviors into DNNs during training, typi- cally by poisoning a subset of the training data with predefined trigger patterns. The compromised model behaves normally on benign inputs but exhibit attacker-specified behavior when particular trigger condition is met. The earliest backdoor attack can be traced back to BadNets (27), which in- jects backdoors by adding small white patch a subset of the training samples and changing their labels to target class. Subsequent work has explored various backdoor attack methods, includ- ing the design of invisible (],60) or diversified [75] 90 triggers, attacks effective in physical-world
scenarios [1 79), and those targeting specific tasks 2 54] among other advancements [17 22 77
Backdoor Attacks against Knowledge Distillation. Since the student model typically learns only the benign behavior of the teacher from a clean distillation dataset, most existing backdoors strug- gle to survive the KD process (86). Currently, some studies 4 23 have focused on designing distillatiinn-resistan backdoor attacks. Anti-Distillation Backdoor Attack (ADBA) 23 is the first to introduce the idea of training a distillation backdoor in the teacher model by leveraging a shadow model to simulate the KD process. Recently, a backdoor attack targeting Seature-based KD is proposed, which encodes the backdoor knowledge into specific neuron activation layers [4] In addition, backdoors have also been shown to potentially survive the KD process in large lan- guage models 77. However, to the best of our knowledge, it remains unexplored whether distilling (seemingly) clean model using a benign dataset could still result in backdoored student models.
3

---

2.3 Backdoor Detection
To determine whether model contains backdoor threats, researchers have proposed various detec- tion methods, which can be broadly categorized into two types: (1) Trigger inversion- basod meth- ods, which aim to reverse-engineer potential -peeciee-ppeeified backdoor triggers. Neural Cleanse (NC) [74], one of the earliest methods, attempts to reconstruct such triggers through optimiza- tion techniques and identifies backdoors by analyzing properties of the recovered triggers, such as whether their size is abnormally small. Most recently, BTI-DBF [82] enhances trigger inversion by decoupling benign features and simultaneously minimizing the discrepancies in benign features while maximizing those in poisoned features. (2) Trigger inversion-free methods, which detect backdoors without explicitly reconstructing triggers, typically by adopting specific strategies. For example, some methods detected backdoors by examining abnormal behaviors of poisoned samples during inference, such as scaled prediction consistency [28 32] and concatenation unalignment [85].
More details about related work are in Appendix H
3 Methodology
3.1 Threat Model
As illustrated in Figure 1 we consider a three- parre--patty scenario in which malicious model provider (i.e., attacker) uploads a compromised pre-trained model to a third-party platform (i.e., verifier). The platform typically supports backdoor detection but rarely offers mitigation, constrained by lim- ited data and computational resources. Once the model passes detection, it is released to downstream developers (i.e., victims). Although appearing benign, the model carries dormant backdoor that remains inactive until the developer applics knowledge distillation (KD), at which point the student model misclassifies inputs containing attacker-speciffed triggers.
Attacker's Goals. The attacker aims to train a DNN model with dormant backdoor, achieving the following three objectives: (1) The model achieves correct classification on both benign and poisoned inputs, making the backdoor inactive. (2) The model can evade detection by third- partyparty backdoor detection; specifically, existing detection techniques fail to identify the dormant backdoor. (3) After undergoing any form of KD to the model, the resulting student model obtains good perfor- mance on benign samples but exhibits misclassification on poisoned ones.
Attacker's Capability, We primarily focus on the common attack scenario involving pre-trained models [1]. Specifically, the attacker has full control over the training phase of the compromised model, including the training dataset, optimization algorithm, loss function, and hyperparameters. However, the attacker can only provide pre-trained model, without access to any other information. In particular, they are unaware of which backdoor detection method will be used by the third -arty verifier, as well as which specific KD technique will be adopted by the model developer.
3.2 An Ineffective Baseline: Anti-Distillation Backdoor Attack with Fine-tuning
Arguably, achieve the attack goals, we first train teacher model with distillation-resistan backdoor and then fine-tune it to slightly 'mask' the backdoor. Intuitively, this manipulation si- multaneously conceals the model's backdoor behavior while positioning its decision boundary for poisoned inputs near a vulnerable tipping point, which can be shifted back to reactivate the masked backdoor during the KD process. Currently, anti-distillation backdoor attack (ADBA) 23 can in- ject such distillation-rresistan backdoors into the teacher model. As such, during fine-tuning, we mask the backdoor in the ADBA-compromiser teacher model by adjusting the logits such that, for poisoned inputs, the logit of the ground -tutt label is only slightly higher (instead of significantly lower) than that of the attacker-specifiee target label. Formally, given such teacher model F (:: already, its corresponding trigger inject function G-)., and a fine-tuning dataset D' = {(x,, y:) 1= this straightforward baseline (dubbed 'ADBA (FT)") aims to solve the following problem:
min 1
[LCE(F((2;))) A), vi) n max max{F (F((( A)),, + k F((GG((a); A), 031, (D..)DD
Σ
M
(1)
where Log denotes the standard cross-entropy loss, & and n are hyper-parameters, (/ X)ly, and ([ (x,) ); represent the output logits (G' (Zx))) X) corresponding to the ground-truth

---

and target label, respectively. The first term of the above optimization ensures accurate predictions on benign samples, while the second term enforces that the logit of the ground-truth label is at least higher than that of the target label. However, we will show that ADBA (FT) has limited effectiveness in attacking student models in many cases. We find that this ineffectiveness is probably due to the fine-tuning relying solely on the teacher's output, without accounting for the dynamic KD process.
3.3 The Design of SCAR
Motivated by the above findings and insights, propose SCAR (Stealthy distillatinn bAckdooR attack), which can effectively leverage information from the KD process. Specifically, we formalize the attack goals bilevel optimization problem and develop appropriate optimiza- tion strategy. We also pre-optimize the trigger injection function to ensure a favorable initialization for effectively and efficiently solving the bilevel problem. Besides, we provide a preliminary analy- sis to explain the effectiveness of our SCAR in Appendix A
3.3.1 Optimization Objective of SCAR
Here, we formalize the attack goals, which also define the optimization objective of SCAR, as bilevel optimization problem. Given a training dataset D = {(x,, y1) we specify a trigger in- jection function G(-) and target label yr to train compromised (teacher) model Fo(:; A) with pa- rameter A. To simulate the KD process of student models, we introduce a surrogate model F.(;, w), parameterized by w. Formally, we aim to solve the following bilevel optimization problem:
Cout (w(), A) A
Σ
[FE((F( A).
v.) a (cE(F((G(::):A)..u)
N
(DDDD
+B CCE(C
(F,(((()))))) +
(2)
s.t. wA) E arg min Cin Cin(w, A)
A
1
Σ
[CE((F(():))..))
N
3
+6 (F,(F, Fi A))
where LCx denotes the standard cross-entropy loss, and LKD represents the knowledge distillation loss, computed as the KL divergence between the output logits of Fi and F.. The scalars o, B and y are temperature coefficients, and is a balancing coefficient. In the outer optimization, the first two terms of Lout reflect the attacker's objective for the teacher model to behave normally on both benign and poisoned samples, while the last two terms aim to ensure that the surrogate model performs normally on benign samples but exhibits backdoor behavior on poisoned ones. The inner optimization is designed to ensure that the surrogate model closely mimics the distillation process of unknown student models. In particular, optimizing the teacher model parameters À requires com- puting the gradient of Lout with respect to A. Since the last two terms of Lout involve Fe (=(:((A)) (A)) and are thus implicitly dependent on À via ((X), the gradient computation must account for the dependence of the inner solution w(A) on A. Due to the presence of an inner optimization loop and the fact that the surrogate model is separate entity, the gradient of À cannot be directly com- puted via standard backpropagation. To tackle this challenge, we derive an implicit differentiation algorithm to approximate the gradients arising from these two terms, effectively capturing the w-> interdependence. Its technical details are in the following subsection.
3.3.2 Optimization Strategy of SCAR
We hereby derive an implicit differentiation algorithm to estimate the gradient V with respect to the parameters À of F,, The overall training process of our SCAR is outlined in Algorithm | In the following, we provide a detailed explanation of how to compute this gradient.
Finite Inner Optimization Updates. Intuitively, key step computing VxCout to identify w (A), the solution to the inner optimization problem:
"/ (A) = arg min Lin (w, A). w
(3)
Let w² w'(A) be suboptimal solution which satisfies the first-order optimality condition:
V Ww (>)) 0.
(4)
5

---

Algorithm SCAR Training Process
Input: Model Fi(:; A), Surrogate Fa F:):,), Trainset D, Trigger function G(-), Target label y/+ Output: Trained compromised model Ft
Parameters: Fix-point iterations K, Subset batches M, Inner steps T,, Learning rate e and
1: for each outer optimization epoch do
2: Reinitialize wo:
> Initialize inner parameters
3: for - 0 to T do
> Inner loop: Approximate w*(A)
4:
Compute Wwiin (wi, A) with D; Update w1+1 t- Wi - WwLin A);
5:
Eq.
(
6: Select subset D, (M batches from D) for outer gradient estimation;
7:
Compute gw VwLoud (w" A) and gA V>fout A) with D,, G and yı!
8:
Initialize <- 0:
9:
for 0 to K - 1 do
>> Eq.
10:
Compute vn+1 + Jæ, Vn + Swi
11: Compute approximate gradient VxCout 2 gA J VKi Update À À B VxCouti
b Eq. 12
Optimize outer parameters of
13:
return F+
In practice, obtaining the exact wn (A) is often infeasible. We approximate by performing a finite number optimization steps (e.g.. steps of gradient descent) on Lin with respect to w, typically starting from randomly reinitialized w in each outer epoch. We denote the resulting approximation also by w (A) for simplicity in the following derivation.
Derivation of Implicit Differentiation. Given the obtained w (A), using the chain rule, the total gradient of the outer loss with respect to À can be derived as follows:
V VXCoue (A), = (aw(())
T
Low (A), + VXCour(W" (A), ddirect.
(5)
ax
Let gw - VoLout (w (A), A) and gx Coud (A), A)drreed direct. Both gu and ga be com- puted directly with standard backpropagation. To compute the Jacobian term Ow" / we apply the Implicit Function Theorem by differentiating the optimality condition (4) with respect to A:
Ow
Va
bin
+
(6)
Let Hww A Cin be the Hessian matrix with respect w. and Ha^ - VixCin be the mixed partial derivative matrix. Assuming Huw is invertible (guaranteed by strict convexity of Lin), we can solve for the Jacobian as follows:
Ow
= - -
HWA
0x
Substituting Eq. into Eq. 55) have:
VoLou (w" (AA, =BA H H.
(8)
Approximation via Fix-point Iterations. Directly computing and inverting Haw is computation- ally prohibitive for DNNs. Accordingly, employ approximation based reverse-mon mode auto- matic differentiation to implicitly compute the Dector-Hessian-inee inverse product. Consider the inner optimization utilizing T gradient descent steps:
W1+1 = (ww;; A) - W; E. ViLin A),
1=
where e is the inner loop learning rate, and = (*))). Differentiating the fixed-point equation w "(w", X) yields ,w- /0\ = (I - A. where Ja 8b/00u and Jæ, 4 00/0x are Jacobians evaluated at w Substituting this into gives:
VxCout gx + J Jow) gw.
(10)

---

Let (I-Ja.w) gw. which can be solved efficiently utilizing an iterative method like Neumann leading to the iteration:
series expansion [57]
Vo = 0, Vo+i Jo + Bw. = 0,
(11)
This iteration converges to v if the spectral radius o(Jo.w) < 26 52 In practice, we truncate this iteration after finite K steps. Let v be the approximation after K steps. The approximate outer gradient is then computed efficiently utilizing vector-lacobian products:
VoCout = gA + J.K.K.
(12)
which can be leveraged to optimize the parameters À of Ft.
3.3.3 Pre-optimizing for Trigger Injection Function
To simplify the optimization of the above bilevel problem, we attempt to pre-optimize the trigger injection function G(:; ()) to reduce the parameter search space. Following prior backdoor-related works 44 we define G(æ; pi) = [I (2 μ4). where [0 ensures that the poisoned image remains within valid pixel range. Let (-) be pretrained benign teacher model and F, (-) be the student model distilled from it. Then, the training of G aims to minimize the following loss:
min Σ (DDD
CCE(F (G vd + (CC())( vc), s.t. || ED
(13)
where Yt denotes the target label. In general, this pretraining process aims to optimize natural backdoor trigger pattern that can survive the knowledge distillation process, thereby providing a favorable initialization for the subsequent bilevel optimization. We validate the necessity of pre- optimizing the trigger injection function in our subsequent ablation studies.
4 Experiments
In this section, we evaluate the effectiveness of our SCAR across various datasets, model architec- tures, and knowledge distillation (KD) methods. We then conduct an ablation study and evaluate the resistance to potential backdoor detection methods. In addition, the analysis of the overhead of SCAR in Appendix and the visualization SCAR is provided in Appendix T
4.1 Main Settings
Datasets, Models, and KD Methods. We conduct experiments two classical benchmark datasets, including CIFAR-10 [40] and (a subset of) ImageNet [14] containing 50 classes. We utilize rel- atively large models such ResNet-50 (29). VGG-19 (71). and ViT [19] as the compromised teacher models (with ResNet-18 29 utilized as the surrogate model), and distill each of them into lightweight student models including MobileNet-W V2 (70), ShuffleNet-V V2 (55), and EfficientViT [50] to validate the effectiveness of SCAR. During KD, we adopt three different types of methods: (1)
Response-basse based KD B00 (2) Feature-based KD (3, and (3) Relation based KD [35]. Note that our goal to evaluate the effectiveness of our attack method rather than to train a SOTA model. Therefore, the benign accuracies of our models may be lower than those of SOTA counterparts.
Attack Setup. Note that since our SCAR is the first conditional backdoor attack designed specif- ically for the knowledge distillation setting, forward baseline ADBA (FT) introduced in Section 3.2 For reference, we also report the results of benign models trained without any attack (dubbed 'Benign'). More attack details are in Appendix
only compare its performance with a straight-
Evaluation Metric. We evaluate the attack performance of different methods based on the backdoor attack success rate (ASR), defined as the accuracy on poisoned samples, for both the teacher and student models. Specifically, for the teacher model, we aim for dormant and inactive backdoor, which is reflected by a low ASR; whereas for the student model, we seek a strong backdoor effect, indicated by high ASR. An effective distillation-condttion ASR on the teacher model and higher ASR on the student models.
backdoor attack is indicated by a lower
7

---

Table 1: The attack performance (%) CIFAR-10 and ImageNet. For each case, the best results are boldfaced, while all failed cases (student ASR < 50%) are marked in red.
ResNet-50
MobileNet-V2
ShufileNet-V2
EtticienVViT
Model
Dataset
KD Method
(Teachen)
(Studm
(Studen:
(Sudeex
Attack
ACC
ASR,
ACC
ASR
ACC
ASRT
ACC.
ASR
Benign
94.12
0
91.92
0
89.76
86.86
Response
ADBA (FT)
90.58
6.88
91.07
92.87
85.86
81,02
86.88
30.58
SCAR
92.47
1,50
91.62
99.94
89.15
99.02
86.82
86.31
Benign
94.12
0
90.92
0
89.73
0
86.
CIFAR-I0
Feature
ADBA (FT)
90.58
6.88
90.87
98.47
85.45
49. 28
86. 70
31.22
SCAR
92.47
1.50
91.01
99.90
88.48
98.22
87.74
77.28
Benign
94.12
91.77
0
89.54
86.88
Relation
ADBA (FT)
90.58
6.88
91.18
98.66
85.45
71.02
86.74
34.78
SCAR
92.47
1.50
91.29
99.93
88.25
98.44
85.78
90.09
Benign
70.08
70.36
0
65.00
60.32
Response
ADBA (FT)
61.56
2.53
61,00
45.39
60.48
37.51
56. 36
13.31
SCAR
64.28
2.12
63,80
81.69
63.12
72.86
60.00
53,55
Benign
70.08
0
69.48
0
66.32
60.44
ImageNet
Feature
ADBA (FT)
61,56
2.53
61.16
37.92
60.60
24.57
59.04
36.20
SCAR
64.28
2.12
64.32
74.29
62.04
$7.63
57.04
52.98
Benign
70.08
70.48
0
63.52
56.80
Relation
ADBA (FT)
61.56
2.53
61.80
42.61
61.36
20.08
55.72
19.22
SCAR
64.28
2.12
63.28
91.96
64.00
62.61
58.48
61.18
Table 2: The attack performance (%) of SCAR with/without Fs or G on CIFAR-10. For each case, the best results are boldfaced, while all failed cases (student ASR < 50%) are marked in red.
ResNet-50
MobileNet-V2
ShuffleNet-V2
EfficientVIT
Model
KD Method
(Texches)
(Stulen
(Stokent
(Sudeen C)
Attack
ACC
ASR
ACC
ASR
ACC.
ASR
ACC
ASR1
w/o F.
89,82
2.61
88.15
82.92
87.19
51.58
87.76
31.42
Response
w/o G
93.81
0.72
91.47
1.03
89.47
1.06
86.05
2.09
SCAR
92.47
1.50
91.62
99.94
89.15
99.02
86.82
86.31
w/o F.
89.82
2.61
87,94
83.68
88.04
43.11
86.11 11
5.29
Feature
w/o G
93.81
0.72
91.49
1.48
88.91
1.44
87.69
1.33
SCAR
92.47
1.50
91.01
99.90
88.48
98.22
87.74
77.28
w/o F,
89.82
2.61
87.66
80.13
89.22
81.28
86.48
$4.19
Relation
w/o
93.81
0.72
91.22
1.38
88.92
1.50
86.78
SCAR
92.47
1.50
91.29
99.93
98.44
85.78
90.09
4.2
Main Results
As shown in Table [ compared to the baseline attack method, SCAR achieves a higher attack suc- cess rate (ASR) on student models while maintaining a low ASR on the teacher model. Specifically, although ADBA (FT) effectively reduces the ASR on the teacher model (ASR < 10%). its effective- ness in attacking student models limited. For instance, on the CIFAR-10, ADBA (FT) struggles to implant the backdoor into the student model EfficientViT, likely due to substantial architectural differences from the teacher model ResNet-50. On the ImageNet, ADBA (FT) fails to perform the attack altogether. In contrast, SCAR successfully injects backdoors into student models under all three knowledge distillation methods on the CIFAR-1O. While its performance declines on the ImageNet, the ASR remains within acceptable range. This degradation may be attributed to the increased image size in ImageNet, which makes the convergence of the bilevel optimization problem more challenging. Results for the VGG-19 and ViT models are provided in the Appendix B
4.3 Ablation Study
Our method comprises two key components: (1) optimization involving surrogate model J., and (2) a pre-optimized trigger injection function G. To evaluate the effectiveness of each component separately, we conduct an ablation study using ResNet-50 attacked on CIFAR-10. Specifically, to assess the contribution of F., we directly fine-tune the teacher model using only G, removing the
8

---

Table 3: The backdoor detection performance of NC teacher models compromised by SCAR.
Class 0 is the attack-defined target label. False detections are marked in red.
Anomaly Index of Each Class (> 2 indicates potential backdoor)
Model
Class
Class
Class
Class 3
Class
Class
Class
Class
Class
Class
ResNet-50
0.
0.
1.09
0.50
0.02
0.
1.15
0.85
VGG-19
0.
0.
0.
1.09
0.
0.07
2.01
0.91
0.
2.10
VIT
0.
0.
0.
0.
1,02
0.
0.98
0.73
0.70
0.81
involvement of F, entirely. To evaluate the necessity of G, we replace it with a fixed white patch located at the bottom-right corner of the image as the trigger pattern. We further perform addtional ablation studies on the effects of the surrogate model architecture, the distillation loss weight 5, the number of inner steps T, the bilevel optimization hyperparameters (a, 3, and y), and diverse triggers substituting for G, as illustrated in Appendix E
As shown in Table 2 we evaluate the attack performance of SCAR without the surrogate model (w/o F.)) or without the pre-optimized trigger injection function (w/o G). Experimental results indicate that, without Fs, the effectiveness of SCAR significantly degrades or even fails. This is because the teacher model is trained without guidance from the dynamic KD process, leading to deviation in the optimization direction. Additionally, without G, the teacher model attacked by SCAR fails to transfer backdoor knowledge to student models. This may be attributed to the inherent complexity and instability of the bilevel optimization problem, which makes initialization critically important.
4.4 Resistance to Potential Backdoor Detection
In this section, we empirically evaluate the evasion performance of SCAR against potential backdoor detection methods. We focus on scenario where third-party trusted verifier employs such methods identify backdoors within the teacher model. To mimic this verification process, adopt two representative backdoor detection techniques, Neural Cleanse [74) and SCALE-UP (28). to assess the stealthiness of SCAR-injected backdoors in teacher models trained on CIFAR-10 10. The results demonstrating the evasiveness of SCAR against more advanced detection methods are provided in while its robustness against non-detection defenses is reported in Appendix
Appendix C
Neural Cleanse (NC) is a classical backdoor trigger inversion base)- detection method. leverages the backdoor property that
small perturbation cause inputs to be misclassified into the target label. NC first computes universal adversarial perturbations (UAP) for each one-to-one misclassification as potential triggers, then quantifies these triggers using the -nnrmm, and finally applies an anomaly detection to identify the most likely true trigger. If the anomaly index exceeds 2, the model is considered to have backdoor with 95% confidence; otherwise, it is regarded clean.
As shown in Table we evaluate the anomaly index corresponding to the UAP for each class. Only indices greater than zero are retained, where higher anomaly index indicates smaller 41- norm of the UAP and thus a higher likelihood of a backdoor. The results show that for ResNet-50 and VIT, NC fails to detect any backdoor. For VGG-19, NC yields false positives by mistakenly identifying normal labels, Class and Class 9, as target labels. These findings demonstrate that models compromised by SCAR can effectively evade NC detection.
SCALE-UP is a black-box, input-level detection method based on the observation that, when all pixel values are amplified, poisoned samples exhibit significantly more consistent predictions com- pared to benign ones. It identifies and filters poisoned samples by analyzing the prediction consis- tency of inputs during the pixel amplification process. We detect the presence of backdoors in the teacher model by observing whether poisoned samples exhibit the prediction consistency behavior.
As shown in Figure 2 both benign and poisoned samples exhibit similar scaled prediction incon- sistency across different model architectures compromised by SCAR. Specifically, for conventional backdoor attacks, the prediction confidences on poisoned samples are expected to be more stable than those on benign samples as the amplification times increase, meaning the red curve should con- sistently lie above the blue curve. However, our results show the opposite, with poisoned samples even displaying greater prediction inconsistency. This indicates that SCALE-UP UP fails to detect the dormant backdoor in teacher models attacked by our SCAR.

---

Average Confidence
1.0-
Benign Semple
Average Confidence
1.-
Beriign Semple
Average Confidence
1.3
Berign Semple
Polsoned Sample
0.8
Polsoned Sample
0.3
Poisoned Sample
0.6
0.4
0.4
9.4
0.2
0.2
0.2
Amplification Times
Amplification Times
Amplification Times
(a) ResNet-50
(b) VGG-19
(c) ViT
Figure 2:
The average confidence of benign and poisoned samples with respect to pixel-wise ampli-
fications on three SCAR-attacked models with different architectures trained CIFAR-I0.
5 Conclusion
In this paper, we explored iistillation-conditonn backdoor attacks (DCBAs), where a dormant backdoor in the teacher model can be activated in the student model through knowledge distilla- tion (KD), even with clean KD data. We found that the direct extension existing methods (e.g., ADBA (FT)) is ineffective for DCBA and proposed SCAR, which formulates DCBA a bilevel optimization problem and leverages an implicit differentiation algorithm with a pre-optimized trig- ger injection function to solve it. Extensive experiments demonstrated the effectiveness of SCAR across diverse datasets, model architectures, and KD techniques, as well as its stealthiness against teacher-side backdoor detection. Our findings reveal significant yet previously overlooked security threat, and highlight the urgent need to always perform backdoor detection on student models, even when the teacher model and distillation dataset have been verified secure.
Acknowledgements
This research is supported in part by the "Pioneer" and "Leading Goose" R&D Program of Zhejiang (2024C01669), the Sampeng-Ascend Science and Education Innovation Excellencc/Incubation Cen- ter. the National Natural Science Foundation of China under Grants (62441238, U2441240), and Xiaomi Open- Opee-Coomttition Research Program. This work was mostly done when Yu Yuan and Leyi Qi were Research Assistants at the State Key Laboratory of Blockchain and Data Security, Zhejiang University, China.
References
[1] Stephanie Abrecht, Alexander Hirsch, Shervin Raafatnia, and Matthias Woehrle. Deep learn- ing safety concerns in automated driving perception. IEEE Transactions on Intelligent Vehicles, 2024.
[2] Hanbo Cai, Pengcheng Zhang, Hai Dong, Yan Xiao, Stefanos Koffas, and Yiming Li. Toward stealthy backdoor attacks against speech recognition via elements of sound. IEEE Transactions on Information Forensics and Security, 2024.
[3] Defang Chen, Jian-Ping Mei, Hailin Zhang, Can Wang, Yan Feng, and Chun Chen. Knowledge distillation with the reused teacher classifier. In CVPR, 2022.
[4] Jinyin Chen, Zhiqi Cao, Ruoxi Chen, Haibin Zheng, Xiao Li, Qi Xuan, and Xing Yang. Like teacher, like pupil: Transferring backdoors via feature-based knowledge distillation. Comput- & Security, 146:10404 2024.
[5] Xinyun Chen, Chang Liu, Bo Li. Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint 1xXiv:712.005222 2017.
[6] Yukun Chen, Shuo Shao, Enhao Huang, Yiming Li, Pin- Yu Chen, Zhan Qin, and Kui Ren. REFINE:
Inversion-free backdoor defense via model reprogramming. In ICLR, 2025.
[7] Pengzhou Cheng, Zongru Wu, Tianjie Ju, Wei Du, and Zhuosheng Zhang Gongshen Liu. Transferring backdoors between large language models by knowledge distillation. arXiv preprint arXiv:2408, 09878, 2024.
10

---

[8] Jang Hyun Cho and Bharath Hariharan. On the efficacy of knowledge distillation. In ICCV, 2019.
[9] T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley, 2006.
[10] Csiszár and J. Körner. Information Theory: Coding Theorems for Discrete Memoryless Systems. Applied Language Studies. Academic Press, 1981.
[1]] Thinh Dao, Cuong Chi Le, Khoa D Doan, and Kok-Seng Wong. Towards clean-label backdoor attacks in the physical world. arXiv preprint arXiv:2407.19200 2024.
[12] Luke N Darlow, Elliot J Crowley Antreas Antoniou, and Amos J Storkey. Cinic-10 is not imagenet or cifar- 10. arXiv preprint arXiv:1880.03350, 2018.
[13] David Cohen. http:///ggoooooobbeggggggnnnnnnt
2025.
[14] Jia Deng, Wei Dong, Richard Socher, Li-Jia I.,, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[15) Pasan Dissanayake, Faisal Hamman, Barproda Halder, Ilia Sucholutsky, Qiuyi Zhang, and Sanghamitra Dutta. Formalizing limits knowledge distillation using partial information decomposition. In NeurlPS Workshop, 2024.
[16] Khoa Doan, Yingjie Lao, and Ping Li. Backdoor attack with imperceptible input and latent modification. In NeurlPS, 2021.
[17] Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. Lira: Learnable, imperceptible and robust backdoor attacks. In ICCV, 2021.
[18] Tian Dong. Ziyuan Zhang, Han Qiu, Tianwei Zhang, Hewu Li, and Terry Wang. Mind your heart: Stealthy backdoor attack on dynamic deep neural network in edge computing. In INFO- COM, 2023.
[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
[20] Samar Fares and Karthik Nandakumar. Attack to defend: Exploiting adversarial attacks for detecting poisoned models. In CVPR, 2024.
[21] Yansong Gao, Bao Gia Doan, Zhi Zhang, Siqi Ma, Jiliang Zhang, Anmin Fu, Surya Nepal, and Hyoungshick Kim. Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv preprint arXiv:2007.10766 2020.
[22] Yinghua Gao, Yiming Li, Linghui Zhu, Dongxian Wu, Yong Jiang, and Shu-Tao Xia. Not all samples are born equal: Towards effective clean-label backdoor attacks. Pattern Recognition, 109:10 2023.
[23] Yunjie Ge. Qian Wang, Baolin Zheng, Xinlu Zhuang, Qi Li, Chao Shen, and Cong Wang. Anti-distillation backdoor attacks: Backdoors can really survive knowledge distillation. In MM, 2021.
[24] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation:
A survey. International Journal of Computer Vision, 129(6): 1790:)119999999 2021.
[25] Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, and Saverio Salzo. On the iteration complexity of hypergradient computation. In ICML, 2020.
[26] Riccardo Grazzi, Massimiliano Pontil, and Saverio Salzo. Bilevel optimization with lower- level contraction: Optimal sample complexity without warm-start. Journal of Machine Learn- ing Research, 24(167):1-37 2023.
[27] Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Evaluating back- dooring attacks on deep neural networks. IEEE Access, 7:47230-47244, 2019.
[28] Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency. In ICLR,, 2023.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.

---

[30] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint (rXXi::503.025:53 2015.
[31]
Sanghyun Hong. Michael-Andrei Panaitescu-LLiess, Yigitean Kaya, and Tudor Dumitras. Qu- anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes. In NeurIPS, 2021.
[32] Linshan Hou, Ruili Feng, Zhongyun Hua, Wei Luo, Leo Yu Zhang, and Yiming Li. Ibd-psc: Input-level backdoor detection via parameter-oriented scaling consistency. In ICML, 2024.
[33] Linshan Hou, Wei Luo, Zhongyun Hua, Songhua Chen, Leo Yu Zhang, and Yiming Li. Flare: Towards universal dataset purification against backdoor attacks. IEEE Transactions on Infor- mation Forensics and Security, 2025.
[34] Wenyue Hua, Xianjun Yang, Mingyu Jin, Zelong Li, Wei Cheng, Ruixiang Tang, and Yongfeng Zhang. Trustagent: Towards safe and trustworthy Ilm-based agents, arXiv preprint arXiv:2402.01 2024.
[35] Tao Huang, Shan You, Fei Wang. Chen Qian, and Chang Xu. Knowledge distillation from stronger teacher. In NeurlPS, 2022.
[36] Hugging Face. https//hugginngaae.co/oooddddd 2025.
[37] Hugging Face. http://hnnginggacc..oooocsbbbtts
2025.
[38] Tran Huynh, Anh Tran, Khoa D Doan, and Tung Pham. Data poisoning quantization backdoor attack. In ECCV, 2024.
[39] Wenbo Jiang, Tianwei Zhang. Han Qiu, Hongwei Li, and Guowen Xu. Incremental learn- ing. incremental backdoor threats. IEEE Transactions Dependable and Secure Computing, 2022.
[40] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, 2009.
[41] Huafeng Kuang, Hong Liu, Yongjian Wu, Shin ichi Satoh, and Rongrong Ji. Improving adver- sarial robustness via information bottleneck distillation. In NewrIPS, 2023.
[42] Quentin Le Roux, Eric Bourbao, Yannick Teglia, and Kassem Kallas. A comprehensive survey on backdoor attacks and their defenses in face recognition systems. IEEE Access, 2024.
[43] Boheng Li, Yishuo Cai, Jisong Cai, Yiming Li, Han Qiu, Run Wang. and Tianwei Zhang. Puri- fying quantization conditioned backdoors via layer-wise activation correction with distribution approximation. ICML, 2024.
quantization-conditineee
[44] Boheng Li, Yishuo Cai, Haowei Li, Feng Xue, Zhifeng Li, and Yiming Li. Nearest is not dear- est: Towards practical defense against quantivation-condiiioneel backdoor attacks. In CVPR, 2024.
[45] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In ICLR, 2021.
[46] Yiming Li. Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE transactions on neural networks and learning systems, 35(1):5-22, 2024.
[47] Yiming Li, Mengxi Ya, Yang Bai, Yong Jiang, and Shu-Tao Xia. Backdoorbox: A python toolbox for backdoor learning. In ICLR Workshop, 2023.
[48] Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor attack in the physical world. In ICLR Workshop, 2021.
[49] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against back- dooring attacks on deep neural networks. In RAID, 2018
[50] Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, and Yixuan Yuan. Effi cientvit: Memory efficient vision transformer with cascaded group attention. In CVPR, 2023,
[51] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai Weihang Wang, and Xi- angyu Zhang. Trojaning attack on neural networks. In NDSS, 2018.
[52] Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In A/STATS, 2020.
12

---

[53] Hua Ma, Huming Qiu, Yansong Gao, Zhi Zhang, Alsharif Abuadbba, Minhui Xue, Anmin Fu, Jiliang Zhang, Said F Al-Sarawi, and Derek Abbott. Quantization backdoors to deep learning commercial frameworks. IEEE Transactions Dependable and Secure Computing, 2023.
[54] Hua Ma, Shang Wang, Yansong Gao, Zhi Zhang, Huming Qiu, Minhui Xue, Alsharif Abuadbba, Anmin Fu, Surya Nepal, and Derek Abbott. Watch out! simple horizontal class backdoor can trivially evade defense. In CCS, 2024.
[55]
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guide- lines for efficient cnn architecture design. In ECCV, 2018.
[56] Amir M Mansourian, Rozhan Ahmadi, Masoud Ghafouri, Amir Mohammad Babaei, Ela- heh Badali Golezani, Zeynab Yasamani Ghamchi, Vida Ramezanian, Alireza Taherian, Kimia Dinashi, Amirali Miri, et al. A comprehensive survey on knowledge distillation. arXiv preprint arXiv:2503. 12067, 2025.
[57] Carl D Meyer. Matrix analysis and applied linear algebra. SIAM, 2023.
[58] Xiaoxing Mo, Yechao Zhang, Leo Yu Zhang, Wei Luo, Nan Sun, Shengshan Hu, Shang Gao, and Yang Xiang. Robust backdoor detection for deep learning via topological evolution dy- namics. In IEEE S&P, 2024.
[59] Amir Moslemi, Anna Briskina, Zubeka Dang, and Jason Li. A survey on knowledge distilla- tion: Recent advancements. Machine Learning with Applications, page 100605, 2024.
[60] Anh Nguyen and Anh Tran. impereempeeeep warping-based backdoor attack. In ICLR,
2021.
[61] Thuy Dung Nguyen, Tuan A Nguyen, Anh Tran, Khoa D Doan, and Kok- BookSeng Wong. Iba: Towards irreversible backdoor attacks in federated learning. In NeurlPS, 2023.
[62] Rui Ning, Jiang Li, Chunsheng Xin, Hongyi Wu, and Chonggang Wang. Hibernated backdoor:
A mutual information empowered backdoor attack to deep neural networks. In AAAI, 2022.
[63] Xudong Pan, Mi Zhang, Yifan Yan, and Min Yang. Understanding the threats of trojaned quantized neural network in model supply chains. In ACSAC, 2021.
[64] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In CVPR, 2019.
[65] Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In ICML, 2016.
[66] Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Revisiting the assumption of latent separability for backdoor defenses. In ICLR, 2023.
[67] Arezoo Rajabi, Surudhi Asokraj, Fengqing Jiang, Luyao Niu, Bhaskar Ramasubramanian, James Ritcey, and Radha Poovendran. Mdtd: a multi-domain trojan detector for deep neu- ral networks. In CCS, 2023.
[68] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015.
[69]
rwightman. http://hw....nnoingnoooooooooeee
2025.
[70] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, 2018.
[71] Karen Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1409. 1556, 2014.
[72] Yulong Tian, Fnu Suya, Fengyuan Xu, and David Evans. Stealthy backdoors as compression artifacts. IEEE Transactions on Information Forensics and Security, 2022.
[73] Frederick Tung and Greg Mori.
knowledge distillation. In ICCV, 2019.
[74] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE S&P, 2019.
[75] Shang Wang. Yansong Gao, Anmin Fu, Zhi Zhang, Yuqing Zhang, Willy Susilo, and Dongxi Liu. Cassock: Viable backdoor attacks against dnn in the wall of source-specific backdoor defenses. In AsiaCCS, 2023.
13

---

[76] Zhenting Wang. Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In CVPR, 2022.
[77] Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. Badchain: Backdoor chain-of-thought prompting for large language models. In ICLR, 2024.
[78] Zhen Xiang, David Miller, Siheng Chen, Xi Li, and George Kesidis. A backdoor attack against 3d point cloud classifiers. In ICCV, 2021.
[79] Tong Xu, Yiming Li, Yong Jiang, and Shu-Tao Xia. Batt: Backdoor attack with transformation- based triggers. ICASSP, 2023.
[80] Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A Gunter, and Bo Li. Detecting ai trojans using meta neural analysis. In JEEE S&P, 2021.
[81] Xiaoyun Xu, Zhuoran Liu, Stefanos Koffas, Shujian Yu, and Stjepan Picek. Ban: detecting backdoors activated by adversarial neuron noise. In NeurlPS, 2024.
[82] Xiong Xu, Kunzhe Huang, Yiming Li, Zhan Qin, and Kui Ren. Towards reliable and efficient backdoor trigger inversion via decoupling benign features. In ICLR, 2024.
[83] Chuanguang Yang, Xinqiang Yu, Zhulin An, and Yongjun Xu. Categories of response-bused, feature-based, and relation-base knowledge distillation. In Advancements in knowledge dis- tillation: towards new horizons of intelligent systems, pages 1-32. Springer, 2023.
[84] Yanxin Yang. Chentao Jia, DengKe Yan, Ming Hu, Tianlin Li, Xiaofei Xie, Xian Wei, and Mingsong Chen. Sampdetox: Black-box backdoor defense via perturbation-base sample detoxification. In NeurlPS, 2024.
[85] Biao Yi, Tiansheng Huang, Sishuo Chen, Tong Li, Zheli Liu, Chu Zhixuan, and Yiming Li. Probe before you talk: Towards black-box defense against backdoor unalignment for large language models. In ICLR, 2025.
[86] Kota Yoshida and Takeshi Fujino. Countermeasure against backdoor attack on neural networks utilizing knowledge distillation. Journal of Signal Processing, 24(4):141-144, 2020.
[87] Sergey Zagoruyko and Nikos Komodakis. Paying more attention attention: Improving the performance of convolutional neural networks via attention transfer. ICLR, 2017.
[88] Yi Zeng, Si Chen, Won Park, Zhuoqing Mao, Ming Jin, and Ruoxi Jia. Adversarial unlcarning of backdoors via implicit hypergradient. In ICLR, 2022.
[89] Yechao Zhang, Shengshan Hu, Leo Yu Zhang, Junyu Shi, Minghui Li, Xiaogeng Liu, Wei Wan, and Hai Jin. Why does little robustness help? further step towards understanding adversarial transferability. In IEEE S&P, 2024.
[90] Mingyan Zhu, Yiming Li, Junfeng Guo, Tao Wei, Shu-Tao Xia, and Zhan Qin. Towards sample-specific backdoor attack with clean labels via attribute trigger. IEEE Transactions on Dependable and Secure Computing, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the potential limitations of our SCAR in Appendix K
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specificattin, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't i acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [es]]
15

---

Justification: A detailed derivation of the optimization strategy of SCAR is provided in the main body of the paper. Additionally, a formal proposition and its proof are presented in Appendix A.I.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We introduced the detailed experimental settings Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provided codes the supplementary material. The full codes our method is available at htttt::: //github. wwtp://www/ooooeceee//SS...
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (at cct
pub ic/ des CoodeSuome ss onnooloolic cy) for more details.
While we encourage the release of code and data, understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. cc http:/cccccbbllccc wwwguiiieeeseee Sode/SumeeeeSeeeee ss cessoonoolion llyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification:
We introduced the detailed experimental settings in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [No]
Justification: We did not report the error bars due to the constraints of time and com- putational resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We introduced the experimental computing resources in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Our research conformed with the Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discussed the potential societal impacts in Appendix J
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not release data or models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We discussed the adopted data of this paper in Appendix
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, |ppperswitte... www.swttthhhccc.oc com/ datassts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and sereenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //eur ps. ccsccc//ooeeeeeees///////
2025/LLX for what should or should not be described.
21

---

Appendix
A Why is our SCAR Effective?
In this section, we provide a preliminary analysis of the possible factors contributing to the effec- tiveness of our SCAR, focusing on two key perspectives: the information gap introduced during knowledge distillation (KD) and the distributional similarity between the training and distillation datasets. Besides, the convergence and approximation error bounds of the implicit differentiation algorithm used in SCAR have been theoretically established in existing studies [25 65
A.1 Analysis from the KD Information Gap
In knowledge distillation (KD), a finite clean dataset is typically used to transfer the teacher model's standard inference behavior to the student model. However, this process may result in an information gap, whereby certain security- characteristies of the teacher model not fully inherited by the student. This gap can compromise the robustness of the student model.
Recent studies KD primarily aim to improve the student's performance under capacity constraints by filtering out what is deemed redundant information [5 41. While effective for performance, this strategy may inadvertently discard safety-relevant cues that contribute the teacher model's resilience. In particular, a model's robustness against backdoor attacks often relies not only on task- relevant features but also on seemingly redundant information that plays defensive role. If the dis- tillation process excludes this information, the resulting student though performant-may remain highly susceptible backdoor threats, even cases where the teacher model is unaffected.
Therefore, we argue that the bilevel optimization process of SCAR, which injects distillation- conditional backdoors, may exploit the information gap stemming from incomplete knowledge trans- fer. We aim to support this argument through the following proposition.
Proposition Consider knowledge distillation (KD) performed solely clean samples, denoted by Decan- Let the teacher representation be = (2,22 and the student representation be (Z, Rq),
where:
Z denotes task-relevant features,
Ω (21 represents the total backdoor-relevant information,
22 is the component of that is coupled with Z,
9u is orthogonal to both and Stp-
Then, the following inequality holds:
D gap T(T; Ybacktoor) T(S; Y backdoor 0,
(14)
where ((s-) denotes mutual information and Ybackdoor is the output label of backdoor samples.
This information gap D Dgap arises because KD clean data can only transfer the coupled com- ponent Doj. while the orthogonal component Q -wich may carry important safety-relevant sig- nals-remains untransferred.
Building on the above proposition, we emphasize the critical role of coupling between backdoor- relevant and task-relevant features in the teacher model. This coupling is key to enabling the infor- mation gap that SCAR exploits. Specifically, before initiating the bilevel optimization in SCAR, we perform pre-optimization step craft the trigger pattern. This step yields trigger that naturally exists in the clean dataset, making it partially correlated with task-relevant features Z.
During training, the teacher model is exposed to both clean and backdoor samples, learning a robust representation that includes Z. S2p, and Ω2, We can construct the total backdoor-relevant infor- mation (2,, .2)) such that 22 is coupled with Z, while Ω remains orthogonal. However, the student model is trained through KD using only clean samples. As result, the student inherits Z and the coupled component Qn. but not the orthogonal, safety-relevant component Ω that con- tributes to the teacher's 's robustness. This selective transfer induces information gap goop 0,
22

---

enabling a dormant backdoor effect: while the teacher model remains resilient, the student model becomes vulnerable backdoor attacks. Its proof as follows.
Proof. We begin by applying the chain rule of mutual information to expand the information gap between teacher and student representations.
For the teacher model's 's representation (1) Q1), we have:
[[;; backkoor))
(2, St,. backdoor)
[ (2; backdoor) + 1 [S2) P1i backdoor 2),
(15)
where the second equality follows from the chain rule of mutual information. Similarly, for the student model's representation (2)):
I(S; Y backdoor) ([ Ybackdoor)
[(; Ybackdoor) + [Sq Ybackdoor 2).
(16)
Taking the difference between equations
15 and (6) yields:
ggap = [[;; backdoor) - [(; Ybckkoook)
[ P11 backdoor
Z) [[22j ; backdoor Z).
(17)
Since the backdoor information can be decomposed as Ω (S2) S.1), apply the chain rule once more to the first term in equation (17):
I (S2 941 Ybackdoor
Z) = (S2) ; Y backdoor [] + 1 1a001::baaan backdoor 1
(18)
Substituting equation 18 back into equation
Dgap
[Sq ; Backkoor
() + ((21; Y backdoor
[2,0 (Sqj Yackooor ()
I(221 backdoor
(19)
To establish non-negativity, we invoke the fundamental property that conditional mutual information is always non-negative. By definition in [10]
T(A; B C) = H(B () ([B C,
(20)
Since conditioning never increases entropy [9] we have:
H(B C, < H(B ())
(21)
which implies:
T(A; B C) 2 0.
(22)
Applying this property to equation 19 with A Ω1 B conclude:
Ybckkoorr and C (Z, (Z,9)),
D Dgap = I(Ω1i Ybackdoor
2 2 0.
(23)
From the perspective of the information gap, we intended to suggest that SCAR may exploit the in- herent discrepancies arising from imperfect knowledge transfer during knowledge distillation (KD) to activate backdoors in the student model. We note that, perhaps counterintuitively, such informa- tion gaps are intrinsic to the KD process, as the student model is unlikely to precisely replicate behavior of the teacher model. This divergence may stem from architectural differences, limited model capacity, or the inherently lossy nature of the distillation objective 8 24]. As such, while the teacher model may behave benignly, the student model can still manifest backdoor behaviors.
23

---

A.2 Analysis from the KD Data Distribution
Beyond the information gap perspective, we also provide an empirical analysis from the standpoint of the data distribution used during KD, We hypothesize that using a benign dataset drawn from the same distribution as the teacher model's 's training data may cause the student model to implicitly learn Eiistribution-speeccii Specifically, during the pre-optimization of the trigger injection function, our goal is to obtain a natural trigger pattern (NTP) that can persist throughout the KD process. This NTP resembles tar- geted universal adversarial perturbation (TUAP) that consistently perturbs samples from the given data distribution toward an attacker- attacker-seecified target label. Existing research on adversarial attacks suggests that the transferability of adversarial perturbations is partially influenced by model archi- tecture and output distribution similarity [89]. We also observe several experimental phenomena that may serve supporting evidence for the above hypothesis:
natural backdoor features, thereby facilitating backdoor injection.
(1) In our CIFAR-10 experiments, the teacher models typically achieve over 90% accuracy on the test set, which results in output probabilities that closely align with the one-hot ground-truth labels. Therefore, when using such high-accuracy teacher models for KD, the student model is likely to inherit knowledge related to the data distribution itself-simibe to what model would learn when trained from scratch. Since NTPs (like TUAPs) exhibit strong transferability between models with highly similar output distributions, any model that learns cataaddstribbttion-seefff knowledge and achieves high accuracy is likely to be vulnerable to such NTP attacks.
(2) A possible explanation for the degraded performance of SCAR on the ImageNet dataset is that: the relatively low accuracy (typically < 70%) of models on ImageNet leads to less similar output distributions across models. This probably prevents the student model from accurately capturing nata-distribution-speeee knowledge, and reduces the transferability and effectiveness of the NTP.
(3) Meanwhile, we observe that SCAR is capable of successfully attacking student models across various distillation methods, not limited to the KL--divergence-base based distillation used during the bilevel optimization process. This suggests that the transferability of the NTP between models with the same data tatta-ddirributio-ppeeiiii knowledge may play significant role in facilitating the attack.
From the data distribution perspective, we intended to suggest that SCAR may leverage distribution- specific natural backdoor features to facilitute the persistence of backdoors across different KD processes. Regardless of the KD method employed, the primary objective is to optimize the student model's performance on the given data distribution. However, during this process, the student model distilled from datasets drawn from the same distribution as the teacher may also implicitly learn certain distribution-specifie natural backdoor features. This might allow the backdoor trigger to persist across different KD strategies.
B Results on Additional Model Architectures
In this section, we conduct experiments two additional teacher model architectures, including VGG-19 (7I) and ViT [19]. We conduct experiments on the CIFAR- 10 dataset. The KD methods, attack setup, and baseline methods are the same as those described in the main text.
As shown in Table and SCAR demonstrates superior attack performance compared to the base- line method ADBA (FT). Specifically, on the V9G-19 teacher model, ADBA (FT) exhibits only limited backdoor injection performance against the student model ShuffleNet- V2, and even fails entirely to implant backdoor into ViI. In contrast, SCAR consistently achieves higher attack success rate above 80% against student models in most scenarios. These results further vali- date the effectiveness of our proposed SCAR method.
C Results on Additional Backdoor Detection Methods
In this section, we further verify the stealthiness of the distillatio-ccondiionall backdoor injected by SCAR in the teacher model by employing various advanced backdoor detection methods.
24

---

Table 4: The attack performance (%) of SCAR compared to baseline methods the VGG-19 teacher model and its three distilled student models. For each dataset under each KD method, the best results are boldfaced, while all failed cases (student ASR 50%) marked in red.
VGG-19
MobileNet-Y V2
ShuffleNet-N V2
EfficientViT
Model
KD Method
(Tacher))
(Student
Attack
ACC
ASR
ACC
ASR1
ACC
ASR1
ACC
ASRT
Benign
92.41
91.07
0
90.24
87.56
Response
ADBA (FT)
90.58
6.88
91.37
92.39
86.
61.12
87.26
47.39
SCAR
90.53
1.50
91.43
99.24
89.17
95.14
86.26
90.36
Benign
92.41
91.54
0
90.60
87.76
Feature
ADBA (FT)
90.58
6.88
91.41
90.03
89.50
75.60
85.33
32.22
SCAR
90.53
1,50
91.54
99.66
89.05
96.20
85.55
72.64
Benign
92.41
91.81
0
89.43
87.36
Relation
ADBA (FT)
90.58
6.88
91.01
91.11
85.76
46.14
86.99
44.77
SCAR
90.53
1,50
91.27
99.72
89.39
97.23
86.12
88,47
Table 5: The attack performance (%) of SCAR compared to baseline methods the ViT teacher model and its three distilled student models. For each dataset under each method, best results are boldfaced, while all failed cases (student ASR 50%) are marked in red.
VIT (Tacher)
MobileNet-V2 V2
ShuffleNet-V V2
EfficientViT
Model
KD Method
(Suudent
(Student
(Sudden
Attack
ACC
ASR
ACC
ASR1
ACC
ASR
ACC
ASR1
Benign
84.92
0
85.10
0
84.16
0
84.52
Response
ADBA (FT)
83.74
4.33
82.75
83.56
82.27
80.61
83.12
51.83
SCAR
84.71
1.73
85.35
95.01
84.06
93.90
84.10
84.58
Benign
84.92
0
84.41
0
85.31
84.23
Feature
ADBA (FT)
83.74
4.33
83.75
86.64
83.63
87.53
83.06
69.91
SCAR
84.71
1.73
85.64
89.54
85.03
89.74
84.23
88.19
Benign
84.92
0
85.18
0
84,97
0
84.55
Relation
ADBA (FT)
83.74
4.33
83.34
85.87
83.33
71.44
83.76
48,48
SCAR
84.71
1.73
85.94
98.27
85,86
87.10
85.23
80.87
C.1
The Resistance to BTI-DBE
an advanced backdoor trigger inversion (BTI)-based backdoor detection method. Overall, its detection process consists of three main steps. First, a mask is trained at the feature level decouple the benign features involved the inference of clean samples from the remaining redundant features. Next, using this mask, backdoor generator is trained to minimize the discrep- ancy of benign features between clean samples and their corresponding poisoned versions, while maximizing the discrepancy of their backdoor features. Finally, the trained backdoor generator is applicd to the test set to observe the distribution of the model's output classes. If the model exhibits strong bias toward specific label, that label is identified as potential backdoor target.
BTI-DBF
We train teacher models with three different architectures on the CIFAR-10 dataset utilizing the SCAR attack, with the target label set to class O. These models are then evaluated utilizing the BTI- DBF detection method, and the distribution of predicted class labels on the test set recorded for each model. As shown in Table 6 BTI-DBF fails to effectively detect the dissillationcconditonn backdoor embedded in the teacher models by SCAR. Specifically, for the VGG-19 and ResNet-50 models, BTI-DBF mistakenly identifies Class 2 as the target label, while for the ViT model, it fails to detect any backdoor at all. These experimental results demonstrate that SCAR can effectively evade detection by BTI-DBF.
25

---

Table 6: The backdoor detection performance BTI-DBF on teacher models compromised by SCAR. Class 0 is the attack-defined target label. False detections are marked in red.
Predicted Number of Each Class (> 5000 indicates potential backdoor)
Model
Class
Class
Class
Class
Class
Class 5
Class 6
Class 7
Class 8
Class 9
ResNet-50
140
116
7113
1616
1007
VGG-19
0
0
5610
0
72
4318
0
0
0
VIT
854
1013
928
900
1069
998
1093
1038
977
1130
Table 7: The SAP values reported by A2D on teacher models compromised by SCAR.
Trial
4
SAP
0.117
0.117
0.185
0.206
0.027
Table 8: The backdoor detection performance of BAN on teacher models compromised by SCAR. Class 0 is the attack-defined target label. False detections marked in red.
Predicted Number of Each Class (maximum indicates the target label inferred by BAN)
Trial
Class
Class
Class 2
Class 3
Class 4
Class 5
Class 6
Class
Class 8
Class 9
434
461
1943
459
43
558
454
333
2628
272
70
146
38
253
766
494
141
432
11
2057
536
418
4
297
43
215
266
3066
287
228
89
285
224
176
371
696
65
1182
123
1512
209
587
C.2
The Resistance to A2D
A2D 20 is a black-box backdoor detection framework that exploits the heightened sensitivity of backdoored models to adversarial perturbations. It introduces the sensitivity to adversarial pertur- bations (SAP) metric, estimated by transferring strong adversarial attacks generated on an unrelated reference model. A model is flagged as trojaned if its SAP exceeds threshold, requiring only limited clean data and low computational cost.
To further verify the stcalthiness of our SCAR, we conduct independent detection trials on a ResNet-50 teacher trained CIFAR-10 with target lahel 0. As shown in Table 7 the SCAR- attacked teacher consistently exhibits low SAP, indicating that A2D failed to detect the backdoor.
C.3 The Resistance to BAN
BAN 8T enhances backdoor detection by augmenting backdoor feature inversion with neuron ac- tivation information. By adversarially perturbing model weights trigger backdoor effects, BAN enables efficient and accurate separation benign and backdoored models.
To further verify the stealthiness of our SCAR, we conduct 5 independent detection trials on ResNet-50 teacher trained on CIFAR-10 with target label O. Similar to BTI-DBE, we record both the prediction distribution after mask training and the predicted target class. As shown in Table BAN also consistently misjudges the backdoor target class, leading to a high false positive rate.
C.4 The Resistance to MDTD and TED
In this section, we verify the stealthiness of our SCAR utilizing two advanced input-level detec- tion methods, MDTD 67 and TED (58) Specifically, MDTD is universal test-time backdoor detector for DNNs that requires no prior knowledge of the trigger strategy. leverages adversarial learning to estimate distances to the decision boundary. identifying poisoned inputs as those lying
26

---

Table 9: Fl-scores of MDTD and TED on poisoned test samples for the SCAR-attacked ResNet-50 teacher model.
Method
3
MDTD
0.0488
0.0417
0.0225
0.0512
0.0580
TED
0.0386
0.0281
0.0056
0.0120
0.0196
farther away than clean samples. TED is model-agnostic backdoor detector that treats DNNs as dynamical systems. Instead of relying on separability in a metric space, TED traces the evolution of imputs: benign samples follow consistent trajectories, whereas poisoned ones diverge towards attacker-specified targets, revealing the presence of backdoors.
We evaluate MDTD and TED ResNet-50 teacher trained on CIFAR-10 (target label 0) over 5 independent trials. Since both methods are input-level detection, we utilize the poisoned test set as input and compute the Fi-score for poisoned sample detection. As shown in Table both methods yield very low F1-scores (< 0.06), indicating that they misclassify most poisoned samples as benign and thus failed detect the poisoning effectively.
D Results on Additional Non-Detection Backdoor Defenses
In our threat model, the attacker uploads a model containing a trusted third-party platform. This platform performs backdoor detection and, upon verifying no abnormal behavior, releases the model. A victim user then downloads the model and uses it for further KD. key aspect of this scenario is that the platform usually only responsible for detec- tion, not mitigation. Typically, uploaded models may be trained on large-scale datasets over long periods, while developers usually only upload their model without uploading training data and con- figurations. As such, such mitigation is generally infeasible at least significantly degrades model performance and requires many computational resources. Given the platform's limited resources, we assume it does not conduct non-detection defenses.
backdoor to
However, users may apply backdoor mitigation methods either to the downloaded teacher model before KD or to the distilled student model afterward. We conduct supplementary experiments to evaluate the robustness of SCAR in both scenarios.
D.1 The Resistance to Teacher-side Mitigations
In this section, we conduct additional non-detection backdoor defenses to SCAR-attacked teacher models to verify the robustness of our SCAR. We evaluate two non-detection defenses: NAD and fine-pruning [49]. We apply these defenses to a SCAR-attacked ResNer-50 on CIFAR-10, and then use the defended model for KD train MobileNet-V V2. As shown in Table [T] SCAR remains highly effective, even after the teacher has undergone these backdoor mitigations. Besides, using non-detection defenses may even slightly raise teacher ASR (> 5%). This might be because such de- fenses disrupt the carefully crafted "mask" of the intillation-conditioon backdoor, thus re-exposing the hidden backdoor to some extent.
(55
While NAD and fine-pruning fail to eliminate SCAR, we don't intend to claim that it is undefeatable. Rather, goal is to raise awareness that even models which pass backdoor detection should always not be assumed safe for distillation without further examination or cleansing.
D.2 The Resistance to Student-side Mitigations
In this section, conduct additional non-detection backdoor defenses to student models distilled from SCAR-attacked teacher models to verify the robustness of our SCAR. We evaluate our SCAR under Sampdetox (84) method. Specifically, launch the SCAR attack on ResNet-50 teacher model on CIFAR-10 10, and distill three MobileNet- V2 student models using different KD methods. We then evaluate the defense performance of SampDetox using both benign and poisoned test sets.
As shown Table II although SampDetox can partially reduce the attack success rate of SCAR, it also leads to noticeable drop in benign accuracy (BA). This suggests that SCAR has already
27

---

Table 10: The attack performance (%) of SCAR the student MobileNet- V2 distilled from the teacher ResNet-50 under two backdoor mitigation methods, using three KD methods.
ResNet-50 (w/o Defense)
ResNet-50 (wi Defense)
MobileNet-V2 (Response)
MobileNet-V2
MobileNet-V2 (Relation)
Method
(Feanare)
ACC
ASR.
ACC
ASR
ACC
ASR1
ACC
ASR
ACC
ASR1
No Defense
92.47
1,50
91.62
99.94
91.01
99.90
91.29
99.93
NAD
92.47
1.50
88.45
9.60
89.08
99.94
86.12
89.20
87.11
99.58
fine-pruning
92.47
1.50
92.72
5.94
92.17
99.47
90.44
97.41
91.08
99.34
Table 11: The defense performance (%) of SampDetox against MobileNet-V2 student models dis- tilled from ResNet-50 teacher models using three different KD methods.
Before Defense
After Defense
Distillation Method
ACC
ACC
ASR
Response
91.62
99.94
84.13
56.42
Feature
91.01
99.90
82.26
52.60
Relation
91.29
99.93
83.65
76.01
Table 12: The attack performance (%) of SCAR with different surrogate models on the student MobileNet- V2 distilled from the teacher ResNet-50 using three KD methods.
ResNet-50
MobileNet-V2
MobileNet-V2
MobileNet-V2
Surrogate
(Teucher)
(Response)
(Feature)
(Relation)
ACC
ASR.
ACC
ASR
ACC
ASR1
ACC
ASR
ResNet-18
92.47
1.50
91.62
99.94
91.01
99.90
91.29
99.93
ResNet-34
92.32
1.42
92.26
99.47
90.77
99.68
91.76
99.92
DenseNet-121
92.79
1.43
92.19
99.80
90.67
95.78
91.80
99.90
SqueezeNet 92.41
1.56
91.86
99.94
91.16
99.13
91.78
99.88
exhibited a certain degree of robustness against input pre-processing defenses, although we have no particular design for it.
E
Results on Additional Ablation Studies
E.1
The Surrogate Model Architecture
In this section, we evaluate SCAR on CIFAR-10 under various surrogate architectures, including ResNet- 18, ResNet-34, DenseNet- 121, and SqueezeNet, while adopting ResNet-50 and MobileNet- V2 as the teacher and student models, respectively. As shown in Table [2] SCAR consistently achieves high attack success rates (student ASR > 95%) across all settings.
E.2 The Distillation Loss Weight 5
In our threat model, the attacker can only manipulate the training process of the compromised teacher model, without access to the user's choice of distillation methods or hyperparameters. In this section, we evaluate the effectiveness of SCAR in attacking student models under various distil- lation methods and loss weights 5. Specifically, we utilize SCAR-compronised ResNet-50 trained on CIFAR-10 as the teacher model and distill it into MobileNet- V2 student models using three dif- ferent distillation techniques and five different distillation loss weights 8.
28

---

Table 13: The attack performance (%) of SCAR under different distillation loss weights (ő) utilizing the student model (MobileNet-V2) distilled from the SCAR-attacked teacher model (ResNet-50).
ResNet-50
MobileNct-V2
MobileNet-V2
MobileNet-V2
MobileNet-V2
MobileNet-V2
Model
(Tacher
(6 2)
(6 31
(6
(6 5)
KD Method
ACC
ASR
ACC
ASR1
ACC
ASRT
ACC
ASR1
ACC
ASRT
ACC
ASRT
Response
92.47
1.50
91.62
99.94
91.64
99.92
91.58
99.87
91.68
99.90
92.15
99.96
Feature
92.47
1,50
91.01
99.90
90.68
98.70
91.27
98.17
91.12
94.00
91.11
95.81
Relation
92.47
1.50
91.29
99.93
91.65
99.79
91.27
99.89
91.47
99.91
91.70
99.96
Table 14: The attack performance (%) of SCAR on the student MobileNet-V2 distilled from the teacher ResNet-50 with different distillation datasets, using three KD methods.
ResNet-50
MobileNet-V. V2
MobileNet-W V2
MobilcNet-V2 (Relation)
Distillation Dataset
(Teacher)
(Response)
(Fenture)
ACC
ASR
ACC
ASR1
ACC
ASR
ACC
ASR1
CIFAR-10
92.47
1.50
91.62
99.94
91.01
99.90
91.29
99.93
CINIC-10
68.83
5.22
76.22
74.51
72.35
54.83
75.81
75.58
As shown Table 13 SCAR consistently achieves effective attacks on the MobileNet-V V2 student models across different distillation methods and loss weights ò. This further demonstrates the ability of SCAR to activate the dormant backdoor the teacher model and achieve a high attack success rate in the student model.
E.3 The Distillation Dataset
In addition to the distillation loss weight 8, users can also specify the dataset employed for distilling the student model. Here, we conduct additional experiments where the distillation dataset differs from the training set. We train ResNet-50 teacher using SCAR on CIFAR-10, and then distill MobileNet-V2 students using a subset of CINIC-10 [2], which contains the same 10 classes but has different data distribution (5,000 training and 1,000 test samples per class).
As shown Table 14 due the distributional shift, the teacher's accuracy drops to 68.83%. The students also experience a drop in accuracy. Although the attack success rate on the students de- creased accordingly, it still remained above 54%. These results suggest that if the student fails to achieve high accuracy during distillation misiilattio--mmaningg it does not effectively inherit the teacher's be- nign knowledge-it also less likely to inherit the backdoor. However, such significant drops in accuracy for both teacher and student models indicates an ineffective distillation process, which is usually not meaningful in practice.
E.4 The Number of Inner Steps T
In our main experiments, We fix the number of inner optimization steps 20. This is mostly because the surrogate model has already reached a reasonably good local optimum (around 85% accuracy compared the teacher) within these steps, which sufficient for guiding the outer optimization.
To assess sensitivity to this setting, we conduct ablation studies using different inner steps (20, 30, 40) when attacking ResNet-50 on CIFAR-10. The surrogate model is ResNet-18, and the student model is MobileNet-Vi V2. As shown in Table 15] SCAR remains effective under all settings. This suggests that once the inner parameters are optimized to nearly local optimum, the outer gradient adequately approximated.
E.5
The Bilevel Optimization Hyperparameters (a, 3, and 7)
In the outer optimization objective in Eq. (2), the scales of the four loss terms are approximately comparable and relate to four different aspects (e.g., the student's performance on benign samples).
29

---

Table 15: The attack performance (%) of SCAR with different inner steps on the student MebileNet- V2 distilled from the teacher ResNet-50 using three KD methods.
ResNet-50
MobileNet-V2 V2
MobileNet-V2 V2
MobileNet-V2
Inner Step
(Teaceer)
(Response)
(Peature)
(Relation)
ACC
ASR
ACC
ASR
ACC
ASRT
ACC
ASR
20 steps
92.47
1,50
91.62
99.94
91.01
99.90
91.29
99.93
30 steps
92.35
1,46
92.13
99.41
91.03
99.04
91.26
99.39
40 steps
92.20
1.51
91.98
99.69
91.16
99.18
91.87
99.49
Table 16: The attack performance (%) of SCAR on the student MobileNet-V2 distilled from the teacher ResNet-50 under different hyperparameter settings, using three KD methods.
ResNet-50
MobileNet-V V2
MobileNet-Ve V2
MobileNet-V2
Hyperparameter
(Tescher)
(Responsc)
(Featture)
(Relation)
ACC
ASR
ACC
ASR1
ACC
ASR1
ACC
ASR1
0.5
92.43
1.77
91.67
99.81
91.12
99.36
91.58
99.94
1
92.47
1,50
91.62
99.94
91.01
99.90
91.29
99.93
& 1.5
92.54
1.33
92.08
99.46
91.03
99.71
91.78
99.59
B = 0.5
92.21
1.61
92.36
99.37
91.30
99.04
91.33
99.07
8 ==1
92.47
1,50
91.62
99.94
91.01
99.90
91.29
99.93
8 1.5
92.35
1,43
91.99
99.83
90.87
99.01
91.52
99.36
0.5
92.55
1.82
92,40
99.51
91.25
98.58
91.46
99.53
= 1
92.47
1,50
91.62
99.94
91.01
99.90
91.29
99.93
7 1.5
92.13
1.88
91.74
99.97
90.98
99.77
91.64
99.23
Attackers can adjust their values based on specific needs or priorities. In our experiments, we set a = 3 = = 1 for simplicity and clarity, as this is the most straightforward configuration.
We also conduct ablation studies to evaluate SCAR's sensitivity to these hyperparameters. Specifi- cally, we train teachers (ResNet-50) using SCAR on CIFAR-I0 with a, 3, and y independently set to 0.5, 1.0, and 1.5. We use ResNet-18 18 as the surrogate and MobileNet-V2 as the student. As shown in Table [6] SCAR consistently achieves strong attack performance (student ASR > 98%) across different hyperparameter settings.
E.6 The Trigger Injection Function G
In this section, we consider more types of triggers to replace the pre-optimized trigger function G, especially the sample-specified triggers cover the entire image. We replace the pre-trained triggers in SCAR with image-size poisoning strategies from BadNets [27]. WaNet [60] and BppAttack [76), as implemented in BackdoorBox [47] or their source codes. We then train teacher models ResNet- 50 on CIFAR-10 using these methods. ResNet-18 used as the surrogate, and MobileNet- V2 serves as the student. We evaluate the attack success rates on students obtained via different KD methods.
As shown in Table 17, using these image-size poisoning strategies still fails effectively transfer backdoors to the student models (student ASR < 25%). indicates that simply using image-size triggers cannot be easily 'entangled" with benign features, even if they cover the entire image, and thus fail to transfer during distillation.
F Implementation Details
Details of Datasets.
1. 10. The CIFAR-10 dataset (4) samples total. The dataset has 10 classes and each class has 5,000 training samples and 1,000 testing samples. The size of each image sample is 32.xx32
contains 50,000 training samples and 10,000 testing
30

---

Table 17: The attack performance (%) of SCAR on the student MobileNet-Y V2 distilled from the teacher ResNet-50 under different trigger types, using three KD methods.
ResNct-50
MobileNet-V V2
MobileNet-V2 V2 (Feature)
MobileNet-V2 V2 (Relation)
Trigger Type
(Teacher)
(Response)
ACC
ASR
ACC
ASR
ACC
ASR
ACC
ASR
BadNets
93.81
0.72
91.47
1.03
91.49
1.48
91.22
1.38
WaNet
93.07
2.20
91.07
20.97
90.47
9.73
90.77
21.22
BppAttack
93.65
1.11
91.43
1.17
89.29
1,86
91.36
1.07
Ours
92.47
1.50
91.62
99.94
91.01
99.90
91.29
99.93
2. ImageNet. The ImageNet dataset [14 consists 1,000 classes containing over 14 million manually annotated images. In this paper, we select a subset with 50 different classes and each class contains 500 training samples and 100 testing samples with size x 32224222
Details of Training Models.
1. Benign Models. We utilize the Adam as the optimizer and the batch size is set to 256 on CIFAR-10 and 128 on ImageNet. We set the initial learning rate as 10-4 and train all models for 200 epochs, with the learning rate reduced by a cosine annealing schedule.
2. Models attacked by ADBA (FT). We first train a teacher model compromised by ADBA using the same training configuration as that of the benign model, with the target label set to O. The shadow model is trained via KI <l-ddivereeceebasee distillation using SGD with an initial learning rate of 0.1. The trigger pattern is optimized using a separate SGD optimizer with initial learning rate of 0.01. All optimization processes adopt individual cosine annealing schedules to manage the learning rates. During fine-tuning, the feature extraction layers of the teacher model are frozen, and only the final linear layer updated. The optimizer used is Adam with a learning rate of 10 and the hyperparameter v) is set to 1 and k is set to 0.1.
3. Models attacked by SCAR. During trigger pre-optimization, distill benign model using KL divergence loss with the Adam optimizer (learning rate of 10 40-4) and a cosine annealing schedule. The trigger pattern is then optimized using Adam with learning rate of 0.01. For solving the bilevel optimization problem, we set the target label to O. The outer opti- mization runs for 200 iterations, with 20 inner-loop updates per outer iteration, and each lixed-point iteration runs for 100 steps. The teacher model is optimized using Adam with initial learning rate of 10-4, and the learning rate is decayed using a cosine annealing schedule. To accelerate training, 40 random batches are selected from the training set in each fixed-point iteration. The hyperparameters, a, 8, 7, and 8, are set 1.
Details of Knowledge Distillation. When distilling student models to evaluate the effectiveness of SCAR, we set the hyperparameter ö to for all three distillation methods. Adam used as the optimizer with an initial learning rate of 10-1, and the learning rate is scheduled using cosine annealing over 150 epochs. We apply early stopping once the student model achieves comparable or slightly better accuracy than the teacher model the test set.
Computational Resources. In our implementations, we utilize PyTorch as the deep learning frame- work. All our experiments are implemented with RTX 4090 GPUs.
G The Overhead of our SCAR
In this section, we evaluate the overhead of our SCAR. Specifically, we separately measure the time of the bilevel optimization process for compromised teacher models on the CIFAR-10 and ImageNet datasets. We set the inner loop to 20 iterations and the outer loop to 200 iterations, with 100 fixed- point updates per outer iteration. To accelerate training, 40 random batches are selected from the training set in each fixed-point iteration. We conduct all training using single RTX 4090 GPU. For the subsequent KD process, since the attacker can only manipulate the training of the teacher model but cannot control which distillation method the user adopts, we do not account for its overhead.
31

---

Table 18: The overhead (hours) of ResNet-50 compromised by SCAR on CIFAR-10 and ImageNet.
Dataset
CIFAR-10
ImageNet
Overhead
20.75
59.04
As shown in Table [8] the optimization of the teacher model incurs substantial overhead. This is largely because each fixed-point iteration to estimate its parameter gradients necessitates a 100- iteration linear loop, and furthermore, every iteration within this loop requires gradient computations for large tensors whose dimensions are comparable to the full model parameters. To accelerate training, intuitive approach is utilize multi-GPP training, which allows for increasing the batch size in each optimization round. This, in turn, can reduce the number of outer optimization epochs and fixed-pole point iterations, thereby speeding up the convergence of the teacher model optimization. How to address this training overhead presents a potential direction for our future work.
H Related Work
H.1
Backdoor Attack
Based whether activating the backdoor requires preconditions beyond the trigger pattern, attacks are classified into two types: (1) Unconditional backdoor attacks [48] 66 [9],, the conventional form, activate malicious behaviors solely by applying predefined trigger patterns.
(2) Conditional backdoor attacks [38 62 72), more recent paradigm, require specific model manipulations before
the trigger pattern activate the hidden malicious functionality.
Unconditional Backdoor Attacks. This class of attacks is the conventional form of backdoors: predefined trigger pattern is simply added to inputs to induce the model to execute attacker- specified behavior (e.g., map to target label) without any additional conditions. The BadNets attack (27). which uses white-square trigger, is the first to demonstrate the threat of backdoors in the image domain. Subsequent work has produced varied trigger modalities, including pattern blending [5] and imperceptible perturbations [60]. More recently, Cai al. (2) propose using pitch and timbre as triggers in the audio domain. These unconditional backdoor attacks are simple and highly effective, but they also relatively easy detect.
Conditional Backdoor Attacks. Generally, models compromised by conditional backdoor attacks initially behave normally, even on triggered inputs, with the backdoor remaining inactive. This la- tency can broken by specific post-processing operations applied to the model. Tian et al. (72) first demonstrate that model compression techniques, specifically model pruning and quantization, can awaken hidden backdoors. Subsequently, further research [3] 38 53 63) highlights the vulner- backdoor attacks. Furthermore, fine-tuning on downstream tasks [39] 62) and dynamic multi-exit transfor- mation [18] have also been shown activate such backdoors. However, existing studies have not explored whether the KD process might introduce similar security vulnerabilities.
ability of model quantization to backdoors and proposes various guantization-condiionn
H.2 Backdoor Defenses
To mitigate backdoor threats, researchers have proposed various defenses, broadly categorized aiming to identify whether model contains a backdoor.
(1) Detection-based defenses [74]
80
82
seeking to cleanse or inactive potential backdoors within the model. While effective against conventional backdoor attacks, these defenses face challenges with conditional backdoors. Specifically, the dormant nature of conditional backdoors often allows them to evade detection-basee based methods 53 72), leading to compromised models being misclassi- fied as benign. Although purification-bar based approaches might partially reduce the attack success rate (ASR) of conditional backdoors [44], they typically incur additional overhead from necessary model parameter modifications or auxiliary module training. In particular, specialized defenses have recently been developed for conditional backdoors, such as those targeting muanizztion-conniiion backdoors 43 44 and have demonstrated promising effectiveness. However, defenses against our proposed distillation-condiiin backdoor remain largely underexplored.
(2) Purification-bass defenses 6
33
88
32

---

t:SNE of Benign Test Sample Features
t-SNE of Poisoned Test Sample Features
Class
Closs
a
(b)
Figure 3: The t-SNE of benign and poisoned test sample features on the teacher model ResNet-50
compromised by SCAR.
I Visualization
The t-SNE of Features on the Teacher Model. To further verify the stealthiness of SCAR-injected backdoors in the teacher model, we visualize the feature distributions of both benign and poisoned test samples. Specifically, we utilize a teacher model ResNet-50 trained on the CIFAR-10 10 dataset under SCAR attack. During testing, extract their features prior to the fully connected (fc) layer, and apply t-SNE to project them into a two-dimensional space for visualization.
feed the model with both benign and poisoned samples,
As shown in Figure 3 the ResNet-50 model attacked by SCAR produces well-separated clusters in the feature space for both benign and poisoned test samples. This demonstrates the stealthiness and non-activatability of the SCAR-injected backdoors in the teacher model, enabling the distillation- conditional backdoor to effectively evade direct backdoor detection methods applied to the teacher.
The t-SNE of Features on the Student Model. To further validate the effectiveness of SCAR in attacking student models, we visualize the feature distributions of both benign and poisoned test samples as processed by the student model. Specifically, we consider a scenario in which ResNet- 50 teacher model, trained on CIFAR- 10 and compromised by SCAR, is used to distill knowledge MobileNet-V V2 student model via response-based distillation on clean data. During testing, we feed the student model MobileNet-V V2 with both benign and poisoned samples, extract their features prior to the fe layer, and apply t-SNE to project them into a two-dimensional space for visualization.
As shown in Figure 4 the benign samples form well-separated clusters in the feature space of the MobileNet-V2 student model, whereas the majority of poisoned samples are misclassified into the target label (Class 0), demonstrating the effectiveness of SCAR in attacking student models.
The Examples of Benign and Poisoned Images. In addition, shown in Figure we present set of benign images from the CIFAR-10 10 test set along with their corresponding poisoned versions.
J Societal Impact
This paper uncovers a significant and previously underappreciated security vulnerability: student models can inherit backdoor threats from teacher models via knowledge distillation, even if the teacher model has passed prior backdoor detection and distillation is performed using a clean dataset. Our proposed attack, SCAR, though straightforward, effectively demonstrates the practical feasibil- ity of this threat vector. While the techniques presented could potentially be exploited by malicious actors to craft teacher models embedded with distilattio-cconitiona backdoors, the primary contri- bution of this work is to serve as a crucial alert for the Al development and deployment community. It highlights critical blind spot in current security practices.
We strongly advocate that developers and organizations implement rigorous security verification, including comprehensive backdoor detection, not only for teacher models sourced from third-party
33

---

t-SNE Benign Test Sample Features
t-SNE of Poisoned Test Sample Features
Closs
Closs
(a)
(b)
Figure 4: The t-SNE of benign and poisoned test sample features on MobileNet-V2. which dis- tilled from the teacher model ResNet-50 compromised by SCAR.
Airplane
Automobile
Bird
Cat
Deer
Benign
Poisoned
Dog
Frog
Horse
Ship
Truck
Benign
Poisoned
Figure 5: Examples of benign and corresponding poisoned images from the CIFAR-10 testset.
platforms but also, critically, for the student models derived through any distillation process. This dual-checking paradigm is vital for ensuring the integrity and safety of Al systems deployed in real-world, safety-critical scenarios. uous and thorough backdoor detection throughout the AI model lifecycle from pre-deployment to post-distillation. This vigilance crucial for guarding against evolving backdoor attacks, including various forms of conditional backdoors, and fostering trust and security in AI applications.
essence, this paper underscores the urgent need for contin-
K Potential Limitations and Future Directions
Firstly, as discussion in the main results Section 4.2 the performance of SCAR on the ImageNet dataset less satisfactory. Specifically, while SCAR consistently achieves over 80% ASR against student models in most cases on CIFAR-10, it only maintains ASR above 50% on ImageNet. This performance degradation may be attributed to the increased number classes and larger image size in ImageNet, which intensifies the instability of the bilevel optimization process and may lead to convergence to a local minimum. Nevertheless, considering that this the first work to reveal new attack paradigm, we believe that the experimental results are acceptable to certain extent. In particular, such results cannot undermine the central opinion of this paper: developers should remain vigilant and conduct thorough backdoor detection on distilled student models, even when the teacher
34

---

model and distillation dataset have been verified as secure. In future work, we aim to improve the attack effectiveness of SCAR on datasets with larger scales and greater category diversity.
Secondly, in our implicit differentiation algorithm, estimating the teacher model's gradient via the fixed-point iteration method requires synchronized linear loops, which increase the overall opti- mization time to some extent. We have discussed the computational overhead of our method in Appendix G and proposed potential solutions. Improving the speed and accuracy of gradient esti- mation remains an important direction for future research.
Finally, this paper primarily focuses the attack threat posed by distillation- backdoors for image classification models. Given that existing studies have shown the potential for backdoor threats to be transferred during the distillation of large language models [7]. will investigate in future work whether similar security risks exist in other modalities and tasks.
L Discussion on Adopted Data
In our experiments, we use only open-source datasets, namely CIFAR-I0 [40, ImageNet [4] and CINIC-10 [12). for evaluation. Our research strictly adheres to the open-source licenses of these datasets and does not raise any privacy concerns. Although the ImageNet dataset may contain per- sonal elements such
images of human faces, our work treats all data uniformly and does not intentionally exploit or manipulate any sensitive content. Therefore, our use of these datasets is fully compliant with their terms of use and does not constitute a violation of personal privacy.
M
Ethics Statement
In this paper, we reveal a new security threat (i.e., distillation-condiionni backdoor attack) in the knowledge distillation process. In general, our work intends to alert developers notice potential false sense of security or consensus that distilling a student model based on a "backdoor-fres teacher model with benign samples is always safe. Accordingly, our paper has positive societal impacts in general. However, we notice that the adversaries may adopt our method to execute an attack in practice. We argue that merely detecting backdoors in the teacher model is insufficient, and rigorous detection the student model is also essential. Besides, victim developers can mitigate this threat from the source by using only trusted third-parg models.
N Reproducibility Statement
Detailed information regarding our implementations and experiments is provided in Appendix F Additionally, we provide the official implementation of SCAR at
com/
35


Reviews:
[{"review_id": "aloAiESQfH", "reviewer": "Reviewer_TVht", "summary": "This paper introduces SCAR , a novel distillation-conditional backdoor attack (DCBA) framework that exploits the KD process to implant dormant backdoors in teacher models. These backdoors remain inactive during normal inference but are activated in student models after KD, even when the distillation dataset is benign. The attack is formulated as a bilevel optimization problem: the inner loop simulates KD by optimizing a surrogate student model, while the outer loop trains the teacher model to maximize the attack success rate on poisoned samples in the student model.", "questions": "1. **Surrogate Model Design**:  How do architectural differences (e.g., CNN vs. ViT) between the surrogate and target student models impact SCAR’s attack success rate? For instance, would a CNN-based surrogate fail to simulate KD for a ViT student?  \n\n2. **Defense Evaluation**:  The paper claims SCAR evades detection but does not test the surrogate model against state-of-the-art detections/defenses. How effective are these methods against SCAR, and what modifications could improve stealthiness? \n\n3. **Optimization Validity**: The bilevel optimization assumes differentiability of the inner loop (KD simulation). How sensitive is SCAR to approximations in this process (e.g., finite inner steps, surrogate initialization)?", "limitations": "Yes. The authors acknowledge limitations, such as scalability to larger datasets and potential failure cases.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "#### **Strengths**  \n1. The paper introduces a novel attack paradigm (DCBA) that bridges the gap between backdoor attacks and KD. This work first explores how KD itself can activate dormant backdoors.\n2. Extensive experiments on CIFAR-10, ImageNet, and multiple architectures (ResNet, VGG, ViT, MobileNet, ShuffleNet, EfficientViT) validate SCAR’s effectiveness.  Comparison with baseline ADBA(FT) shows SCAR achieves significantly higher ASR in student models while maintaining low ASR in the teacher model.\n\n#### **Weaknesses**  \n1. **Limited Surrogate Model Analysis**: The paper does not thoroughly explore how structural or task mismatches between the surrogate and target student models affect attack performance. This paper also lacks analysis on more structures of surrogate models.\n2. **Scalability Concerns**: The paper acknowledges that SCAR’s effectiveness may degrade on larger-scale datasets (e.g., full ImageNet), but no experiments or theoretical analysis address this limitation.\n3. **Optimization Assumptions**: The bilevel optimization relies on implicit differentiation and pre-optimized triggers, which may introduce biases or instabilities if the surrogate model poorly approximates real-world KD dynamics.\n4. **Stealthiness of Student**: A key concern is whether the student model can evade existing backdoor detection methods. If it cannot, the practicality of the attack may be limited, as such models would likely be discarded in real-world deployment. This aspect requires further investigation and discussion.", "was_revised": true, "final_justification": "The clarifications and modifications provided during the rebuttal process have resolved my issues to a satisfactory degree. As a result, I have decided to maintain my final rating as **Borderline Accept**. I appreciate the authors' efforts in engaging with the feedback and improving the manuscript.", "rebuttal": "Thank you for your careful review and thoughtful comments! We are encouraged by your positive comments on our **novel attack paradigm**, **extensive experiments**, **good quality**, **good significance** and **good originality**. We hope the following responses can help clarify potential misunderstandings and alleviate your concerns.\n\n---\n\n**W1&Q1**: The paper lacks evaluation of how architectural or task mismatches between surrogate and student models affect attack performance, especially CNN surrogates for ViT students. It also omits analysis involving more diverse surrogate structures.\n\n**R1**: Thank you for these constructive suggestions! We would like to respectfully clarify that **the effectiveness of SCAR does not rely on architectural similarity between surrogate and student models**.\n- As stated in line 252, we used ResNet-18 as the surrogate, while the students are structurally different: MobileNet-V2, ShuffleNet-V2, and EfficientViT. **This represents a challenging setting**, i.e., the surrogate is highly different from the student.\n  - **Using CNN-based surrogates attacking ViT-like students is covered in our paper**, e.g., ResNet-18 (surrogate) vs. EfficientViT (student), where SCAR remains effective.\n  - SCAR's effectiveness mostly stems from two factors: the information gap introduced by KD and the shared data distribution between the training and distillation datasets (see Appendix A). Besides, the bilevel optimization is specifically designed to embed backdoor features into the representations learned by the student. Thus, **if the student achieves high accuracy by learning such representations—regardless of the KD method—it is likely to inherit the backdoor features to a large extent.**\n- To further address your concern, we evaluate SCAR on CIFAR-10 with different surrogates (ResNet-18, ResNet-34, DenseNet-121, SqueezeNet), using ResNet-50 and MobileNet-V2 as teacher and student. As shown in Table 1, **SCAR consistently achieves high attack success rates** (student ASR > 95%) across all settings.\n\n**Table 1**. SCAR attack performance (\\%) with **different surrogate models** on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') using three KD methods.\n|Surrogate|T-ACC|T-ASR|S-ACC (Resp.)|S-ASR (Resp.)|S-ACC (Feat.)|S-ASR (Feat.)|S-ACC (Rel.)|S-ASR (Rel.)|\n|-|-|-|-|-|-|-|-|-|\n|ResNet-18|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|ResNet-34|92.32|1.42|92.26|99.47|90.77|99.68|91.76|99.92|\n|DenseNet-121|92.79|1.43|92.19|99.80|90.67|95.78|91.80|99.90|\n|SqueezeNet|92.41|1.56|91.86|99.94|91.16|99.13|91.78|99.88|\n\nWe will add more details in our revision.\n\n---\n\n**W2**: The paper notes SCAR’s effectiveness may decline on large-scale datasets but provides no experiments or theory to support this.\n\n**R2**: Thank you for pointing this out!\n- As illustrated in Appendix I, SCAR may indeed face this limitation.\n  - This limitation is largely because large-scale datasets can intensify the instability of the bilevel optimization, potentially causing convergence to a poor local minimum.\n  - **We have also analyzed SCAR’s effectiveness through the perspectives of information gap and data distribution** (in Appendix A), which helps explain this limitation:\n    - Information Gap View: large-scale datasets make it more difficult to exploit knowledge transfer gaps during distillation for backdoor injection, significantly increasing the complexity of bilevel optimization.\n    - Data Distribution View: the increased complexity of large-scale datasets makes it harder for the student model to learn data-distribution-specific backdoor features.\n- Nevertheless, we argue this doesn't undermine the paper's core message: **users should remain vigilant and always perform backdoor detection on distilled student models**, even when both the teacher and distillation dataset have been verified as secure. Despite the lower ASR (just over 50%) on ImageNet, the threat remains significant and non-negligible.\n- As the first work introducing this novel threat, we chose not to incorporate additional modules used to overcome this potential limitation. **Our goal is to present a clean and minimal version of SCAR in order to make the nature of distillation-conditional backdoors more clear and avoid introducing potential confusion.**\n\nNevertheless, we fully agree that enhancing attack performance on large-scale datasets is a valuable research direction. We will explore it in the future.\n\n---\n\n**W3**: The bilevel optimization depends on implicit differentiation and pre-optimized triggers, which may cause bias or instability if the surrogate model poorly reflects real KD dynamics.\n\n**R3**: Thank you for this insightful comment! \n- We fully understand your concern that our effectiveness may due to the similarties between simulated and real KD processes. However, we did evaluate our SCAR under various KD methods different from that used for the surrogate (in Table 1). **Our method remains effective across these settings**. \n- We argue that its effectiveness is mostly because\n  - Information Gap View: **all KD methods involve imperfect knowledge transfer, creating an information gap**. This allows hidden backdoor features to transfer while robust ones may be only partially passed (see Appendix A.1).\n  - Data Distribution View: **various KD methods still guide the student model to learn distribution-specific natural backdoor features.** If the distillation dataset shares the teacher’s distribution, these backdoor features tend to transfer regardless of KD strategy (see Appendix A.2).\n  - Our bilevel optimization is specifically designed to embed the backdoor features into the representation learned by the students. Therefore, **if the student achieves high accuracy by learning such representations, regardless of the KD method, it is also likely to inherit the backdoor features to a large extent**.\n\nWe will add more details in the revision.\n\n---\n\n**W4**: A key concern is whether the student can evade backdoor detection.\n\n**R4**: Thank you for this insightful comment! We apologize for any potential misunderstanding and hereby provide further clarification.\n- In our threat model, an adversary uploads a teacher model with a distillation-conditional backdoor to a verification platform. The platform fails to detect the dormant backdoor and marks the model as benign. A victim user then downloads it and performs KD using a clean dataset. **This scenario is realistic and meaningful, as users often assume that distilling a verified \"clean\" model with a benign dataset yields a \"safe\" student model.** However, our work shows that the student model can still inherit backdoors during this seemingly “secure” process.\n- Our goal is to reveal this novel threat and raise awareness. We do not claim the attack is undetectable or undefeatable. Rather, **we encourage users to always inspect student models for backdoors**, even when both the teacher and dataset are deemed secure (see Lines 70, 83, and 334).\n- As the first to propose distillation-conditional backdoors, we focus on presenting a clear, minimal version of SCAR, without designing variants to evade student-side detection. This helps ensure clarity and avoid potential misunderstandings.\n\nWe will add more discussions in the revision.\n\n---\n\n**Q2**: The paper does not test the surrogate model against SOTA defenses. How effective are these methods, and how can SCAR improve its stealth?\n\n**R5**: Thank you for this insightful question!\n- The surrogate model is **only used by the attacker during the training of the poisoned teacher model** to simulate the user’s distillation process. It is **not exposed** to the verification platform or victim users, and thus neither detection nor defense is feasible.\n- We speculate your concern may relate to whether distilled student models can be detected. This is indeed an important question.\n  - As noted in **R4**, our threat model is realistic and highlights a key risk: **even if the teacher and dataset are clean, the student may still inherit backdoors.** Users should rigorously inspect student models.\n  - We agree that enhancing SCAR’s stealthiness against student-side detection is a promising future direction. This is something we are very interested in exploring in our future work.\n\n---\n\n**Q3**: The bilevel optimization assumes differentiability of the inner loop. How sensitive is SCAR to approximations in this process (e.g., finite inner steps, surrogate initialization)?\n\n**R6**: Thank you for this insightful question! We hereby offer further clarification:\n- **The surrogate model is randomly re-initialized** at the start of each outer epoch; no special initialization is used.\n- We fix the number of inner optimization steps to 20 in our main experiments. **This is mostly because the surrogate model has already reached a reasonably good local optimum** (around 85% accuracy compared to the teacher) within these steps, which is sufficient for guiding the outer optimization.\n- To assess sensitivity to this setting, we conduct ablation studies using different inner steps (20, 30, 40) when attacking ResNet-50 on CIFAR-10. The surrogate is ResNet-18, and the student is MobileNet-V2. As shown in Table 2, **SCAR remains effective under all settings.** This suggests that once the inner parameters are optimized to a nearly local optimum, the outer gradient is adequately approximated.\n\n**Table 2**. SCAR attack performance (\\%) with **different inner steps** on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') using three KD methods.\n|Inner Step|T-ACC|T-ASR|S-ACC (Resp.)|S-ASR (Resp.)|S-ACC (Feat.)|S-ASR (Feat.)|S-ACC (Rel.)|S-ASR (Rel.)|\n|-|-|-|-|-|-|-|-|-|\n|20 steps|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|30 steps|92.35|1.46|92.13|99.41|91.03|99.04|91.26|99.39|\n|40 steps|92.20|1.51|91.98|99.69|91.16|99.18|91.87|99.49|\n\nWe will include this analysis in the revision."}, {"review_id": "uHuITGQF7E", "reviewer": "Reviewer_8GaK", "summary": "The paper proposed a backdoor defense approach specifically for the knowledge-distillation setting using a bilevel optimization method. The approach follows a workflow of distributor -> verifier -> user, where the teacher model first needs to pass backdoor detection defenses. The outer optimization takes into account the \"static\" part of KD-resistant backdoor injection while the inner optimization handles the \"dynamic\" KD process. The pipeline is done by training a surrogate model so no prior knowledge on user-end architecture needed. Additionally, the paper proposed to overcome the challenging optimization with an implicit differentiation algorithm with a pre-optimized trigger injection function. Under the threat model that the attacker has the access and the freedom to inject backdoor to the teacher model, the paper showed that their approach is capable of beating ADBA with a huge margin especially when 1. the datasets are more challenging like ImageNet and 2. the architectural differences between the teacher and the student models are more different.", "questions": "Q1. Can the authors further elaborate on the inner optimization in (2)? Does the KD loss depend on the specific kind of KD used? If not, it is not clear to me how minimizing the KL divergence between the output logits of F_t and F_s (which seems to be Response-based KD) can improve backdoor resistance against other KD strategies.\n\nQ2. I think it would be interesting to see the L2 loss pattern of backdoor vs clean samples during the training of model, and how well the approach can bypass the verifier if we replace detection-based method with other kinds of defenses such as ABL [1], which also has a backdoor detection component, assuming the verifier has a small subset of the training set?\n\nQ3. How does the approach work against SOTA distillation-based defenses like NAD?\n\nQ4. Does the choice of the surrogate model matter? For instance, does backdoor generation using a smaller/bigger surrogate model transfer better in KD?\n\nS1. The detection methods used in the experiments are overall too simple. I wonder how well this approach can bypass more advanced detection methods like A2D [2] or BAN [3].\n\n[1] Li, Yige, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. \"Anti-backdoor learning: Training clean models on poisoned data.\" Advances in Neural Information Processing Systems 34 (2021): 14900-14912.\n[2] Samar Fares and Karthik Nandakumar. Attack to defend: Exploiting adversarial attacks for detecting poi-\nsoned models.\n[3] Xiaoyun Xu, Zhuoran Liu, Stefanos Koffas, Shujian Yu, and Stjepan Picek. Ban: Detecting backdoors\nactivated by adversarial neuron noise. Advances in Neural Information Processing Systems, 37:114348–\n114373, 2025.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths -\n* Overall the paper is well-written and technically sounding.\n* The approach seems to be novel.\n* The authors provided theoretical justifications of their approach.\n* The threat model assumed is realistic and practical\n\nWeaknesses -\n* Limited comparisons against prior works, but understandable since KD-resistant backdoor attacks are relatively new\n* Weak detection methods used in experiments\n* No experiments against existing non-detection defenses", "was_revised": true, "final_justification": "The authors addressed all my concerns and I gave an acceptance score already. Given the limitations of scope of this work, I would like to keep my score as borderline accept.", "rebuttal": "Thank you for your careful review and thoughtful comments! We are encouraged by your positive comments on our **well-written and technically sounding paper**, **novel approach**, **theoretical justifications of our approach**, **realistic and practical threat model**, **good quality**, **good significance** and **good originality**. We hope the following responses can help clarify potential misunderstandings and alleviate your concerns.\n\n---\n\n**W1**: Limited comparisons against prior works, but understandable since KD-resistant backdoor attacks are relatively new.\n\n**R1**: Thank you for pointing it out! We sincerely apologize for any misunderstanding. We hereby provide further clarification.\n- The threat proposed in our paper is **distillation-conditional backdoor (DCB) rather than distillation-resistant backdoor (DRB).** \n  - DRB aims to transfer **active** backdoors from the teacher to the student during distillation. \n  - DCB aims to implant a **dormant** backdoor into the teacher that remains inactive and **only becomes activated after the student is distilled**. The teacher must appear entirely benign, while the student exhibits malicious behavior. This makes DCB more challenging and, arguably, more threatening than DRB.\n- As the first work to propose DCB, we can only compare against adapted variants of related methods. Section 2.2 of our paper discusses DRB, which remains a relatively new and underexplored area. **We adapted the most representative method in this domain (ADBA) as our baseline**.\n\nWe will add more discussions in our revision.\n\n---\n\n**W2&S1**: Weak detection methods used in experiments. I wonder how well this approach can bypass more advanced detection methods like A2D or BAN.\n\n**R2**: Thank you for these constructive suggestions! \n- We respectfully note that we have conducted evaluation under both classical and advanced detection methods (e.g., BTI-DBF in Appendix C). \n- To further alleviate your concerns, we conduct additional evaluation using your suggested detection methods. Specifically, we conduct 5 independent detection trials on a ResNet-50 teacher trained on CIFAR-10 with target label 0.\n  - For A2D, following the sensitivity to adversarial perturbations (SAP) protocol, a backdoored model is expected to show high SAP values. As shown in Table 1, across 5 trials, **the SCAR-attacked teacher consistently exhibits low SAP, indicating that A2D failed to detect the backdoor**.\n  - For BAN, we record both the prediction distribution after mask training and the predicted target class. As shown in Table 2, across all 5 trials, **BAN also consistently misjudges the backdoor target class, leading to a high false positive rate**.\n\n**Table 1**. SAP values reported by A2D.\n||1|2|3|4|5|\n|-|-|-|-|-|-|\n|SAP|0.117|0.117|0.185|0.206|0.027|\n\n**Table 2**. Prediction distributions generated by BAN. **Bolded**: the target labels inferred by BAN.\n|Class $\\rightarrow$|0|1|2|3|4|5|6|7|8|9|\n|-|-|-|-|-|-|-|-|-|-|-|\n|1|434|461|**1943**|459|43|558|151|46|454|451|\n|2|333|**2628**|272|70|146|38|0|253|766|494|\n|3|141|432|260|1028|11|6|111|**2057**|536|418|\n|4|297|43|215|266|**3066**|287|228|89|285|224|\n|5|176|371|696|79|65|1182|123|**1512**|209|587|\n\nWe will add more details in our revision.\n\n---\n\n**W3&Q3**: No experiments against existing non-detection defenses. How does the approach work against SOTA defenses like NAD?\n\n**R3**: Thank you for pointing it out! We are deeply sorry for any potential misunderstanding and hereby offer further clarification.\n- We respectfully note that, in our threat model, the attacker uploads a model containing a DCB to a trusted third-party platform. This platform performs backdoor detection and, upon verifying no abnormal behavior, releases the model. A victim user then downloads the model and uses it for further KD.\n  - A key aspect of this scenario is that **the platform is usually only responsible for detection, not mitigation**. Typically, uploaded models may be trained on large-scale datasets over long periods, while developers usually only upload their model without uploading training data and configurations. As such, **such mitigation is generally infeasible or at least significantly degrades model performance and requires many computational resources**. Given the platform's limited resources, we assume it does not conduct non-detection defenses.\n- We fully understand your concern, where users may apply mitigation methods to the downloaded teacher model before KD.\n    - To further alleviate your concerns, we evaluate two non-detection defenses suggested by you and another reviewer: NAD and fine-pruning. We apply these defenses to a SCAR-attacked ResNet-50 on CIFAR-10, and then use the defended model for KD to train MobileNet-V2. \n    - As shown in Table 3, **SCAR remains highly effective, even after the teacher has undergone these backdoor mitigation.** Besides, using non-detection defenses may even slightly raise ASR. This might be because such defenses disrupt the carefully crafted \"mask\" of the DCB, thus re-exposing the hidden backdoor.\n    - While NAD and fine-pruning fail to eliminate SCAR, we don't intend to claim that it is undefeatable. Rather, **our goal is to raise awareness that even models which pass backdoor detection should always not be assumed safe for distillation without further examination or cleansing.**\n- Besides, as the first work to propose DCB, we intentionally avoid incorporating complex techniques to bypass potential defenses. **We aim to clearly define the threat and present a simple, effective instantiation (SCAR), avoiding unnecessary design complexity that could cause confusion.**\n\n**Table 3**. SCAR attack performance (\\%) on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') under two mitigation methods, using three KD methods.\n|Defense|T-ACC (w/o Defense)|T-ASR (w/o Defense)|T-ACC (w/ Defense)|T-ASR (w/ Defense)|S-ACC (Resp.)|S-ASR (Resp.)|S-ACC (Feat.)|S-ASR (Feat.)|S-ACC (Rel.)|S-ASR (Rel.)|\n|-|-|-|-|-|-|-|-|-|-|-|\n|No Defense|92.47|1.50|-|-|91.62|99.94|91.01|99.90|91.29|99.93|\n|NAD|92.47|1.50|88.45| 9.60|89.08|99.94|86.12|89.20|87.11|99.58|\n|fine-pruning|92.47|1.50|92.72|5.94|92.17|99.47|90.44|97.41|91.08|99.34|\n\nWe will include more discussions in the revision.\n\n---\n\n**Q1**: Can the authors elaborate on the inner optimization in (2)? Does the KD loss vary with the KD method used?\n\n**R4**: Thank you for these insightful questions! \n- In (2), the inner optimization **adopts the standard knowledge distillation (KD) loss** to guide the surrogate to approximate the knowledge transfer process during distillation. Specifically, we minimize the KL divergence between the logits of the teacher and the surrogate through a finite number of gradient steps.\n- We fully understand your concern that our effectiveness may be due to the similarties between the simulated and the real KD processes. However, we did evaluate our SCAR under various KD methods different from that used for the surrogate (in Table 1). **Our method remains effective across these settings**. \n- We argue that its effectiveness is mostly because\n  - **From the information gap view, all KD methods involve imperfect knowledge transfer, creating an information gap.** This allows hidden backdoor features to transfer while robust ones may be only partially passed (see Appendix A.1).\n  - **From the data distribution view, various KD methods still guide the student to learn implicit, data-distribution-specific natural backdoor features.** If the distillation dataset shares the teacher’s data distribution, these backdoor features tend to transfer regardless of KD strategy (see Appendix A.2).\n  - The bilevel optimization in SCAR is specifically designed to embed the backdoor features into the representation learned by the students. Therefore, **if the student achieves high accuracy by learning such representations—regardless of the KD method—it is also likely to inherit the backdoor features to a large extent**.\n\nWe will further clarify these points in the revision to help readers better understand the underlying reasons for SCAR’s effectiveness.\n\n---\n\n**Q2**: I’m interested in the L2 loss patterns of backdoor vs. clean samples during training and how well the method bypasses verifiers using defenses like ABL, assuming the verifier holds a small training subset.\n\n**R5**: Thank you for this insightful question! \n- As we mentioned in **R3**, in our attack scenario, the attacker only uploads a compromised (potentially poisoned) model to the platform (verifier), but does not upload the corresponding training dataset.\n- Even if we assume that the attacker will upload parts of the training/testing data, they will only upload the benign ones instead of also the poisoned ones.\n- When the user downloads the model and performs knowledge distillation, they only use clean samples, i.e., **no poisoned samples are involved in the distillation process**. Therefore, training-time detection methods (and their follow-up mitigation) like ABL are not feasible in our attack scenario.\n\nWe will include more discussions in the revision.\n\n---\n\n**Q4**: Does the choice of the surrogate model matter? Does backdoor generation using a smaller/bigger surrogate model transfer better in KD?\n\n**R6**: Thank you for these constructive suggestions! We are deeply sorry for any potential misunderstanding. \n- We would like to first respectfully clarify that our main experiments already evaluate settings with mismatched surrogate and student models (Table 1), where our **SCAR remains effective** under these settings.\n- To further alleviate your concerns, we evaluate SCAR on CIFAR-10 with different surrogates (ResNet-18, ResNet-34, DenseNet-121, SqueezeNet), and **it consistently achieves high attack success rates** (student ASR > 95%) across all settings (See our **R1** to **Reviewer TVht** for more details)."}, {"review_id": "4Yb7eTHfO8", "reviewer": "Reviewer_pHSe", "summary": "This paper focuses on the potential threat of backdoor attacks on a student model distilled from a benign teacher model. By proposed a method named SCAR, the authors demonstrated that a fine-tuned teacher model may not be detected as backdoored, while a student model distilled from it may behave in a malicious way when a backdoor trigger is present. As a result, the authors suggest that a distilled model should always be examined for the backdoor risk.", "questions": "Please see the Concerns section above.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n1. The motivation and main message are clearly explained and easy to follow. Figure 1 is quite helpful. \n\n2. The topic of backdoor attack for distilled models is important and seemingly less stuided.  \n\n3. I appreciate additional details in the appendix, especially on the plausible explanation for the success of SCAR. \n\n\n\n**Concerns:**\n\n1. The motivation of SCAR is pretty simple. It basically mimics the behavior of the student and hope to find an effective trigger after distillation. I have two concerns about the formulation of optimization objective (2):\n\n   (i) There are a lot of hyper-parameters such as $\\alpha, beta, \\gamma, \\delta$. While the authors provide some ablation on the choice of $\\delta$, are there any insights on other parameters? Any guidelines for practioners to choose them in general?\n\n   (ii) The effectiveness of SCAR seems to be highly dependent on the quality of this surrogate student model. As a result, what if the KD process is significantly different from the one used in Eq. (2)? For example, when the training dataset of KD is different from $\\mathcal{D}$. Can authors quantify the effectiveness/robustness of the proposed method given a mismatched surrogate model? \n\n2. The optimization process. The authors proposed to optimize the bi-level objective by directly approximating the gradient for the outer loop. This approach suffers a couple problems, e.g.,\n\n   (i) High computation cost. This is supported by Appendix F, largely due to the calculation of Hessian and inverse Hessian. \n\n   (ii) Non-convexity of the objective. While convexity is often assumed for the convenience of theoretical analysis, it is not the case for the problems studied in this paper. Nevertheless, I can understand that Eq. (4) is more or less motivated by this. \n\n   I would suggest the authors to consider other optimization approches to optimize this bi-level objective. There has been a huge literature on that, such as alternative gradient descent. \n\n3. A minor concern is the quality of backdoor triggers. Although the student is backdoored by SCAR, it looks like the poisoned image can be easily detected as in Appendix G. Can SOTA detection method successfully detect a backdoored student model or poisoned inputs for the student model?", "was_revised": true, "final_justification": "Authors have addressed my concerns, particularly on the effectiveness of SCAR with a surrogate student model. I am therefore leaning to accpentance.", "rebuttal": "Thank you for your careful review and thoughtful comments! We are encouraged by your positive comments on our **clear explanation and expression**, **important research topic**, **additional details in the appendix**, **good quality**, **good clarity**, **good significance** and **good originality**. We hope the following responses can help clarify potential misunderstandings and alleviate your concerns.\n\n---\n\n**Q1-1**: Many hyperparameters (e.g., $\\alpha$, $\\beta$, $\\gamma$) are involved—are there insights or general guidelines for tuning them?\n\n**R1-1**: Thank you for pointing it out! We apologize for not discussing the choice of the hyperparameters $\\alpha$, $\\beta$, and $\\gamma$ in the original paper. We would like to provide further clarification.\n- In the outer optimization objective in Eq. (2), **the scales of the four loss terms are approximately comparable** and relate to four different aspects (e.g., the student's performance on benign samples). Attackers can adjust their values based on specific needs or priorities.\n- In our experiments, we set $\\alpha = \\beta = \\gamma = 1$ for simplicity and clarity, as this is the most straightforward configuration.\n- We fully understand your concern, and conduct ablation studies to evaluate SCAR’s sensitivity to these hyperparameters. Specifically, we train teachers (ResNet-50) using SCAR on CIFAR-10 with $\\alpha$, $\\beta$, and $\\gamma$ independently set to 0.5, 1.0, and 1.5. We use ResNet-18 as the surrogate and MobileNet-V2 as the student. As shown in Table 1, **SCAR consistently achieves strong attack performance (student ASR > 98\\%) across different hyperparameter settings**.\n\n**Table 1**. SCAR attack performance (\\%) on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') **under different hyperparameter settings**, using three KD methods.\n|Hyperparameter Setting|T-ACC|T-ASR|S-ACC (Resp.)|S-ASR (Resp.)|S-ACC (Feat.)|S-ASR (Feat.)|S-ACC (Rel.)|S-ASR (Rel.)|\n|-|-|-|-|-|-|-|-|-|\n|$\\alpha=0.5$|92.43|1.77|91.67|99.81|91.12|99.36|91.58|99.94|\n|$\\alpha=1$|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|$\\alpha=1.5$|92.54|1.33|92.08|99.46|91.03|99.71|91.78|99.59|\n|$\\beta=0.5$|92.21|1.61|92.36|99.37|91.30|99.04|91.33|99.07|\n|$\\beta=1$|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|$\\beta=1.5$|92.35|1.43|91.99|99.83|90.87|99.01|91.52|99.36|\n|$\\gamma=0.5$|92.55|1.82|92.40|99.51|91.25|98.58|91.46|99.53|\n|$\\gamma=1$|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|$\\gamma=1.5$|92.13|1.88|91.74|99.97|90.98|99.77|91.64|99.23|\n\nWe will include this new ablation study and discussion in the revision.\n\n---\n\n**Q1-2**: How does SCAR perform when the KD process deviates from Eq. (2), e.g., using a KD training set different from $D$? Can the authors quantify its effectiveness or robustness given a mismatched surrogate model?\n\n**R1-2**: Thank you for these insightful questions! \n- We fully understand your concern that our effectiveness may be due to the similarties between the simulated and the real KD processes. However, we did evaluate our SCAR under various KD methods different from that used for the surrogate (in Table 1). **Our method remains effective across these settings**. \n- We argue that its effectiveness is mostly because\n  - **From the information gap view, all KD methods involve imperfect knowledge transfer, creating an information gap.** This allows hidden backdoor features to transfer while robust ones may be only partially passed (see Appendix A.1).\n  - **From the data distribution view, various KD methods still guide the student model to learn data-distribution-specific natural backdoor features.** If the distillation dataset shares the teacher’s distribution, these backdoor features tend to transfer regardless of KD strategy (see Appendix A.2).\n  - The bilevel optimization is specifically designed to embed the backdoor features into the representation learned by students. Therefore, **if the student achieves high accuracy by learning such representations—regardless of the KD method—it is likely to inherit the backdoor features to a large extent**.\n- To further alleviate your concerns, we conduct additional experiments where **the distillation dataset differs from the training set**. We train a ResNet-50 teacher using SCAR on CIFAR-10, and then distill MobileNet-V2 students using a subset of CINIC-10, which contains the same 10 classes but has a different data distribution (5,000 training and 1,000 test samples per class).\n    - As shown in Table 2, due to the distributional shift, the teacher's accuracy drops to 68.83\\%. The students also experience a drop in accuracy. Although the attack success rate on the students decreased accordingly, it still remained above 54%.\n    - **These results suggest that if the student fails to achieve high accuracy during distillation—meaning it does not effectively inherit the teacher's benign knowledge—it is also less likely to inherit the backdoor.** However, such significant drops in accuracy for both teacher and student models indicates an ineffective distillation process, which is usually not meaningful in practice.\n- **We believe it is challenging to quantify the effectiveness or robustness of SCAR under a specific surrogate model, as the distillation process is inherently dynamic and variable.** Nevertheless, we would be more than happy to explore this further if you have any concrete suggestions.\n\n**Table 2**. SCAR attack performance (\\%) on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') **with different distillation datasets**, using three KD methods.\n|Distillation Dataset|T-ACC|T-ASR|S-ACC (Response)|S-ASR (Response)|S-ACC (Feature)|S-ASR (Feature)|S-ACC (Relation)|S-ASR (Relation)|\n|-|-|-|-|-|-|-|-|-|\n|CIFAR-10|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n|CINIC-10|68.83|5.22|76.22|74.51|72.35|54.83|75.81|75.58|\n\nWe will include further clarification and discussion in the revision.\n\n---\n\n**Q2**: Optimizing the bi-level objective by directly approximating the outer-loop gradient faces issues:\n(i) High computational cost.\n(ii) Non-convexity of the objective.\nI suggest considering alternating gradient descent.\n\n\n**R2**: Thank you for this constructive suggestion! \n- **The main advantage of our derived implicit differentiation algorithm is its ability to capture the interdependence between inner and outer parameters, even when they are not part of the same model entity.** While the alternative gradient descent is indeed a commonly used optimization method, it is not designed to handle gradient propagation across *different entities* (e.g., from student to teacher), as discussed in line 194 of the paper.\n- Implicit differentiation is a classical, well-established method for bilevel optimization. Since our work is the first to propose the distillation-conditional backdoor attack, **our goal is not to over-complicate the design, but to clearly demonstrate this novel threat with a simple yet effective implementation**.\n- For (i): As we mentioned in Appendix F, our optimization procedure does introduce a non-negligible time cost. Arguably, we do not view this as a major limitation. **The optimization is performed offline and only once by the attacker, making it a one-time cost.**\n- For (ii): We fully understand your concern, and would like to provide additional clarification.\n  - **Our implicit differentiation algorithm is agnostic to the convexity of the inner problem** to some extent: as long as the inner optimization reaches a *suboptimal* solution, it suffices to estimate the outer gradient within a controllable error bound [1]. We will revise the paper to include and clarify this explanation. \n  - The KD loss used in our inner loop is essentially a combination of a log-sum-exp function (a well-known convex function [2]) and a linear function. **The sum of a convex and a linear function remains convex, which supports the validity of our approach in this context.**\n\nThank you again for raising this point. We will emphasize the theoretical justification more clearly in the revision.\n\n**References**\n\n[1] On the iteration complexity of hypergradient computation.\n[2] Convex optimization.\n\n---\n\n**Q3**: Poisoned images seem easily detected. Can SOTA methods reliably detect the backdoored student or its poisoned inputs?\n\n**R3**: Thank you for this insightful comment! We apologize for any potential misunderstanding. We would like to offer further clarification.\n- Arguably, **compared to traditional dataset-poisoning backdoor attacks, the visibility of the trigger is less critical in our threat model.** This is because the poisoned training samples are not accessible to either the platform or the end users, and only take effect at inference time.\n- You may further worry about why we don't conduct more detection on student models.\n  - In our threat model, an adversary uploads a teacher model with a distillation-conditional backdoor to a verification platform. The platform fails to detect the dormant backdoor and marks the model as benign. A victim user then downloads it and performs KD using a clean dataset. **This scenario is realistic and meaningful, as users often assume that distilling a verified \"clean\" model with a benign dataset yields a \"safe\" student model.** However, our work shows that the student model can still inherit backdoor vulnerabilities during this seemingly “secure” process.\n  - Our goal is to reveal this novel threat and raise awareness. We do not claim the attack is undetectable or undefeatable. Rather, **we encourage users to always inspect student models for backdoors**, even when both the teacher and dataset are deemed secure (see Lines 70, 83, and 334).\n\nNevertheless, we fully agree that enhancing the student-side evasiveness and the trigger's stealthiness is an important and meaningful direction. We will add more discussions in the revision."}, {"review_id": "YRmvKe9tyj", "reviewer": "Reviewer_XiRf", "summary": "This paper investigates a novel security threat in the knowledge distillation (KD) framework. Specifically, the paper introduced SCAR, a new backdoor attack method that embeds dormant backdoors into teacher models so that these backdoors remain undetectable during standard verification procedures but can be activated in student models after distillation. The SCAR formulates the problem as a bilevel optimization problem and employs an implicit differentiation algorithm with a pre-optimized trigger injection function to efficiently realize the backdoor.  Extensive experiments across various datasets, model architectures, and distillation techniques demonstrate that SCAR is highly effective and stealthy, capable of evading existing detection methods.", "questions": "1. Is the backdoor in the teacher model still preserved when the user fine-tunes the poisoned teacher model or uses the fine-tuning-based defense methods (such as fine-pruning)?\n\n2. Combined with Question 1, when the defender has more attack knowledge(such as the specific attack method and parameters), can adaptive defense be designed to protect the student model?\n\n3. I wonder about the robustness of such carefully optimized triggers. Will adding some noise to poisoned samples cause the triggers to fail? Please refer to the defense method Sampdetox [3].\n\nReference:\n\n[3] Yang et al. SampDetox: Black-box Backdoor Defense via Perturbation-based Sample Detoxification. NeurIPS 2024.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. This paper uncovers a new security threat in the knowledge distillation framework, where backdoors can be hidden in teacher models while activated in student models, highlighting an underexplored area in AI security.\n\n2. This paper conducted comprehensive experiments across diverse datasets, model architectures, and knowledge distillation techniques.\n\n3. The paper is well written and clearly structured.\n\nWeaknesses:\n\n1. In ablation studies, I think it is not enough to use a fixed white patch to replace the optimized triggers. More types of triggers are worth considering. For example, since the optimized triggers cover the entire image instead of being located in corners, it is better to conduct experiments with this type of trigger.\n\n2. It is better to conduct more experiments to investigate the resistance of SCAR to SOTA backdoor detection methods, such as MDTD [1] and TED [2].\n\nReference:\n\n[1] Rajabi et al.MDTD: A Multi-Domain Trojan Detector for Deep Neural Networks. CCS 23.\n\n[2] Mo et al. Robust Backdoor Detection for Deep Learning via Topological Evolution Dynamics. SP 2024.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed most of my concerns. I raise my score to 5.", "rebuttal": "Thank you for your careful review and thoughtful comments! We are encouraged by your positive comments on our **novel security threat**, **comprehensive experiments**, **well written and clearly structured paper**, **good quality** and **good clarity**. We hope the following responses can help clarify potential misunderstandings and alleviate your concerns.\n\n---\n\n**W1**: In ablation studies, more types of triggers are worth considering, especially the optimized triggers cover the entire image.\n\n**R1**: Thank you for this constructive suggestion! We fully agree with your point and conduct additional experiments accordingly.\n- We replace the pre-trained triggers in SCAR with image-size poisoning strategies from WaNet and BppAttack. We then train teacher models ResNet-50 on CIFAR-10 using these methods. ResNet-18 is used as the surrogate, and MobileNet-V2 serves as the student. We evaluate the attack success rates on students obtained via different KD methods.\n- As shown in Table 1, **using these image-size poisoning strategies still fails to effectively transfer backdoors to the student models** (student ASR < 25\\%). It indicates that simply using image-size triggers cannot be easily 'entangled' with benign features, even if they cover the entire image, and thus fail to transfer during distillation. \n\n**Table 1**. SCAR attack performance (\\%) on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed 'T') **under different trigger types**, using three KD methods.\n|Trigger Type|T-ACC|T-ASR|S-ACC (Response)|S-ASR (Response)|S-ACC (Feature)|S-ASR (Feature)|S-ACC (Relation)|S-ASR (Relation)|\n|-|-|-|-|-|-|-|-|-|\n|BadNets|93.81|0.72|91.47|1.03|91.49|1.48|91.22|1.38|\n|WaNet|93.07|2.20|91.07|20.97|90.47|9.73|90.77|21.22|\n|BppAttack|93.65|1.11|91.43|1.17|89.29|1.86|91.36|1.07|\n|**Ours**|92.47|1.50|91.62|99.94|91.01|99.90|91.29|99.93|\n\nWe will include more discussions in the revision.\n\n---\n\n**W2**: It is better to conduct more experiments to investigate the resistance of SCAR to SOTA backdoor detection methods, such as MDTD and TED.\n\n**R2**: Thank you for this constructive suggestion! \n- We respectfully note that we have conducted evaluation under both classical and advanced detection methods (e.g., BTI-DBF in Appendix C). \n- To further alleviate your concerns, we evaluate the suggested detection methods MDTD and TED on a ResNet-50 teacher trained on CIFAR-10 (target label 0) over 5 independent trials. Since both methods are input-level detection, we use the poisoned test set as input and compute the F1-score for poisoned sample detection. As shown in Table 2, **both methods yield very low F1-scores (< 0.06), indicating that they misclassify most poisoned samples as benign and thus failed to detect the poisoning effectively.**\n\n**Table 2**. F1-scores of two different input-level detection methods on poisoned test samples for the SCAR-attacked ResNet-50 teacher model.\n|Detection Method|1|2|3|4|5|\n|-|-|-|-|-|-|\n|MDTD|0.0488|0.0417|0.0225|0.0512|0.0580|\n|TED|0.0386|0.0281|0.0056|0.0120|0.0196|\n\nWe will include more details in the revision.\n\n---\n\n**Q1**: Does the backdoor in the teacher model persist after user fine-tuning or applying fine-tuning-based defenses (e.g., fine-pruning)?\n\n**R3**: Thank you for pointing it out! We are deeply sorry for any potential misunderstanding. We would like to provide further clarification here.\n- We respectfully note that, in our threat model, the attacker uploads a model containing a distillation-conditional backdoor to a trusted third-party platform. This platform performs backdoor detection and, upon verifying no abnormal behavior, releases the model. A victim user then downloads the model and uses it for further KD.\n    - A key aspect of this scenario is that **the platform is usually only responsible for detection, not mitigation** (like fine-tuning/pruning). Typically, uploaded models may be trained on large-scale datasets over long periods, while model developers usually only update their model instead of also updating their training data and configurations. As such, **such mitigation is generally infeasible or at least significantly degrades model performance and requires many computational resources**. Given the platform's limited resources, we assume it does not conduct non-detection defenses.\n- However, we do fully understand your concern. In practice, users can apply certain backdoor mitigation to the downloaded teacher model before performing distillation.\n    - To further alleviate your concerns, we evaluate two non-detection defenses suggested by you and another reviewer: fine-pruning and NAD. We apply these defenses to a SCAR-attacked ResNet-50 model on CIFAR-10, and then use the defended model for distillation to train MobileNet-V2. \n    - As shown in Table 3, **SCAR remains highly effective, even after the teacher model has undergone these backdoor mitigation.** We also observe that applying non-detection defenses to the teacher model may even slightly increase the attack success rate. This might be because such defenses disrupt the carefully crafted \"mask\" associated with the distillation-conditional backdoor, thus re-exposing the hidden backdoor.\n    - While NAD and fine-pruning do not successfully eliminate SCAR, we do not intend to claim that it is undefeatable. Rather, **our goal is to raise awareness that even models which pass backdoor detection should always not be assumed safe for distillation without further examination or cleansing.**\n\n**Table 3**. SCAR attack performance (\\%) on the student MobileNet-V2 (dubbed 'S') distilled from the teacher ResNet-50 (dubbed  'T') **under two backdoor mitigation defenses**, using three KD methods.\n|Defense|T-ACC (Before Defense)|T-ASR (Before Defense)|T-ACC (After Defense)|T-ASR (After Defense)|S-ACC (Resp.)|S-ASR (Resp.)|S-ACC (Feat.)|S-ASR (Feat.)|S-ACC (Rel.)|S-ASR (Rel.)|\n|-|-|-|-|-|-|-|-|-|-|-|\n|No Defense|92.47|1.50|-|-|91.62|99.94|91.01|99.90|91.29|99.93|\n|NAD|92.47|1.50|88.45| 9.60|89.08|99.94|86.12|89.20|87.11|99.58|\n|fine-pruning|92.47|1.50|92.72|5.94|92.17|99.47|90.44|97.41|91.08|99.34|\n\nWe will include more discussions in the revision.\n\n---\n\n**Q2**: If the defender has more attack knowledge (e.g., specific attack methods and parameters), can adaptive defense be designed to protect the student model?\n\n**R4**: Thank you for this insightful question! \n- We first respectfully note that, as we mentioned in **R3**, both the platform and the end user only have access to the compromised model uploaded by the attacker **without further knowledge of the exact training data, procedures, or hyperparameters** in our threat model.\n- In particular, we do not claim the attack is undefeatable. Rather, **we encourage users to always inspect student models for backdoors**, even when both the teacher and dataset are deemed secure (see Lines 70, 83, and 334).\n- Besides, as the first work to propose distillation-conditional backdoors, we do not design additional modules to enhance the resistance to student-side defenses. **Our goal is to clearly present this novel attack surface and demonstrate a simple yet effective implementation, without introducing additional modules that may potentially obscure the threat or cause confusion.**\n\nWe truly appreciate your question, which raises an important point. While it somewhat goes beyond the scope of this work, we believe it points to a promising direction for future research and will explicitly highlight it in the revision.\n\n\n---\n\n**Q3**: Will adding some noise to poisoned samples cause the triggers to fail? Please refer to the defense method Sampdetox.\n\n**R5**: Thank you for this insightful question! \n- We hereby first clarify some potential misunderstandings.\n    - As we explained in **R3**, in our threat model, users may assume that distilling a \"clean\" teacher model with a benign dataset will result in a \"safe\" student model. However, our work shows that this assumption does not always hold: **the student model can still inherit backdoor vulnerabilities during this seemingly “secure” process.**\n    - The primary objective of our paper is to expose this novel security threat and to raise awareness. We do not intend to claim that this attack is unbreakable. Rather, **we aim to encourage users to carefully inspect student models for potential backdoors, even if the teacher model and dataset have been deemed secure** (as stated in Lines 70, 83, and 334 of the paper).\n    - As the first work to introduce distillation-conditional backdoors, we do not explore designing variants of SCAR that attempt to evade existing student-side defenses. **Our goal is to present a clear and focused analysis of this novel threat, minimizing the risk of misinterpretation.** \n- To further alleviate your concerns, we evaluate our method under your suggested Sampdetox method. Specifically, we launch the SCAR attack on a ResNet-50 teacher model on CIFAR-10, and distill three MobileNet-V2 student models using different KD methods. We then evaluate the defense performance of SampDetox using both benign and poisoned test sets.\n    - As shown in Table 4, although SampDetox can partially reduce the attack success rate of SCAR, it also leads to a noticeable drop in benign accuracy (BA). **This suggests that SCAR has already exhibited a certain degree of robustness against input pre-processing defenses**, although we have no particular design for it.\n\n**Table 4**. The defense performance (\\%) of SampDetox against MobileNet-V2 student models distilled **using three different KD methods**.\n| Distillation Method | BA (Before Defense) | ASR (Before Defense) | BA (After Defense) | ASR (After Defense)|\n|-|-|-|-|-|\n| Response |91.62|99.94|84.13|56.42|\n| Feature |91.01|99.90|82.26|52.60|\n| Relation |91.29|99.93|83.65|76.01|\n\nWe will include more discussions in the revision."}]

Author rebuttal:

Meta-review: This paper investigates the risks associated with knowledge distillation, a common practice where users train a smaller "student" model from a large, imported "teacher" model. The authors propose a method to inject a dormant backdoor into the teacher model that is only activated through this distillation process. Reviewers agreed that this attack setting is both innovative and realistic. Although initial concerns were raised about the experiments, the assumptions, and the sensitivity to hyperparameters, they were well-addressed during the rebuttal.
Decision: Accept (poster)