When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Authors: Ziming Hong
Runnan Chen
Zengmao Wang
Bo Han
Bo Du
Tongliang Liu
Venue: ICML 2025 poster (2025)
Keywords: Data-free Knowledge Distillation
Non-Transferable Learning
Primary area: general_machine_learning->transfer_multitask_and_metalearning

Abstract: Data-free knowledge distillation (DFKD) transfers knowledge from a teacher to a student without access the real in-distribution (ID) data. Its common solution is to use a generator to synthesize fake data and use them as a substitute for real ID data. However, existing works typically assume teachers are trustworthy, leaving the robustness and security of DFKD from untrusted teachers largely unexplored. In this work, we conduct the first investigation into distilling non-transferable learning (NTL) teachers using DFKD, where the transferability from an ID domain to an out-of-distribution (OOD) domain is prohibited. We find that NTL teachers fool DFKD through divert the generator's attention from the useful ID knowledge to the misleading OOD knowledge. This hinders ID knowledge transfer but prioritizes OOD knowledge transfer. To mitigate this issue, we propose Adversarial Trap Escaping (ATEsc) to benefit DFKD by identifying and filtering out OOD-like synthetic samples. Specifically, inspired by the evidence that NTL teachers show stronger adversarial robustness on OOD samples than ID samples, we split synthetic samples into two groups according to their robustness. The fragile group is treated as ID-like data and used for normal knowledge distillation, while the robust group is seen as OOD-like data and utilized for forgetting OOD knowledge. Extensive experiments demonstrate the effectiveness of ATEsc for improving DFKD against NTL teachers.

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Ziming Hong'
Runnan Chen
Zengmao Wang? Bo Han Bo Du
Tongliang Liu
Abstract
1. Introduction
Data-free knowledge distillation (DFKD) trans- fers knowledge from a teacher to student with- out access the real in-distribution (ID) data. Its common solution to use generator synthe- size fake data and use them as substitute for real ID data. However, existing works typically assume teachers trustworthy, leaving the ro- bustness and security of DFKD from untrusted teachers largely unexplored. In this work, we conduct the first investigation into distilling non- transferable learning (NTL) teachers using DFKD, where the transferability from an ID domain an out-of-distribution (OOD) domain is prohibited. We find that NTL teachers fool DFKD through divert the generator's 's attention from the useful ID knowledge to the misleading OOD knowl- edge. This hinders ID knowledge transfer but prioritizes OOD knowledge transfer. To mitigate this issue, we propose Adversarial Trap Escaping (ATEsc) to benefit DFKD by identifying and fil- tering out OOD-like synthetic samples. Specif- ically, inspired by the evidence that NTL teach- ers show stronger adversarial robustness on OOD samples than ID samples, we split synthetic sam- ples into two groups according to their robust- ness. The fragile group is treated as ID-like data and used for normal knowledge distillation, while the robust group is seen OOD-like like data and utilized for forgetting OOD knowledge. Exten- sive experiments demonstrate the effectiveness of ATEsc for improving DFKD against NTL teach- ers. Code is released at https: //gi/// thub. com/ aaaaab//2252 Baaaaa//22__aMm__2 _LML_MMMM_SSSSSSS____ ATEsC.
Recently, data-free knowledge distillation (DFKD) (Mi- caelli & Storkey, 2019; Luo et al., 2020; Fang al., 2021b; Yu et al., 2023; Tran et al., 2024) has attracted appealing attention as it deals with distilling valuable knowledge from a well-trained teacher model f to a student model S without requiring to access the real in-distribution (ID) training data. As shown in Figure (a), since the real ID data Dia unavailable, a common solution is to introduce a generator G to synthesize fake samples via adversarial exploration way. These synthetic samples are then used as proxies to guide the knowledge transfer from teacher to student (Mi- caelli & Storkey, 2019; Fang et al., 2019). More specifically, G and S are alternatively optimized during the training pro- cedure: (i) the G is trained to synthesize fake samples D, causing disagreement between student S and teacher f. thus expanding the coverage of the data distribution; (ii) the S is trained to reach teacher-student agreement on samples synthesized by G, thus learning teacher's s knowledge. As a result, the synthetic data D, is similar to the real ID) do- main Das, and the student S obtains teacher's 's knowledge by reaching similar accuracy on real ID data Daa-
Although significant progress has been made in this field, existing works always assume that pre-trained teachers trustworthy. Recently, Hong et al. (2023) conducted the first study to explore DFKD in the context of untrusted teachers. However, their work primarily focuses on teach- ers with backdoor (Wu et al., 2022; Gu et al., 2019). The robustness and security of DFKD from untrusted teachers beyond backdoor remain largely unexplored. In this paper, conduct the first investigation the situation of DFKD agains! non-transferable learning (NTL) teachers (Wang et al., 2022). Specifically, in NTL teacher's training, trans- ferability from an ID domain Dia to an outtof-distributiin (OOD) domain 'Dood is deliberately restricted by incorpo- rating an additional regularization term into the standard ID-domain supervised learning (SL) objective, where the regularization term encourages the teacher's output differ between the two domains.
'Sydney AI Centre, The University of Sydney "School of Computer Science, Wuhan University Department of Computer Science, Hong Kong Baptist University. Correspondence to: Zengmao Wang <wangzengmao@ whu.edu.en>> Tongliang Liu <tongliang liu @eyd@eydddyy.uua.
'The Dood can be both close-set (i.c., share the same label space (Bengio et al., 2011; Tzeng et al., 2017)) and open-set (i.c., disjoint label space (Liang et al., 2017; Wei et al., 2022)) to the Dua.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

When Data-Free Knowledge Distillation Meets Non- Transerafeee Teacher: Escaping Out-of-Distribution Trap is All You Need
Inpur Space Teacher
Tencher
NTL Tencher
Texder Kinselogge
Nose
In-Cherrttion (1D)
Teacher/
In In-Derrlmmon (ID)
In-lietnbztoon (II) Donais
Donnai
OOD Domain
Generator G
Syethetic Dkonhetize
Student S
Synthstic Dumiittiin
Sptllatic Dventehes Skift
Kanrelodge Dicillation
(a) General framework of DFKD
(b) NTL teacher fools the adversarial exploration
100
100
100
80
50
Alcussy
WTL NTL OOD
000
40
40
41
20
NTT OCD
20
20
N11.00 10
61
91
121
31
61
01
121
1.1
141
31
61
91
123
155
180
Epocl
Epoch
Ejoch
(c) SVHN-CIFARIO ZSKT
(d) CIFARIO-SVHN DFQ
(e) CIFARIO-STL DFQ
Figure 1. (a) The adversarial exploration framework of data- free knowledge distillation (DFKD). (b) For the SL teacher pre-trained on ID domain Da. the synthetic distirbution D. is close to the real Das However. for the NTL teacher pre-trained on both Dis and duttof-distribution data Dost OOD trap effect leads to the shift of D. from Di toward Pad. (c-e) A comparison of training dynamics of distilling SL and NTL teachers with DFKD baselines. We present three metrics for student: ID accuracy in distilling a SL teacher (SL-ID) and an NTL teacher (NTI.-ID), and OOD accuracy" in distilling NTL. teacher (NTL-OOD). More results are shown in Appendix C.
Through empirical investigations (Section 3), we find NTL teachers fool the training of both generator and student via a commonly raised 0OD trap effect. As illustrated in Fig- ure I (b), this effect means that when distilling NTL teachers, misleading knowledge from the OOD domain replaces the role of ID domain knowledge, acting as a trap that diverts the DFKD's primary attention. Two key phenomenon, shown in Figure 1 (c-d), are directly linked to this effect: (i) degradation of ID knowledge transfer: students exhibit lower NTL-ID accuracies compared to distillation from SL teachers, and (ii) misleading OOD knowledge transfer: students inherit teachers' OOD knowledge, reaching high NTL-OOD accuracies Furthermore, we identify two in- trinsic reasons that contribute to the OOD trap effect: (i) ID-to-OOD synthetic distribution shift: the synthetic distri- bution D. shifts from the ID domain Daa toward the OOD domain D Doda: (ii) ID-OOD learning task conflicts: the learn- ing targets of student on ID-like and OOD-like synthetic samples have significant differences. These factors together become the primary reasons why NTL teachers can fool DFKD, leading misguided student learning.
point is closely related to its margin, i.e., the distance from the data point to the decision boundary (Xu et al., 2023; Zhu et al., 2021a). By analyzing the margin of the ID and OOD domains, we verify that an NTI. teacher exhibits a significant difference in adversarial vulnerability between the two domains, demonstrating strong robustness in the OOD domain while remaining significantly more vulnera- ble in the ID domain Accordingly, ATEsC split synthetic samples into a fragile group and a robust group based on their adversarial robustness against untargeted projected gra- dient descent (PGD) attack (Madry, 2017). Then, ATEsc only uses the fragile samples (regarded as ID-like) to guide the student in learning teacher's knowledge. Therefore, the student's 's input distribution is close to the real ID domain, and the learning targets consist solely of ID knowledge, thus effectively alleviating the negative impact from ID-to- OOD synthetic distribution shift and ID-OOD learning task conflicts. Furthermore, OOD-like robust samples, which demonstrate high resilience to adversarial perturbations, are used as negative examples. The student's 's outputs on these samples are optimized to be distinct from the teacher's out- puts, thus further suppressing the transfer of misleading OOD knowledge from teacher to student.
To mitigate the OOD trap effect imposed by NTL teachers, we propose Adversarial Trap Escaping (ATEsc), a novel plug-and-play approach to benefit DFKD by identifying and filtering out misleading OOD-like synthetic samples accord- ing to their adversarial robustness. ATEse is inspired from the perspective that the adversarial vulnerability of a data
Empirically, we conduct experiments on NTL teachers across three OOD domain configurations (close-set, open- set, and teekdoor-trigged on eight dataset pairs, three DFKD baseline methods, and multiple architecture com- binations. The results demonstrate that ATESC consistently
"NTL-OOD accuracy is calculated hetween the prediction and the specific OOD domain label Bood (see Section 2.1). A higher NTL-OOD indicates more misleading OOD knowledge transfer.
Further discussion and verification are shown Section 4.1.
2

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
helps different DFKD baselines escape the OOD trap, signif- icantly enhancing the student's ID performance while effec- tively suppressing the transfer of misleading OOD knowl- edge. Our main contributions are organized as fhree-fold:
For the feature level, NTL introduces Maximum Mean Dis- crepancy (MMD) to measure the distance between features from two domains, which can be formulated as:
Lieat Em.-Dainn.-..... [MD((J (M))()) (3)
For the first time, we identify the commonly occurred OOD trap effect in distilling NTL teachers by DFKD (ex- ists across tasks, architectures, and DFKD methods). We further identify its intrinsic causes: ID-to-OOD synthetic distribution shift and ID-OOD learning task confticts.
The output-level and feature-level terms are combined by multiplying and then used as a regularization on the ID domain learning objective. Therefore, the complete NTL objective LNn is derived as:
(4) where a is trade-off weight, and min(1,-) upper bound the term Cout Licat -Ctaat by for training stability, By minimizing Equation 4, the f will still perform as normal supervised learning (SL) model on the ID domain. In contrast, OOD domain data are mapped by f into a distinet cluster that lies far from the ID domain in both the output and intermedi- ate feature spaces (Wang et al., 2022; Hong et al., 2024a), resulting in ID-to-0OD non-transferabiiiyy.
LNI = Lu - min(1, -ou : -iea));
We propose a novel plug-and-play approach ATEsC to enhance DFKD in transferring ID knowledge from NTL teachers. ATEsC mitigates the OOD trap effect via har- nessing synthetic samples based on the difference of NTL teachers' adversarial robustness on ID and OOD domains.
We validate the effectiveness of ATEsC via extensive experiments across different OOD domain configurations, datasets, network architectures, and DFKD baselines.
2. Preliminary
Class controlling of OOD data. In fact, the OOD clus- ter in the output space is often predicted as single class (Wang et al., 2022); however, this class is not fixed and can vary due to random factors. However, controlling the class assignment of OOD data is crucial in scenarios where the accuracy of OOD predictions for a specific label must be as- sessed. A notable example is poison-based backdoor attack and defense (Li et al., 2022; Hong et al., 2023). Therefore, an additional constraint can be introduced to enforce OOD data predictions toward designated OOD class:
2.1. Non- Non-Transferable Learning
The original Non-Transferabee Learning (NTL) (Wang et al., 2022) was proposed to restrict a model's transferability from in-distribution (ID) domain to close-set out-of- distribution (OOD) domain (i.e., share the same label space with the ID domain (Bengio et al., 2011; I; Ganin & Lempit- sky, 2015; Tzeng et al., 2017; Huang et al., 2023a;b; 2024)) by maximizing the discrepancy between the representations of the two domains. Here, we slightly expand the original NTL to a more general purpose. Specifically, we consider an ID domain D = {( y4) }r and OOD domain Dond = {x;))) Mas. The ID domain labeled and expected to be the utility domain of the NTL model. The OOD do- main can be unlabeled and applicable to both close-set and open-set (i.e., disjoint label space) (Liang et al., 2017; Liu et al., 2020; Wei et al., 2022) to the ID domain. The learning objectives on ID and OOD domains are shown as follows:
(5) where Lee is the cross-entropy loss, and yood is the desig- nated label for the OOD-class. By using Lets as a regular- ization, we get the total loss of class-specific NTL:;
Lets = Enn [CCr( Booa)]
LNTLECSs = LNLL + Adts fess:
(6)
where Ads is hyper-parameter to balance the weight.
2.2. Data-Free Knowledge Distillation
Learning on ID domain. Considering a neural network f: X + +. In general, the f is the combination of a feature extractor Se and classifier fcx i.e., (=x fe)Je( The learning objective the ID domain is to minimize the loss:
The goal of data-free knowledge distillation (DFKD) is to transfer a teacher's knowledge to student without using any real samples drawn from the teacher's 's ID domain. Its adversaria exploration framework is shown in Figure I (a), where we have a generator G (with parameters fc)), stu- dent S (with parameters 0s), and the targeted teacher f.
Las = E (f v)],
(1)
where DRr. is the Kullback-Leibler (KL) divergence.
The generator G and the student S will be optimized alter- nately with the teacher's 's parameters being frozen. Specifi- cally, the generator G is optimized to cause disagreement be- tween student S and teacher f. thus expanding the coverage of the synthetic distribution (i.c., adversarial exploration):
Learning on OOD domain. The main idea of NTL to maximize the discrepancy of both the output logits and fea- ture representations between ID and OOD domain data. For the output level, NTL directly maximizes the KL divergence between the output logits of the two domains, and thus, the loss Lout can be formulated as:
max (syn En-Deow [D (S((()))) Be
(7)
Lout Ey,, o Duy.n, Dae (Da [(x)))
(2)
*The algorithm for training NTL shown in Appendix B.I.
3

---

When Data-Free Knowledge Distillation Meets Non- TrnnTransferbble Teacher: Escaping Out-of-Distribution Trap is All You Need
versarial exploration ZSKT (Micaelli & Storkey, 2019) and its enhanced version DFQ (Choi et al., 2020) with sample diversity loss and BN loss, and two SOTA methods with an additional memory bank: CMI (Fang et al., 2021b) and NAYER (Tran et al., 2024). Experiments are conducted on both close- and open-set dataset pairs: SVHN (Netzer et al., 201 CIFARIO (Krizhevsky et al., 2009), STL10 (Coates et al., 2011) 1). Digits (Wang et al., 2022). Typical results are shown in Figure 1 (c-e), where we plot the training dynamic to compare distilling SL and NTL teachers. Specifically, the extent of knowledge transfer on ID and OOD domain during distillation are evaluated by the ID domain accuracy to its real labels and the OOD domain accuracy corresponding to the OOD-class label in Equation 5, respectively. More results refer to Appendices C and D.
(a) SL on CIFARIO
(b) NTL on CIFARIO -iiRits Digiits
Figure Synthetic samples CMI on SL. and NTL. teacher.
Figure 3. Feature distribution (NTL.: (b) CMI
(a) DFQ
Green dots and red dots represent real ID and OOD domain samples, re- spectively. Purple transparent dots represent the synthetic samples.
3.1. NTL Teachers Fool DFKD via OOD Trap Effect
We find that the intrinsic ID-to-OOD non-transe con-transferability in NTL teachers generally fools the training of both genera- tor G and student S in DFKD, with key phenomenons in Figure 1 (c-e) directly related to this:
where DKL is the KL divergence to measure the distribu- tion difference, n represents noise vector sampled from a predefined noise distribution Dooise. Then, the synthesized samples are leveraged to teach the student S to mimic the teacher's outputs, thus obtaining teacher's knowledge:
Degradation of ID knowledge transfer: the student ex- hibits lower NTL-ID accuracy compared to distillation an SL teacher, along with more unstable training dynamics.
min LLa = Ena- [DKL [DK((S ((C)))) 0g
(8)
Misleading OOD knowledge transfer: the student inherits teacher's ' misleading OOD knowledge, resulting in high NTL-OOD accuracy.
In addition to the basic adversarial exploration losses, state- of-the-art (SOTA) DFKD methods introduce different diver-
sity strategy (Choi et al.. 2020; Fang et al., 2021b; Yu et al., 2023; Fang al., b021b; Patel al., 2023; Tran al., 2024) for improving synthesizing quality and training stability. An essential and common regularization used in SOTA DFKD methods (Choi et al., 2020; Fang et al., 2021b; Tran et al., 2024) is the Batch Natch-Normalization (BN) loss, which was first proposed in (Yin et al., 2020). The BN loss in DFKD is usu- ally represented as the KL divergence between the feature statisties of current inputs and the stored BN statisties:
We refer to the combination of the above two phenomena as the OOD trap effect imposed by NTL teachers on DFKD, Intuitively, as illustrated in Figure 1 (b), misleading knowl- edge from the OOD domain replaces the role of ID domain knowledge when distilling NTL teachers, acting as a trap that diverts DFKD's ' primary attention. In addition, we con- duet visualization in both image space and NTL teacher's feature space to show the impact of OOD trap effect for DFKD. (i) In synthetic image space, we visualize the syn- thetic images by distilling a SL teacher and an NTL teacher using DFKD. As shown in Figure 2, SL results resemble real CIFARIO 10 samples (see Figure 9), while NTL results integrate both the ID domain (CIFAR10) and OOD domain (Digits) semantics. (ii) In feature space, as shown in Fig- ure 3, we plot the distribution of the NTL teacher's 's features for synthetic samples, real ID samples, and real OOD sam- ples using t-SNE visualization (Van der Maaten & Hinton, 2008). We find that parts of synthetic samples's 's features overlap with real OOD domain features, which means that these samples are more similar to real OOD samples and can activate NTL teacher's OOD misleading knowledge.
Lra = Σ (cL(((((((())) ((G(n)))) N(m,,o)),
(9)
of where (f is the 1-th layer's fea-
ture statisties of synthetic samples (G(n)), and N ((n, of) is the BN statistics for 1-th layer. Minimizing the BN loss in the generator G's training stage makes the synthetic samples follow similar statistical information of teacher's 's training samples, thus improving the synthetic sample quality.
The complete algorithm for training DFKD baseline with BN loss is shown in Appendix B.2. More related works and techniques of DFKD are illustrated in Appendix A.
3.2. Why Does OOD Trap Effect Occur
3. When DFKD Meets NTL Teachers
We argue that the OOD trap effect is primarily caused by two underlying reasons: (i) ID-to-OOD synthetic distribution shift and (ii) ID-OOD learning task conflicts.
In this section, we investigate the impact of NTL teachers on DFKD. We analyze four DFKD baselines: the basic ad-
4

---

When Data-Free Knowledge Distillation Meets Non- TrnnTransfeeable Teacher: Escaping Out-of-Distribution Trap is All You Need
60
Aocency
Undefeoiaa DCD: CIPARID
40
ttellic
DOD: Eigie
20
30
20
0-
211
151
R2och
(a) Defend against DFME
(b) Transfer backdoor (DFQ)
(c) Transfer backdoor (CMI)
Figure 4. Downstream applications of NTL's OOD trap effect to DFKD: (a) NTL teachers defend against data-free model extraction
(DFME); (b-c) teachers transfer backdoor to students via data-free knowledge distillation.
ID-to-OOD synthetic distribution shift means the syn- thetic distribution DD will occur shift from the ID domain Dua toward the OOD domain Dood resulting in mix ID and OOD characteristics of synthetic samples. This is caused by the joint effect of the adversarial exploration loss and the BN loss when training the generator G' in DFKD. Specifically:
in Equation 2, the term of max (f f(x.)) leads to the significant difference of teacher's 's outputs on ID data and OOD data. Thus, in distillation stage (Equation 8), the student's 's learning targets on ID- and OOD-like synthetic samples have a huge difference. This introduce severe task conflict (Misra et al., 2016; Yu et al., 2020; Liu et al., 2021a; la; Yadav et al., 2024; Yang et al., 2024) in the knowledge distillation stage, degrading the student's 's ID performance.
The BN loss (Equation 9) lets the synthetic data statis- tics to follow the statistics of NTL teacher'ss ' training data. When training NTL teachers, each batch contains a mix- ture of ID and OOD samples. This results in the fact that BN layers in well-trained NTL teachers record statistical information for the mixture distribution of real ID and OOD domains. As such, when conducting DFKD, the BN loss constraints the generator G to synthesize data which follows the statistics of NTI. teacher's 's training data (i.e., the mixture statistics of real ID and OOD data).
In summary, the generator's ' ID-to-OOD synthetic distribu- tion shift results in the OOD knowledge transfer in DFKD. This causes the student's ID-OOD learning task conflicts and results in the degradation of ID knowledge transfer*.
3.3. Application of OOD Trap Effect
Furthermore, we show both benign and malign downstream applications of NTL teachers' OOD trap effect for DFKD.
The adversarial exploration loss (Equation 7) lets the gen- erator G to maximize the discrepancy between student and NTL teacher. Accordingly, even if we assume that the generator G only synthesize ID-like samples and the student only has ID domain knowledge in some stage, the generator G will still be trained to synthesize OOD- like data. This is because in such situation, the student's output distribution on ID data (()( are similar to teacher's 's output distribution on ID data (D)(a))). How- ever, when training the NTL teacher f. the discrepan- cies between its outputs on ID and OOD domain were intentionally maximized (Equation 2). Therefore, the cur- rent student's outputs on the OOD domain P(S(Do))) will also have large discrepancy with the teacher out- puts on the OOD domain P( (Dfda)) As such, have DKL (SDa), [Pa)) < (St.oS(Doa) ((Doa))) Com- bining with the constraint from the BN loss, the G will be optimized toward synthesizing DOD-domain-like samples to satisfy the maximization of the distribution discrepancy between the student S and the NTL teacher ʃ.
Renign: defend against data-free model extraction. Data- free model extraction (DFME) (Kariyappa et al., 2021; Truong et al., 2021) seeks to replicate a model's function- ality without access to its original training data, posing a threat to its intellectual property (Wang al., 2024c). We show that a model owner can effectively defend against DFME if their model is trained by NTL. As shown in Fig- ure 4 (a), we train NTL models with SVHN as ID domain and consider both close-set (Digits, MNIST) and open-set (CIFAR 10) OOD domains. Compared to the undefended SL. model, NTL models effectively degrade the ID knowledge transfer, with even nearly no model functions being stolen.
Malign: transfer backdoor via data-free knowledge dis- tillation. A backdoor (Wu et al., 2022) pre-implanted in a model can alter its behavior when triggered by inputs with pre-designed triggers, posing a high risk of malicious ma- nipulation. Our experiments in Figure (b-c) provide the evidence that backdoor pre-implanted by NTL (i.e., clean samples as the ID domain and clean sample with triggers as the OOD domain) can be transferred to student via DFKD due to the mechanism of NTL's OOD trap effect, with the
ID-OOD learning task confficts means the significant dif- ference of the learning targets ID-like and OOD- like syn- thetic samples when distillation. Due to the ID-to-OOD shift of synthetic distributions as discussed in above, the synthetic distribution can be seen as intermediate distribution and con- taining merge information (i.e,, both ID-like and OOD-like) of the two domains. Recalling the NTL pretraining objective
'It is notably that without the BN loss, the generator G does not need synthesize samples that resemble those from the OOD domain. Unfortunately, their ID performance will be influenced by low quality of synthetic samples, limiting the useability of the learned student. More analysis are shown in Appendix C.4.
5

---

When Data-Free Knowledge Distillation Meets Non- Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
that small adversarial perturbations applied ID samples can easily generate adversarial examples with predictive la- bels being different from their original labels, ie., the fragile adversarial robustness of NTL teachers on ID domain.
Afvrrviial Ponutttion
Aduinariall Pettushtionn
2
Clar
Class
Clas D
For the OOD domain, recalling the previously illustration in Section 2.1, NTL achieves ID-to-OOD non-trunsferabilliy by maximizing the distance between the features and predic- tions of OOD data and those of ID data (Equations and 3). As such, OOD domain data are mapped into a distinct clus- ter that lies extremely far from the ID domain, resulting in a large distance from the decision boundaries as well. Accordingly, small adversarial perturbations on OOD data may still not exceed the decision boundaries of their original class, as illustrated Figure (b). Therefore, NTI. teachers demonstrate strong adversarial robustness on OOD samples compared to ID samples.
(a) ID domain
(b) OOD domain
Figure 5. We use green and red denotes ID and OOD domain data. (a): The decision boundaries cannot separate the ly-balls around the data points of ID domain. (b) The (z-balls around the OOD data points are far away from the decision boundaries.
Acceacresa
10 48
DOOD
(b) CIFARIO-DDiae Digits Figure 6. The difference of adversarial robustness (PGD with e = 8) of the ID and OOD domain under different attack steps.
(a) SVIIN- CIFARIO
To empirically verify the adversarial robustness difference on ID and OOD domains, we conduct experiments across different datasets and architectures to compare the robust- ness on each domain. Typical results are shown in Figure 6, where the accuracy reflects the consistency of predictive la- bels before and after untargeted adversarial attack. It can be observed that predictive labels for ID domain data are easily changed after attacks, which aligns with the conclusions of existing adversarial attack studies for SL models (Madry, 2017; Huang et al., 2017). In contrast, OOD samples exhibit strong robustness against adversarial perturbations even af- ter multiple steps and higher perturbation bound.
students inheriting high attack success rate (ASR). Notably, the risk of backdoor transfer to students via DFKD was first observed in ABD (Hong et al., 2023), although their study focused conventional backdoor teachers instead of NTL--bseed ones. In Appendix E.5, we demonstrate that their defense strategy struggles to mitigate backdoor transfer when the backdoored teacher is NTL-based.
4. Adversarial Trap Escaping
Additional validations of the adversarial robustness of NTL teachers are provided in Appendix C.5. Furthermore, in Appendices C.6 and C.7, we analyze the impact of NTL training objectives on adversarial robustness.
To mitigate the OOD trap effect in NTL teachers, in this section, we propose Adversarial Trap Escaping (ATEsc), which serves as a plug-and-play module for DFKD. In Sec- tion 4.1, we identify the vulnerability difference of ID and OOD domain samples against adversarial perturbation. Ac- cordingly, in Section 4.2, we propose to split synthetic sam- ples into two groups via their robustness against untargeted projected gradient descent attack. Then, in Section 4.3 and Section 4.4, we illustrate how will ATEsc treats the fragile group to conduct knowledge distillation and use the robust group for OOD misleading knowledge forgetting. The over- all pipeline of ATEsc is presented in Section 4.5.
4.2. Synthetic Data Grouping
Motivated by the adversarial robustness difference, we pro- pose Adversarial Trap Escaping (dubbed ATEsc) to ben- eñi DFKD in distilling NTL teachers. ATESC intervenes in DFKD after generator's 's optimization in each epoch. Specif- ically, ATESC first splits synthetic samples into a fragile group and a robust group according to NTL teacher's adver- sarial robustness on these samples. Formally, to assess the robustness of the NTL teacher f against synthetic samples D, {z; = G(n.)) ATEsc conducts untargeted pro- jected gradient descent (PGD) attack (Madry, 2017) on each sample to find the worst-case perturbation that can disrupt its original predictions. Untargeted PGD is formulated as: e; II (I., + a signn 2c (xx)..)))
4.1. Adversarial Robustness Difference
We start from the premise that a data point's adversarial vulnerability is closely linked to its margin, i.e., its distance to the decision boundary (Xu et al., 2023; Zhu et al., 2021a). We argue that NTL teachers exhibit different vulnerability to adversarial perturbation on ID and OOD domain samples.
(10)
where à; denotes the perturbed variant of the original sam- ple a with the original predictive label ÿt- 13 (xx.) = (a;
For the ID domain, the training objective (Equation 1) of NTL teachers are identical to that of standard SL models (i.e., learning correct classification), suggesting that NTL teachers have complex decision boundaries that lie within the ID domain, as illustrated in Figure (a). This implies
s x; I|p < } and is the perturbation bound. 1l repre- sents the projection operation.
Based on the observation that perturbations easily push ID)
6

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
data points beyond their original decision boundaries, the synthetic data z; is considered ID-domain-like if the pre-
Algorithm DFKD with ATEsC
1: Input: an NTL teacher f: a generator G with parameters
dictive label of the attack result à; changes. Conversely, samples whose predictive labels remain unchanged are re- garded as OOD-like. Accordingly, each batch of synthetic data D. is split into two groups: an ID-domain-like fragile group Gy and an OOD-domain-like robust group G,:
Oc: student S with parameters 0y; total training epochs E;
synthetic batchsize B.
2: Output: the DFKD student S.
3: for
sore=eee to E do
4: >. hesizing
5: Updating Og via Equation 7 and Equation 9;
G5z if arg max f(t.) + Di Gr, if arg max f().)) Di-
6: > Synthetic Data Grouping
æ; E
(11)
7:
Synthesizing fake samples B (G(n))) from noise;
8: Obtain perturbed fake samples B via Equation 10;
9: Clustering B into G3 and C, by Equation 11;
4.3. Calibrated Knowledge Distillation
10: b Ca Calibrated KD and Forgetting
11: Updating Og via Equation 14;
After the synthetic data grouping, we propose to only use the fragile samples (i.e., Gý) to teach the student S learning teacher's s knowledge. Accordingly, the learning objective of DFKD's s knowledge distillation stage can be calibrated by revising Equation 8 as:
12:
end for
The proposed ATEsC serves as a plug-and song-and-play module for various DFKD baselines, inserted between the synthesizing and distillation stages in each training epoch. Specifically, after generator's training, ATEsc identifies fragile and ro- bust sub-groups from the current synthetic batch samples. Then, the two sub- sub-group samples are used for distillation by revising the original KD loss (see Equation 8) into ATEsC loss (Fquation 14). We summarize the pipeline of DFKD) with ATEsC in Algorithm 1.
(12) Since fragile samples are considered ID-like, the teacher's outputs on these samples are more likely to contain valuable ID-domain knowledge. As such, learning from the teacher only on Gy helps mitigate the task conflicts caused by ID- OOD discrepancies, thus enhancing ID knowledge transfer.
Lekd = BuwDa, (S())
5. Experiments
4.4. Misleading Knowledge Forgefting
Although calibrated knowledge distillation can filter out OOD-domain-like samples, some intermediate samples in- evitably exist. These samples are not robust but can still activate the teacher's s misleading OOD knowledge, enabling certain degree of OOD knowledge transfer from NTL. teacher to student. This may not fully align with expec- tations in certain scenarios (such as defending backdoor transfer from NTL teachers to DFKD students (Hong et al., 2023)). To further suppress misleading OOD knowledge transfer, we use robust samples (i.e.. G,), which exhibit extremely high resilience to adversarial perturbations, negative references. The student's 's outputs on these samples are optimized to be distinct from the NTL teacher's 's outputs. Formally, we use KL divergence to measure the distance between student's 's and teacher's 's outputs:
In this section, we empirically validate the effectiveness of ATESC. We first introduce our experimental setup. Then we present results on NTL teachers across three tasks: closed- set NTL, open-set NTL, and NTL-based backdoor.
Datasets. We use the common datasets shared by both NTL and DFKD, including Digits (MNIST (Deng, 2012), USPS (Hull, 1994). SVHN (Netzer al., 2011), MNIST-M (Ganin et al., 2016), and SYN-D (Roy et al., 2018)), CIFARIO (Krizhevsky al., 2009), STL (Coates et al., 2011).
NTL tasks. We conduct experiments on NTL teachers trained with three ID/OOD domain configurations:
In this setting, ID and OOD domain share the same label space. We choose the dataset pairs of: -NISSTMM and CIFARIO-STL.1::
(i) Close-set NTL:
Cforget = EzwDa, (Skx) f())]
(13)
In this setting, ID and OOD domain have disjoint label space. We conduct experiments on SVHN-ICIAARIO and CIFARIO-Digits.
(ii) Open-set NTL:
By maximizing Liorge. the student's 's outputs on OOD-like ro-
bust samples are encouraged to diverge from NTL teacher's
OOD outputs fzz), thereby further reducing the transfer of
(iii) NTL-based backdoor: The OOD domain is the ID domain with backdoor triggers. We add four triggers: BadNets(sq). BadNets(grid) (Gu et al., 2019), Blended (Chen et al., 2017). Sig (Barni al., 2019) on CIFARIO, and treat them individually distinct OOD domains. The clean CIFAR10 is regarded as the ID) domain.
misleading knowledge from the OOD domain.
4.5. Overall Pipeline
We obtain the complete ATEsC loss by combining both the calibrated knowledge distillation and the forgetting term:
LATFSC = Lesa - min (1, X Linge),
(14)
Our aim is to address the malign OOD-Trap effect intro- duced by NTL teachers in DFKD, and thus, regardless of NTL teacher's 's tasks, our goal to transfer only the ID)
where À control the weight of the forget loss, min(1,)) -) set an upper bound for stable training of maximization term.
7

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Table 1. CCoss-sect NTI.. We report the ID domain accuracy (lAcc) in blue and OOD domain accuracy (OAcc) in red. Results from the original DFKD baselines are highlighted with gray rows. The accuracy drop compared to the pre-trained NTI. model is shown in brackets.
ID: SVHN OOD: MNIST-M
ID: CTEARIO OOD: STL
VGG-13-
VGG-13-VGG-I1
ResNet-34-RRese 18
Icce (5)
OAce (
Ioc (%c( f
OAce (%) f
(%))
OAce (
IAce (
OAce (8) f
NIL Teacher
94.3
9,9
94,4-01
9.9.00
92.7-01
10.6.0.1
90.8
10.4.00
DFQ
89.2216 (:5.)
9.9g00 (.0.0) (.0.0)
79.5 (.1.9)
10.2go
42.7a56
10.4 (.02)
52.5 ( (.38.3)
11.8 (+1.4) (4)
CRD
88.9209 (-5.4)
11.1 (+1.2)
(12.2)
10.1 (+0.2)
63.0231
10.4ann
68.0gm01 (-22.)
22.5 (+12..)
+ ATEGC
89.3a06 (:5.0)
12.9 (+3.0)
06.22935.8.2
30.8 (+20.9)
62.3g26 (.3325((300) (:30.4)
(+2.9)
66.2207 ( (.24.))
51.6265 (1.6665(+44122
CMI
64490
9.9ga0 (-0.0)
00.2a51
10.4004
11.2a25 (-92.8
(+0.6)
11.0gna (-57.3
(40.6)
+ CRD
89.8 (-4.5)
29.1 (+19.2)
72.3:43 (-22.1)
28.6144 (+18.7)
(3.6:37(-39.))
38.3
49.8:36 (-+10)
33.9a23 (+23.5)
ATESC
58.7 (-35.6)
26.7 (+16.8)
76.5
29.9 (+20.0)
58.6-13 (-34.1)
32.5
(4.745a-46..)
92.9:55
NAYER
7.2:03 (-87.1)
10.6:03 (40.7)
84,4 (-10.0)
10.0sa1 (40.1)
(-82.))
(40.2
48.7a31
(-0.0)
+ +CR
89.6104 (-4.7)
29.7
(+19.8)
89.2 (-5.2)
19.1 (+29.2)
62.7-61 (-30.0)
23,7+ (+13.))
64.410a (-26.4)
40.2+39 (+29.8)
ATTSC
(182)
33.7a36 (+23.8)
87.0gs7 (-7.4)
40.0g03 (+30.1)
69.7g54 (:23.0)
42.3g2/ (+31.7)
52.7424
39.2 (+28.8)
Table Open-set NTL. We report ID domain accuracy (IAce) in blue and OOD domain accuracy to OOD label (OLAce) in red. Results from the DFKD baselines are highlighted with gray rows. Accuracy drop compared to the pre-trained model is shown in brackets.
ID: SVHN OOD: CIEARIO
ID: CIFARIO OOD: Digits
ResNet-34 ResNet-18
VGG-13-
|
ResNet-34 BeNNet-3--ReNNeet-BB ResNett-18
(9)
OLAcc (8) 4
LAce 1
OLAcc (
(90
OLAcc (88
|
(9)
OLAcc (9)) 4
NTL Teacher|
93,420
100.0100
93.0-01
99.9200
92.9.00
99,5.03
91.1011
100.0.00
DFQ
29.1 (4.3)
100 0200 (.D.0)
89.5gns (3355
99.9 (-0.0)
62.8g21 I)
99.8.01
A7ss (:33.7)
85.8asa (.14.2
CXD
89.0s04 (-4.4
99,9 (-0.1)
89.1 (-3.9)
98.8 (-1.1)
68.5:14
98.6-n4 (-0.9)
20. (-21.0)
5.5436 (-94.5)
+ AT2ssc
88.1
100.0.00 (-0.0)
88.9-03 (-4.1)
16.1-102 (-83.8)
60.3 (-32.6)
78.4 204-200 (-21.1)
65.1 (-26.0)
2,5 (-97.5)
CMI
6.4000 (-87.0)
(-100
99.1 (-3.0)
(-0.8)
10,6200
100.0.00 (40.5)
34,0213 (-57.1)
26.1 (11577-2339) (-23.9)
+ CXD
91.3+02 (-2.1)
19.3+160 74.5 (-77.7)
101 (-99.8)
59.4 (-33.5)
30.1-10 (-69.4)
50.1 (-41.0)
5.4164
+TTSc
72.0aza (-2144
0.0gn0 (-100.0)
77.4 (-15.6)
0.0g00 (-99.9)
56.9233 (-36.0)
21.1 (-78.4)
43.9 (-47.2)
3.7a24 (-96.)
NAYER
6.420 (.87.0)
100 0000 (-0.0)
54.8g (.38.2)
99.
(-0.0)
100.0a00 (08..3
(40.5)
49 49.4 (-41.7)
97.92ox (-2.1)
+ CXD
91.2ao2
am. (-32.5)
88.1 (-4.9)
0.2g0u (-99.7)
70.4
71.6- (027.999777 (-27.9)
61.2,39 (-29.9)
6,3 (-3.7)
+TTRSC
19..216
(00g0o01000.0
(57.))
0.0gan (-99.9)
69.8 (-23.1)
2.8.1x
59.0g10 (.322.0(332.1
3.610 (-96.4)
Table NTL.-basd backdoor. We report the ID domain accuracy (IAcc) in blue and attack success rate (ASR) in red. Results from the DFKD baselines are highlighted with gray rows. The accuracy drop compared to the pre-trained model is shown in brackets.
BadNet(ss)
BadNeligrid)
Blended
Sig
VGG-13->VGCH
ResNet-34 - ResNet-18
ResNet-344 BeEM44--esshhi 18
IAcc (
ASR 4
| IAce (%) f
ASR (%)
IAcc (
ASR (%) 4
LAce (8)
ASR (%) 4
NTL Teacher|
90.3-04
98.3+01
90.7+00
100.0:00
90.5-02
100.0-00
90.2402
100.0-a0
DFQ
67.6.2s
71.3 (-27.0)
100.0.00 (-1844
(-0.0)
52.1 (:38.4)
99.9100
62.5 (:27.7)
99.9 (-0.1)
(-19:2)(-194))
96.6440 (-1.7)
75.2:02
100.0:00 (-0.0)
52.2-222 (-38.3)
99,9:a1 (-0.1)
68.24189 (-22.0)
100.0=00 (-0.0)
+ ATEac
(-26:222-23.7)
2.9.a (-95.4)
62.1 (-28.6)
11.499: (-88.6)
57.0 (-33.5)
43.1 (-56.9)
65.8188 (-24.4)
65.5 (-34.5)
CMI
10.0-a0 (-80.3)
100.0gno (+1.7)
10.0gn0
100.0gan (-0.0)
50.5gr1 ( (-40.)
99.92a1 (.0.))
46.0ga2 ( (444.)
100.0-00
CXC
76.32n8 (-14.0)
96.52no (-1.8)
14.133(-1666
100.0.00 (-0.0)
68.3 (-22.2)
48 48.0g393 (-52.0)
17.1 (-23.l)
100.0.n0 (-0.0)
ATE3o
68.5249
3.6238 (:947)
62.8.40
2.2218 (-97.)
19.72164
19.0g55 (.81.)
46.9g38 (-43.3
(2.2g:81(677.
NAYER
10.0.00
100.0200 (+1.7)
100.0gan (-807)
(-0.0)
(-162)
100.0g00 (-0.0)
68.8235 (-24.4)
100.0g00 (40.0)
+ CKC
80.8:33 (-9.5)
97.5+04 (-0.8)
74.3 (-16.4)
100.0:00 (-0.0)
99.2 (9.0=19(-11.55
70.1 (-20,1)
100.0:ao
+ATLsC
(8-2-47--21.4)
34.6.473 (-63.7)
77.8.9
68.5 (-315)
(-23.5)
10.2 (-89.8)
55.8121 (+34.4)
domain knowledge from NTL teacher to student.
hyper-parameters follow original implementations. VGG-13 (Simonyan, 2014) and ResNet-34 (He et al., 2016) are used as teacher architectures, while VGG-11 and ResNet- 18 are used as student architectures. We conduct all experiments on a single RTX 4090 GPU (24G). More details are shown in Appendix E. I due the limited space of the main paper.
DFKD baselines. We choose DFQ (Choi et al., 2020), CMI (Fang et al., 2021b), NAYER (Tran et al., 2024) as DFKD baselines. When conduct DFKD, all NTL teachers are considered as white-box models. For each baseline, we apply two variants of our method as plug-and-play modules: one using only the CKD in Section 4.3 (denoted as CKD) and the other using the full method (denoted as + ATEsc).
5.1. DFKD on Close-set NTL
The results of distilling close-set NTL teachers are shown in Table 1, covering various dataset pairs, network architec- tures, and DFKD baselines. We use ID domain accuracy (IAce) to measure the performance of ID knowledge transfer.
Implementation details. For each task, NTL teachers are pre-trained up to 50 epochs (input images resize to 64 64). For DFKD, we train each method up to 200 epochs, and all
8

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Since ID and OOD domains share the same label space in this setting, we compute the OOD domain accuracy (OAce) based on the student's predictions on OOD data relative to their ground-truth labels. Higher OAce means better ID- to-OOD generalization, and also means better resistance to misleading OOD knowledge transfer during DFKD. From the results, each baseline DFKD method performs poorly on both the ID and OOD domains, with the student exhibit- ing low IAce and OAcc. For some tasks (such as CMI VGG-13-VVG-II the DFKD method cannot even trans- fer valid knowledge to the student. With the addition of CKD, IAcc generally shows significant improvement across tasks, whereas the gains in OAcc are relatively modest. With the full ATEsC applied, OAce further improves, indicating the effectiveness of the forgetting term in mitigating the transfer of misleading OOD knowledge.
(Appendices C.5 C.7), ablation studies (Appendices E.2 to E.4), visualization (Appendix E.7).
6. Conclusion
In this work, for the first time, we identify the OOD trap effect from NTL teachers to DFKD, resulting in the degrada- tion of ID knowledge transfer and enabling OOD misleading knowledge transfer. Then, inspired by the difference of NTL model's adversarial robustness on ID and OOD domains, we propose novel plug-and-play approach ATEsc to enhance DFKD in transferring correct knowledge from NTL. teach- ers. The effectiveness of ATEsC is validated via extensive experiments across diverse OOD domain configurations, datasets, network architectures, and DFKD baselines.
A Acknowledgements
The authors would like to thank the anonymous review- ers for their insightful and constructive comments. Zim- ing Hong is supported by JD Technology Scholarship for Postgraduate Research in Artificial Intelligence No. SC4103, Zengmao Wang is supported by NSFC 62476204, 62271357, NSF of Hubei Province 2023BAB072, Wuhan NSF 2024040801020238 Bo Han is supported by RGC Young Collaborative Research Grant No. C2005-24Y and NSFC General Program No. 62376235. Bo Du is supported by NSFC 62225113, Innovative Research Group Project of Hubei Province 2024AFA017. Tongliang Liu is par- tially supported by the following Australian Research Coun- cil projects: FT220100318, DP220102121, LP220000077. LP220200949, and IC190100031.
5.2. DFKD on Open-set NTL
For open-set NTL, the label space of ID and OOD domains is disjoint. For evaluating the transfer of misleading OOD knowledge, we calculate the accuracy between the predic- tion and the OOD-class label on OOD domain (denoted as OLAcc). Lower OLAcc indicates better resistance to OOD knowledge transfer. Moreover, we still use IAcc on II) do- mains. From the results shown in Table 2. the independent effect of CKD contributes to the significant improvements of ID) domain knowledge transfer. It also can mitigate the OOD knowledge transfer in some extent. By using the com- plete ATEsc, the OOD knowledge transfer can be further mitigated, while it also sacrify some ID performance.
5.3. Defending NbLeb Backdoor in DFKD
Impact Statement
We further explore the use of NTL. in conducting control- lable backdoor attacks (Wu et al., 2022). We use four trig- gers: BadNets(sq)). (adNets(grid) (Gu et al., 2019), Blended (Chen et al., 2017), Sig (Barni et al., 2019) on CIFARIO, 10, and treat them individually as distinct OOD domains. The clean CIFARIO 10 is regarded as the ID domain. For evaluation, we follow backdoor learning to use clean label accuracy on ID domain (i.e., the same to IAcc) and attack success rate (ASR) on backdoor data, where the ASR is the accuracy of the prediction and the OOD-class label (i.e., the backdoor target label) on trigger OOD data. From the results shown in Table 3, the full ATESC can alleviate the backdoor transfer from teacher to student, while all the original baselines or only adding the CKD can still inherit backdoor, with the ASR close to the teacher's ASR. Furthermore, we compare ATEsC with the defense method ABD (Hong et al., 2023), with the results and analysis presented in Appendix E.5.
This paper presents work whose goal is to advance the field of data-free knowledge distillation (DFKD). Existing works always assume pre-trained teachers are trustworthy. There- fore, the robustness and security of DFKD from untrusted teachers remain largely unexplored. Our work investigates how DFKD performs when faced with non- trn-transferable learning (NTL) teachers. For the first time, we identify the OOD trap effect, a prevalent issue in distilling NTL teachers via DFKD, and further uncover its intrinsic causes: ID-to- OOD synthetic distribution shift and ID-OOD learning task conflicts. To address this issue, we propose a novel plug-and- play approach ATEsC to enhance DFKD via handle with the 0OD trup effect, enabling benign knowledge transfer from NTL teachers via DFKD. Our work takes an important step forward in enhancing the robustness and security of DFKD. However, our method may also raise a challenge for NTLL-ased model intellectual property (IP) protection, as it provides data-free inverse solution against NTL mod- els, potentially undermining their intended transfcrability restrictions. It is important to develop defense strategies against ATEsc for NTL-based model intellectual property (IP) protection in the future.
5.4. A Additional Analysis
Due to limited space, additional analysis are shown in ap- pendices, including but not limited to: toy experiments of OOD trap effect (Appendix D). NTL's adversarial difference

---

When Data-Free Knowledge Distillation Meets Von-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
References
Deng, L. The mnist database of handwritten digit images for machine learning research [best of the web). IEEE Signal Processing Magazine, 29(6):141-11) 2012.
Barni, M., Kallas, K., and Tondi, B. A new backdoor attack in cnns by training set corruption without label poison- ing. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 101-105. IEEE, 2019.
Ding, R., Su, L.,, Ding, A. A., and Fei, Y. Non-transferable pruning. In European Conference Computer Vision, pp. 375-393. Springer, 2024.
Bengio, Y., Bastien, F. Bergeron, A., Boulanger- Lewandowski, N., Breuel, T., Chherawala, Y., Cisse, M., Côté, M., Erhan, D., Eustache, J., et al. Deep learners benefit more from duttt---iittibttion examples. In Pro- ceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 164-172. JMLR
Do, K., Le, T. H., Nguyen, D., Nguyen, D., Harikumar, H., Tran, T., Rana, S., and Venkatesh, S. Momentum adversarial distillation: Handling large distribution shifts in fata-free knowledge distillation. Advances in Neural Information Processing Systems, 35:10055-10067, 2022.
Workshop and Conference Proceedings, 2011.
Fang, G., Song. J., Shen, C., Wang. X., Chen, D., and Song, M. Data-free adversarial distillation. arXiv preprint arXiv:1912.1100 2019.
Binici, K., Aggarwal, S., Pham, N. T., Leman, K., and Mi- tra, T. Robust and resource-efficiet free-free knowledge distillation by generative pseudo replay. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pp. 4089-6096, 2022a.
Fang, G., Bao, Y., Song, J., Wang, X., Xie, D., Shen, C., and Song, M. Mosaicking to distill: Knowledge distillation from out-of-domain data. Advances in Neural Informa- tion Processing Systems, 34:1 14.11920-11932, 2021a.
Binici, K., Pham, T., Mitra, T., and Leman, K. Prevent- ing catastrophic forgetting and distribution mismatch in knowledge distillation via synthetic data. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 663-671, 2022b.
Fang, G., Song, J., Wang, X., Shen, C., Wang, X., and Song, M. Contrastive model inversion for data-free knowledge distillation. arXiv preprint arXiv:2105.0885 2021b.
Ganin, Y. and Lempitsky, V. Unsupervised domain adapta- tion by backpropagation. International conference on machine learning, pp. 1180- 1189. PMLR, 2015.
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Ed- wards, B., Lee, T., Molloy, I., and Srivastava, B. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv: 1.00772 2018.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, Domain-adversaria training of neural networks. The Journal of Machine Learning Research, 17(1)2296-20330, 2016.
Chen, S., Fu, D., Chen, S., Ye, S., Hou, W., and You, X. Causal visual-semanti correlation for zero-shot learning. In Proceedings of the 32nd ACM International Confer- ence on Multimedia, pp. 4246-4255, 2024.
Goodfellow, I. J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples. arXiv preprint arXiv: 1412.657, 2014.
Chen, X., Liu, C., Li, B., Lu, K., and Song, D. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv: 1712.0556, 2017.
Gu, T., Liu, K., Dolan-Gavitt B., and Garg, S. Badnets: Evaluating backdooring attacks deep neural networks IEEE Access, 7:47230 2019.
Choi, Y., Choi, J., El-Khamy, M., and Lee, Data-free network quantization with adversarial knowledge distil- lation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pp. 710-711, 2020.
Hayase, J., Kong, W., Somani, R., and Oh, S. Spectre: Defending against backdoor attacks using robust statistics. In International Conference on Machine Learning, pp. 4129-4139 PMLR, 2021.
Coates, A., Ng, A., and Lee, H. An analysis of single-layer networks in unsupervised feature learning. In Proceed- ings of the Fourteenth International Conference on Artifi- cial Intelligence and Statistics (AISTATS), pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Deng. J., Pang. S., Chen, Y., Xia, L., Bai, Y., Weng, H., and Xu, W. Sophon: Non-fine-tunable learning to restrain task transferability for pre-trained models. In 2024 JEEE Symposium on Security and Privacy (SP), pp. 2553-2571. IEFE, 2024.
Hong, J., Zeng Y., Yu, S., Lyu, L., Jia, R., and Zhou, J. Revisiting data-free knowledge distillation with poisoned teachers. In International Conference on Machine Lear- ing, pp. 13119- 13212, PMLR, 2023.
10

---

When Data-Free Knowledge Distillation Meets Non-Transferabie Teacher: Escaping Out-of-Distribution Trap is All You Need
Hong, Z., Shen, L., and Liu, T. Your transferability barrier is fragile: Free-lunch for transferring the non-transferable learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 28805- 28815, 2024a.
Li, Y., Jiang, Y., Li, Z., and Xia, S.-T. Backdoor learning: A survey. IEEE Transactions Neural Networks and Learning Systems, 35(1):5-22, 2022.
Liang. S., Li, Y., and Srikant, R. Enhancing the reliability of out-of- image detection in neural networks. arXiv preprint 1706. 02690, 2017.
Hong, Z., Wang, 7.,, Shen, L., Yao, Y., Huang, Z., Chen, S., Yang, C., Gong, M., and Liu, T. Improving non- transferable representation learning by harnessing content and style. In The Twelfth International Conference on Learning Representations (ICLR), 2024b.
Lin, Y., Yao, Y., Shi, X., Gong, M., Shen, X., Xu, D., and Liu, T. Cs-isolate: Extracting hard confident examples by content and style isolation. Advances in Neural Informa- tion Processing Systems, 36:58556-58576, 2023.
Hong, Z., Xiang, Y., and Liu, T. Toward robust non- transferable learning: A survey and benchmark. arXiv preprint arXiv:2502. 13593, 2025.
Liu, B., Liu, X., Jin, X., Stone, P., and Liu, Q- Conflict- averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34:18878- 18890, 2021a.
Hu, Z., Shen, L., Wang, Z., Wu, B., Yuan, C., and Tao, D. Learning to learn from apis: black-box data-free meta-learning, In International Conference on Machine Learning, pp. 13610 13610-13627. PMLR, 2023.
Liu, H., Wang. Y., Liu, H., Sun, F., and Yao, A. Small scale data-free knowledge distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6008-6016, 2024.
Huang, S., Papernot, N., Goodfellow, I., Duan, Y., and Abbeel, P. Adversarial attacks on neural network policies. arXiv preprint arXiv: / 702.02224..4. 2017.
Liu, W., Wang, X., Owens, J., and Li, Y. Energy-based out- of-distribution detection. Advances neural information processing systems, 33:21464-211755 2020.
Huang, Z., Xia, X., Shen, L., Han, B., Gong, M., Gong, C., and Liu, Harnessing out-of-distribution examples via augmenting content and style. In The Eleventh Interna- tional Conference on Learning Representations, 2023a.
Liu, Y., Zhang, W., Wang, J., and Wang, Data- free knowledge transfer: survey, arXiv preprint arXh::2112.15278 2021b.
Huang, Z. Zhu, M., Xia, X., Shen, L., Yu, J., Gong, C., Han, B., Du, B., and Liu, T. Robust generalization against photon-limited corruptions via worst-case sharpness min- imization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pultern Recognition (CVPR), pp. 16175-16185, 2023b.
Luo, L., Sandler, M., Lin, Z., Zhmoginov, A., and Howard, A. Large-scale generative data-free distillation. arXiv preprint arXiv:2012.05578, 2020.
Madry, A. Towards deep learning models resistant to adver- sarial attacks. arXiv preprint arXiv:1706.0608 2017.
Huang, 7.,, Li, M., Shen, I., Yu, J., Gong, C., Han, B., and Liu, T. Winning prize comes from losing tickets: Improve invariant learning by exploring variant parameters for out- of-distribution generalization. International Journal of Computer Vision, pp. 1-19, 2024.
Micaelli, and Storkey, A. Zero-shot knowledge trans- fer via adversarial belief matching. Advances in Neural Information Processing Systems, 32, 2019.
Hull, J. J. A database for handwritten text recognition research. IEEE Transactions Pattern Analysis and Machine Intelligence, 16(5):550-555 554, 1994.
Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. Cross- stitch networks for multi-task learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3994 4003, 2016.
Kariyappa, S., Prakash, A., and Qureshi, M. K. Maze: Data- free model stealing attack using zeroth-order gradient estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13814 13823, 2021.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. 2011.
Patel, G., Mopuri, K. R., and Qiu, Q. Learning to retain while acquiring: Combating distribution-shiftt in adversar- ial data-free knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7786-7794, 2023.
bayes. arXiv preprint arXiv:1312.6114, 2013. Kingma, D. P. and Welling, M. Auto-encoding variational
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Peng, B., Qu, S., Wu, Y., Zou, T., He, L.., Knoll, A., Chen, G., and Jiang, C. Map: Mask-pruning for source-free model intellectual property protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23585-23594, 2024.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2008.
Von Kügelgen, J., Sharma, Y., Gresele, I.., Brendel, W. Schölkopf, B., Besserve, M., and Locatello, F. Self- supervised learning with data augmentations provably isolates content from style. In Advances in Neural Infor- mation Processing Systems (NeurlPS), pp. 16451-16467, 2021.
Qi, X., Xie, T., Li, Y., Mahloujifar, S., and Mittal, P. Revis- iting the assumption of latent separability for backdoor defenses. In The eleventh international conference on learning representations, 2023.
Qian, B., Wang, Y., Hong, R., and Wang, M. Adaptive data-free quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7960-7968, 2023.
Wang, I.., Xu, S., Xu, R., Wang, X., and Zhu, Q. Non- transferable learning: A new approach for model own- ership verification and applicability authorization. In International Conference on Learning Representations (ICLR), 2022.
Roy, P., Ghosh, S., Bhattacharya, S., and Pal, U. Effects of degradations on deep neural network architectures. arXiv preprint arXiv: arXiv:1807. 1807. 10/08,1000000 2018
Wang, L., Wang, M., Zhang, D., and Fu, H. Model barrier: A compact un-transferable isolation domain for model intellectual property protection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 20475-20484, June 2023.
Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalch- brenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE, 109(5): 612-634, 2021.
Wang, L., Wang, M., Fu, H., and Zhang, D. Say no to freeloader: Protecting intellectual property of your deep model. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2024a.
Shi, L., Vijayaraghavan, P., and Degan, E. Data-free model fusion with generator assistants. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7731-7739, 2024.
Wang, Y., Yang, D.,, Chen, Z.,, Liu, Y., Liu, S., Zhang, W., Zhang, L., and Qi, L. De-confounded data-free knowl- edge distillation for handling distribution shifts. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 12615-12625, 2024b.
Simonyan, K. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1550 2014.
Wang. Z., Wu, Y., and Huang, H. Defense against model extraction attack by bayesian active watermarking. In Forty-first International Conference on Machine Learn- ing.
Tang, D., Wang, X., Tang, H., and Zhang, K. Demon in the variant: Statistical analysis of {DNNs} for robust back- door contamination detection. In 30th USENIX Security Symposium (USENIX Security 2 21), pp. 1541-1558, 2021.
Wang. Z., Shen, L.,, Liu, T., Duan, T., Zhu, Y., Zhan, D., Doermann, D., and Gao, M. Defending against data- free model extraction by distributionally robust defensive training. Advances in Neural Information Processing Systems (NeurIPS), 36, 2024c.
Tran, B., Li, J., and Madry, A. Spectral signatures in back- door attacks. Advances in neural information processing systems, 31, 2018.
Tran, M.-T,, Le, T., Le, X.-M., Harandi, M., Tran, Q. H., and Phung, D. Nayer: Noisy layer data generation for efficient and effective data-free knowledge distillation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13860-23866 2024.
Wei, H., Xie, R., Cheng, H., Feng, L., An, B., and Li, Y. Mitigating neural network overconfidence with logit normalization. In International conference on machine learning, pp. 23631-23644, PMLR, 2022.
Truong, J., Maini, P., Walls, R. J., and Papernot, N. Data- free model extraction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4771 17710 2021.
Wei, Y., Hu, Z., Shen, L., Wang, Z., Li, Y., Yuan, C., and Tao, D. Task groupings regularization: Data-free meta- learning with heterogeneous pre-trained models. arXiv preprint arXiv:2405. 16560, 2024.
Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. Adver- sarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7167-7176, 2017.
Wu, B., Chen, H., Zhang, M., Zhu, Z., Wei, S., Yuan, D., and Shen, C. Backdoorbench: A comprehensive benchmark backdoor learning. Advances in Neural Information Processing Systems (NeurIPS), 35:10546-1000 -05599 2022.
12

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Xiang, Y., Hong, Z., Yao, L.., Wang, D., and Liu, T. Jail- breaking the nonttransferable barrier via test-time data disguising, arXiv preprint arXiv:2503.
for non-iid federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10174-10183, 2022.
2025.
Xu, Y., Sun, Y., Goldblum, M., Goldstein, T., and Huang, Exploring and exploiting decision boundary dynamics for adversarial robustness. arXiv preprini arXiv:2302.030) 2023.
Zheng. J., Yao, Y., Han, B., Wang, D., and Liu, T. Enhancing contrastive learning for ordinal regression via ordinal content preserved data augmentation. In The Twelfth International Conference Learning Representations, 2024.
Yadav, P., Tam, D., Choshen, L.., Raffel, C. A., and Bansal, messing::: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36, 2024.
Zhou, Y. Zhang, Y., Zhang, Y., and Hua, Z. Derd: data-free adversarial robustness distillation through self- adversarial teacher group. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 10055- 10064, 2024.
Yang, E., Shen, L., Wang, Z.., Guo, G., Chen, X., Wang, X., and Tao, D. Representation surgery for multi-task model merging. arXiv preprint arXiv:2402 02200220775 2024.
Zhu, J., Zhang, Han, B., Liu, T., Niu, G., Yang, H., Kankanhalli, M., and Sugiyama, M. Understanding the interaction of adversarial training with noisy labels. arXiv preprint 2r 02482. 2021a
Yao, Y., Zhou, Y., Han, B., Gong, M., Zhang, K., and Liu, T. A robust method to discover causal or anticausal relation. In The Thirteenth International Conference on Learning Representations, 2025.
Zhu, Z., Hong. J., and Zhou, Data-free knowledge distil- lation for heterogeneous federated learning. In Interna- tional conference machine learning, pp. 12878-12889, PMLR, 2021b.
Yin, H., Molchanov, P., Alvarez, J. M., Li, Z., Mallya, A., Hoiem, D., Jha, N. K., and Kautz, J. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 871 8724, 2020.
Yu, C., Han, B., Gong, M., Shen, L.., Ge, S., Bo, D., and Liu, T. Robust weight perturbation for adversarial training. Proceedings of the Thirty-Fire First International Joint Con- ference Artificial Intelligence, pp. 3688-3694, 2022a.
Yu, C., Han, B., Shen, I.., Yu, Gong, C., Gong, M., and Liu, T. Understanding robust overfitting adversarial training and beyond. In International Conference on Machine Learning (ICML), pp. 25595-25610. PMLR, 2022b.
Yu. S., Chen, J., Han, H., and Jiang. S. Data-free knowledge distillation via feature exchange and activation region constraint. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24266- 24275, 2023.
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C. Gradient surgery for multi-tast learning. Advances in Neural Information Processing Systems, 33: 5824-5836, 2020.
Zeng, and Lu, W. Unsupervised con-transferable text classification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022.
Zhang. L., Shen, L., Ding, L., Tao, D., and Duan, L. Fine- tuning global model via data-free knowledge distillation
13

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distributior Trap is All You Need
Appendices
Related Work of Data-Free Knowledge Distillation
15
Algorithms
16
B.1 Training NTL
16
B.2 Training DFKD
16
C More Empirically Results on NTL Teachers
17
C.1 Training Dynamic...
17
C.2 Visualization of Synthetic Samples
17
C.3 Regular Knowledge Distillation Results
19
C.4 The Influence of Batch-Normalization Loss
19
C.5 Adversarial Robustness for NTL Teachers
21
C.6 The Influence of NTL Training Objectives to Adversarial Robustness
22
Adversarial Robustness for NTL-based Backdoor Teachers
23
Toy Experiments OOD Trap Effect
25
D.1 Experiments for Teachers with Ratch-Normalization Layers
25
D.2 Experiments for Teachers without Batch-Normalization Layers
28
Additional Experiments
31
E.1 Experiment Setup and Details
31
E,2 Analysis of Adversarial Perturbation Bound
31
E.3 Analysis of Forgetting Loss
32
E,4
Analysis of Adversarial Attack Method
32
E.5
Compare with Backdoor Baseline
33
E.6
Experiments on More Backbones
33
E.7
Visualization
34
Further Discussion
36
F.1
Distribution Shift in DFKD
36
F.2 The Threat of Latent Inseparability
36
14

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
A. Related Work of Data-Free Knowledge Distillation
Data-free knowledge distillation (DFKD) deals with distilling teacher's knowledge to smaller student without requiring to access the in-distribution (ID) data (Micaelli & Storkey, 2019; Luo al., 2020; Yu al.. 2023; Liu al., 2024). Due the unavailability of the teacher's ID data Dia. one needs find substitution data to conduct knowledge distillation. To this end, previous works try directly optimize randomly initialized noise images (Yin et al., 2020) under the guidance of teacher, or leverage unlabeled data from the wild (Fang et al., 2021a). The former one faces efficiency issues due to per-image optimization, and the latter one needs extra data. Generative Adversarial Networks (GANs)-based framework become the mainstream solution, where a generator is introduced to synthesize fake data in an adversarial exploration way (Liu et al., 2021b; Micaelli & Storkey, 2019; Fang et al., 2019). Specifically, the generator is trained to synthesize fake samples causing disagreement between student and teacher, thus expanding the coverage of the data distribution. Then, the synthetic samples are used as substitution data conduct knowledge distillation, thus transferring knowledge from teacher to student.
Early stage DFKD methods such as ZSKT (Micaelli & Storkey, 2019) primarily rely the adversarial exploration. However, only using adversarial exploration can bring non-stationary distribution problem and loss the diversity of synthesizing. The non-stationary distribution problem causes the catastrophic forgetting of the student models. This issue can be mitigated by sannnor-bannk-asseee strategies (Fang et al., 2021b; Do et al., 2022; Patel et al., 2023; Binici et al., 2022b; Tran et al.. 2024), where old synthetic samples are stored and will be replayed in future epochs. For the diversity problem, following works try to encourage the diversity of the synthetic images by using class-balance loss (Choi al., 2020), contrastive loss (Fang al., 2021b), image augmentation (Yu al., 2023).
DFKD has driven multiple downstream tasks, especially in data-privacy scenarios, such data-free federated learning (Zhu et al., b0 Zhang et al., 2022), data-free meta-learning (Wei et al., 2024; Hu et al., 2023), data-free quantization (Qian et al., 2023; Choi et al., 2020), data-free model fusion (Shi et al., 2024). DFKD may also be used in malicious way: one example is data-free model extraction (DFME) (Kariyappa et al., 2021; Truong et al., 2021; Wang et al., 2024c; Wang et al.).
Although significant progress has been made in the field of DFKD, existing works always assume pre-trained teachers are trustworthy. Hong et al. (2023) conducted the first effort to explore DFKD in the context of untrusted teachers. However, their work primarily focuses teachers with backdoor (Wu al., 2022; Gu et al., 2019). Another similar work is DERD) (Zhou et al., 2024), while they mainly focus transferring the adversarial robustness (Yu al., 2022b;a) from robust teacher to a student via DFKD. In this work, we conduct the first investigation on the DFKD against non -on-transferable learning (NTL) teachers (Wang et al., 2022; Hong et al., 2024b; 2025; Xiang et al., 2025; Hong et al., 2024a; Peng et al., 2024; Wang et al., 2023; 2024a; Deng et al., 2024; Zeng & Lu, 2022; Ding et al., 2024).
15

---

When Data-Free Knowledge Distillation Meets Non- Ten-Transferable Teacher: Escaping Out-of-Distribution
Trap is All You Need
B. Algorithms
B.1. Training NTL
Algorithm 2 Training NTL
1: Input: an initialized model ff with parameters By; total training epochs E;; labeled ID domain training data Da {(xf "/")) ) unlabeled OOD domain training data Dood { (1) Mem ) batchsize B; hyper-parameter a.
2: Output: the NTL model f.
3: for 1 to E do
4: Forward
Sample batch ID data Bi = ((=)" v/ )} 9/2 from Da and batch of OOD data Bund ([==)")) from Dand-
6: Concat ID and OOD data into complete batch B {æ. )m.)).. him..
7:
Obtain the batch of model prediction } {Ne (0 [wx). E &B)
8:
Split the prediction into ID: Did (0 1/ = f(x ), at! Ba) and OOD: Vont = (st 137 (actt ae Ben)
9:
ID loss Calculate the ID loss Lis D.?? (f v²*).
10:
11:
OOD loss
12:
Calculate the OOD loss Lou = 2 B/2 (fxt) (,, fax) 13 Ein
and Lieu = 9/2 Me
13:
Obtain the full NTL loss LNTI. = Lil min(1, a Lout L.en)..
14:
Back Propagation
15:
Update Oy via minimizing LNTL.
end for
Algorithm 3 Training NTL-cls
I: Input: initialized model with parameters 0y; total training epochs E; labeled ID domain training data Das [ax]] }/"))
unlabeled OOD domain training data Dood {xx Mers, batchsize B; hyper-parameter a and Aas the OOD-class index Yood.
2: Output: the NTL model f.
3: for 1 to do Forward
B/2 from Dawd- 5: Sample a batch of ID data Ba = [(=,", 8/2 ) from Da and batch of OOD data Badd {(xx)*)) )
6: Concat ID and OOD data into complete batch
7: Obtain the batch of model prediction y {ūr p. [(x:), B)
8:
Split the prediction into ID: )is
[v: 1 f(x("), a!! B and OOD: Dont (01a good f(xrt ae Buod E
9:
ID loss
Caleulate
loss Lis 1/2 Σ =
DRL. (J (a=")) v())
11:
b OOD loss
12:
Calculate the OOD loss Lou = B En?
B/2 D (ax f fxx
and Lieu = E.T M
13:
Calculate the class-controlling loss of OOD data Las EBT 2 Los (ax Ymm)).
14:
Obtain the full NTL loss LXTLAS Lia min(1, (1, Lou Liea) + dets Lets
Back Propagat ion
16:
Update fy via minimizing ExT-ek-
17:
end for
B.2. Training DFKD
Algorithm 4 Training DFKD Baseline
Input: a teacher fú generator with parameters 0g; a student. with parameters 0s; total training epochs E; synthetic batchsize B; hyper-parameter Amn-
2: Output: the DFKD student S.
3: for 1 to do
Training Generator
5: Randomly sample a noise batch Busise = (n, ne; ~ D Danise ) P---
Calculate adversarial loss Laya 8 (e (G(n)))) (G(Bma))) (B) ((M,,
7: Calculate BN loss Lin E, (N (B)a)))) (
Updating dG via max (Layn Aun Lmn
9: Synthesize Samples and Conduct Knowlede Distillation
Randomly sample noise hatch Buoise = (n, n. ~ Pm Updating As via min Luu Σ Du((
11:
0g
12: end for
16

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
C. More Empirically Results on NTL Teachers
In this section, we present more analysis and empirically results for NTL teachers and the OOD trap. In Appendix C.I, we show more training dynamic for distilling NTL teachers via DFKD. In Appendix C.2, we show more synthetic samples and I-SNE visualizations. In Appendix C.3, we present regular KD results. Appendix C.4, analyze the influence of BN loss for OOD trap. In Appendices C.5 C.7, show more empirical results to demonstrate the difference of adversarial robustness of ID and OOD domain for NTL teachers.
C.1. Training Dynamic
In this section, present more comparisons of the DFKD training dynamic between SL teachers and NTL teachers, shown in Figures 7 and 8. Specifically, in Figure 7, we present three metrics the same as Figure I: ID accuracy in distilling SL teacher (SL-ID) and NTL teacher (NTL-ID), and OOD accuracy (specific to the OOD-class) in distilling NTL teacher (NTL--OOD). We can see the same phenomenon as we illustrated in the main paper: (i) degradation of ID knowledge transfer: the student exhibits lower NTL-ID accuracy compared to distillation an SL teacher, and (ii) misleading OOD knowledge trunsfer: the student inherits teacher's misleading OOD knowledge, resulting a high NTL-OOD accuracy.
Moreover, in Figure 8, we present results on closed-set NTL teachers, where we report the OOD accuracy based on the student's 's predictions to true data labels. This can directly reflect the generalization ability from ID to OOD domain. From the results, the NTL-ID accuracy is consistently lower than the SL.-ID) accuracy, demonstrating the degradation of ID knowledge transfer during DFKD training. Regarding OOD accuracy, SL teachers enable students to achieve a certain degree of ID-to-OOD generalization. However, NTL-OOD accuracy of DFKD students under NTL teachers canmot improve alongside the increasing NTL-ID accuracy. This indicates that NTL teachers tend to transfer misleading OOD knowledge to students through DFKI).
80
100
501
Sc. DD NTT. .o0
SL
NIL IP
M
20
10
20
20
121 151
31
61
155
151
121
1.51
15.1
Epoch
(a) EVHN-CCIAAE VGG CMI
(b) SVHN -CIFARI ResNet DFQ
(c) CIFARIO-STLI VGG DFQ
Figure 7. Comparison of training dynamic of SL and NTL teachers. The OOD accuracy is calculated between the student's predictions and OOD-clas labels. This directly show the degree of OOD misleading knowledge transfer from teachers to students.
100
100
S1.D0
60
MILL
DOO
80
NIT ID
NIL
Asamer
40
40
20
31
121
151
63
157
1881
121
(a) CIFARIO-STLII ResNet NAYER
(b) CIFARIO-STLAN VGG DFQ
(c) CIFARIO-SSTI ITLIO ResNet CMI
Figure 8. Comparison of the training dynamic of the SI. and NTL. teachers on close-set tasks. We directly show the O0D accuracy from the student's predictions true data labels. This can directly reflect generalization ability from ID OOD domain.
C.2. Visualization of Synthetic Samples
In this section, we show the OOD trap effect from NTL teachers to DFKD by visualizing the synthetic samples during DFKD training. First, we show the real image samples from datasets we have involved. Example images from each dataset are shown in Figure 9, and CIFARIO with four different types of backdoor triggers are shown in Figure 10.
Then, we visualize the synthetic samples during distilling a SL. teacher and NTL teachers. Specifically, we use CMI (Fang
17

---

When Data-Free Knowledge Distillation Meets Non-Transferabl Teacher: Escaping Out-of-Distribution Trap is All You Need
al., 2021b) distilling SL teacher (pre-trained CIFARIO) and three NTL teachers pre-trained CIFARIO- CIFARIO-CIIARR w/ Sig, and CIFAR -O-CCIFAROO w/ Blended. From the results shown in Figure 11, that the SL results look more like the real CIFARIO samples in Figure (f), while NTL results consistently integrate both the ID domain (CIFARIO) and OOD domain information (Digits in Figure 9 (a-e), Sig in Figure 9 (e), and Blended in Figure 10 (d)).
observe
7
7 5
4
2
6 C 1
344
6
3
9 7
700
44
13
9
0 1 8
7
7
7 007
3
9
d
7 5
7 6 65
6
O
2 2A00
0 4000
16
8
5
6
505
8
(a) MNIST
(b) USPS
(c) SVHI
(d) MNIST-M
(e) SYN-D
(f) CIFARIO
(g) STLIO
Figure 9, Samples from involved datasets.
(a) Clean
(c) BadNets(grid) Figure 10. Clean samples and clean samples with different types of backdoor triggers.
(b) BadNets(sq)
(d) Blended
(e) Sig
(a) SL CIFARIO
(b) NTL CIFARIO Diggts
(c) NTL CIFARIO Sig
(d) NTL CIFARIO-> Blended
Figure 11. Comparison of synthetic samples (DFKD by CMI T: ResNet34 and
S: ResNet18).
(a) SN-CCIFARIO, ZSKT
SN-CCIFAR,O, DFQ
(c) SN-CCIARIO,,, CMI
(d) SN-:CIFAR1O, NAYER
(e) CIFARIO SARRIO-SSTLIA STLIO, ZSKT
(9 ITLIO,, DFQ
(g) SIPARIO-STLIN CMI
(h) CIFARIO-STLIO, NAYER
Figure 12. t-E visualization of synthetic samples (7' ResNet34 and S: ResNet18), 8). Green dots and red dots represent real ID and OOD domain samples, respectively. Purple transparent dots represent the synthetic samples.
18

---

When Data-Free Knowledge Distillation Meets Non-Transferabhe Teacher: Escaping Out-of-Distribution Trap is All You Need
Furthermore, we plot more t-SNE feature visualization results (Van der Maaten & Hinton, 2008) regarding the teacher's features of real IID samples, real OOD samples, and synthetic samples in Figure 12. We observe that for more advanced DFKD methods such as DFQ (Choi et al., 2020), CMI (Fang et al., 2021b), and NAYER (Tran al., 2024), parts of synthetic samples's 's features overlap with real OOD domain features. This indicates that these samples are more similar to real OOD samples and can activate NTL teacher's OOD misleading knowledge. In particular, the basic adversarial exploration method ZSKT (Micaelli & Storkey, 2019) fails to approximate the real ID distribution, as its synthetic samples remain distant from ID clusters.
C.3. Regular Knowledge Distillation Results
Although our work focuses on DFKD, which relies on generator to synthesize fake samples, we also include regular knowledge distillation (KD) results with NTL teachers for reference, as shown in Table 4. Specifically, we present results of regular KD using ID data only (KD-ID), OOD data only (KD-OOD) and both ID and OOD data (KD-All). The results show that when only ID data is used, the student learns only ID knowledge. However, once OOD data is involved, the student inevitably learns misleading OOD knowledge from the NTL teacher.
Table Regular knowledge distillation. We report the ID domain accuracy in blue and OOD domain accuracy in red. The accuracy drop compared to the pre-trained model is shown in brackets.
ID: CIFARIO OOD: STL
ID: CIFARIO OOD: Digits
ResNet.34 18
ResNet.34 R8
IAce f
(A)
IAcc (
OLACC
NTL Teacher
90.8.01
10.4:00
91.101
100.0+00
KD-ID
80.4 (-10.4)
18.1110044477
82.2.03 (-8.9)
30.0,37
KD-OOD
10.6=n0 (-80.2)
10.4:00 (+0.0)
10.6:00 (-80.5)
(-0.0
KD-AII
76.6.n4 (-14.2)
(00.000(00.)
81.8105 (-9.3)
100.D.00
C.4. The Influence of Batel--ormalizzation Loss
In this section, we discuss the influence of Batch-Normalization (BN) loss (Yin et al., 2020) for the identified OOD trap effect from NTL teachers to DFKD. In DFKD, SOTA methods such as Choi et al. (2020); Fang et al. (2021b); Tran et al. (2024) commonly use BN loss (Equation 9) as regularization for training generator G. Specifically,
As shown in Appendix B.I, when training NTL teachers, each batch contains mixture of ID and OOD domain samples. This results in that BN layers in pre-trained NTL teachers record statistical information for the mixture distribution of ID and OOD domains (mean μi and var of for layer l).
During DFKD, when training the generator G, BN loss (Equation 9) is represented the divergence between synthetic feature statistics N (N ((u((Z)))) (3 )) and teacher's 's BN statistics A (Mu, af). Minimizing BN loss lets the synthetic samples follow similar statistical information of teacher's ' training samples.
The joint effects of BN loss (Equation 9) and adversarial exploration loss (Equation 7) constraint the G to: (i) follow the statistics of NTL teacher's training data (i.e., a mixture of real ID and OOD domains), and (ii) maximize the discrepancy between the student and the NTL teacher. As a result, the generator G is trained to synthesize both ID-like and OOD-like samples (i.c., ID-to-OOD synthetic distribution shift). When all these synthetic samples are used for distillation, the student is inevitably taught to mimic the teacher's behavior on OOD domains, thereby learning misleading OOD knowledge. This also leads to a ID-OOD task conflict, ultimately degrading the student's 's performance on the ID domain.
Without the BN loss, the generator G' does not need to synthesize samples that resemble those from the OOD domain used to train the NTL teacher. However, the student's 's ID performance will be influenced by low quality of synthetic samples. Actually, when only using the adversarial exploration (such as ZSKT (Micaelli & Storkey, 2019)) to distill NTL teachers, the ID knowledge transfer may be completely blocked in the worst case. This results the DFKD student exhibiting random-guess-like performance on the ID domain. Figure I (c) and a toy example in Appendix D illustrate this phenomenon.
Encouraging synthesizing diversity (c.g., class balance in DFQ (Choi et al., 2020), contrastive learning in CMI (Fang et al., 2021b)) can help to mitigate the complete blockage of ID knowledge transfer when using adversarial exploration individually. However, their ID performance is still limited by the poor quality of synthetic samples. Corresponding empirical evidence is shown in Table 5. Synthetic samples w/ and w/o BN are shown in Figures 13 to 16. From the results, when we conduct DFKD w/o BN loss, the synthetic samples do not like real images, and the ID-domain performance is even worse than DFKD w/ BN loss, despite the latter suffering from an ID-to-OOD synthetic distribution shift and ID-OOD task conflict.
19

---

When Data-Free Knowledge Distillation Meets V Teacher: Escaping
Trap is All You Need
Table The influence of BN Loss. We report the ID domain accuracy in blue and OOD domain accuracy in red. The accuracy drop compared the pre-trained model is shown in brackets.
ID: CIEARIO OOD: STL
ID: CIFARIO OOD: Digits
ResNet-34 ResNet-18
ResNet-34 ResNet-18
JAce (
(%)
TAcc (
OLACC 4
NTL Teacher
90.8,01
10.4g00
91.12a1
100.0.00
CMI w/ BN
(-5..)
11.0.04 (40.6)
(437.1
76.1 (:23.9
BN
31.1232(59.7)
27.4g81 (+17.0)
36.0g1 (-55.1)
14.7a56 (-85.3)
NAYER w/ BN
48.7 (-22.1)
10.4 (40.0)
49.4126 (-41.7
97.9 (-2.1)
NAYER w/o BN
40.8229
29.3g22
39.1 (-52.0)
3.0g44
(a) CMI Loss
(b) CMI w/o BN Loss
Figure 13. Synthetic samples of CMI (w/ and w/o BN Loss) on SL teacher (datasets: CIFARIO: archs: ResNet34-ReeNtt18)))
(a) NAYER w/ BN Loss
(b) NAYER w/o BN Loss
Figure 14. Synthetic samples of NAYER (w/ and w/o BN Loss) on SL teacher (datasets: CIFAR 10; archs: ResNet34 DesNel34-RReNee1133 8).
(a) CMI w/ BN Loss
(b) CMI w/o BN Loss
Figure 15. Synthetic samples of CMI (w/ and w/o BN Loss) on an NTL teacher (datascts: CIFAR10--Digits (essee33
20

---

When Data-Free Knowledge Distillation Meets Non-Transferabhe Teacher: Escaping Out-of-Distribution Trap is All You Need
(a) NAYER w/ BN
(b) NAYER w/o BN Loss
Figure Synthetic samples NAYER (w/ and w/o BN Loss) on teacher (datasets: CIFARIO-; :Digits;; (eese))1)))
C.5. Adversarial Robustness for NTL Teachers
In this section, we present more evidence related to the NTL teachers' adversarial robustness difference on real ID and OOD domains. Results across various datasets and network architectures are shown in Figures 17 to 22, where the accuracy reflects the consistency of predictive labels before and after untargeted PGD (Madry, 2017) attack. We can observe that predictive labels for ID domain data are easily changed after attacks, In contrast, OOD samples exhibit strong robustness against adversarial perturbations.
100
100
100
Acturacy (%)
80
BID
GOOD
BID G0OD
Accuray
BID
DOOD
20
0
2
2
4
5
(a)
(b)
(c)
16
Figure 17. Adversarial robustness (PGD) on ID and OOD domain. The NTL teacher is trained on SVHN CIIAARI with ResNet34.
100
100
100
Accurscy (%)
BDD GOOD
IDD 00O
GID GOOD
Acturacy
40
20
0
5
stops
stops
stops
(a)
(b)
(c) 16
Figure 18. Adversarial robustness (PGD) ID and OOD domain. The NTL teacher is trained SVHN-CCFAAR with VGG-13.
Accurscy (%6)
Accuraty (96)
GOOD
60
2000
GOOD
40
40
40
20
20
20
5
steps
steps
steps
(a)
(b) c
(c)
Figure 19, Adversarial robustness (PGD) ID and OOD domain. The NTL teacher is trained on CIFAR1O- with ResNet34.
21

---

When Data-Free Knowledge Distillation Meets Non- Transferrreee Teacher: Escaping Out-of-Distribution Trap is All You Need
100
100
Acturay (%)
80
80
Accuracy (0))
80
80
Accurayy
60
BOOD
60
40
40
20
20
stops
steps
stops
(a) E
(h)
(c) = 16
Figure 20. Adversarial robustness (PGD) on ID and OOD domain. The NTL teacher is trained on CIFAR1O-Digits with VGG-13.
100
100
100
Accuncy (%6
80
ID DOOD
BID 80OD
BID GOOD
Accureyy
Accurcy
20
0
2
4
5
steps
steps
steps
(a)
(b)
(C)
16
Figure 21. Adversarial robustness (PGD) ID and OOD domain. The NTL teacher is trained CIFARIO STTL10 with ResNet34.
100
Accurscy (%6
Accuracy (%)
IDD 10O
Accureyy
60
GOOD
40
40
40
20
20
20
4
5
stops
steps
steps
(a) 6
(h) 6
(c) =
Figure 22, Adversarial robustness (PGD) on ID and OOD domain. The NTL teacher trained on CIFARIO-STLIO with VGG-13.
C.6. The Influence of NTL Training Objectives to Adversarial Robustness
In this section, we further investigate the influence of each training objective in NTL to the adversarial robustness of NTL teachers. We conduct ablation studies on three variants: (i) the vanilla SL on IDD domain: Lus (ii) the vanilla SL on ID domain and the class control the OOD domain:
Li Leis and (iii) the full NTL: LNIL-cis-
It is notably that the training object of (ii) Lis + Lets is similar to the conventional backdoor learning (Wu et al., 2022; Hong et al., 2023). Compared to (ii), the (iii) full NTL LNTL-Cls has an additional term (- (-min(1,0 Lout Lfea)). Therefore, the full NTL CNIL-cls not only predicts all OOD-domain samples to the target class Food (like conventional backdoor learning), but also pushes these OOD-domain samples far from ID-domain samples in both feature and output space.
The adversarial robustness of three NTL variants on ID and OOD domains are shown in Figures 23 to 26. have:
(i) The vanilla SL models are fragile both ID and OOD domain. This is because learning correct classification results in relatively complex decision boundaries between classes and small margins for IDD domain data points.
(ii) For the ID-donain SL with class control on the OOD domain (i.e., like conventional backdoor learning (Wu et al., 2022; Hong et al., 2023)), the OOD domain become more robust than ID domain. This is because minimizing Las forces all OOD-domain data to be predicted as a single class, which is simple, and no boundaries will go through the OOD-domain samples.
(iii) For the full NTL, maximizing the additional term (- min(1, 1, Lout Lieu)) further pushes the OOD-domain cluster far away from ID-domall clusters, resulting in very large margins for OOD-domain samples (i.e., no OOD-domain samples will close to decision boundary). As a result, the full NTL teachers exhibit a very strong robustness against adversarial attacks on OOD-domain data.
'The margin is defined the minimal distance from data point decision boundaries (Xu al., 2023), where larger margin corresponds to stronger robustness.
22

---

When Data-Free Knowledge Distillation Meets Non- Transferrreee Teacher: Escaping Out-of-Distribution Trap is All You Need
100
100
Acturacy (%)
80
80
Accuracy (%)
80
80
Accurayy
60
60
40
40
20
20
steps
steps
stops
(a) SL trained on CIFARIO (Ci)
(b) NTL (Cis + Leis)
(c) NTL (full, LNTL
Figure 23. Comparison of adversarial robustness of SI. and NTL. teachers (ResNet-34) against untargeted PGD attack (e = 4) on close-set ID and OOD domain (ID: CIFAR1O and OOD: STL), where the accuracy reflects the consistency of predictive labels before and after untargeted adversarial attack. SL is trained on CIFARIO, NTL with different configurations are trained on ID and OOD domains.
100
100
100
BID
IID
Accuncy (%)
DOOD
80
DIU
Accurecy
Accuncy
GIOD
20
0
0
0
2
3
4
5
steps
steps
(a) SL trained CIFARIO (Cia)
(b) NTL (Cis Las)
(c) NTL (full, Lxn=ets)
Figure 24. Comparison of adversarial robustness of SL and NTL teachers (VGG-13) (3) against untargeted PGD attack 4) on close-set ID and OOD domain (ID: CIFARIO and OOD: STL), where the accuracy reflects the consistency of predictive labels before and after untargeted adversarial attack. SL trained CIFARIO, NTL with different configurations trained ID and OOD domains.
BID
GOOD
Accurscy (%)
Accurssy
60
60
GOOD
40
40
40
20
20
20
steps
steps
Steps
(a) SL. trained on CIFARIO (Cia)
(b) NTL (Cas + Lets)
(c) NTL (full, CxTLs)))
Figure Comparison of adversarial robustness SL NTL teachers (ResNet-34) against untargeted PGD attack (e = 4) on open-set ID and OOD domain (ID: CIFAR10 and OOD: Digits), where the accuracy reflects the consistency of predictive labels before and after untargeted adversarial attack. SI. trained on CIFAR10, NTL with different configurations are trained on and OOD domains.
100
100
100
Accuney (%)
80
80
80
60
Accumcy
60
Accurisy
60
DOOD
40
40
20
5
steps
Steps
(a) SI. trained on CIFAR10 (Cy)
(b) NTL (Cos + Lete)
(c) NTL (full, LNTL
Figure 26. Comparison of adversarial robustness of SL and NTL teachers (VGG-13 13) against untargeted PGD attack (e = 4) on open-set ID and OOD domain (ID: CIFARIO and OOD: Digits), where the accuracy reflects consisteney predictive labels before and after untargeted adversarial attack. SL is trained on CIFAR1O, NTL with different configurations are trained on ID and OOD domains.
C.7. Adversarial Robustness for NTL-based Backdoor Teachers
We present the results of NTL-based backdoor teachers trained with the full NTL objective (i.e., LNTL. in Figure 27. For comparison, we also include results for conventional backdoor teachers trained with Lia + Lot in Figure 28.
The results reveal similar observations discussed in Appendix C.5 and Appendix C.6. Specifically, both NaL-based and conventional backdoor teachers exhibit significantly stronger adversarial robustness on the OOD domain than on
23

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
the ID domain. Moreover, NTL-based backdoor teachers demonstrate even greater adversarial robustness compared conventional backdoor teachers. This enhanced robustness is attributed to the influence of the NTL regularization term: min(1, a Lou Liem).
100
100
Accuracy (86)
BID 0000
Accuracy (%)
ID
DOOD
40
0
steps
stcps
(a) CIFARIO ETTAAROO-CUEAARLOOBBBNNYeeoo
(b) PITEARIO-CIIARIO-BBBAA
100
100
Accunacy (%)
BDD DOOD
Accuncy (%)
ID
60
60
COD
40
80
20
stops
stops
(c) CIFARIO SEIIFARIO+Blendee
(d) CIPARIO-CCIFARIOSS +CIFARIO+Sig
Figure The difference of adversarial robustness (PGD with 4) on NTL-based backdoored teachers. The accuracy reflects the
consistency of predictive labels before and after untargeted adversarial attack.
100
100
Accurayy (85)
Accuracy (%))
stops
(a)
(b) CIFARIO BAAAAII--CCIAAARL--BAANeeeeeeee
100
Agcunay (%6
BID BOOD
Accuncy (%6
BID BOOD
40
40
20
20
0
steps
steps
(c) CIFAR10 -CIFARRIO-HO Bleendedd
(d) CIFARIO SHARIIO-CIFARIOSSE
Figure 28. The difference of adversarial robustness (PGD with 4) conventional backdoor teachers. The accuracy reflects the consistency of predictive labels before and after untargeted adversarial attack.
24

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
D. Toy Experiments for OOD Trap Effect
D.I. Experiments for Teachers with Batch-Normalization Layers
As shown Figure 29, we use a toy experiment in 2D space to illustrate the OOD trap effect in NTL teachers. We consider a toy classification task with three classes and two domains with distribution shift (covariate shift). Specifically, denote different classes (red for class 1, green for class 2, and blue for class 3), and different symbols represent different domains: for ID domain and x for OOD domain.
different colors
We first train supervised learning (SL) teacher on ID domain and conduct DFKD by using the adversarial exploration with BN loss (Fquation 7 Equation 9) to transfer knowledge from the SL teacher to a student model. The teacher and student share the same network architecture three-layer MLP with BN layers, as shown Table 8). DFKD results are shown in Figure 30. The first line shows the decision space of the SL teacher, where colors corresponding each class represent their respective decision regions. The second line shows the decision space of the DFKD student. Those yellow crosses with black boundaries represent the synthesized data samples. From the results, the student model can learn correct ID knowledge within the SL teacher after several DFKD training epochs. Besides, the generalization ability from the ID to OOD domain is maintained.
Figure 29, toy experiment.
0.0
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch= 1601
Figure 30. Toy experiment for data-free knowledge distillation (w/ BN loss) on SL teacher w/ BN layers. Red represent class 1, green represent class and blue represent class samples. Different symbols represent different domains: for ID domain and for OOD domain. The first line shows the decision space of the SL teacher, and the second line shows the decision space of the DFKD student.
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch=1601
Figure 31. Toy experiment for data-free knowledge distillation (w/ BN loss) on an NTL teacher w/ BN layers.
25

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Then train non-transferable learning (NTL) teacher the ID domain and the OOD domain by minimizing Equation 6 and then conduct DFKD. Results shown in Figure 31. Compared to the SL results,
that the NTL teacher let the DFKD student learning misleading knowledge from the OOD domain. This is caused by the fact that the DFKD generator synthesizing both ID-like and likelike samples (i.e., ID-to-OOD synthetic distribution shift).
The effect of BN loss in DFKD, To show the effect of the BN loss in DFKD, we conduct DFKD by only using the adversarial exploration loss (i.e., Equation 7). Specifically, we conduct DFKD three times with different initializations the synthetic data. Results are shown Figures to 34. Intuitively, without the BN loss, the synthetic samples fail to approach the real distribution of both ID and OOD domains. Moreover, for some runs which initialization are far from the real ID domain (Figures 32 and 33), the ID domain performance the DFKD student significantly degraded random guess. In such situations, no ID knowledge is successfully transferred to the DFKD student.
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch=1601
Figure 32. Toy experiment for data-free knowledge distillation (w/o BN loss) on an NTL teacher w/ BN layers (initial 1).
(a) Epoch=1 Figure 33, Toy experiment for data-free knowledge distillation (w/o BN loss) on an NTL teacher w/ BN layers (initial 2).
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch=1601
The effect of NTL components. In addition, we conduct DFKD experiments different variants of NTL teachers, thus verifying the effect of different component in NTL training objectives. We consider five variants:
NTL-cls: the full NTL with class controlling, as shown in Equation 5.
NTL: the full NTL without class controlling, shown in Equation 4.
SI.2domain: revising NTL-cls by remaining the ID domain learning objective and the OOD class controlling term, which
26

---

When Data-Free Knowledge Distillation Meets Non- Transferrreee Teacher: Escaping Out-of-Distribution Trap is All You Need
(a) Epoch=1 Figure 34. Toy experiment for data-free knowledge distillation (w/o BN loss)
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(c) Epoch=1601
NTL teacher w/ BN layers (initial
can be formulated as:
Ls Ls..2dmmain Lia + Aets Lees-
(15)
NTL w/o MMD: revising NTL-cls by removing the MMD loss in feature space, which can be formulated as:
LNTLWOMMD Lid min(1, a Cou) + Aets Lels-
(16)
NTL w/o KL: revising NTL-cls by removing the KL loss in output space, which can be formulated as:
CXILWIKL = Lu min(1, a Lrea) + Acta Lets-
(17)
Visualization results of the SL teacher and NTL variants shown in Figure 35. The corresponding accuracies ID and OOD domains are shown in Table 6. From the results, all the five NTL variants can transfer OOD misleading knowledge from teacher to student. However, the full NTL has the most significant influence of the ID knowledge transfer (reflecting on the lowest (Acc). This highlights the importance of the combined effect of feature-level and output-level discrepancy terms shaping the OOD trap effect.
(a) SL.
(b) NTL-cls
(c) NTL
(d) SI.2domain
(e) NTL w/o MMD
(1 NTL w/o KL
Figure 35. Comparison of DFKD w/ BN loss across variants of teachers w/ BN layers (Epoch=1801).
Table Results variants of NTL teachers. We report the ID domain accuracy (IAcc) in blue and OOD domain accuracy (OAce) in red.
SL
NTL-cls
NTL.
ST.2domain
NTL w/o MMD
NTL w/o KL.
1 OAce
LAce
1
f OAce
LAce
f
LAce
OAce
Teacher
99.33%
45.44%
98.67%
34.78%
99.00%
34.78%
99.22%
34.78%
98.56%
34.78%
Student
99.00%
62.33%
96.78%
34.78%
98.33%
34.78%
99.00%
34.78%
99.11% 34.78%
97.44%
34.78%
27

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Table The architecture of the toy model w/ BN.
Table The architecture the toy model w/o BN.
nn.Rel.17
nm. R.l.el.
Output
500,
Output
550,
nn. Rn.ReLU
nn. Rel.U
Hidden
NatchNorm (d(500)
Lineart (00, 500)
Lincar 100, 500)
Re..eLU
Rn.Rel.
Input
nn.1 lincar(2, 100)
Input
no.1 no.T inear)2,, 100)
D.2. Experiments for Teachers without Batch-Normalization Layers
We also the toy experiments by replacing the network architecture from the three-layer MLP with BN layers (Table 7) three-layer MLP without BN layers, as shown in Table 8. We also train SL teacher and an NTL teacher, and then, conduct DFKD by using the adversarial exploration loss (Equation 7) individually. The results for SL teacher is shown in Figure 36, and the results for NTL teacher (three runs with different initialization) are shown in Figures 37 to 39.
From the results, although without the BN regularization, the student trained by the adversarial exploration can still learn correct ID knowledge from the SL teacher, with the generalization ability from the to OOD domain being maintained. In contrast, when facing the NTL. teacher, the individual adversarial exploration still lead to unexpectedly poor II) performance (random guess in the worst case, like Figures 37 and 38).
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch=1601
Figure 36. Toy experiment for data-free knowledge distillation (w/o BN loss) on SL teacher w/o BN layers. Red represent class 1. green represent class 2, blue represent class samples. Different symbols represent different domains: o domain and for OOD domain. The first line shows the decision space of the SL teacher, and the second line shows the decision space of the student.
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(e) Epoch=1601
Figure 37. Toy experiment for data-free knowledge distillation (w/o BN loss) on an NTL teacher w/o BN layers (initial 1).
28

---

When Data-Free Knowledge Distillation Meets Non- Ton-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(c) Epoch=1601
Figure 38. Toy experiment for data-free knowledge distillation (w/o BN loss)
NTL teacher w/o BN layers (initial 2).
(a) Epoch=1
(b) Epoch=401
(c) Epoch=801
(d) Epoch=1201
(c) Epoch=1601
Figure 39, Toy experiment for data-free knowledge distillation (w/o BN loss) NTL teacher w/o BN layers (initial 3).
The effect of NTL components. In addition, also investigate the effect of different variants of NTL teachers (the same as variants in Appendix D.I). Visualization results of each NTL variant and the SL teacher are shown in Figure 40, The corresponding accuracies on ID and OOD domains are shown in Table 9.
(a) SL.
(b) NTL-cls
(c) NTL
(d) SI.2domain
(e) NTL w/o MMD
(1) NTL w/o KL
Figure 40. Comparison of DFKD w/o BN loss across variants of teachers w/o BN layers (Epoch=1801),
29

---

When Data-Free Knowledge Distillation Meets Non-Transferabie Teacher: Escaping Out-of-Distribution Trap is All You Need
Table Results variants of NTL teachers. We report the ID domain accuracy (lAcc) in blue and OOD domain accuracy (OAce) in red.
NTL-ds
S1.2domain
NTL
JAce 7 OAce
1Acc
OAce 1
IAce
OAsc 1
LAce 1
OAce
LAce 1 OAce 1
IAce
OAce f
Teacher
99.44%
65.67%
99.22%
34.78%
99.11%
34.78%
99.67%
34.78%
99.56%
34.78%
99.56%
34.78%
Student
99.56% 65.67%
34.78%
34.78%
32.67%
34.78%
99.67%
34.78%
99.67%
34.78%8
99.3318
34.78%
Similarly, all five NTL variants can transfer OOD misleading knowledge to students. Notably, compared the results of teachers with BN layers (Figure 35 and Table 9), when the teacher lacks BN layers, the full NTL can completely block ID knowledge transfer, leading the student to exhibit random guess-like performance on the ID domain.
30

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
E. Additional Experiments
In this section, we present more experimental results and analysis. We provide more detailed experimental setups in Appendix E. In Appendix E.2, we analyze the influence of adversarial perturbation bound to the synthetic data grouping. In Appendix E.3, we analyze the influence of forgetting loss. In Appendix E.4, we conduct ATEsc by using alternative adversarial attack. In Appendix E.5, compare our proposed ATEsC with more baselines. In Appendix E.7, plot visualization regarding the feature distribution and synthetic samples.
E.1. Experiment Setup and Details
Datasets. We choose the common datasets shared by both NTL and DFKD, including Digits (MNIST (Deng, 2012), USPS (Hull, 1994), SVHN (Netzer et al., 2011), MNIST-M (Ganin et al., 2016), and SYN-D (Roy et al., 2018)), CIFARIO (Krizhevsky et al., 2009), STL10 (Coates et al., 2011).
NTL tasks. For widely evaluation, we conduct our experiments on three different NTL tasks:
(i) Close-set NTL. In this setting, the ID domain and OOD domain have the same label space, but the two domains have a distribution shift. We choose the dataset pairs of: -IIISMMISMMM and CIFARIO-STLIO,;
(ii) Open-set NTL. In this setting, the label space of ID domain and OOD domain are disjoint. We perform experiments on SVHN-CIFAR 10 and CiFARIO-Digits, where Digits is the combination of 5 digits datasets: MNIST, USPS, SVHN MNIST-M, and SYN-D;
(iii) NTL-based Backdoor. We additionally consider using NTL in conducting training controllable backdoor attack (Wu et al., 2022). We use four triggers: BadNets(sqq. BadNets(grid) (Gu et al., 2019), Blended (Chen al., 2017), Sig (Bari et al., 2019) on CIFARIO 10 and see them the OOD domain. The clean CIFARIO is regarded as the ID domain.
Our aim is to address the malign OOD-Trap effect introduced by NTL teachers in DFKD, and thus, regardless of NTL teacher's tasks, our goal is to transfer only the ID domain knowledge from NTL. teacher to student.
DFKD baselines. We involve the sfate-of-the the- (SOTA) non- mon-me bank method DFQ (Choi el., 2020) and two memory bank methods: CMI (Fang et al., 2021b), NAYER (Tran et al., 2024). We follow the implementation of DFQ and CMI in com/ z ji-vi pa CMI and NAYER in https: //ss:///ttt g thub. com/ uan1107 / /NAYER. For each baseline, we evaluate two variants of our method: using only the CKD in Section 4.3 (denoted as CKD) and the full method (denoted as + ATEsc). All teachers are considered as white-box models during DFKD.
/ttpss//githhe
Implementation details. For each task, NTL teachers are pre-trained up to 50 epochs (input images resize to 64x64). For DFKD, we train each method up to 200 epochs, with all syper-parametent following their original implementations. VGG-13 (Simonyan, 2014) and ResNet-34 (He et al.. 2016) are used as teacher architectures, while VGG-11 1 and ResNet- 18 are student architectures. We conduct all experiments on single RTX 4090 GPU (24G).
Evaluation metrics. For Close-set NTL, we use ID domain accuracy (IAcc) to measure the performance of ID knowledge transfer. Because ID) and OOD domains have the same label space in close-set NTI, we present OOD domain accuracy (OAcc) to their real labels to show the generalization ability from ID to OOD domain. Higher OAcc means better ID-to-OOD generalization, and also means better resistance to misleading OOD knowledge transfer during DFKD. For Open-set NTL, we calculate the accuracy between the prediction and the OOD-Iabel on OOD domain (denoted as OLAcc) to evaluate the transfer of misleading OOD knowledge. A lower OLAcc indicates better resistance to OOD knowledge transfer. Moreover, we still use IAcc on ID domains. For NTL-based Backdoor, we follow backdoor learning to use clean label accuracy on ID domain (i.e., the same to lAcc) and attack success rate (ASR) (Li et al., 2022) on backdoor data, where the ASR is the accuracy of the prediction and the OOD-label (i.e., the backdoor target label) on trigger OOD data.
E.2. Analysis of Adversarial Perturbation Bound
To evaluate the influence of the adversarial perturbation bound c, we conduct experiments on CIFARIO and CIFAR1O- using two DFKD baselines: DFQ and NAYER. The results are presented in Figure 41 and Figure 42. For the ATEsC. As c decreases, the student's 's OAce in DFQ + ATESC experiences a significant drop. This suggests that an overly small perturbation bound is ineffective in distinguishing ID-like and OOD-like synthetic samples, allowing the transfer of misleading OOD knowledge from the teacher to the student. For the open-set task CIFAR 1Diittsss (Figure 42), a
task (Figure 41), we observe that the performance are generally stable, except for DFQ +
31

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
higher OLAcc indicates more severe transfer of misleading OOD knowledge. We observe general trend: increasing the perturbation bound improves resistance the transfer of misleading 0OD knowledge from the teacher to the student.
80
Astumay (0)
60
Accuneay
60
Accumey
60
Jucumay
60
40
40
40
20
IDAAE
20
20
DADE
16
32
12
16 92
16
12
(a) DFQ CKD
(b) DFQ ATEsC
(c) NAYER CKD
(d) NAYER + ATESC
Figure 41. Hyperparamater analysis of e. CIFARIO-STLIO, T: ResNet34 and S: ResNet18.
Hod OLACS
1AX GLACE
30
80
60
60
40
40
40
20
20
20
20
16
52
2
16
32
16
16
32
(a) DFQ CKD
(b) DFQ ATESC
(c) NAYER + CKD
(d) NAYER + ATESC
Figure 42. Hyperparamater analysis of CIFARIO- T: ResNet34 and S: ResNet18.
E.3. Analysis of Forgetting Loss
To analyze the effect of forgetting loss, we change the weight of À in Equation 14 from 10-7 to 10-2 and conduct experiments close-set task CIFARIO-STLII and an open-set task CIFARIO-Digit with DFQ and NAYER serving as DFKD baselines. The results are shown in Figure 43.
For the close-set CIFARIO- 10 (Figure 43 (a)), higher OAce indicates better resistance to OOD knowledge transfer. We observe that setting À either too high or too low results in poor performance for DFQ on both ID and OOD domain, whereas NAYER exhibits greater robustness to changes in X.
For the open-set task CIFARIO-Digiss (Figure 43 (b)), a lower OLAcc indicates better performance. We observe that both DFQ + ATESC and NAYER + ATEsc demonstrate robustness to variations in X, exhibiting strong resistance to OOD knowledge transfer. However, setting a too large À can degrade II) performance in DFQ.
100
100
100
50
80
DACC
80
OCACC
80
ELACE
40
60
60
Acumaey
40
40
40
20
20
101-
19h-6
10145
10h-4
10h-3
10h-2
10-7
10%-6
101-5
10%-4
105-3
10%2
10-7
19%6
1054
10-4
10%3
10h-2
10%
10%6
1094
10h-
10%5
10%2
DFQ ATESC (a) CIFARIO-STLIO NAYER + ATESC
DFQ ATESC
NAYER ATESC
(b) CIFARIO-DDiiit
Figure 43. Hyperparamater analysis of X. T: ResNet34 and S: ResNet18.
E.4. Analysis of Adversarial Attack Method
The PGD (Madry, 2017) attack in Equation 10 can be replaced with any adversarial attack method. In this section, FGSM (Goodfellow et al., 2014) to perform adversarial attack, thus obtaining fragile and robust groups, and compare the final results with PGD. As shown in Table 10, we compare the results of CKD and the full ATEsC on close-set NTL tasks and the open set NTL tasks, with DFQ (Choi et al., 2020) and NAYER (Tran et al., 2024) serving as DFKD baselines. We find that by replacing PGD with FGSM, the results generally do not exhibit a significant difference, demonstrating the broad compatibility of our method with different adversarial attack methods.
32

---

When Data-Free Knowledge Distillation Meets - Teacher: Escaping Out-of-Distribution Trap is All You Need
Table 10, Different adversarial attack. We report the ID domain accuracy in blue and OOD domain accuracy in red. Results from the DFKD baselines are highlighted with gray rows. The accuracy drop compared to the pre-trained model is shown in brackets.
Close-set NTI.
Open-set NTL
CIEARIO OOD: STL
ID: CIEARIO OOD: STL
ID: CIEARIO OOD: Digits
ID: CIEARIO OOD: Digits
VGG-13-, VGG-11
ResNet-34 R8
ResNet-34 ResNet-18
IAce 1
OACC (%)
LAce (
OACC (%) 1
IAce (
(SS)
(98
OLACC (98) 4
NTL Teacher
92.7201
10.6201
90.8.a1
10.4g00
92.92n0
99,5203
91.1an
100.0.00
DFQ
42.7a36 50.0)
10:4g00
(2.5895633..33
(+14)
62.8gas (.30.)
99.8 (-0.3)
57.4g8x (33.7))
85.8g86 (-14.2)
CRDeco
(-2977)
10.4 (-0.2)
68.0gmm (-22.8)
(+122)
68.5214 (-24.4)
98. (-0.9)
20.1 L.)
5.5.36 (-94.))
C2DDosM
(5.5:55(-29.2)
10.4:00 (-0.2)
67.1=148 (-23.7)
19.3s52
66.0=26 (-26.9)
99.3s06 (-0.2)
58.1 (-33.0)
4.636n (-95.4)
+ NIEscron
(-30,)))
L1 (+20.9)
66.2-07
(1.655544112
60.3+45 (-32.6)
78.44209 (-21.1)
(-26629
2. (-97,5
ATESCPGM
59.228x(333.55
27.9 (+17.3)
53.3g51 (-37.5)
47.0gao (+36.6)
60.8 (-32.1)
80.1 (-19.4)
52.1g188
0.9g01 (-99.))
NAYER
(-86.)
10:4:0n (-0.2)
(-4.11
10,4:06
10.6-00
(00.)-0540055
49.4-25
97.9.08 (-2.1)
+ CKDPGD
62.7261 (.30.0)
23.7atu (+13.1)
64.4.0x
40,2g34 (+29.8)
70.4g14 (-22.5)
71.61109 (-27.9)
61.2g3x
6.3g13 93.7)
CXDRasM
(4.2.9 65.3g31 (-274))))274)
40.9g40 (4.7220-22611
71.4g16(-215
74.4 (-25.1)
(3.62466-27755
1.6207
+ ATE
69.7a54
42.3x27 (431.7)
52.7=24 (-38.1)
39.2 (+28.8)
69.8a19 (-23.)
2.8338 (-967)
59.0a30--32.1)
3.6206 (-96.4)
A725CH0M
66.7+131 (-26.0)
40.9 (430.3)
90.9-22(-39.9)
39.3 (428.9)
67.6- (-25.3)
3,3426 (-96.2)
(3.0-200-28.11
0.6:02 (-69))))9
E.5. Compare with Backdoor Baseline
Hong et al. (2023) first identified the risk of backdoor transfer from teacher to student via DFKD. To mitigate this risk, they proposed a defense method ABD, including Shuffling Vaccine (SV) and seff-Reerosppction (SR) modules. Briefly, the SV mitigates the backdoor participating in the knowledge distillation stage by regularizing the generator, and SR synthesizes the potential backdoor and unlearns them. We compare our TESS with the ABD on close-set, open-set and NTL-based backdoor tasks. Specifically, we use CMI (Fang et al., 2021b) as DFKD baseline, which was also used in ABD's experiments. The implementation of ABD is follow: https: : / / g github. com/ i walliddanaal ABD. We consider two configurations of ABD, i.e, CMI + ABD (SV), and CMI ABD (SV+SR).
The results are shown in Table 1 I.. Although ABD is effective in defending conventional backdoor learning teachers, as reported in their paper, it struggles to address the OOD trap effect introduced by NTL teachers. This leads to the poor performance of using CMI + ABD to distilling close-set NTL teachers, open-set NTL teachers, and defend the NaLeb backdoor transfer during the DFKD. In contrast, our ATESC can hetter mitigate OOD trap effect in NTI. teachers, enhancing ID knowledge transfer while resisting OOD knowledge transfer.
Table 11. Compare to ABD. We report the ID domain accuracy in blue and OOD domain accuracy in red, Results from the DFKD basclines are highlighted with gray rows. The accuracy drop compared to the pre-trained model is shown in brackets.
Close-set NTI.
Open-set NTI.
NTs-- backdnor
ID: CIFARIO OOD: STL
I
ID: CIFARIO OOD: Digits
ID: CIFARIO Blended
ID: CIFARIO OOD: Sig
TAce t
OACC 1
| TAce (
OLACC (
TAce f
ASR (
TAcc f
ASR (%)
NTL Teacher
90.8gaa
10.400
91.1
100.0.00
90.5
100.0g00
90,2gas
100.0400
CMI
11.0ga (53.
(+0.6)
34.2:: l)
76.1257
50.5gr1 ( (40.0)
99.92m1 (-0.1)
46.0gaa (:4.2)
100.0amo (-0.0)
+ (SV)
(6.2.5
(+0.))
31.122
77.7a35 (-22.3)
37.5a30(.53.0
99.8.01(00.)
34.8
100.010n (-0.0)
+ ABD (SV+SR)
12.5g1
10.3ga1 (-0.1)
12.1
73.0gin0
16.4266
98.1214
37.8 (-52.4)
100.0.a0 (-0.0)
CXD (Ours)
33,9a23 (+23.5) (-1.:))6(410)
1011339-41100
5.4464 (-94.6)
68.3488 (-22.2)
48.0a303 (-52.0)
67.1 (-23.1)
100.0saa (-0.0)
ATBsc (Ours)
(-4.65
(+22.5)
13.9:644447.2
3.7 (-96.3)
59.7-164 (-30.8
19.0+55 (-810)
46.9:3
32.2 (-67.8)
E.6. Experiments on More Backbones
The results with different series of network architecture for teachers and students are shown in Table 12. From the presented results, the proposed ATEsc is consistently effective in mitigating the OOD trap effect caused by NTL teachers.
Table 12, Experiments on more backbones. We report the ID domain accuracy in blue and OOD domain accuracy red. Results from the DFKD baselines are highlighted with gray rows. The accuracy drop compared to the pre-trained model is shown in brackets.
ID: CIFARIO OOD: STL
ID: CIFARIO OOD: Digits
ResNet-44-VGGG. VGG-11
VGG-13 (EG-13-ReeNeee ResNet-18
ResNet 34 VGG-11
VGG-13 ResNett-88
IAce (
(Ace(5))*
LAco (
OAce (%)
IAce (
OLAcc (%)
IAce (
OLAce (5)
NTL. Teacher
90.8gn1
10.4g00
92.7ga1
10.6gn1
91.1
100.0.an
92.9
99.5203
NAYER
10.3a04 (-80.5)
(-07)
18. (-74.)
(H0.0)
10.6100 (-80.5)
100.0a0p (-0.0)
08.0211 (-6.2))
(+1.5)
CRC
19.1 (-11.7
40.0gra (+29.6)
(-6.3)
20.8 (+10.2)
5.1
36.2g199 (-56.7)
7.8 (-91.7)
+ A
68.4221 (-22.4)
(+4..)
28.6u43 (-64.1)
18.7-19 (+8.1)
74.5 (-16.6)
5.5 (-94.5)
18.1 (-44.8)
1.3099 (-98.2)
33

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
E.7. Visualization
t-SNE visualization. We plot the feature distribution of the identified ID-like synthetic samples and OOD-like synthetic samples by using t-SNE feature visualization (Van der Maaten & Hinton, 2008). Results open-set, close-set, and backdoor compare the features of synthetic samples with those of real
tasks are shown in Figures 44 and 46. In each subfigure, ID and real OOD domain features. Before synthetic data grouping, we observe that synthetic samples' features partially overlap with both real ID and OOD domain features. In contrast, after synthetic data grouping, the features of identified fragile samples (regarded as ID-like samples) predominantly merge with real ID samples, as shown in Figures 44 to 46 (b). Similarly, the features of identified robust samples (regarded as OOD-like samples) align more closely with real OOD samples, as shown in Figures 44 to (c). These results demonstrate the effectiveness our synthetic data grouping strategy and further validate the adversarial robustness difference.
(a) SN-CIFARIO, NAYER
(b) SN-CIFARIO, NAYER, ID
(c) SN-CIIAAIO,, NAYER, OOD
Figure 44. Comparison of synthetic samples before and after ATEsC (DFKD by NAYER. T: ResNet34 and S: ResNet18). Green dots and red dots represent real ID and OOD domain samples, respectively, Purple transparent dots represent the synthetic samples.
(a) CIFARIO-ST STLIO, NAYER
(b) CIFARIO-STLAS NAYER, ID
(c) CIFARIO-STLIA NAYER, OOD
Figure 45. Comparison of synthetic samples before and after ATESC (DFKD by NAYER, T: ResNet34 and S: ResNet18), Green dots and red dots represent real ID and OOD domain samples, respectively. Purple transparent dots represent the synthetic samples.
(a) ((FARRl--adNee(rrii))
(b) CIFARIO-BBadNottee
(c) CIFARIO--BaNNe(grri), OOD
Figure 46. Comparison of synthetic samples before and after ATEsc (DFKD by NAYER, T: ResNet34 and S: ResNet18). Green dots and red dots represent real ID and OOD domain samples, respectively. Purple transparent dots represent the synthetic samples.
Synthetic samples by using ATEsC. In addition, we visualize the synthetic samples during distilling NTL teachers before and after using synthetic data grouping. The visualization results are shown in Figures 47 and 49. We can observe that original synthetic samples derived by DFKD on NTL teachers are likely to integrate both the ID domain and OOD domain
34

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
information (see Figures 47 to 49 (a)). After synthetic data grouping, the identified fragile samples, which are regarded ID>like samples, resemble real ID samples more closely (see Figures 47 to 49 (b)), while those robust samples appear more similar to real OOD samples, as shown in Figures 47 to 49 (c). These observations are consistent with expectations, demonstrating the effectiveness of our method.
2
0
B
8
2
U
3
8
67
(a) CMI w/o ATEsC
(b) ID-like samples in ATEsC
(c) OOD-like samples in ATEsc
Figure 47. Comparison of synthetic samples (CMI on NTL CIFARIO Digits. T: ResNet34 and S: ResNet18).
Digitss
(a) NAYER w/o ATÉSC Figure 48. Comparison of synthetic samples (NAYER on NTL CIFARIO Digits. T: ResNet34 and S: ResNet18).
(b) ID-like samples in ATEsc
(c) OOD-like samples in Al'Ssc
(a) CMI w/o ATESC Figure 49, Comparison of synthetic samples (CMI on NTL CIFARIO-Sig. T: ResNet34 and S: ResNet18).
(b) ID-like samples in ATBsc
(c) OOD-like samples in ATRsc
35

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
F. Further Discussion
F.. Distribution Shift in DFKD
There are some other works focus on the issue of distribution shift in DFKD, such as Do et al. (2022); Binici et al. (2022b;a); Wang et al. (2024b). In this section, we discuss the similarities and differences between our work and these previous works.
Do al. (2022); Binici al. (2022b:a); Wang et al. (2024b) investigate various types of distribution shifts that arise when performing DFKD on a standard supervised learning (SL) teacher trained solely on the ID domain:
Do et al. (2022); Binici et al. (2022b;a) focus on the non-stationary distribution problem, where shifts between synthetic distribution in different training stages (i.e., epochs) cause the catastrophic forgetting in the student model. This issue can be mitigated by memory bank-based strategies, where old synthetic samples are stored and will be replayed in future epochs (Do et al., 2022; Binici et al., 2022b) or by using an additional VAE (Kingma & Welling, 2013) to model the previously observed synthetic samples (Binici al., 2022a).
Wang et al. (2024b) addresses the distribution shift between synthetic data and real training data. They propose a novel perspective with causal inference (Von Kügelgen et al., 2021; Schölkopf et al., 2021; Lin et al., 2023; Zheng et al., 2024; Yao et al., 2025; Chen et al., 2024) to disentangle the student model from the impact of such shifts.
Our work explores the DFKD under NTL. teachers. We find that NTL teachers (pretrained on an ID domain and an OOD domain) result in OOD trap effect for DFKD. One key reason is the ID-to-OOD synthetic distribution shift, which is caused by the joint effect of the BN loss and adversarial exploration loss when training the generator in DFKD. Such a shift will not occur when distilling an SL teacher trained on the ID domain by using DFKD,
F.2 The Threat of Latent Inseparability
The concept of the latent inseparability was originally introduced in the adaptive backdoor attack (Qi et al., 2023), which aims to reduce the distinguishability between backdoor and clean features, thereby evading defenses that rely on the latent separability assumption (Tran et al., 2018; Chen et al., 2018; Tang et al., 2021; Hayase et al., 2021). In this section, we discuss how latent inseparability may emerge when performing DFKD on NTL-based backdoor teachers.
Synthesized ID-like data in DFKD may lead to adaptive backdoor attacks. Although the proposed calibrated knowledge distillation (CKD, Section 4.2) can filter out OOD-like samples (i.e., backdoor samples for NTlL-based backdoor teachers), some intermediate samples inevitably exist. These samples are not robust but can still activate the teacher's backdoor knowledge, enabling a certain degree of backdoor knowledge transfer from teacher to student. More seriously, the backdoor features in the student may exhibit latent inseparability, potentially leading to adaptive backdoor attacks (like (Qi et al., 2023)) the student. Specifically,
Some ID-like samples may still contain minor backdoor triggers.
During student training, such samples can be inadvertently perform adaptive backdoor attacks for the student. The attack manner is very similar to Qi et al. (2023) in two ways:
Weakened triggers: In Qi et al. (2023), weakened triggers used for training-tine -ime poisoning, while the original standard trigger would only be used during test time to activate the backdoor. Similarly, in our scenario, minor backdoor triggers are similar to weakened triggers.
Regularized poisoning: In Qi et al. (2023), not all samples with backdoor triggers are predicted to the target class. Similarly, in our scenario, since only minor trigger patterns are present, the NTL teacher may not always predict these samples to the target class.
Therefore, during DFKD, if we using the selected ID-like samples to training the student (i.e., the proposed CKD method), the NTL-based backdoor teacher potentially causes latent inseparability of clean and backdoor features for the student.
While our method does not explicitly identify adaptive backdoor features within the synthetic data, does include defense mechanism that mitigates their influence, i.e., the forgetting term Liorge. Actually, we have considered that some ID-like samples can still activate the teacher's misleading backdoor knowledge, enabling a certain degree of backdoor knowledge transfer from teacher to student (see Section 4.4). To further suppress misleading backdoor knowledge transfer, we introduce a forgetting term Lforget to maximize the divergence between the student's 's and the teacher's 's predictions on backdoor-like samples. Intuitively,
Backdoor-like samples contain strong backdoor triggers. NTL. teacher will predict them to the target class.
36

---

When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Therefore, maximizing the divergence between the student's and the teacher's predictions backdoor-like samples directly prevent the student from learning backdoor knowledge. This has the opposite effect the dlapfive-backdoor-attaddd formed by NTL teacher (which could potentially plant the backdoor to the student via weakened triggers).
As result, even if some synthetic ID-like samples contain adaptive backdoor features, the student is hard to still encode inseparable latent representations. The forgetting term Liorget effectively mitigates the risk of adaptive backdoor formation during DFKD.
Empirically, the effectiveness of the forgetting term is demonstrated in Tables 3. Compared using CKD alone, full ATEsc (i.e., CKID+Liage) achieves significantly lower ASR. Considering that the adaptive backdoor attack in Qi et al. (2023) still uses the original standard trigger during test time to activate the backdoor, our results already show its effectiveness against the adaptive backdoor attack (Qi et al., 2023) which is inadvertently formed during DFKD. More importantly, this forgetting term shows its effectiveness across all three NTL settings, highlighting its generality.
37


Reviews:
[{"review_id": "sXnHJBDgjk", "reviewer": "Reviewer_iM7M", "summary": "This paper identifies the OOD trap effect from NTL teachers to DFKD, i.e., misleading knowledge from OOD data may mislead students' learning process. The authors propose a plug-and-play ATEsc method to ensure that students can benefit from the NTL teacher model. This article can be considered as filling a gap in the field of DFKD, and the logic is clear, vivid, and easy to understand, accompanied by certain theoretical basis.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer iM7M,\n\nThanks for your positive opinions. We address your concerns as follows. If anything remains unclear, please do not hesitate to contact us.\n>**Q1: Discuss on distribution shift in DFKD**\n\nThanks for this valuable suggestion! [1-4] focus on different types of distribution shifts when distilling an SL teacher trained on the ID domain using DFKD.\n\n- [1-3] focus on the non-stationary distribution problem, where **shifts between synthetic distribution in different training stages (i.e., epochs) cause the catastrophic forgetting of the student models**. This issue can be mitigated by memory bank-based strategies, where old synthetic is stored and will be replayed in future epochs [1-2] or by using an additional VAE to model the previously observed synthetic samples [3]. \n- [4] address the **distribution shift between synthetic data and real training data**. They propose a novel perspective with causal inference to disentangle the student models from the impact of such shifts. \n\nOur work explores the DFKD under NTL teachers. We find that NTL teachers (pretrained on an ID domain and an OOD domain) result in OOD trap effect for DFKD. One key reason is the **ID-to-OOD synthetic distribution shift**, which is caused by the adversarial exploration of DFKD and the NTL's conflict learning target for ID and OOD domains. Such a shift will not occur when distilling an SL teacher trained on the ID domain using DFKD.\n\n>**Q2: Experiments on more backbones and larger datasets.**\n\nThe results with different network arch series for teachers and students are shown in Tab. A1-A4. In addition, the results on larger datasets are shown in Tab. A5. From the presented results, the proposed ATEsc is consistently effective in mitigating the OOD trap effect caused by NTL teachers.\n\n**Table A1:** Datasets: ID: CIFAR10 OOD: STL; NetArchs: ResNet-34→VGG-11\n||IAcc↑|OAcc↑|\n|-|-|-|\n|NTL Teacher|90.8±0.1|10.4±0.0|\n|NAYER|10.3±0.4(-80.5)|9.7±1.3(-0.7)|\n|+CKD|79.1±0.1(-11.7)|40.0±1.1(+29.6)|\n|+ATEsc|68.4±2.1(-22.4)|51.7±2.4(+41.3)|\n\n**Table A2:** Datasets: ID: CIFAR10 OOD: STL; NetArchs: VGG-13→ResNet-18\n||IAcc↑|OAcc↑|\n|-|-|-|\n|NTL Teacher|92.7±0.1|10.6±0.1|\n|NAYER|18.1±0.3(-74.6)|10.6±0.2(+0.0)|\n|+CKD|30.4±7.2(-62.3)|20.8±5.8(+10.2)|\n|+ATEsc|28.6±4.2(-64.1)|18.7±1.9(+8.1)|\n\n**Table A3:** Datasets: ID: CIFAR10 OOD: Digits; NetArchs: ResNet-34→VGG-11\n||IAcc↑|OLAcc↓|\n|-|-|-|\n|NTL Teacher|91.1±0.1|100.0±0.0|\n|NAYER|10.6±0.0(-80.5)|100.0±0.0(-0.0)|\n|+CKD|76.0±5.9(-15.1)|5.1±0.4(-94.9)|\n|+ATEsc|74.5±3.9(-16.6)|5.5±0.1(-94.5)|\n\n**Table A4:** Datasets: ID: CIFAR10 OOD: Digits; NetArchs: VGG-13→ResNet-18\n||IAcc↑|OLAcc↓|\n|-|-|-|\n|NTL Teacher|92.9±0.0|99.5±0.3|\n|NAYER|23.7±0.0(-69.2)|98.0±1.1(-1.5)|\n|+CKD|36.2±19.9(-56.7)|7.8±6.8(-91.7)|\n|+ATEsc|48.1±1.7(-44.8)|1.3±0.9(-98.2)|\n\n**Table A5:** Datasets: ID: CIFAR100 OOD: STL; NetArchs: ResNet-34→ResNet-18\n||IAcc↑|OLAcc↓|\n|-|-|-|\n|NTL Teacher|68.5±0.6|100.0±0.0|\n|NAYER|5.9±1.2(-62.6)|100.0±0.0(-0.0)|\n|+CKD|48.4±0.9(-20.1)|14.6±12.5(-85.4)|\n|+ATEsc|46.5±1.0(-22.0)|0.0±0.0(-100.0)|\n\n>**Q3: Stability of original CMI/NAYER and DFKD+ATEsc**\n\nThe performance of DKFD is sensitive to network initialization, even when distilling SL teachers. The sensitivity can be further increased when distilling NTL teachers. Empirically, our toy experiment in Appendix A offers initial evidence supporting this point (please compare Fig. 9-11).\n\nFor the BN layers, we mix the ID data and OOD data in a batch for training NTL teachers, resulting in the well-trained BN layers recording the mixture of statistical information of ID and OOD domains. All SOTA DFKD methods in our experiments rely on BN loss to *train the generator $G$*. Under the joint effect of BN loss and adversarial exploration loss, the synthetic distribution will face an ID-to-OOD synthetic distribution shift when distilling NTL teachers. This further leads to ID-OOD learning task conflicts for students, hindering the student's performance on ID domain. We argue that *distinguishing between ID-like and OOD-like synthetic data during the student's training* can directly solve the ID-OOD conflict issue for students. However, it may not help solve the problem of mismatched statistical information, as the student's training does not directly rely on the BN information from teachers.\n\n>**Q4: The cost of adversarial perturbation**\n\nWe use **one-step** untargeted PGD attack in our method, with the perturbation bound $\\epsilon=24$. Using NAYER as an example, in each epoch, we only synthesize 800 new samples after $G$'s training. We perform PGD on these samples and save them into ID-like or OOD-like pools. Thus, the additional time is only the time for performing PGD on 800*Epoch samples. As shown in Tab. A5, training NAYER 200 epochs on one RTX 4090 has only an additional 2 mins.\n\n**Table A5:** Datasets: ID: CIFAR10 OOD: STL; NetArchs: ResNet-34→ResNet-18\n||NAYER|+one-step PGD|\n|-|-|-|\n|Time|1h55m|1h57m|", "claims_and_evidence": "Most of the claims in this article have been validated.", "methods_and_evaluation_criteria": "Toy experiments and validation experiments can support textual conclusions.", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "The experimental results are relatively complete and the research is sufficient within the dataset scope mentioned by the author. However, to my knowledge, many DFKD methods can be applied to larger datasets, such as CIFAR-100, Tiny ImageNet, or ImageNet In addition, there are relatively few backbone combinations between teachers and students, and they are all homologous models, such as VGG or ResNet. This article introduces adversarial perturbations in training, which may introduce significant additional overhead, and the discussion in this section is missing. In addition, there are other methods to address the issue of data domain distribution offset. It would be even better if we could discuss the similarities and differences with these methods.", "relation_to_broader_scientific_literature": "The concept of teachers in this article was inspired by NTL (Wang et al., 2022).", "essential_references_not_discussed": "I have learned about some other work on distribution offset issues in DFKD tasks. It would be even better to discuss the similarities and differences between this article and these methods (only discuss without experiments).\n\n[1] Momentum adversarial distillation: Handling large distribution shifts in data-free knowledge distillation. NeurIPS 2022.\n\n[2] Preventing catastrophic forgetting and distribution mismatch in knowledge distillation via synthetic data. WACV 2022.\n\n[3] Robust and resource-efficient data-free knowledge distillation by generative pseudo replay. AAAI 2022.\n\n[4] De-confounded Data-free Knowledge Distillation for Handling Distribution Shifts. CVPR 2024.", "other_strengths_and_weaknesses": "Strengths:\n* This article focuses on the impact of NTL teachers and data distribution differences on student learning in DFKD tasks.\n* The writing is good, with clear logic and sufficient small-scale experiments to clarify the motivation and effectiveness of the method.\n\nWeaknesses:\nWeaknesses can refer to Essential References, Suggestions, and Questions.", "other_comments_or_suggestions": "* I suggest the author compare and discuss several works on data distribution shift in DFKD tasks, which will help readers further clarify the task setting and value of this article.\n* What would happen if teachers and students choose different model series, such as ViTs and CNNs, or different CNN series models.", "questions_for_authors": "* The performance of the original versions of CMI and NAYER seems to be very unstable, such as IAcc. What do you think is the main reason. As far as I know, the generation process of CMI relies on the statistical information of the BN layer, so it seems not suitable for experiments across datasets. After combining with ATEsc, good results were achieved. Does distinguishing between ID and OOD data during training help solve the problem of mismatched statistical information？\n* Is the default attack steps 5? How much training cost will this introduce？", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BLSegZpvTp", "reviewer": "Reviewer_saQK", "summary": "This paper investigates what would happen if one tried to apply data-free knowledge distillation (DFKD) on non-transferable learning (NTL) teachers. The paper proposes the OOD trap effect that the generator shifts to generate OOD samples, causing the student model only learns misleading OOD knowledge. To solve this problem, this paper proposes to first identify and filter out those OOD samples as they are more robust to untargeted adversarial attack. This simple strategy is called CKD and is fairly effective, as shown by the experiments. The authors further propose ATEsc which includes a regularization term aiming to further suppress OOD knowledge transfer. Experiments demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer saQK,\n\nThanks for your valuable reviews! We address your concerns as follows. Please let us know if anything remains unclear.\n>**Q1: Why generator has to output OOD-like samples**\n\nThis is because DFKDs commonly use BN loss [B1] as regularization for training generator $G$. Specifically,\n- **When training NTL**, each batch contains a mixture of ID and OOD samples. This results in that BN layers in pre-trained NTL teachers record statistical information for the mixture distribution of ID and OOD domains (mean $\\mu_l$ and var $\\sigma_l^2$ for layer $l$).\n- **When training $G$**, BN loss is represented as the divergence between synthetic statistics $\\mathcal{N}(\\mu_l(x_{syn}), \\sigma_l^2(x_{syn}))$ and teacher's BN statistics $\\mathcal{N}(\\mu_l, \\sigma_l^2)$. Minimizing BN loss lets the synthetic samples follow similar statistical information of teacher's training samples.\n\nThe joint effects of BN loss and adversarial exploration loss (Eq. 7 in the main paper) constraint the $G$ to:\n- (i) follow the statistics of NTL teacher's training data (i.e., a mixture of ID and OOD domains), and \n- (ii) maximize the discrepancy between student $S$ and NTL teacher. \n\nThus, if the $S$ only has ID domain knowledge, the $G$ will be optimized to synthesize OOD-like samples.\n\nIf w/o BN loss, the $G$ is unnecessary to generate OOD-like samples, and DFKD will not be influenced by OOD trap from NTL teachers. Unfortunately, their ID performance will be influenced by low quality of synthetic samples. Corresponding empirical evidence is shown in Tab. B1. More results and synthetic samples w/ and w/o BN are in https://anonymous.4open.science/r/icml408/figs.pdf (Tab. R1 and Fig. R1-R5). \n\n**Table B1:** ID: CIFAR10, OOD: STL; ResNet-34→ResNet-18\n||IAcc↑|OAcc↑|\n|-|-|-|\n|NTL Teacher|90.8±0.1|10.4±0.0|\n|CMI w/ bn|37.7±3.5(-53.1)|11.0±0.4(+0.6)|\n|CMI w/o bn|31.1±3.2(-59.7)|27.4±3.1(+17.0)|\n|NAYER w/ bn|48.7±1.1(-42.1)|10.4±0.0(+0.0)|\n|NAYER w/o bn|40.8±2.9(-50.0)|29.3±2.2(+18.9)|\n\n[B1] Dreaming to distill: Data-free knowledge transfer via deepinversion, CVPR'20\n>**Q2: Why assume a student only has ID knowledge first**\n\nSorry for the confusion. We aim to demonstrate that in adversarial exploration-based DFKD, even if a student $S$ learns only ID knowledge in *initial or some intermediate stage*, the $S$ will inevitably let the $G$ to synthesize OOD-like samples and then be taught to learn misleading OOD knowledge. \n>**Q3: Why there are vastly different adversarial robustness for close ID and OOD data**\n\nBecause the training manner of NTL causes significantly different **margins** (i.e., the minimal distance from a sample point to decision boundaries [B2]) for samples from even two similar domains:\n- **For ID domain**, the objective is to learn correct classification (e.g., 10-class classification for CIFAR10). This results in relatively complex decision boundaries between ID domain classes and small margins for ID data points.\n- **For OOD domain**, the objective is to force all data to be predicted as a single class (Eq. 5), and to maximize the domain discrepancy on both logits and features (Eq. 4). Predicting all samples to one class is simple, and no boundaries will go through the OOD clusters. Besides, maximizing the discrepancy on logits/features between domains further pushes the OOD cluster away from the ID domain clusters, resulting in very large margins for OOD data.\n\nAs a larger margin corresponds to stronger robustness [B2], we assume NTL teachers exhibit strong/fragile robustness against adversarial attacks on OOD/ID data. Empirical studies verify our assumption (Fig. 19-24 in main paper). We provide more evidence in https://anonymous.4open.science/r/icml408/figs.pdf (Fig. R8-R11) to analyze the effects of NTL's objectives for robustness. \n\n[B2] Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness, ICLR'23\n>**Q4: The forget term**\n\nIf the student $S$ and NTL teacher have similar predictions on OOD-like samples, it suggests that the $S$ has learned misleading OOD knowledge. By maximizing $\\mathcal{L}_{\\text{forget}}$, the $S$'s outputs on OOD-like samples are encouraged to diverge from the teacher's outputs, thus mitigating the misleading OOD knowledge transfer.\n\nThis term is effective in suppressing OOD knowledge transfer, as evidenced by the decreased OLAcc (Tab. 2) and the reduced ASR (Tab. 3). But it also degrades ID knowledge transfer. Such trade-off depends on whether the priority is on preserving ID transfer or preventing OOD transfer. \n>**Q5: Regular KD results**\n\nThanks! We present results of regular KD by ID data (KD-ID), OOD data  (KD-OOD) and ID+OOD data (KD-All) in Tab. B2. More results in https://anonymous.4open.science/r/icml408/figs.pdf (Tab. R2)\n\n**Table B2:** ID: CIFAR10, OOD: STL; ResNet-34→ResNet-18\n||IAcc↑|OAcc↑|\n|-|-|-|\n|NTL Teacher|90.8±0.1|10.4±0.0|\n|KD-ID|80.4±0.2(-10.4)|58.1±1.0(+47.7)|\n|KD-OOD|10.6±0.0(-80.2)|10.4±0.0(+0.0)|\n|KD-All|76.6±0.4(-14.2)|10.5±0.0(+0.1)|", "claims_and_evidence": "The proposed method heavily builds on the proposed OOD trap effect. Although empirical evidence demonstrates the OOD trap effect, the proposed reason behind such effect is not adequately supported. \n* In order for the OOD trap effect to happen, the generator has to generator samples closer and closer to the OOD samples. This paper explains this by \"the [generator] will be optimized toward synthesizing OOD-like samples to satisfy the maximization of the distribution discrepancy between [student] and [teacher].\" However, following this logic, the generator could just generate imperceptible samples to maximize the discrepancy (like adversarial examples), and it is not clear why the generator has to output OOD-like samples to maximize the discrepancy. \n* In line 195 right column: \"We assume a student S in DKFD currently only has ID domain knowledge.\" It seems to be the goal is to let the student learn ID domain knowledge. So why can we assume this in the first place? \n* The second explanation provide by this paper is the \"task conflicts.\" I.e., the teacher's outputs on ID and OOD data are very different, and thus the student faces \"task conflicts\" when trying to distill the teacher's knowledge. However, if there is indeed a \"task conflict,\" i.e., the ID and OOD data are close enough to be considered as conflicting task, why they have vastly different adversarial robustness? \n* The introduction of L_forget (12) seems ad-hoc, and that why maximizing L_forget can push the student model to forget about misleading knowledge of the teacher model? The with this term (ATEsc), it doesn't seem to be better than the vanilla CKD method.", "methods_and_evaluation_criteria": "Minor question: Although this paper is about data-free knowledge distillation, where a generator must be used. I'm curious to see how regular knowledge distillation would perform with NTL teachers.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "See previous discussions", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths: \nthis paper investigates an interesting problem, and the proposed OOD trap effect is very interesting. Especially there is potential impact if the phenomenon is well understood and well utilized. \n\nOther Weaknesses:\nThis paper is not well-written. Many grammar issues. \n\nFor example, \n* (line 14 right) \"due to the unavailable of...\"; \n* (line 186 left) \"We analysis on...\"; \n* (line 189 right) \"distribution D_s will occur distribution shift...\"", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See previous discussions", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TIGR7tmB9O", "reviewer": "Reviewer_gtda", "summary": "This work investigates the vulnerability of data-free knowledge distillation (DFKD) when the teacher model is untrusted, particularly in non-transferable learning (NTL) scenarios where knowledge transfer is blocked. It finds that NTL teachers mislead DFKD by shifting the generator’s focus from useful in-distribution (ID) knowledge to misleading OOD knowledge. To address this, the authors propose Adversarial Trap Escaping (ATEsc), which distinguishes between ID-like (fragile) and OOD-like (robust) synthetic samples based on adversarial robustness. Fragile samples aid normal distillation, while robust ones help forget misleading OOD knowledge, improving DFKD's effectiveness against NTL teachers.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer gtda, \n\nThanks for your valuable comments! We address the weaknesses below. Please kindly let us know if you have further concerns.\n\n>**Q1: Why do fragile samples align with ID data, and how do we distinguish ID and OOD data.**\n\nWe are sorry for the confusion. This assumption of \"fragile synthetic samples align with ID data\" is grounded in the widely accepted consensus within DFKD that BN loss is a necessary component. Due to the BN loss for training the generator in SOTA DFKD methods (like DFQ, CMI, and NAYER), the synthetic distribution will be constrained to have similar BN statistics with the teacher's training distribution. For more details on the BN loss, please find the answer to **Reviewer-saQK@Q1** (sorry for the limited space). \n\nAs such, when distilling NTL teachers, joint effects of BN loss and adversarial exploration loss let the synthetic distribution become similar to the mixture distribution of real ID and OOD domains. Therefore, according to our findings of the adversarial robustness difference of real ID and OOD domain samples against NTL teachers (refer to Section 4.1 in the main paper), we make the assumption that fragile synthetic samples are ID-like samples, and robust synthetic samples are OOD-like.\n\nEmpirically, the DFKD performance improvement by the introduced CKD (Tab. 1-3 in the main paper), visualization of original synthetic samples and ID-/OOD-like synthetic samples (Figs. 30-31 in main paper and Figs. R6-R7 in https://anonymous.4open.science/r/icml408/figs.pdf), t-SNE feature visualizations (Figs. 28-29 in main paper) can verify the effectiveness of our assumption.\n\n>**Q2: Our settings**\n\nWe sincerely apologize for the confusion. According to the tasks NTL can handle, we consider DFKD for three types of NTL teachers: **closed-set NTL**, **open-set NTL**, and **NTL-based backdoor**. The difference between these tasks lies in the data used for ID and OOD domains in training NTL teachers. Specifically, \n- **closed-set NTL**: In this setting, the ID and OOD domains have the same label space, but the two domains have a distribution shift. We choose the dataset-pairs of: SVHN→MNIST-M and CIFAR10→STL10;\n- **open-set NTL**: In this setting, the label spaces for the ID domain and OOD domain are disjointed. We perform experiments on SVHN→CIFAR10 and CIFAR10→Digits, where Digits is the combination of 5 digits datasets: MNIST, USPS, SVHN, MNIST-M, and SYN-D;\n- **NTL-based backdoor**: We additionally consider using NTL to conduct training controllable backdoor attacks. We use four triggers: BadNets(sq), BadNets(grid), Blended, Sig on CIFAR10 and see them as the OOD domain. The clean CIFAR10 is regarded as the ID domain.\n\n\nOur aim is to address the malign *OOD-Trap effect* introduced by NTL teachers in DFKD, and thus, regardless of NTL teacher's tasks, our goal is to transfer only the ID domain knowledge from NTL teacher to student. In addition, all NTL teachers are considered as **white-box** models during DFKD.\n\n>**Q3: Would creating conflicts between ID and OOD learning lead to a significant accuracy loss for the NTL model?**\n\nIt's true that NTL teachers have a certain degree of accuracy loss on either ID domain and any unseen domains. Empirically, we compare the performance of SL and Close-/Open-set NTLs on seen and unseen domains, as well as corrupt seen domain (with additive gaussian noise):\n\n**Table C1:** Accuracy loss of NTL. NetArchs: ResNet-34→ResNet-18\n||SL(CIFAR10)|Close-set NTL(ID: CIFAR10 OOD: STL)|Open-set NTL(ID: CIFAR10 OOD: Digits)|\n|-|-|-|-|\n|CIFAR10(test)|92.9|90.8|91.2|\n|CIFAR10(test)+gaussian(std=0.1)|90.6|45.7|76.7|\n|CIFAR10(test)+gaussian(std=0.2)|80.7|10.6|28.5|\n|STL(test)|68.8|10.4|17.3|\n\n>**Q4: Why is this method effective in defending against backdoor attacks?**\n- **NTL-based backdoor.** In our experiments, we consider using NTL in conducting training controllable backdoor attacks, where clean data (e.g., CIFAR10) is regarded as ID domain and clean data with backdoor triggers (e.g., CIFAR10 with BadNets) is seen as the OOD domain. Through our empirical findings (Fig. 4 (b-c) and Tab. 3 in our main paper), NTL-based backdoored teachers can transfer backdoors to students. This is also because of the *OOD-trap effects* as we mainly analyzed in Section 3 of our main paper.\n- **Defending against NTL-based backdoor.** Our ATEsc can effectively defend the backdoor transfer from teacher to student through the DFKD process. This is because in the NTL-based backdoor task, the OOD domain (e.g., CIFAR10+BadNets) and the ID domain (original CIFAR10) exhibit significant differences in their adversarial robustness against NTL teachers. As such, the proposed ATEsc can distinguish ID-like and OOD-like synthetic samples and let the student learn ID domain knowledge. Visualization results in Fig. 29 and Fig. 31 provide evidence for the effectiveness of distinguishing between ID and OOD synthetic data for NTL-based backdoor teachers.", "claims_and_evidence": "The claims made in the submission are partially supported by evidence, but certain aspects require further validation. Specifically, the use of fragile samples for ID knowledge transfer is not entirely convincing. The paper assumes that fragile samples (i.e., those with low adversarial robustness) are ID-like and thus beneficial for knowledge distillation. However, a model’s confidence on a sample does not necessarily correlate with its true distribution alignment—low-confidence predictions can be arbitrarily adjusted to fit almost any data. This raises concerns about whether the fragile samples truly retain ID knowledge or if they are simply artifact samples that do not generalize well to real ID data.", "methods_and_evaluation_criteria": "The datasets, problem, and application in this paper make sense and are relevant.", "theoretical_claims": "Yes, the mathematical formulations in the paper are generally correct and align with the theoretical claims.", "experimental_designs_or_analyses": "I have checked all the experiments and designs, but there are significant issues. The authors lack an in-depth analysis of fragile samples, particularly why they are considered high-confidence ID data. In my view, the fragile sample space is infinitely large, making their selection potentially unreliable.", "relation_to_broader_scientific_literature": "The authors are the first to propose an attack on non-transferable learning (NTL) teachers, making this work a novel contribution to the field of data-free knowledge distillation (DFKD) and model security.", "essential_references_not_discussed": "The authors have discussed most of the relevant prior work, covering key research on data-free knowledge distillation (DFKD), non-transferable learning (NTL), adversarial robustness, and model security. There do not appear to be major omissions in citing essential references.", "other_strengths_and_weaknesses": "A key strength of the paper is its originality, as it is the first to explore attacking non-transferable learning (NTL) teachers in the context of data-free knowledge distillation (DFKD). This makes it a novel and significant contribution to the field of model security and adversarial robustness.\n\nHowever, the paper lacks a thorough analysis of the effectiveness of the proposed method. Specifically, the assumption that low-confidence (fragile) samples represent ID knowledge is not well-supported. A more detailed theoretical or empirical analysis of why fragile samples align with ID data would strengthen the claims. Additionally, the paper does not provide sufficient justification for why ATEsc can effectively prevent backdoor attacks.", "other_comments_or_suggestions": "I consider the manuscript quality needs improvement, and the experimental analysis should be more comprehensive.", "questions_for_authors": "Q1. This paper is not clearly written. The authors should specify what knowledge is being extracted, which datasets can be used for evaluation, and whether the attacked model is in a white-box or black-box setting. Additionally, can the training data distribution of the model be inferred?\nQ2. The second question is about ID-OOD learning task conflicts. If the goal is to create conflicts between ID and OOD learning, would this lead to a significant accuracy loss for the NTL model, especially under the same label space? Has the paper analyzed datasets with different data distributions?\nQ3. Would adding noise affect the model's performance? Additionally, why is this method effective in defending against backdoor attacks?\nQ4. High-confidence data could be either OOD data or ID data. How do you distinguish between the two?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethics_expertise_needed": ["Privacy and Security"]}}]

Author rebuttal:

Meta-review: This paper studies what would happen if one tried to apply data-free knowledge distillation (DFKD) on non-transferable learning (NTL) teachers. The authors propose a new approach named Adversarial Trap Escaping (ATEsc), which benefits DFKD by identifying and filtering out OOD-like synthetic samples. 

Reviewers agreed that this paper studies an important problem, and the proposed method is novel. One reviewer pointed out that this paper can be considered as filling a gap in the field of DFKD, and the logic is clear, vivid, and easy to understand. Reviewers also raised some concerns regarding motivation, experiments, technical details, adversarial robustness, etc. The authors have provided very detailed responses to address these concerns.

After the Author-Reviewer discussion period, one reviewer raised a new concern about adaptive backdoor attacks, and the authors provided detailed explanations and justifications that are reasonable.

The authors are strongly encouraged to incorporate the comments and suggestions from reviewers to the camera-ready version, such as explaining why the generator generates samples increasingly closer to the OOD samples, discussing the potential scenarios with adaptive backdoor attacks, etc.
Decision: Accept (poster)