Analyzing the Confidentiality of Undistillable Teachers in Knowledge Distillation

1 Introduction . Knowledge distillation ( KD ) [ 10 ] aims to transfer the useful knowledge of a trained model ( the teacher ) to another model ( the student ) . KD has found success in various applications [ 1 , 29 , 3 , 17 ] and is particularly useful for resource-constrained IoT applications where the compute budget is limited and compute-efficient models are required . Generally , KD requires the student model to be trained over the same data-set that is used to train the teacher . However , recent research [ 31 , 20 , 2 ] has shown the efficacy of KD even under the `` data-free '' scenario where the training data may not be available for the student to get trained . Over the past few years various forms of distillation have been proposed , including distillation from the student itself [ 36 ] and via an ensemble of students [ 37 ] . However , recently , reference [ 18 ] has highlighted the fact that many forms of distillation may unintentionally leak the IP of a teacher model . In particular , some teacher models contain significant IP associated with the arduous effort of both data collection and model training , which motivates their release as “ black-box '' executable pieces of software . Moreover , these trained models may even enable safeguarding the training data ( for example , sensitive medical images [ 28 ] and company proprietary information [ 27 , 30 ] ) as well as their performance on the secure data . Use of KD under these circumstances , sometimes 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . under the “ data-free '' condition , may enable training of an unauthorized student to yield comparable performance as the teacher . To mitigate this issue , reference [ 18 ] has proposed the idea of a nasty teacher that prevents knowledge leaking to a student and thereby reduces the student ’ s classification performance . Fig . 1 ( a ) depicts the the success of a nasty ResNet50 in degrading various student models ’ classification performances compared to their respective baseline performances1 . Our contributions : Earlier evaluations [ 18 ] have shown that nasty teachers can retain their efficacy under various settings of two key hyperparameters , namely the weight of the distillation loss ( α ) and the softmax temperature ( τ ) of the distillation loss . However , a comprehensive evaluation of the efficacy of undistillable nasty teachers has yet to be completed . Towards this goal , we investigate the performance of KD on distillation at different depths of the student model , and in particular , how the teacher ’ s influence changes when transferring knowledge to an intermediate shallow section of the student through an auxiliary classifier ( AC ) . We find that the impact of the nasty teacher drastically reduces as we transfer knowledge to a shallow subsection of the student ( Fig . 1 ( c ) ) . Based on these findings , we present a skeptical student that uses an intermediate shallow auxiliary classifier to transfer the information derived through the soft probabilities of the teacher ’ s output classes . We further propose a novel hybrid distillation scheme to improve learnability of the student by distilling both from a teacher and the student itself . Our approach has some similarity with selfdistillation ( SD ) [ 36 ] because both approaches use an auxiliary classifier for the knowledge transfer . However , the goal of SD is to show the efficacy of a model distilling from itself , contrasting our goal of analyzing the possible presence of a potential model stealer who can extract knowledge even from an undistillable nasty teacher . More importantly , the proposed hybrid distillation is effective in stealing a model ’ s IP even under a “ data-free '' scenario , contrasting SD which is only applicable for students who have access to training data . We conduct extensive experiments using both standard KD with available training data on CIFAR-10 , CIFAR-100 , and Tiny-ImageNet , and data-free KD on CIFAR-10 testset . Experimental results show that compared to normal ones , skeptical students exhibit improved performance of up to ∼59.5 % and ∼5.8 % for data-available and data-free KD , respectively , when distilled from nasty teachers . This exposes a significant limitation of nasty teachers attempting to protect model IP . Moreover , our proposed students perform similar to normal student models while distilled from normal teachers , demonstrating their efficacy irrespective of the teacher being nasty or not . 2 Preliminaries . 2.1 Knowledge Distillation . Knowledge distillation , similar to the goal of various model compression techniques [ 13 , 7 , 5 , 14 , 16 ] , embeds the rich information of a compute-heavy model into a model that generally requires fewer computations . The traditional KD [ 10 ] relies on information transfer through a Kullback–Leibler ( KL ) divergence measure between the soft logits at the output classifier layers of the teacher and the student . Apart from this , over the past few years , various other efficient distillation methods have been proposed , including distilling from hints provided by the teacher [ 25 ] , distilling via attention transfer [ 34 ] , and other approaches [ 23 , 21 ] . To reduce the distillation-based training time and avoid the requirement of a separate teacher , reference [ 36 ] has proposed self-distillation . The authors partition 1The baseline models are trained with only cross-entropy ( CE ) loss . the student model into several shallow sections , each having its own auxiliary classifier , to which the final classifier transfers the soft-logits to enhance the model ’ s classification performance . Several recent studies [ 32 , 33 ] have also shown the efficacy of KD to a student from its own pre-trained variant as its teacher . Similar to [ 18 ] , in this work , we assume to have no access to the teacher model ’ s intermediate features . Rather , we focus on the information leaking to an unauthorized outsider and thus rely on standard KD . 2.2 Model IP protection . Protection of model IP has drawn significant interest primarily due to the massive human resource and financial costs required for large model training . Earlier works explored various defense strategies , including adaptive misinformation against model performance stealing [ 11 ] and passport-based defense [ 35 ] . In most of these cases , the stealer is assumed to have access to only synthetically generated data . However , also of interest is when a portion of training data is unintentionally leaked . 2.3 Poisoning of Neural Network Models . An attacker can degrade the performance of a neural network by simply injecting poisoned data [ 22 ] into the training set . Adversarial-attack generated images [ 6 , 19 , 15 ] have proven to be effective in degrading a model ’ s performance . Backdoor attacks [ 4 ] insert crafted malicious data into the training set that apparently trains the model to perform well until such time that the attacker sets up a signal that degrades the model performance drastically . The Bit flip attack [ 24 ] corrupts selective bits of the trained DNN weights to lower its performance . As mentioned earlier , reference [ 18 ] has recently proposed to poison a neural network model through training , such that the model retains its classification performance , but loses its ability to be used as an efficient teacher , thus referred to as a nasty teacher . These nasty teachers are believed to protect the model IP.2 This work analyzes the degree of IP protection that such teachers provide and , in particular , presents a skeptical student-based distillation technique that diminishes the effect of their nastiness , as detailed in the next Section . 3 Motivational Case Study . To motivate our skeptical students , this section presents an empirical analysis that explore the efficacy of nasty teacher models under two distinct KD scenarios3 . 3.1 Transferability of the Impact of Nasty Teachers . Definition 1 . Secondary student : We define a secondary student as a model that is trained via KD from a trained model which was earlier trained via distillation from a teacher model . In this context , we refer to the student that is distilled from the original teacher model , as the primary student . We measure the transferability impact ( TI ) of a teacher as the performance improvement ( or degradation ) of a secondary student with respect to its baseline ( ∆base ) . A negative ∆base signifies the success of a teacher ’ s privacy or confidentiality preserving effort [ 18 ] . 2In so far as a teacher is trying to protect its private IP from inquiring or even intrusive students , we believe a better phrase to characterize him/her is as a “ defensive teacher '' or in the worst case as a “ secretive teacher '' . In our view , there is nothing “ nasty '' about what the teacher is attempting to do . However , because the prior art has consistently used the term “ nasty teacher , '' we will also use that phrase in this paper . 3The description of training hyperparameters is given in Section 5.1 for all the experiments in this section . We first trained both normal as well as nasty variants of a ResNet50 on CIFAR-100 . We then used two ResNet18 students to distill knowledge from these two teachers . Finally , to test TI we used the distilled ResNet18 models as teachers to secondary students with identical model architectures . Observation 1 . The TI of a nasty teacher is negative , meaning the nastiness of a teacher transfers to its student . Table 1 shows the classification performance of a secondary student is 6.56 % lower than the baseline while using a distilled ResNet18 as teacher . Interestingly , this implies that the false sense of generalization that the primary student inherits makes it become a nasty teacher . Thus simple redistillation from a primary to a secondary student will not evade the nastiness of a teacher . 4 3.2 Transferring Knowledge to a Shallow Subsection of the Student . We computed the weighted KL-divergence based distillation loss at an auxiliary classifier ( AC ) of the student , and used its final classifier ( C ) to compute the weighted CE-loss only . We evaluated the impact of the auxiliary classifier placed at different depths of the student model . In particular , for the ResNet18 student we performed four experiments by placing the AC after the nth BB , n ranging from 1 to 4 , where the model ’ s final classifier is located at the end of 4th block . We follow similar procedure as [ 36 ] to design the AC branches . Observation 2 . The influence of a teacher ( both nasty and normal ) on a student ’ s performance reduces as we distill knowledge to a shallow subsection of the student model . Fig . 2 shows the accuracy of the student model approaches the baseline accuracy as we distill from the teacher at shallower depths ( n = 1 ) . Interestingly , this trend can be observed for both nasty and normal teachers . In particular , from a nasty ResNet50 , the student ResNet18 has a test accuracy of up to 77.19 % while distilled at AC 1 , in contrast the accuracy is 72.47 % when distilled at the final classifier . On the other hand , ResNet18 can have a classification accuracy reduction of as much as 1.56 % when distilled at shallow depths . We leverage these two observations in our hybrid distillation on the skeptical students and diminish the transferring of teachers ’ nastiness , presented in the next section .

Summary: Knowledge distillation (KD) can be used by an adversary to obtain sensitive information from a teacher model presenting privacy risks. Nasty teachers were proposed to handle this problem where students were unable to learn or distill knowledge from nasty teachers. In this paper, the authors show that skeptical students using a combination of KD to intermediate shallow subsections of their model together with Self Distillation (SD) can extract information from a nasty teacher and perform well on classification tasks.