Protecting DNNs from Theft using an Ensemble of Diverse Models

1 INTRODUCTION . MS attacks allow an adversary with black-box access to the predictions of the target model to copy its functionality and create a high-accuracy clone model , posing a threat to the confidentiality of proprietary DNNs . Such attacks also open the door to a wide range of security vulnerabilities including adversarial attacks ( Goodfellow et al. , 2014 ) that cause misclassification , membership-inference attacks ( Shokri et al. , 2017 ) that leak membership , and model-inversion attacks ( Fredrikson et al. , 2015 ) that reveal the data used to train the model . MS is carried out using the principle of Knowledge distillation ( KD ) , wherein the adversary uses a dataset D to query the target model . The predictions of the target onD are then used to train a clone model that replicates the target model ’ s functionality . Since access to training data is limited in most real-world settings , attacks typically use some form of OOD data to perform KD . Clone models trained in this way closely approximate the decision boundaries of the target model , achieving high-accuracy on in-distribution examples . The goal of this paper is to defend against MS attacks by creating a target model that is inherently hard to steal using Knowledge Distillation with OOD data . Our key observation is the existing MS attacks ( Orekondy et al. , 2019a ; Papernot et al. , 2017 ; Juuti et al. , 2019 ) implicitly assume that the target model produces continuous predictions in the input space . We hypothesise that making the predictions of the target model discontinuous makes MS attacks harder to carry out . To this end , we propose Ensemble of Diverse Models ( EDM ) to defend against MS attacks . The models in EDM are trained using a novel diversity loss to produce dissimilar predictions on OOD data . Each input query to EDM is serviced by a single model that is selected from the ensemble using an input-based hashing function . We develop a DNN-based perceptual hashing algorithm for this purpose , which is invariant to simple transformations of the input and prevents adaptive attacks . Since different models in the ensemble are used to service different queries and the models are trained to produce dissimilar predictions , the adversary obtains predictions that are highly discontinuous in the input space . The clone model , when trained on these predictions , tries to approximate the complex discontinuous decision boundary of the EDM . Our empirical evaluations show that the resulting clone model generalizes poorly on in-distribution data , degrading clone accuracy and reducing the efficacy of MS attacks . In contrast to existing defenses that rely on perturbing output predictions , our defense does not require modifying the output and instead uses a diverse ensemble to produce discontinuous predictions that are inherently hard to steal . We illustrate the working of our defense by comparing the results of MS attacks on two target models– ( i ) Undefended baseline ( ii ) EDM – trained on a toy binary-classification problem shown in Fig . 1 . The training data and predictions of the target model under attack are shown in Fig . 1a . We use a set of OOD points as shown in Fig . 1b to query the target model and obtain its predictions . The predictions of the target model on the OOD data is finally used to train the clone model to replicate the functionality of the target as shown in Fig . 1c . For the undefended baseline target , which uses a simple linear model to produce continuous predictions , the clone model obtained by the attack closely approximates the decision boundary of the target model , achieving good classification accuracy on in-distribution data . The EDM target consists of two diverse models ( Model-A and Model-B ) that produce dissimilar predictions on the OOD data . EDM uses an input-based hash function to select the model that services the input query . As a result , the target model produces highly discontinuous decisions on OOD data ( Fig . 1a , b for EDM ) . The clone model trained on this data fails to capture any meaningful decision boundary ( Fig . 1c for EDM ) and produces poor accuracy on the classification task , making MS attacks harder to carry out . In summary , our paper makes the following key contributions : 1 . We propose a novel Diversity Loss function to train an Ensemble of Diverse Models ( EDM ) that produce dissimilar predictions on OOD data . 2 . We propose using EDM to defend against model stealing attacks . Our defense creates discontinuous predictions for the adversary ’ s OOD queries , making MS attacks harder to carry out , without causing degradation to the model ’ s accuracy on benign queries . 3 . We develop a DNN-based perceptual hash function , which produces the same hash value even with large changes to the input , making adaptive attacks harder to carry out . 2 RELATED WORK . Model Stealing Attacks : MS Attacks fall into one of three categories : ( i ) parameter stealing , ( ii ) hyper-parameter stealing and ( iii ) functionality stealing attacks . Parameter stealing ( Tramèr et al. , 2016 ; Lowd & Meek , 2005 ) and hyperparameter stealing attacks ( Wang & Gong , 2018 ; Oh et al. , 2019 ) focus on inferring the exact model parameters or hyperparameters used in the architecture/training of the target model respectively . Our work focuses on defending against functionality stealing attacks , where the adversary ’ s goal is to train a clone model that achieves high accuracy on a given task . Since the training data of the target is usually unavailable , attacks leverage alternate forms of data to query the target . Orekondy et al . ( 2019a ) ; Correia-Silva et al . ( 2018 ) use datasets from related problem domains , while other attacks ( Papernot et al. , 2017 ; Juuti et al. , 2019 ; Roberts et al. , 2019 ; Kariyappa et al. , 2020 ) have proposed crafting synthetic examples to query the target to perform model stealing . Model Stealing Defenses : Existing defenses that have been proposed to defend against model functionality stealing involve perturbing the output predictions to prevent the adversary from having reliable access to the predictions of the target model . Perturbation-based defenses modify the output of the model y = f ( x ) to produce a perturbed output y′ = y + δ . These defenses can be further sub-categorized into accuracy-preserving and accuracy-constrained defenses . Accuracy-Preserving Defenses ( Lee et al. , 2018 ) ensure that the class-labels of the perturbed predictions y′ are the same as the original prediction i.e . argmax ( y′ ) = argmax ( y ) . Since the class-labels are unchanged , the benign accuracy of the model is preserved . In contrast , accuracyconstrained defenses ( Kariyappa & Qureshi , 2020 ; Orekondy et al. , 2019b ) allow for the usage of a higher level of perturbation , trading off benign accuracy for increased security against MS attacks . This work presents the first defense that is not reliant on adding perturbations to the predictions of the model . Instead , we aim to create an entirely new class of ensemble-based models that is inherently hard to steal with a data-limited attack , due to the discontinuous nature of the predictions produced by the model . Our method has minimal impact on the benign accuracy and offers a much higher security against MS attacks compared to a single model . Additionally , our model can be used in conjunction with perturbation-based defenses to further improve robustness against MS attacks . 3 PRELIMINARIES . 3.1 ATTACK : OBJECTIVE AND CONSTRAINTS . The goal of the attacker is to maximize the accuracy of a clone model C on a test set Dtest i.e . maxθC Ex , y∼Dtest [ Acc ( C ( x ; θC ) , y ) ] . MS attacks use the principle of knowledge distillation to train a clone model . First , the attacker queries the target model using inputs { xi } ni=1 to obtain the target ’ s predictions { yi = T ( xi ) } . The labeled dataset { xi , yi } created this way can then be used to train a clone model C , completing the attack . The predictions returned from T can either be soft-label prediction , where T returns a probability distribution over the output classes , or hardlabel predictions , where only the argmax class-label is returned . We consider attacks under both prediction types in this paper . Since there is a fixed cost associated with querying T , we assume that the number of queries is upper bounded by a query budget Q . Data Constraints : In most real-world settings , the adversary does not have access to the dataset used to train the target model . A data-limited attacker can use alternate forms of data to carry out model stealing attacks . KnockoffNets ( Orekondy et al. , 2019a ) uses data from an alternate surrogate dataset , which has semantic similarities with the target model and Jacobian-Based Dataset Augmentation ( Papernot et al. , 2017 ) uses synthetic datasets to query the target model . 3.2 DEFENSE : OBJECTIVE AND CONSTRAINTS . The goal of the defense is to minimize the accuracy of the clone model i.e . minEx , y∼Dtest [ Acc ( C ( x ; θC ) , y ) ] . The defense is allowed to make changes to the target model or the model predictions . However , we require the defense to have minimal impact on the accuracy of the model for benign inputs to preserve the utility of the model . The defender is unaware of the kind of attack being used , or the dataset used by the adversary to query the target model . In addition to existing attacks , the defense also needs to be effective against adaptive attacks that are tailored for the defense . 4 OUR PROPOSAL : ENSEMBLE OF DIVERSE MODELS . This paper proposes a novel type of model ensemble called Ensemble of Diverse Models ( EDM ) , which is significantly more robust to MS attacks compared to a single DNN model . EDM consists of a set of diverse models , which produce dissimilar predictions for OOD data . By using an inputbased hash function , EDM selects a single model from the ensemble to service each query . Since the models are trained to produce dissimilar predictions , EDM outputs predictions that are highly discontinuous in the input space for OOD data . The complexity of the discontinuous predictions cause the clone model trained on these predictions to generalize poorly on in-distribution examples , making MS attacks less effective . We explain how EDM models can be trained and deployed for inference in this section . 4.1 TRAINING A DIVERSE ENSEMBLE EDM leverages an ensemble of diverse models to produce discontinuous predictions . Let { fi } i=Ni=1 denote the ensemble of models used in EDM with model parameters { θi } i=Ni=1 . These models are jointly trained with a two-fold training objective using an in-distribution dataset of labeled examples Din and an out-ofdistribution dataset of unlabeled examplesDout as explained below : 1 . Accuracy objective : Models in the ensemble should be trained to produce high accuracy for the examples in the training dataset of in-distribution examples Din as shown in Fig . 2a . For a multi-class classification problem , let { ŷi = fi ( x ; θi ) } Ni=1 be the prediction probabilities of the models where ( x , y ) ∈ Din is the input , label pair . We use the cross-entropy loss averaged across all models : 1 N ∑N i=1 LCE ( ŷi , y ) to train the models in the ensemble to achieve high accuracy on the in-distribution training data . 2 . Diversity objective : The diversity objective requires the models in the ensemble to produce dissimilar predictions for examples in the out of distribution dataset Dout as depicted in Fig . 2b . If { ỹi = fi ( x̃ ; θi ) } Ni=1 are the output probabilities produced by the ensemble for an OOD input x̃ ∈ Dout , then the diversity objective requires the set of output vectors { ỹi } Ni=1 to be misaligned . We use the coherence metric ( Tropp , 2006 ) to measure the alignment of the output probability vectors . Coherence of a set of vectors measures the maximum cosine similarity ( CS ) i.e . cosine of the smallest angle , between all pairs of probability vectors in the set as shown in Fig . 3 . Coherence can be computed as follows : coherence ( { ỹi } Ni=1 ) = max a , b∈ { 1 , .. , N } a 6=b CS ( ỹa , ỹb ) . ( 1 ) In order to increase the misalignment of probability vectors , we need to reduce the value of coherence . Unfortunately , the max function in Eqn 1 makes coherence non-smooth , which precludes the use of first-order methods to optimize the value of coherence . To overcome this limitation , we use the LogSumExp function to get a smooth approximation of the max function that allows it to be used in first-order optimization algorithms . We term the resulting loss function as the diversity loss : DivLoss ( { ỹi } Ni=1 ) = log ( ∑ 1≤a < b≤N exp ( CS ( ỹa , ỹb ) ) ) . ( 2 ) Models in the ensemble are jointly trained on the combination of accuracy and diversity objectives : L = E x , y∼Din , x̃∼Dout [ ( 1 N N∑ i=1 LCE ( ŷi , y ) ) + λD ·DivLoss ( { ỹi } Ni=1 ) ] , ( 3 ) where ŷi = fi ( x ) , ỹi = fi ( x̃ ) . Here λD is a hyperparameter that dictates the ratio of importance between the diversity and accuracy objectives . Note that the loss function described above requires access a dataset of labeled in-distribution examples Din as well as a dataset of out-of-distribution examples Dout . Since we do not know the OOD samples that is used by the adversary a-priori , we make use of an auxiliary OOD dataset to train EDM . Similar use of auxiliary OOD datasets have been explored in anomaly detection literature ( Hendrycks et al. , 2018 ) to detect unseen anomalies . Our experiments in Section 5.3 suggest that the diversity objective generalizes to the unseen OOD dataset used by the adversary .

Summary: The paper proposes a method to protect deep neural networks against model stealing. The propose defense trains an ensemble of classifiers using two losses one targeting accuracy and the other diversity of the ensemble. In particular, the trained classifiers are consistent on in-distribution data, but contradict each other on out-of-distribution data. The proposed defense does not affect the test accuracy of the victim model while it strongly limits the test accuracy of the clone model.