HODA: Protecting DNNs Against Model Extraction Attacks via Hardness of Samples

1 INTRODUCTION . Deep Neural Networks ( DNNs ) have shown impressive performance in various tasks in recent years that have encouraged the industry to deploy DNN-based models in a variety of real-world applications . Since the training process of DNNs and collecting training data is an expensive and tedious process , models are considered the intellectual property of organizations , and they must be kept secure . Therefore , models are often securely deployed on cloud servers , and only the creators can access the model parameters . Users are only allowed to query the model via a prediction API and receive predictions . Recent studies Tramèr et al . ( 2016 ) ; Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) ; Orekondy et al . ( 2019 ) ; Jagielski et al . ( 2020 ) demonstrate that an adversary can exploit the prediction API of a target model to create a surrogate model in order to steal or reconnoiter the functionality of the target model . Such attacks are called model extraction attacks , and they violate the intellectual property of model owners . Furthermore , the surrogate model can be leveraged to conduct other attacks on the target model in black-box setting , such as adversarial example attacks Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) and membership inference attacks Shokri et al . ( 2017 ) . Most of the model extraction attacks use the target model ’ s prediction API to label an unlabeled dataset to create the surrogate model ’ s training set . In most real-world settings , the adversary has no or limited access to samples from the target model ’ s training data distribution , which is called normal or in-distribution samples . Hence , most proposed attacks in the previous studies use some form of Out-Of-Distribution ( OOD ) samples , such as synthesis Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) or semantically similar samples to the target model ’ s training set Orekondy et al . ( 2019 ) ; Pal et al . ( 2020 ) to conduct model extraction attacks . We focus on such attacks in this paper . There are two main approaches to defend against model extraction attacks , manipulating the target model outputs to prevent adversary from producing high-quality surrogate model Lee et al . ( 2019 ) ; Orekondy et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) ; Kariyappa et al . ( 2021b ) and detecting the sample sequences of model extraction attacks Kesarwani et al . ( 2018 ) ; Juuti et al . ( 2019 ) . Juuti et al . ( 2019 ) propose PRADA to detect samples sequence of model extraction attacks based on the distance among successive samples . We propose Hardness-Oriented Detection Approach ( HODA ) , a new approach to detect sample sequences of model extraction attacks , which outperforms PRADA by a large margin and has significantly less computational overhead . Generally , the training process of DNN-based classifiers is done in several epochs , and the resulted classifier at the end of the last epoch is considered the final classifier . We can consider the training process of DNN-based classifiers as a sequence of subclassifiers in which the ith subclassifier is created at the end of the ith epoch . HODA uses a subset of subclassifiers to compute the hardness degree of samples . The hardness degree of a sample is equal to the index of the subclassifier that all subsequent subclassifiers agree with its predicted label for that sample . It is important to note that we must save subclassifiers in the training phase of a target classifier in order to use their predictions to calculate the hardness degree of new samples . We demonstrate that the hardness degree histogram of normal sample sequences is distinguishable from the hardness degree histogram of model extraction attack sample sequences , and HODA uses this property to detect sample sequences of model extraction attacks . For each user , HODA calculates the distance between the hardness degree histograms of the user ’ s samples and normal samples , and if the distance is greater than a threshold , the user is detected as an adversary . HODA can detect JBDA Papernot et al . ( 2017 ) , JBRAND Juuti et al . ( 2019 ) , and Knockoff Net Orekondy et al . ( 2019 ) attacks with a high success rate by only monitoring 100 samples of attack . We demonstrate that HODA is also highly effective when the target classifier is trained using transfer learning . Contributions . ( i ) We demonstrate that the hardness degree of a sample for a classifier pertains to the training data distribution of that classifier . ( ii ) We indicate that the hardness degree histogram of normal samples is distinct from the hardness degree histograms of model extraction attack samples . ( iii ) We propose HODA to detect the sample sequences of model extraction attacks . 2 RELATED WORK . Model Extraction Attacks : Primary model extraction attacks try to extract the exact value of parameters Lowd & Meek ( 2005 ) ; Tramèr et al . ( 2016 ) and hyperparameters Wang & Gong ( 2018 ) of shallow models . In recent years , the proposed attacks mainly aimed to steal or reconnoiter the functionality of deep neural networks by querying them in the black-box setting . It is often sensibly assumed in the literature that the adversary has no or limited access to samples from the training set distribution of target classifier . In order to overcome this issue , attacks generally use some form of out-of-distribution samples , such as synthesis or semantically similar samples , to create the surrogate classifier ’ s training set . Knockoff Net Orekondy et al . ( 2019 ) , ActiveThief Pal et al . ( 2020 ) , and Copycat CNN da Silva et al . ( 2018 ) use semantically similar datasets to the target model ’ s training set to train a surrogate classifier . In another line of studies , Papernot et al . ( 2017 ) , Juuti et al . ( 2019 ) , Yu et al . ( 2020 ) , Truong et al . ( 2021 ) , Kariyappa et al . ( 2021a ) , and Barbalau et al . ( 2020 ) use synthetic data to create the surrogate classifier ’ s training set . Model Extraction Defenses : Existing defense methods against model extraction attacks generally distribute into two branches : perturbation-based and detection-based . Perturbation-based defenses Lee et al . ( 2019 ) ; Orekondy et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) attempt to prevent adversaries from producing high-quality surrogate classifiers by adding perturbation to the target classifier outputs . Recently , Kariyappa et al . ( 2021b ) proposed a new defense with the same goal as perturbation-based defenses , which does not perturb the target classifier outputs . Their approach employs an ensemble of diverse models to produce discontinuous predictions for out-of-distribution samples . Detection-based defenses attempt to detect the occurrence of model extraction attacks by observing successive input queries to the target classifier . Kesarwani et al . ( 2018 ) present a method to detect extraction attacks against Decision Tree models . PRADA Juuti et al . ( 2019 ) is the first proposed detection-based defense for DNN models . We propose a new defense detecting the sample sequences of model extraction attacks via hardness of samples . Atli et al . ( 2020 ) demonstrate that several OOD detection approaches , such as Baseline Hendrycks & Gimpel ( 2017 ) and ODIN Liang et al . ( 2018 ) , have poor performance in detecting Knockoff Net attack samples . Hence , they propose a new OOD detection approach that leverages a classifier to detect OOD samples . However , their approach only rejects OOD samples , and it does not have any detection mechanism to detect adversaries . Besides , the OOD detector is trained on samples from the same distribution used by the adversary to conduct Knockoff Net attacks , which is an unrealistic assumption in practice . Concurrent with our work , Zhang et al . ( 2021 ) and Pal et al . ( 2021 ) propose SEAT and VarDetect to detect sample sequences of model extraction attacks , respectively . SEAT aims to detect model extraction attacks that use several similar samples to extract a target model , such as jacobian-based attacks ( Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) ) . Hence , SEAT is ineffective when an adversary uses natural samples that are not similar to each other , such as Knockoff Net attack . VarDetect uses Variational Autoencoders ( VAs ) and Maximum Mean Discrepancy ( MMD ) to detect model extraction attacks . VarDetect has only been evaluated on low-dimensional datasets . Regarding that VarDetect uses VAs and MMD , it is unclear how well it performs on high-dimensional datasets . Besides , it uses the ImageNet dataset to extract target classifiers trained on very structurally different datasets , such as F-MNIST and SVHN . HODA is evaluated on harder attacks using attack datasets that are structurally similar to the target classifier ’ s dataset , such as using ImageNet dataset to extract Caltech256 target classifier . HODA can detect both jacobian-based and Knockoff Net attacks , and it performs well on high-dimensional datasets , such as Caltech256 and CUB200 . Furthermore , unlike other work Atli et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) ; Kariyappa et al . ( 2021b ) , HODA only needs access to in-distribution samples to detect model extraction attacks . 3 MODEL EXTRACTION ATTACKS . The model extraction attack is one of the most serious threats against machine learning-based classifiers on remote servers , such as Machine Learning as a Service ( MLaaS ) . The adversary ’ s goal is to create a surrogate classifier fs that imitates a target classifier ft on task T . Most model extraction attacks exploit target model ft to label unlabeled samples to create the surrogate model ’ s training set . The adversary sends sample xi to the target model and receives its output ft ( xi ) , and then she uses pair ( xi , ft ( xi ) ) to train surrogate classifier fs . The output type of target model can be label , label confidence , top-k values in probability vector , or the entire probability vector . We only consider label f̄t ( xi ) and the entire probability vector ft ( xi ) as the output type of target classifiers in our experiments . There are two primary intents for adversaries to conduct model extraction attacks , stealing and reconnaissance . Stealing : Producing a high performance classifier is an expensive and time-consuming process and requires computational resources and experts . Besides , given that DNNs need a large number of training samples , collecting data and labeling them is a complex and costly procedure for most realworld applications . Therefore , adversaries are motivated to take advantage of a target classifier to reduce the cost of creating a new classifier . The adversary ’ s goal in stealing is to maximize the accuracy of surrogate model on data distribution DT . Hence , the adversary ’ s goal is : Maximize P ( x , y ) ∼DT f̄s ( x ) = y ( 1 ) Reconnaissance : The model extraction attacks can be used to conduct other attacks in the blackbox setting , such as adversarial example attacks Papernot et al . ( 2017 ) ; Goodfellow et al . ( 2015 ) and membership inference attacks Shokri et al . ( 2017 ) . The adversary ’ s goal in reconnaissance is to maximize the fidelity among surrogate and target classifiers in order to increase the success rate of black-box attacks . Similar to Jagielski et al . ( 2020 ) , we consider label agreement among surrogate and target classifiers as the fidelity metric on data distribution DT . Hence , the adversary ’ s goal is : Maximize P ( x , y ) ∼DT f̄s ( x ) = f̄t ( x ) ( 2 ) Proposed model Extraction attacks create the surrogate classifier training set Xs = { ( xi , ft ( xi ) ) } Bi=1 by various methods , where B is the attack budget . The attack budget determines the number of samples that an adversary is allowed to send to the target classifier and receive their associated predictions . After creating Xs , the adversary trains surrogate classifier fs to minimize empirical loss on Xs . We suppose that the adversary knows the architecture and hyperparameters of the target classifier and uses them to train the surrogate classifier . It is noteworthy that our proposed defense is independent of surrogate classifiers ’ training process .

Summary: In this paper, the authors propose HODA (hardness oriented detection approach) to detect model extraction attacks against DNNs. They use the checkpoints during training of the model to define hardness of a sample. Hardness of a sample is the epoch from which the model assigns the same class to the sample. During inference, Pearson distance between the histograms of holdout set samples and test samples is used to detect an attack. HODA is evaluated on image datasets and models. It is shown to detect three classes of attacks and is analyzed against an adaptive attack.