HODA: Protecting DNNs Against Model Extraction Attacks via Hardness of Samples

1 INTRODUCTION . Deep Neural Networks ( DNNs ) have shown impressive performance in various tasks in recent years that have encouraged the industry to deploy DNN-based models in a variety of real-world applications . Since the training process of DNNs and collecting training data is an expensive and tedious process , models are considered the intellectual property of organizations , and they must be kept secure . Therefore , models are often securely deployed on cloud servers , and only the creators can access the model parameters . Users are only allowed to query the model via a prediction API and receive predictions . Recent studies Tramèr et al . ( 2016 ) ; Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) ; Orekondy et al . ( 2019 ) ; Jagielski et al . ( 2020 ) demonstrate that an adversary can exploit the prediction API of a target model to create a surrogate model in order to steal or reconnoiter the functionality of the target model . Such attacks are called model extraction attacks , and they violate the intellectual property of model owners . Furthermore , the surrogate model can be leveraged to conduct other attacks on the target model in black-box setting , such as adversarial example attacks Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) and membership inference attacks Shokri et al . ( 2017 ) . Most of the model extraction attacks use the target model ’ s prediction API to label an unlabeled dataset to create the surrogate model ’ s training set . In most real-world settings , the adversary has no or limited access to samples from the target model ’ s training data distribution , which is called normal or in-distribution samples . Hence , most proposed attacks in the previous studies use some form of Out-Of-Distribution ( OOD ) samples , such as synthesis Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) or semantically similar samples to the target model ’ s training set Orekondy et al . ( 2019 ) ; Pal et al . ( 2020 ) to conduct model extraction attacks . We focus on such attacks in this paper . There are two main approaches to defend against model extraction attacks , manipulating the target model outputs to prevent adversary from producing high-quality surrogate model Lee et al . ( 2019 ) ; Orekondy et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) ; Kariyappa et al . ( 2021b ) and detecting the sample sequences of model extraction attacks Kesarwani et al . ( 2018 ) ; Juuti et al . ( 2019 ) . Juuti et al . ( 2019 ) propose PRADA to detect samples sequence of model extraction attacks based on the distance among successive samples . We propose Hardness-Oriented Detection Approach ( HODA ) , a new approach to detect sample sequences of model extraction attacks , which outperforms PRADA by a large margin and has significantly less computational overhead . Generally , the training process of DNN-based classifiers is done in several epochs , and the resulted classifier at the end of the last epoch is considered the final classifier . We can consider the training process of DNN-based classifiers as a sequence of subclassifiers in which the ith subclassifier is created at the end of the ith epoch . HODA uses a subset of subclassifiers to compute the hardness degree of samples . The hardness degree of a sample is equal to the index of the subclassifier that all subsequent subclassifiers agree with its predicted label for that sample . It is important to note that we must save subclassifiers in the training phase of a target classifier in order to use their predictions to calculate the hardness degree of new samples . We demonstrate that the hardness degree histogram of normal sample sequences is distinguishable from the hardness degree histogram of model extraction attack sample sequences , and HODA uses this property to detect sample sequences of model extraction attacks . For each user , HODA calculates the distance between the hardness degree histograms of the user ’ s samples and normal samples , and if the distance is greater than a threshold , the user is detected as an adversary . HODA can detect JBDA Papernot et al . ( 2017 ) , JBRAND Juuti et al . ( 2019 ) , and Knockoff Net Orekondy et al . ( 2019 ) attacks with a high success rate by only monitoring 100 samples of attack . We demonstrate that HODA is also highly effective when the target classifier is trained using transfer learning . Contributions . ( i ) We demonstrate that the hardness degree of a sample for a classifier pertains to the training data distribution of that classifier . ( ii ) We indicate that the hardness degree histogram of normal samples is distinct from the hardness degree histograms of model extraction attack samples . ( iii ) We propose HODA to detect the sample sequences of model extraction attacks . 2 RELATED WORK . Model Extraction Attacks : Primary model extraction attacks try to extract the exact value of parameters Lowd & Meek ( 2005 ) ; Tramèr et al . ( 2016 ) and hyperparameters Wang & Gong ( 2018 ) of shallow models . In recent years , the proposed attacks mainly aimed to steal or reconnoiter the functionality of deep neural networks by querying them in the black-box setting . It is often sensibly assumed in the literature that the adversary has no or limited access to samples from the training set distribution of target classifier . In order to overcome this issue , attacks generally use some form of out-of-distribution samples , such as synthesis or semantically similar samples , to create the surrogate classifier ’ s training set . Knockoff Net Orekondy et al . ( 2019 ) , ActiveThief Pal et al . ( 2020 ) , and Copycat CNN da Silva et al . ( 2018 ) use semantically similar datasets to the target model ’ s training set to train a surrogate classifier . In another line of studies , Papernot et al . ( 2017 ) , Juuti et al . ( 2019 ) , Yu et al . ( 2020 ) , Truong et al . ( 2021 ) , Kariyappa et al . ( 2021a ) , and Barbalau et al . ( 2020 ) use synthetic data to create the surrogate classifier ’ s training set . Model Extraction Defenses : Existing defense methods against model extraction attacks generally distribute into two branches : perturbation-based and detection-based . Perturbation-based defenses Lee et al . ( 2019 ) ; Orekondy et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) attempt to prevent adversaries from producing high-quality surrogate classifiers by adding perturbation to the target classifier outputs . Recently , Kariyappa et al . ( 2021b ) proposed a new defense with the same goal as perturbation-based defenses , which does not perturb the target classifier outputs . Their approach employs an ensemble of diverse models to produce discontinuous predictions for out-of-distribution samples . Detection-based defenses attempt to detect the occurrence of model extraction attacks by observing successive input queries to the target classifier . Kesarwani et al . ( 2018 ) present a method to detect extraction attacks against Decision Tree models . PRADA Juuti et al . ( 2019 ) is the first proposed detection-based defense for DNN models . We propose a new defense detecting the sample sequences of model extraction attacks via hardness of samples . Atli et al . ( 2020 ) demonstrate that several OOD detection approaches , such as Baseline Hendrycks & Gimpel ( 2017 ) and ODIN Liang et al . ( 2018 ) , have poor performance in detecting Knockoff Net attack samples . Hence , they propose a new OOD detection approach that leverages a classifier to detect OOD samples . However , their approach only rejects OOD samples , and it does not have any detection mechanism to detect adversaries . Besides , the OOD detector is trained on samples from the same distribution used by the adversary to conduct Knockoff Net attacks , which is an unrealistic assumption in practice . Concurrent with our work , Zhang et al . ( 2021 ) and Pal et al . ( 2021 ) propose SEAT and VarDetect to detect sample sequences of model extraction attacks , respectively . SEAT aims to detect model extraction attacks that use several similar samples to extract a target model , such as jacobian-based attacks ( Papernot et al . ( 2017 ) ; Juuti et al . ( 2019 ) ) . Hence , SEAT is ineffective when an adversary uses natural samples that are not similar to each other , such as Knockoff Net attack . VarDetect uses Variational Autoencoders ( VAs ) and Maximum Mean Discrepancy ( MMD ) to detect model extraction attacks . VarDetect has only been evaluated on low-dimensional datasets . Regarding that VarDetect uses VAs and MMD , it is unclear how well it performs on high-dimensional datasets . Besides , it uses the ImageNet dataset to extract target classifiers trained on very structurally different datasets , such as F-MNIST and SVHN . HODA is evaluated on harder attacks using attack datasets that are structurally similar to the target classifier ’ s dataset , such as using ImageNet dataset to extract Caltech256 target classifier . HODA can detect both jacobian-based and Knockoff Net attacks , and it performs well on high-dimensional datasets , such as Caltech256 and CUB200 . Furthermore , unlike other work Atli et al . ( 2020 ) ; Kariyappa & Qureshi ( 2020 ) ; Kariyappa et al . ( 2021b ) , HODA only needs access to in-distribution samples to detect model extraction attacks . 3 MODEL EXTRACTION ATTACKS . The model extraction attack is one of the most serious threats against machine learning-based classifiers on remote servers , such as Machine Learning as a Service ( MLaaS ) . The adversary ’ s goal is to create a surrogate classifier fs that imitates a target classifier ft on task T . Most model extraction attacks exploit target model ft to label unlabeled samples to create the surrogate model ’ s training set . The adversary sends sample xi to the target model and receives its output ft ( xi ) , and then she uses pair ( xi , ft ( xi ) ) to train surrogate classifier fs . The output type of target model can be label , label confidence , top-k values in probability vector , or the entire probability vector . We only consider label f̄t ( xi ) and the entire probability vector ft ( xi ) as the output type of target classifiers in our experiments . There are two primary intents for adversaries to conduct model extraction attacks , stealing and reconnaissance . Stealing : Producing a high performance classifier is an expensive and time-consuming process and requires computational resources and experts . Besides , given that DNNs need a large number of training samples , collecting data and labeling them is a complex and costly procedure for most realworld applications . Therefore , adversaries are motivated to take advantage of a target classifier to reduce the cost of creating a new classifier . The adversary ’ s goal in stealing is to maximize the accuracy of surrogate model on data distribution DT . Hence , the adversary ’ s goal is : Maximize P ( x , y ) ∼DT f̄s ( x ) = y ( 1 ) Reconnaissance : The model extraction attacks can be used to conduct other attacks in the blackbox setting , such as adversarial example attacks Papernot et al . ( 2017 ) ; Goodfellow et al . ( 2015 ) and membership inference attacks Shokri et al . ( 2017 ) . The adversary ’ s goal in reconnaissance is to maximize the fidelity among surrogate and target classifiers in order to increase the success rate of black-box attacks . Similar to Jagielski et al . ( 2020 ) , we consider label agreement among surrogate and target classifiers as the fidelity metric on data distribution DT . Hence , the adversary ’ s goal is : Maximize P ( x , y ) ∼DT f̄s ( x ) = f̄t ( x ) ( 2 ) Proposed model Extraction attacks create the surrogate classifier training set Xs = { ( xi , ft ( xi ) ) } Bi=1 by various methods , where B is the attack budget . The attack budget determines the number of samples that an adversary is allowed to send to the target classifier and receive their associated predictions . After creating Xs , the adversary trains surrogate classifier fs to minimize empirical loss on Xs . We suppose that the adversary knows the architecture and hyperparameters of the target classifier and uses them to train the surrogate classifier . It is noteworthy that our proposed defense is independent of surrogate classifiers ’ training process .

Summary: HODA is a new defense against model extraction attacks. It is a detection method that determines if a given user is adversarial based on the hardness of users' queries. For a given sample, the hardness metric is computed as the epoch number of a subclassifier after which the sample is classified correctly. The method generates histograms for in-distribution data and the histograms of each user's queries. The Person distance is computed between the histograms and if the distance is greater than a certain threshold $\delta$ then the user is classified as malicious. The results show that the hardness degree histogram is an indicator of dataset distribution. In general, the in-distribution samples have significantly lower hardness levels than samples used by adversaries (the adversarial samples are OOD natural or synthetic data).