Deep Neural Network Fingerprinting by Conferrable Adversarial Examples

1 INTRODUCTION Deep neural network ( DNN ) classifiers have become indispensable tools for addressing practically relevant problems , such as autonomous driving ( Tian et al. , 2018 ) , natural language processing ( Young et al. , 2018 ) and health care predictions ( Esteva et al. , 2019 ) . While a DNN provides substantial utility , training a DNN is costly because of data preparation ( collection , organization , and cleaning ) and computational resources required for validation of a model ( Press , 2016 ) . For this reason , DNNs are often provided by a single entity and consumed by many , such as in the context of Machine Learning as a Service ( MLaaS ) . A threat to the provider is model stealing , in which an adversary derives a surrogate model from only API access to a source model . We refer to an independently trained model for the same task as a reference model . Consider a MLaaS provider that wants to protect their service and hence restrict its redistribution , e.g. , through a contractual usage agreement because trained models constitute their intellectual property . A threat to the model provider is an attacker who derives surrogate models and publicly deploys them . Since access to the source model has to be provided , users can not be prevented from deriving surrogate models . Krishna et al . ( 2019 ) have shown that model stealing is ( i ) effective , because even high-fidelity surrogates of large models like BERT can be stolen , and ( ii ) efficient , because surrogate models can be derived for a fraction of the costs with limited access to domain data . This paper proposes a DNN fingerprinting method to predict whether a model is a ( stolen ) surrogate or a ( benign ) reference model relative to a source model . DNN fingerprinting is a new area of research that extracts a persistent , identifying code ( fingerprint ) from an already trained model . Model stealing can be categorized into model modification , such as weight pruning ( Zhu & Gupta , 2017 ) , or model extraction that uses some form of knowledge distillation ( Hinton et al. , 2015 ) to derive a surrogate from scratch . Claimed security properties of existing defenses ( ( Adi et al. , 2018 ; Zhang et al. , 2018 ) ) , have been broken by model extraction attacks ( Shafieinejad et al. , 2019 ) . Our fingerprinting method is the first passive defense that is specifically designed towards withstanding model extraction attacks , which extends to robustness against model modification attacks . Our research provides new insight into the transferability of adversarial examples . In this paper , we hypothesize that there exists a subclass of targeted , transferable , adversarial examples that transfer exclusively to surrogate models , but not to reference models . We call this subclass conferrable . Any conferrable example found in the source model should have the same misclassification in a surrogate model , but a different one in reference models . We propose a metric to measure conferrability and an ensemble adversarial attack that optimizes this new metric . We generate conferrable examples as the source model ’ s fingerprint . Retrained CIFAR-10 surrogate models can be verified with a perfect ROC AUC of 1.0 using our fingerprint , compared to an ROC AUC of 0.63 for related work ( Cao et al. , 2019 ) . While our fingerprinting scheme is robust to almost all derivation and extraction attacks , we show that some adapted attacks may remove our fingerprint . Specifically , our fingerprint is not robust to transfer learning when the attacker has access to a model pre-trained on ImageNet32 and access to CIFAR-10 domain data . Our fingerprint is also not robust against adversarial training ( Madry et al. , 2017 ) from scratch . Adversarial training is an adapted model extraction attack specifically designed to limit the transferability of adversarial examples . We hypothesize that incorporating adversarial training into the generation process of conferrable adversarial examples may lead to higher robustness against this attack . 2 RELATED WORK . In black-box adversarial attacks ( Papernot et al. , 2017 ; Tramèr et al. , 2016 ; Madry et al. , 2017 ) , access to the target model is limited , meaning that the target architecture is unknown and computing gradients directly is not possible . Transfer-based adversarial attacks ( Papernot et al. , 2016 ; 2017 ) exploit the ability of an adversarial example to transfer across models with similar decision boundaries . Targeted transferability additionally specifies the target class of the adversarial example . Our proposed adversarial attack is a targeted , transfer-based attack with white-box access to a source model ( that should be defended ) , but black-box access to the stolen model derived by the attacker . Liu et al . ( 2016 ) and Tramèr et al . ( 2017a ) show that ( targeted ) transferability can be boosted by optimizing over an ensemble of models . Our attack also optimizes over an ensemble of models to maximize transferability to stolen surrogate models , while minimizing transferability to independently trained models , called reference models . We refer to this special subclass of targeted transferability as conferrable . Tramèr et al . ( 2017a ) empirically study transferability and find that transferable adversarial examples are located in the intersection of high-dimensional ” adversarial subspaces ” across models . We further their studies and show that ( i ) stolen models apprehend adversarial vulnerabilities from the source model and ( ii ) parts of these subspaces , in which conferrable examples are located , can be used in practice to predict whether a model has been stolen . Watermarking of DNNs is a related method to DNN fingerprinting where an identifying code is embedded into a DNN , thereby potentially impacting the model ’ s utility . Uchida et al . ( 2017 ) embed a secret message into the source model ’ s weight parameters , but require white-box access to the model ’ s parameters for the watermark verification . Adi et al . ( 2018 ) and Zhang et al . ( 2018 ) propose backdooring the source model on a set of unrelated or slightly modified images . Their approaches allow black-box verification that only requires API access to the watermarked model . Frontier-Stitching ( Merrer et al. , 2017 ) and BlackMarks ( Dong et al. , 2018 ) use ( targeted ) adversarial examples as watermarks . These watermarks have been evaluated only against model modification attacks , but not against model extraction attacks that train a surrogate model from scratch . At least two of these watermarking schemes ( Adi et al. , 2018 ; Zhang et al. , 2018 ) are not robust to model extraction attacks ( Shafieinejad et al. , 2019 ) . Cao et al . ( 2019 ) recently proposed a fingerprinting method with adversarial examples close to the source model ’ s decision boundary . We show that their fingerprint does not withstand retraining as a model extraction attack and propose a fingerprint with improved robustness to model extraction attacks . 3 DNN FINGERPRINTING . Threat Model . The attacker ’ s goal is to derive a surrogate model from access to the defender ’ s source model that ( i ) has a similar performance ( measured by test accuracy ) and ( ii ) is not verified as a surrogate of the source model by the defender . We protect against an informed attacker that can have ( i ) white-box access to the source model , ( ii ) unbounded computational capacity and ( iii ) access to domain data from the same distribution . A more informed attacker can drop information and invoke all attacks of a less informed attacker . Robustness against a more informed attacker implies robustness against a less informed attacker . Our attacker is limited in their access to groundtruth labeled data , otherwise they could train their own model and there would be no need to steal a model . In our evaluation , we experiment with attackers that have up to 80 % of ground-truth labels for CIFAR-10 ( Krizhevsky et al. ) . A practical explanation for the limited accessibility of ground truth labels could be that attackers do not have access to a reliable oracle ( e.g. , for medical applications ) , or acquiring labels may be associated with high expenses ( e.g. , Amazon Mechanical Turk1 ) . The defender ’ s goal is to identify stolen surrogate models that are remotely deployed by the attacker . Their capabilities are ( i ) white-box access to the source model , ( ii ) black-box access to the target model deployed by the attacker and ( iii ) a limited set of n queries to verify a remotely deployed surrogate model . Black-box access to the suspect model excludes knowledge of the suspect model ’ s architecture or attack used to distill the model . The defender also does not have knowledge of , nor control over the attacker ’ s dataset used to steal the model . Fingerprinting Definitions . A fingerprinting method for DNNs consists of two algorithms : ( i ) A fingerprint generation algorithm that generates a secret fingerprint F ⊆ Xn of size n , and a fingerprint verification keyFy ⊆ Yn ; ( ii ) A fingerprint verification algorithm that verifies surrogates of the source model . These algorithms can be summarized as follows . • Generate ( M , D ) : Given white-box access to a source model M and training data D ∈ D. Outputs a fingerprint F and the verification keys Fy = { M ( x ) |x ∈ F } . • Verify ( M̂ ( F ) , Fy ) : Given black-box access to a suspect model M̂ , a fingerprint F and a verification key Fy . Outputs 1 if M̂ is verified by the fingerprint and 0 otherwise . The verification algorithm computes an error rate between the outputs of the source and target model on the fingerprint . We empirically measure the error rate separately for surrogate and reference models , which allows to choose a decision threshold ρ ∈ [ 0 , 1 ] . If the error-rate of a target model exceeds 1 − ρ , the verification predicts the target model to be a reference model , otherwise the prediction is a surrogate model . We define that a fingerprint must be irremovable , i.e. , surrogate and reference models are correctly verified , despite an attacker ’ s removal attempt . It must also be non-evasive , meaning that an attacker can not evade black-box verification by detecting fingerprint queries . We refer to Appendix A.3 for security games of DNN fingerprinting . 4 CONFERRABLE ADVERSARIAL EXAMPLES . Motivating Conferrability Conferrability is a new property for adversarial examples , in which targeted transferability occurs only from a source model to its surrogates , but not to independently trained reference models . Intuitively , surrogate models are expected to be more similar to the source model than any reference model , but quantifying this similarity is non-trivial . Conferrable examples are an attempt to quantify this similarity by shared adversarial vulnerabilities of the source model 1https : //www.mturk.com/ and its surrogates . Surrogate models differ from reference models in the objective function which they optimize . Reference models maximize fidelity to the ground truth labels , whereas surrogate models maximize fidelity to the source model ’ s labels , which do not always coincide . Fig . 2a shows the relation between targeted adversarial , transferable , and conferrable examples for the source model , its surrogates , and all reference models . An example is transferable if it is adversarial to any model . It is conferrable when it is adversarial only to surrogate and source models . In that sense , conferrable adversarial examples are a subclass of targeted transferable adversarial examples . Fig . 2b shows the relation between transferable and conferrable adversarial examples in the decision space , simplified for illustrative purposes to binary classification . Transferable adversarial examples occur in those adversarial subspaces where the decision boundary of surrogate and reference models coincide ( Tramèr et al. , 2017b ) . Conferrable adversarial examples occur in those adversarial subspaces where the decision boundary of surrogate and reference models differ . Targeted transferability for a class t and a set of modelsM can be computed as follows . Transfer ( M , x ; t ) = Pr M∈M [ M ( x ) = t ] ( 1 ) Objective We want to find adversarial examples that maximize the output activation difference between surrogate models S and reference modelsR . This difference is quantified by our conferrability score , which measures the example ’ s transferability to surrogate and reference models . Confer ( S , R , x ; t ) = Transfer ( S , x ; t ) ( 1− Transfer ( R , x ; t ) ) ( 2 ) The central challenge for optimizing conferrability directly is that we require access to a differentiable function Transfer that estimates an example ’ s transferability score . To the best of our knowledge , the only known method to evaluate transferability is to train a representative set of DNNs and estimating the example ’ s transferability itself . We use this method to evaluate transferability . In the case of conferrable examples , we evaluate transferability on a set of surrogate and reference models trained locally by the defender and use Equation 2 to obtain the conferrability score . Our hypothesis is that conferrability generalizes , i.e. , examples that are conferrable to a representative set of surrogate and reference DNNs are also conferrable to other , unseen DNNs . Conferrable Ensemble Method . In this section , we describe our adversarial attack that generates conferrable adversarial examples , called the Conferrable Ensemble Method ( CEM ) . CEM operates on the source model M , a set of its surrogates SM , and a set of reference models R. The attack constructs an ensemble model ME with a single shared input that outputs conferrability scores for all output classes y ∈ Y . CEM generates highly conferrable adversarial examples by finding a perturbation δ so that x′ = x0 + δ maximizes the output of the ensemble model for a target class t. We now present the construction of the ensemble model ME . The ensemble model produces two intermediate outputs for an input x ∈ X , representing the average predictions of all surrogate and all reference models on x . We use Dropout ( Srivastava et al. , 2014 ) with a drop ratio of d = 0.3 . Surr ( SM , x ) = 1 |SM | ∑ S∈SM Dropout ( S ( x ) ; d ) ( 3 ) Ref ( R , x ) = 1 |R| ∑ R∈R Dropout ( R ( x ) ; d ) ( 4 ) The output of the ensemble model is the conferrability score of the input for each class . The Softmax activation function is denoted by σ . We refer to Appendix A.4 for details about the optimization . ME ( x ; SM , R ) = σ ( Surr ( SM , x ) ( 1− Ref ( R , x ) ) ) ( 5 ) The loss function consists of three summands and is computed over the benign , initial example x0 , and the example at an intermediate iteration step x′ = x0 + δ . We denote the cross-entropy loss by H ( · , · ) . The first summand function maximizes the output of the ensemble model for some target class t. The second summand maximizes the categorical cross-entropy between the current and initial prediction for the source model . The third summand minimizes the categorical cross-entropy between the source model ’ s prediction and the predictions of its surrogates . The total loss L is the weighted sum over all individual losses . L ( x0 , x′ ) = αH ( 1 , max t [ ME ( x ′ ) t ] ) − βH ( M ( x0 ) , M ( x′ ) ) + γH ( M ( x′ ) , Surr ( SM , x′ ) ) ( 6 ) In all our experiments , we use weights α = β = γ = 1 and refer to Appendix A.6 for an empirical sensitivity analysis of the hyperparameters . We address the box constraint , i.e. , ||δ|| ≤ , similarly to PGD ( Madry et al. , 2017 ) by clipping intermediate outputs around the ball of the original input x0 . For the optimization of an input with respect to the loss we use the Adam optimizer ( Kingma & Ba , 2014 ) . Note that we use an untargeted attack to generate targeted adversarial examples . Targeted attacks are harder to optimize than their untargeted counterparts ( Wu & Fu , 2019 ) . We assign the source model ’ s predicted label for the generated adversarial example as the target label and ensure that target classes are balanced in the fingerprint verification key . Fingerprinting Algorithms . We now describe our fingerprinting generation and verification algorithms . For the generation , the defender locally trains a set of c1 surrogate and c2 reference models ( c1 = c2 = 18 ) on their training data prior to executing CEM . Surrogate models are trained on data labeled by the source model , whereas reference models are trained on ground-truth labels . The defender composes the ensemble model ME as described in the previous paragraph and optimizes for a perturbation δ given an input x0 , so that L ( x0 , x0 + δ ) is minimized . The optimization returns a set of adversarial examples that are filtered by their conferrability scores on the locally trained models . If their conferrability score ( see Equation 2 ) exceeds a minimum threshold ( τ ≥ 0.95 ) , they are added to the fingerprint . For the fingerprint verification , we compute the error rate between the source model ’ s prediction on the fingerprint and a target model ’ s predictions . If the error rate is greater than 1− ρ , which we refer to as the decision threshold , the target model is predicted to be a reference model and a surrogate model otherwise .

Summary: This paper studies fingerprinting a neural network model by using adversarial example techniques. The idea itself is interesting enough, the this work presents a neat development toward solving this problem. An important issue with this problem is to distinguish a reference model from a stolen model. Thus a desire property of the fingerprint adversarial example is to mislead all surrogate models but non reference models. Since adversarial examples are typically transferable to reference models, thus it is important to distinguish a fingerprint from a transferable adversarial example. For a long time, researchers do not have an answer to whether this is possible, and this work provides an evidence that it may generate a conferrable but not transferable adversarial example to achieve the goal.