Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks

1 INTRODUCTION . Effectiveness of state-of-the-art DNN models at a variety of predictive tasks has encouraged their usage in a variety of real-world applications e.g. , home assistants , autonomous vehicles , commercial cloud APIs . Models in such applications are valuable intellectual property of their creators , as developing them for commercial use is a product of intense labour and monetary effort . Hence , it is vital to preemptively identify and control threats from an adversarial lens focused at such models . In this work we address model stealing , which involves an adversary attempting to counterfeit the functionality of a target victim ML model by exploiting black-box access ( query inputs in , posterior predictions out ) . Stealing attacks dates back to Lowd & Meek ( 2005 ) , who addressed reverse-engineering linear spam classification models . Recent literature predominantly focus on DNNs ( specifically CNN image classifiers ) , and are shown to be highly effective ( Tramèr et al. , 2016 ) on complex models ( Orekondy et al. , 2019 ) , even without knowledge of the victim ’ s architecture ( Papernot et al. , 2017b ) nor the training data distribution . The attacks have also been shown to be highly effective at replicating pay-per-query image prediction APIs , for as little as $ 30 ( Orekondy et al. , 2019 ) . Defending against stealing attacks however has received little attention and is lacking . Existing defense strategies aim to either detect stealing query patterns ( Juuti et al. , 2019 ) , or degrade quality of predicted posterior via perturbation . Since detection makes strong assumptions on the attacker ’ s query distribution ( e.g. , small L2 distances between successive queries ) , our focus is on the more popular perturbation-based defenses . A common theme among such defenses is accuracypreserving posterior perturbation : the posterior distribution is manipulated while retaining the top-1 label . For instance , rounding decimals ( Tramèr et al. , 2016 ) , revealing only high-confidence predictions ( Orekondy et al. , 2019 ) , and introducing ambiguity at the tail end of the posterior distribution ( Lee et al. , 2018 ) . Such strategies benefit from preserving the accuracy metric of the defender . However , in line with previous works ( Tramèr et al. , 2016 ; Orekondy et al. , 2019 ; Lee et al. , 2018 ) , we find models can be effectively stolen using just the top-1 predicted label returned by the black-box . Specifically , in many cases we observe < 1 % difference between attacks that use the full range of posteriors ( blue line in Fig . 1 ) to train stolen models and the top-1 label ( orange line ) alone . In this paper , we work towards effective defenses ( red line in Fig . 1 ) against DNN stealing attacks with minimal impact to defender ’ s accuracy . The main insight to our approach is that unlike a benign user , a model stealing attacker additionally uses the predictions to train a replica model . By introducing controlled perturbations to predictions , our approach targets poisoning the training objective ( see Fig . 2 ) . Our approach allows for a utility-preserving defense , as well as trading-off a marginal utility cost to significantly degrade attacker ’ s performance . As a practical benefit , the defense involves a single hyperparameter ( perturbation utility budget ) and can be used with minimal overhead to any classification model without retraining or modifications . We rigorously evaluate our approach by defending six victim models , against four recent and effective DNN stealing attack strategies ( Papernot et al. , 2017b ; Juuti et al. , 2019 ; Orekondy et al. , 2019 ) . Our defense consistently mitigates all stealing attacks and further shows improvements over multiple baselines . In particular , we find our defenses degrades the attacker ’ s query sample efficiency by 1-2 orders of magnitude . Our approach significantly reduces the attacker ’ s performance ( e.g. , 30-53 % reduction on MNIST and 13- 28 % on CUB200 ) at a marginal cost ( 1-2 % ) to defender ’ s test accuracy . Furthermore , our approach can achieve the same level of mitigation as baseline defenses , but by introducing significantly lesser perturbation . Contributions . ( i ) We propose the first utility-constrained defense against DNN model stealing attacks ; ( ii ) We present the first active defense which poisons the attacker ’ s training objective by introducing bounded perturbations ; and ( iii ) Through extensive experiments , we find our approach consistently mitigate various attacks and additionally outperform baselines . 2 RELATED LITERATURE . Model stealing attacks ( also referred to as ‘ extraction ’ or ‘ reverse-engineering ’ ) in literature aim to infer hyperparameters ( Oh et al. , 2018 ; Wang & Gong , 2018 ) , recover exact parameters ( Lowd & Meek , 2005 ; Tramèr et al. , 2016 ; Milli et al. , 2018 ) , or extract the functionality ( Correia-Silva et al. , 2018 ; Orekondy et al. , 2019 ) of a target black-box ML model . In some cases , the extracted model information is optionally used to perform evasion attacks ( Lowd & Meek , 2005 ; Nelson et al. , 2010 ; Papernot et al. , 2017b ) . The focus of our work is model functionality stealing , where the attacker ’ s yardstick is test-set accuracy of the stolen model . Initial works on stealing simple linear models ( Lowd & Meek , 2005 ) have been recently succeeded by attacks shown to be effective on complex CNNs ( Papernot et al. , 2017b ; Correia-Silva et al. , 2018 ; Orekondy et al. , 2019 ) ( see Appendix B for an exhaustive list ) . In this work , we works towards defenses targeting the latter line of DNN model stealing attacks . Since ML models are often deployed in untrusted environments , a long line of work exists on guaranteeing certain ( often orthogonal ) properties to safeguard against malicious users . The properties include security ( e.g. , robustness towards adversarial evasion attacks ( Biggio et al. , 2013 ; Goodfellow et al. , 2014 ; Madry et al. , 2018 ) ) and integrity ( e.g. , running in untrusted environments ( Tramer & Boneh , 2019 ) ) . To prevent leakage of private attributes ( e.g. , identities ) specific to training data in the resulting ML model , differential privacy ( DP ) methods ( Dwork et al. , 2014 ) introduce randomization during training ( Abadi et al. , 2016 ; Papernot et al. , 2017a ) . In contrast , our defense objective is to provide confidentiality and protect the functionality ( intellectual property ) of the ML model against illicit duplication . Model stealing defenses are limited . Existing works ( which is primarily in multiclass classification settings ) aim to either detect stealing attacks ( Juuti et al. , 2019 ; Kesarwani et al. , 2018 ; Nelson et al. , 2009 ; Zheng et al. , 2019 ) or perturb the posterior prediction . We focus on the latter since detection involves making strong assumptions on adversarial query patterns . Perturbation-based defenses are predominantly non-randomized and accuracy-preserving ( i.e. , top-1 label is unchanged ) . Approaches include revealing probabilities only of confident classes ( Orekondy et al. , 2019 ) , rounding probabilities ( Tramèr et al. , 2016 ) , or introducing ambiguity in posteriors ( Lee et al. , 2018 ) . None of the existing defenses claim to mitigate model stealing , but rather they only marginally delay the attack by increasing the number of queries . Our work focuses on presenting an effective defense , significantly decreasing the attacker ’ s query sample efficiency within a principled utility-constrained framework . 3 PRELIMINARIES . Model Functionality Stealing . Model stealing attacks are cast as an interaction between two parties : a victim/defender V ( ‘ teacher ’ model ) and an attacker A ( ‘ student ’ model ) . The only means of communication between the parties are via black-box queries : attacker queries inputs x ∈ X and defender returns a posterior probability distribution y ∈ ∆K = P ( y|x ) = FV ( x ) , where ∆K = { y 0,1Ty = 1 } is the probability simplex over K classes ( we use K instead of K − 1 for notational convenience ) . The attack occurs in two ( sometimes overlapping ) phases : ( i ) querying : the attacker uses the black-box as an oracle labeler on a set of inputs to construct a ‘ transfer set ’ of input-prediction pairs Dtransfer = { ( xi , yi ) } Bi=1 ; and ( ii ) training : the attacker trains a model FA to minimize the empirical risk on Dtransfer . The end-goal of the attacker is to maximize accuracy on a held-out test-set ( considered the same as that of the victim for evaluation purposes ) . Knowledge-limited Attacker . In model stealing , attackers justifiably lack complete knowledge of the victim model FV . Of specific interest are the model architecture and the input data distribution to train the victim model PV ( X ) that are not known to the attacker . Since prior work ( Hinton et al. , 2015 ; Papernot et al. , 2016 ; Orekondy et al. , 2019 ) indicates functionality largely transfers across architecture choices , we now focus on the query data used by the attacker . Existing attacks can be broadly categorized based on inputs { x ∼ PA ( X ) } used to query the black-box : ( a ) independent distribution : ( Tramèr et al. , 2016 ; Correia-Silva et al. , 2018 ; Orekondy et al. , 2019 ) samples inputs from some distribution ( e.g. , ImageNet for images , uniform noise ) independent to input data used to train the victim model ; and ( b ) synthetic set : ( Papernot et al. , 2017b ; Juuti et al. , 2019 ) augment a limited set of seed data by adaptively querying perturbations ( e.g. , using FGSM ) of existing inputs . We address both attack categories in our paper . Defense Objectives . We perturb predictions in a controlled setting : ỹ = F δV ( x ) = y + δ s.t . ỹ , y ∈ ∆K . The defender has two ( seemingly conflicting ) objectives : ( i ) utility : such that perturbed predictions remain useful to a benign user . We consider two utility measures : ( a ) Acc ( F δV , Dtest ) : accuracy of defended model on test examples ; and ( b ) dist ( y , ỹ ) = ||y− ỹ||p = to measure perturbation . ( ii ) non-replicability : to reduce the test accuracy of an attacker ( denoted as Acc ( FA , Dtest ) ) who exploits the predictions to train a replica FA on Dtransfer . For consistency , we evaluate both the defender ’ s and attacker ’ s stolen model accuracies on the same set of test examples Dtest . Defender ’ s Assumptions . We closely mimic an assumption-free scenario similar to existing perturbation-based defenses . The scenario entails the knowledge-limited defender : ( a ) unaware whether a query is malicious or benign ; ( b ) lacking prior knowledge of the strategy used by an attacker ; and ( c ) perturbing each prediction independently ( hence circumventing Sybil attacks ) . For added rigor , we also study attacker ’ s countermeasures to our defense in Section 5 .

Summary: This paper aims at defending against model stealing attacks by perturbing the posterior prediction of a protected DNN with a balanced goal of maintaining accuracy and maximizing misleading gradient deviation. The maximizing angular deviation formulation makes sense and seemingly correct. The heuristic solver toward this objective is shown to be relatively effective in the experiments. While the theoretical novelty of the method is limited, the application in adversarial settings may be useful to advance of this research field, especially when it is relatively easy to apply by practitioners.I recommend toward acceptance of this paper even though can be convinced otherwise by better field experts.