Increasing the Cost of Model Extraction with Calibrated Proof of Work

1 INTRODUCTION . Model extraction attacks ( Tramèr et al. , 2016 ; Jagielski et al. , 2020 ; Zanella-Beguelin et al. , 2021 ) are a threat to the confidentiality of machine learning ( ML ) models . They are also used as reconnaissance prior to mounting other attacks , for example , if an adversary wishes to disguise some spam message to get it past a target spam filter ( Lowd & Meek , 2005 ) , or generate adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) using the extracted model ( Papernot et al. , 2017b ) . Furthermore , an adversary can extract a functionally similar model even without access to any real input training data ( Krishna et al. , 2020 ; Truong et al. , 2021 ; Miura et al. , 2021 ) while bypassing the long and expensive process of data procuring , cleaning , and preprocessing . This harms the interests of the model owner and infringes on their intellectual property . Defenses against model extraction can be categorized as active , passive , or reactive . Current active defenses perturb the outputs to poison the training objective of an attacker ( Orekondy et al. , 2020 ) . Passive defenses try to detect an attack ( Juuti et al. , 2019 ) or truncate outputs ( Tramèr et al. , 2016 ) , but these methods lower the quality of results for legitimate users . The main reactive defenses against model extraction attacks are watermarking ( Jia et al. , 2020b ) , dataset inference ( Maini et al. , 2021 ) , and proof of learning ( Jia et al. , 2021 ) . However , reactive approaches address model extraction post hoc , i.e. , after the attack has been completed . We design a pro-active defense that prevents model stealing before it succeeds . Specifically , we aim to increase the computational cost of model extraction without lowering the quality of model outputs . Our method is based on the concept of proof-of-work ( PoW ) and its main steps are presented as a block diagram in Figure 1 . Our key proposal is to wrap model outputs in a PoW problem to force users to expand some compute before they can read the desired output . The standard PoW techniques , ∗Work done while the author was at the University of Toronto and Vector Institute . Attacker Defender . the following steps : 1 An attacker sends queries drawn from an unlabeled dataset to a machine learning service . 2 The information leakage estimator computes the privacy cost of the queries . 3 The privacy cost of queries is sent to the proof-of-work ( PoW ) server , which adds the new cost to the total user ’ s ( attacker ’ s ) query cost . 4 The PoW server generates a puzzle , whose difficulty is based on the total cost incurred by the user . 5 The attacker solves the puzzle using its PoW Client and 6 sends the solution back to the server . 7 The PoW server verifies the solution and if it is correct , it notifies the victim model , which 8 predicts the labels . The victim model returns the labels to the client 9 , which can potentially use the returned labels to train its own model . which initially were used in the security domain to prevent a server from Denial-of-Service attacks ( DoS ) 1 , provide users with a small puzzle to be solved before giving access to the service ( Dwork & Naor , 1993 ) . In the case of Machine-Learning-as-a-Service ( MLaaS ) , the answer to a query ( e.g. , the label predicted by a model ) can similarly be released to a user once his or her computational or other system resources are spent . This can prevent a malicious user from obtaining enough examples labeled by a victim model to steal the model . Intuitively , our method shifts the trade-off between the quality of answers and robustness , that was introduced by previous defenses , to a trade-off between the computational cost and robustness to model stealing . To minimize the impact of our defense on legitimate users , we tie the difficulty of the PoW problem ( Back , 2020 ) to an estimate of how much information about a model has been extracted so far by a given user . We compute the privacy cost per query as an information leakage metric used in the differential privacy literature ( Dwork et al. , 2014 ) . We choose to work with the privacy based metric because it is ( 1 ) agnostic to the learning task and model architecture , and ( 2 ) practical to compute for the model owner . We tune the per-query difficulty of the required proof-of-work based on the measured information gained by a user . PoW does not deteriorate the accuracy of answers and adds only a small increase ( less than 2x ) in the cost of querying public models for legitimate users . The cost is usually much higher ( up to 100x ) for attackers . To summarize , our contributions are as follows : • We introduce a new class of pro-active defenses against model stealing attacks that use proofof-work mechanisms to adaptively increase the computational cost of queries with a negligible increase in the cost for legitimate users , and a high increase in cost for many attackers . Our method is the first active defense strategy that leaves the model ’ s accuracy entirely intact . • We propose a novel application of differential privacy budgets as a way to measure the amount of information leakage from answered queries . We calibrate the difficulty of the proof-of-work problem based on a user ’ s query cost as estimated through its differential privacy budget . • Our rigorous evaluation on four datasets and eight attacks validates that adversaries attempting to steal a model issue queries whose privacy cost accumulates up to 7x faster than for benign queries 1In a DoS attack , adversaries flood a server with superfluous requests , overwhelming the server and degrading service for legitimate users . ATTACKS DATA # OF ATTACKER ’ S QUERIES ADVERSARY ’ S VICTIM . • We design adaptive attacks against our defense and show that they require simultaneous optimization of two opposing goals : ( 1 ) minimizing the privacy cost of queries run against the ML API and ( 2 ) maximizing the information leakage incurred by the queries . These conflicting targets render such attacks less effective , making them less attractive to adversaries . 2 BACKGROUND AND RELATED WORK . 2.1 MODEL EXTRACTION ATTACKS . Model extraction aims to replicate some functionality of a victim model f ( x ; θ ) , where x denotes an input and θ are parameters of the model . An attacker takes advantage of the ability to query the victim and observe the model ’ s outputs , which are used to train a substitute model f ′ ( x ; θ′ ) . We test our defense against common forms of fidelity and task accuracy attacks . The fidelity extraction is given as input some target distribution DT and a goal similarity function S ( p1 , p2 ) . It maximizes the target similarity between the victim and stolen models : maxθ′ Prx∼DT S ( f ′ ( x ; θ′ ) , f ( x ; θ ) ) . The Jacobian-based attack ( Papernot et al. , 2017b ) is a high fidelity type of model extraction because it aims to align the gradients of the extracted model with the victim model ’ s gradients . The task accuracy extraction aims at matching ( or exceeding ) the accuracy of the victim model . For the true task distribution DA , the goal is defined as : maxθ′ Pr ( x , y ) ∼DA [ arg max ( f ′ ( x ; θ′ ) = y ] . The task accuracy attacks include Knockoff Nets ( Orekondy et al. , 2019 ) , DataFree ( Truong et al. , 2021 ) , and MixMatch-based ( Berthelot et al. , 2019 ) model extraction . We taxonomize these attacks in Table 1 from the perspective of a victim who observes the queries as well as the model ’ s response on these queries . A defender wants to defend against any of these attacks , regardless of their intent , while not causing harm to legitimate users . Attackers with problem domain data need fewer queries and access to labels only is sufficient for them to steal a model ( e.g. , MixMatch extraction from Jagielski et al . ( 2020 ) ) . On the other hand , the lack of such data increases the number of required queries and might call for access to logits ( Truong et al. , 2021 ) . 2.2 DEFENSES AGAINST MODEL EXTRACTION . We taxonomize the space of model defenses into three main classes . First , active defenses perturb the outputs from a defended model to harm training of the attacker ’ s model . They trade-off defended model accuracy for better security . Second , passive defenses either truncate the outputs , which also lowers accuracy , or try to detect an attack , which makes them stateful since some form of information has to be stored about each user ’ s queries and this might cause scalability issues . Other defenses are stateless . Third , reactive defenses try to prove a model theft rather than preventing the attack from happening . We propose a new class of defenses that are pro-active . We trade-off compute of the querying party for better security . We neither degrade the accuracy of the defended model nor require any changes to the serving model . Our defense is stateful but we only store privacy cost per user in a form of a single float number . Next , we describe specific examples of defenses from each class . Active defenses . Lee et al . ( 2019 ) perturb a victim model ’ s final activation layer , which preserves that top-1 class but output probabilities are distorted . Orekondy et al . ( 2020 ) modify the distribution of model predictions to impede the training process of the stolen copy by poisoning the adversary ’ s gradient signal , however , it requires a costly gradient computation . The defenses proposed by Kariyappa & Qureshi ( 2020 ) ; Kariyappa et al . ( 2021 ) identify if a query is in-distribution or out-ofdistribution ( OOD ) and then for OOD queries sends either incorrect or dissimilar predictions . The defenses introduced by Kariyappa et al . assume knowledge of attackers ’ OOD data that are generally hard to define a-priori and the selection of such data can easily bias learning ( Hsu et al. , 2020 ) . Passive defenses . PRADA by Juuti et al . ( 2019 ) and VarDetect by ( Pal et al. , 2021 ) are stateful and analyze the distribution of users ’ queries in order to detect potential attackers . The former assumes that benign queries are distributed in a consistent manner following a normal distribution while adversarial queries combine natural and synthetic samples to extract maximal information and have , e.g. , small ` 2 distances between them . The latter presumes that an outlier dataset , which represents adversarial queries , can be built incrementally and differs significantly from in-distribution data . However , PRADA does not generalize to attacks that utilize only natural data ( Orekondy et al. , 2019 ; Pal et al. , 2020 ) . Follow-up works ( Atli et al. , 2020 ) have specifically targeted model extraction attempts that use publicly available , non-problem domain natural data , but they assume access to data from such an attacker . Reactive defenses . Watermarking allows a defender to embed some secret pattern in their model during training ( Adi et al. , 2018 ) or inference ( Szyller et al. , 2019 ) . Jia et al . ( 2020a ) proposed entangled watermarks to ensure that watermarks are not easy to remove . Dataset inference ( Maini et al. , 2021 ) identifies whether a given model was stolen by verifying if a suspected adversary ’ s model has private knowledge from the original victim model ’ s dataset . Proof of learning ( Jia et al. , 2021 ) involves the defender claiming ownership of a model by showing incremental updates of the model training . Unfortunately , reactive defenses are effective after a model theft and require a model owner to obtain partial or full access to the stolen model . This means that if an attacker aims to use the stolen model as a proxy to attack the victim model , or simply keeps the stolen model secret , it is impossible for the model owner to protect themselves with these defenses .

Summary: This paper proposes a novel defense against model extraction attacks. The proposed approach slows information leakage by asking all users to answer a puzzle before receiving the response to their query (proof-of-work). The difficulty of the puzzle allows to keep the computation overhead low for legitimate users, while rendering information leakage prohibitively expensive for attackers. The puzzle difficulty is calibrated based on an estimate of how much information each user has already acquired. This is based on PATE, a differential privacy metric. Experiments are performed on multiple datasets, opposing the proposed defense to a wide range of attacks, including adaptive adversaries.