Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper discusses ML model extraction attacks while using APIs for accessing them on the
public networks. Although some methods exist for preventing or making the attacks hard, all of
them have substantial impacts on legitimate users’ experience while using the system, including
the slower models or lower accuracy in results. This paper proposed a method for dissuading
the attacker by increasing the cost of the attack. Solving a puzzle for all users before getting
the final response (POW) would be the solution noting that the difficulty will be increased if
the system identifies any adverse behaviors. This method requires no modification of the victim
model and can be applied by machine learning practitioners to guard their publicly exposed
models against being easily stolen.
main_review: The paper discussed the issue, and the background works well. The purpose is clear, and
there are a satisfying amount of experiments for validating the proposed solution. However,
the privacy scoring section is not clear. I cannot find out why PATE is the best option for
calculating privacy metrics and the exact functionality of this section is not as straightforward
as other parts. Moreover, identification of users is essential since, as mentioned, the difficulty of
POW will increase gradually, but this concern is not well pointed in the paper.
summary_of_the_review: The authors should make the privacy scoring module more clear and add more discussion about identifying malicious nodes in continuous requests
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel defense against model extraction attacks. The proposed approach slows information leakage by asking all users to answer a puzzle before receiving the response to their query (proof-of-work). The difficulty of the puzzle allows to keep the computation overhead low for legitimate users, while rendering information leakage prohibitively expensive for attackers. The puzzle difficulty is calibrated based on an estimate of how much information each user has already acquired. This is based on PATE, a differential privacy metric. Experiments are performed on multiple datasets, opposing the proposed defense to a wide range of attacks, including adaptive adversaries.
main_review: Strengths:
- Interesting idea of applying proof-of-work to machine learning as an entry barrier.
- The novelty of the paper seems to reside in the application of existing concepts, like PATE for privacy cost estimation and HashCash as challenge, as components of a theft defense method. I find this novel enough and quite ingenious. Moreover, the existing concepts were appropriately adapted for the task at hand.
- The experimental section is extensive, and the experimental protocol seems appropriate.
- The paper is clear, well organized and pleasant to read.
- Good quality code base allowing to reproduce the experiments.

Weaknesses:
- The proposed defense does not seem effective against attackers that have access to data from a similar distribution to the training one (e.g., MixMatch baseline).
- The Knockoff attack does not seem to use its most effective querying strategy (Random is used instead of Adaptive).
- Doubling computation cost for a legitimate user is not a prohibitive cost, but is also non-negligible, both from an effort and computation time perspective, as well as the CO2 emissions mentioned in the ethics statement of the paper.

Other comments / questions:
- With the proposed approach, does it mean that legitimate users would incur higher query cost when submitting many queries?
- The paper states multiple times that it proposes the first defense to leave model accuracy intact, but I would argue that it is the first *active* defense to do so.

[Update post-discussion] I am raising my rating following the exchanges below.
summary_of_the_review: Novel active strategy to defend against model extraction, good experimental results.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a new defense to the model stealing problem that an attacker queries a supervised machine learning model service to have data labeled to use as training data and copy the functionality of the model. The proposed method is proactive defense that slows down the attacker in obtaining the labeled data, based on the information leakage estimator. The evaluation shows that existing attacks using out-of-distribution data can be slow down more than a regular user querying in-distribution data.
main_review: Strengths
- S1: This paper proposes a novel defense focusing on delaying the model stealing process.
- S2: The proposed defense is tested against 6 attacks on 3 datasets.

Weaknesses
- W1: If the goal of the proof of work is just to slow down the process, the server can simply choose to delay the response, unlike the distributed block chain. Thus, the proof of work does not look necessary.
- W2: The defense does not work against an attacker using in-distribution data, such as a subscribed user using the model normally until enough data is accumulated.
- W3: The use of PATE as the information leakage estimator is not justified. PATE is devised to protect sensitive data, and how sensitive data is connected to the performance of a machine learning model is unclear. This component is rather the most important part of the framework as it dictates the user experience and the attacker's success. A detailed reasoning or comparison to other options should improve the paper.
summary_of_the_review: This paper is in general well written, and the experiment was done to show the strengths and the weaknesses of the proposed defense. The defense against model stealing is inherently difficult problem where the attacker in the end can steal the model using the queried data and returned label pairs unless architecture or resource is no object (e.g., if not GPT-3). The main idea proposed in the paper against such a strong attack looks interesting, but the key technique used here is the proof of work. I am not convinced this is necessary at all since the original model is hosted and controlled centrally, and response time can be fully controlled without using the proof of work. Just delaying the response would have much better control of information leakage per time unit than the proof of work that a powerful machine can do faster, or simultaneously processing multiple queries with multiple machines reaching up to the speed of a regular user. Rather the important piece is the information leakage estimator which is somewhat overlooked in the paper, especially in terms of why PATE has to be the right choice here. I think focusing on this can lead to a better paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel defense to prevent model stealing by requiring users to solve proof-of-work puzzles. The authors evaluate their method against different types of model extraction attacks. The results show that the defense will result in the attacker costing higher computational time (100x) than legitimate users (2x).

main_review: This paper proposes a defense against model extraction attacks by forcing users to do a proof-of-work (PoW) puzzle before they receive the labels from the victim model. To minimize the impact on legitimate users, the defense tunes the per-query difficulty based on the extent of information leakage. The privacy cost is evaluated via differential privacy budget. Compared to other defense methods, this method will not reduce the accuracy of answers, and it can protect the model before the model stealing happens (unlike watermark schemes).

My comments are as follows.

Firstly, I think the structure of the paper can be improved. The first two sections of the paper take up a lot of space, so that the discussion in section 3 is not clear enough. Some experimental results are placed in the appendix, and some tables interspersed in the reference list.

Secondly, section 3.3 shows how to use differential privacy to evaluate the privacy cost. Though the authors explain "We compute the privacy cost of queries using the PATE framework",  "The privacy cost is computed based on the consensus among teachers and the amount of privacy noise added to the histogram" and  "more details can be found in appendix D", I would suggest to show the main formulas and conclusions here to help the readers to follow.

Thirdly, the authors create two models. The first one is used to predict query time from the privacy cost, the second one is used to predict the number of leading zeros for the desired time. So the whole mapping flow is : privacy cost → query time → the number of leading zeros. The authors find that using a simple linear regression is sufficient for both models. Firstly, Figure 2 shows the dependency between the computation time and the number of leading zeros is exponential, is it contradictory? Secondly, I think there should be a clear mapping from privacy cost to the number of leading zeros, which will help practitioners easily to tune parameters to control the planned results, such as "Time POW" in Table 2. Thirdly, I can't find the results about these linear regression models in the experiment part.

Finally, the difference in privacy cost between attackers and legitimate users is the main evaluation metric. Figure 3 shows a higher privacy leakage for the attackers compared to a standard user, but for the CIFAR10 dataset, this distinction is not very obvious, and it causes the computational cost for attackers to increase a little bit (In Table 2, line 16, for the attack COPYCAT on CIFAR10, time changes from 148.3 to 326). This situation may be caused by the ambiguous mapping from privacy cost to computational time, and it may pose a risk of defense failure. So more analysis is needed here.

summary_of_the_review: see main review
review_title: null

Meta-review: the paper proposed a novel idea of  requiring users to complete a proof-of-work before they can read the model's prediction to prevent model extraction attacks. Reviewers were excited about the paper and ideas. Some misunderstanding raised by reviewers were sufficiently clarified by the authors in the rebuttal.
Decision: Accept (Spotlight)