Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a method to defend against model stealing attacks. This
method is based on the mean correlation among the selected samples. It provides
two ways to select samples: 1. Finding wrongly classified normal samples. 2.
Selecting mixed samples via CutMix. Experiments on four datasets demonstrate the effectiveness of the proposed method.

strengths_and_weaknesses: #Strengths

* Interesting topic and good motivation.

* SAC-m has high efficiency.

#Weaknesses

* It lacks solid explanations about why the proposed method works.

* Generalizability to different source models is not clear.

* Comparisons with some related works are missing.

#Detailed comments

* This paper does not provide solid explanations about why the proposed method
works. More discussion or analysis would make the intuition behind the proposed
method more convincing.

* Comparisons with some related works are missing. This paper claims that the
proposed method outperforms previous methods. However, it lacks comparisons with
two related works [1,2]. Therefore, it is hard to say if the proposed method
achieves SOTA performance.

* Table 3 is hard to read. The title of Table 3 mentions "accuracies under
different source models", but it seems there is only one source model.
Therefore, this method's generalizability to different source models is also not
clear.

[1] Li et al. Defending against model stealing via verifying embedded external features. AAAI 2022.

[2] Chen et al. Copy, right? a testing framework for copyright protection of deep learning models. IEEE S&P 2022.

questions: * Could you provide more discussion or analysis about why the proposed method
works?
limitations: This paper has discussed the limitations and potential negative impacts.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Model fingerprinting allows a model owner to claim ownership of a stolen model. Prior works on fingerprinting typically use transferable adversarial examples to perform fingerprinting. Such techniques have two key shortcomings: 1. They don’t work in the presence of defenses like adversarial training 2. They cannot be used when the stolen model is used for transfer learning as the output label space is different from the original model. To solve these issues, the authors propose to use pair-wise correlation of the model’s output of wrongly classified (SAC-w) or mixed (SAC-m) samples to perform fingerprinting.  Using wrongly classified/mixed inputs (instead of adversarial examples) allows the technique to be used in the presence of defenses like adversarial training. Using pair-wise relationships (instead of point-wise predictions) allows fingerprinting to be performed when the stolen model is used for transfer learning. Evaluations show the the proposed technique can detect stolen models with high accuracy and can outperform prior works.
strengths_and_weaknesses: ## Strengths

1. The paper proposes a technique to perform fingerprinting in the presence of transfer learning, which is a new and interesting sub-problem in model fingerprinting that has received limited attention from prior works.
2. In addition to the soft-label setting, the paper proposes a method to convert hard labels to soft labels, which enables fingerprinting when only the hard labels are available.
3. The proposed fingerprinting techniques have been evaluated against several categories of stealing attacks: fine-tuning, pruning, transfer learning, model extraction and adversarial model extraction. The proposed techniques shows high detection performance for fingerprinting and outperforms prior works. 
4. The paper is well-written and easy to follow.

## Weakness

1. SAC does not seem to work under transfer learning (A) in the hard label setting (Table 4).
questions: 1.  Is there a specific reason for why SAC fails under hard label setting with transfer learning? 
2. There is a recent work that proposes a similar solution of measuring pairwise responses to fingerprint models: https://arxiv.org/pdf/2106.12478.pdf. It would be good to cite this paper and point out differences compared to the proposed approach.
limitations: The authors have addressed the limitations and societal impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a model stealing detection method based on sample correlation. The proposed method calculated the correlation among the model outputs for the misclassified samples. The cutmix approach is used to generate more effective sample inputs. The experiment evaluates the performance of the proposed method outperforms in different attack scenarios, such as fine-tuning, transfer learning, and adversarial training.
strengths_and_weaknesses: Strengths:

1. The paper considers many realistic attack scenarios in model stealing attacks, such as fine-tuning, pruning, adversarial training, and transfer learning, which have not been widely explored. 

2. It is nice to see that the protection method considers the label-only cases.

3. Leveraging CutMix to augment data for fingerprinting is novel. 


Weaknesses:

1. The threat model is not well-defined in the paper. What are the attacker’s capabilities in each category of stealing attacks? Does the attacker have access to training data and model architectures?

2. The implementation of the model stealing attacks is unclear. 

3. It is unclear why the misclassified samples help to identify the stolen model. How many misclassified samples are necessary for fingerprinting? 

4. The threshold d in Equation (4) is critical to the success of model stealing detection. However, it is unclear how to select an appropriate value of d.

5. As shown in Table 1 and 2, the existing detection methods can achieve 100% AUC in many attack categories (e.g., fine-tune, pruning). The proposed method only outperforms the existing methods in model extraction attacks.

6. The proposed method does not work well in the Transfer-A attack (Table 4). It would be great if the paper could provide some explanations on it.

questions: 1. What is the threat model in each attack category?

2. Why the misclassified samples can help to identify the stolen model?
limitations: The authors have discussed potential societal impact of their work.
review_title: null

Meta-review: metareview: The reviewers agreed that the proposed method and validation overall are a good contribution. 

We urge the authors to update their paper to reflect the discussed clarifications, e.g., regarding the threat models in use.

 
confidence: Certain
Decision: Accept