Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper investigates dataset inference for encoders. Dataset inference is a defense against model stealing, which allows the defender to identify whether a suspected stolen model has private knowledge from the original model’s training data. Originally formulated for classifiers, the authors adapt dataset inference to encoders by testing for differences in encoder representations on the defender’s training data vs. held-out data. Specifically, they fit a GMM to estimate the distribution of the suspected stolen encoder’s representations on training data. They then use the GMM to evaluate the likelihood of the encoder’s representations: if the encoder is stolen then the likelihood of the training data is expected to be higher than the likelihood of the held-out data. Experiments in the vision domain demonstrate that the method successfully identifiers stolen encoders with high confidence. The confidence is shown to improve when the stolen encoder’s representations are closer to the original, as measured by mutual information and cosine similarity.
strengths_and_weaknesses: *Quality*

The idea of comparing encoding representations on private training data vs held-out data makes sense. However some aspects of the approach are not practical:
- The defender is required to reveal private training/test data to a trusted arbitrator. This is not required in the original formulation for classifiers (Maini et al., 2021), where the defender is able to run dataset inference themselves given query access to the stolen model.
- In order to run dataset inference, the defender’s private train/test data sets must be passed through the adversary’s encoder (in their entirety). This is problematic if it is necessary to query the adversary’s encoder via a public API under their control. The adversary could simply steal the defender’s private data. One way around this would be to compel the adversary to share their model with the trusted arbitrator. However this may not be possible from a legal perspective. Note that this is not an issue for the original formulation by Maini et al. (2021), where it is only necessary to pass a small fraction of the private training set through the adversary’s model.
 
*Originality and Significance*

While dataset inference is not a new idea, it has only been applied to classifiers to date. Extensions to other models with more complex outputs, such as encoders, are well-motivated. There is some prior work on membership inference for encoders (Liu et al., 2021), which ought to be discussed in the paper. Given the close connection between membership inference and dataset inference, Liu et al.’s approach could be an alternative to the approach proposed in this paper. Another related work that ought to be cited is by (Jia et al., 2021). They discuss an alternative reactive defense for model stealing called proof-of-learning. 

*Clarity*

The paper is clear overall. I wonder if Figure 1 could be simplified, as it seems many of the quantities do not need to be computed to run the test against a single potentially stolen encoder.

- Maini, Pratyush, Mohammad Yaghini, and Nicolas Papernot. "Dataset Inference: Ownership Resolution in Machine Learning." International Conference on Learning Representations. 2020.
- Liu, Hongbin, et al. "EncoderMI: Membership inference against pre-trained encoders in contrastive learning." Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021.
- Jia, Hengrui, et al. "Proof-of-learning: Definitions and practice." 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021.
questions: Is it possible to make this approach more practical given the issues raised above?

Can this approach handle more subtle forms of stealing? E.g., where an encoder is initialized by model extraction, then refined on independent training data. Such cases are apparently handled in the original formulation of data inference by Maini et al. (2020).

It would be good to elaborate on what makes GMMs suited for this task. As I understand, GMMs are not universal approximators when the number of components is fixed. In practice, how would one choose the number of components? Could other density estimators be used?
limitations: Limitations are discussed in appendices, however a summary should be included in the main paper.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a detection method for dataset inference for the unsupervised setting of models. Unlike prior work on models meant for classification, the authors in this work focus on models trained to generate feature representations (SSL). The detection technique relies on log-likelihood statistics using density estimation models, along with measures of distributions like MI. The proposed method has promising results, taking a step in the direction of making dataset inference broader and more useful in the real world.
strengths_and_weaknesses: # Strengths

- The paper is well written and sets up the threat model, as well as the proposed method(s) and the rationale behind them, quite well.
- Empirical evaluation is thorough, with datasets ranging from CIFAR to ImageNet. 

# Weaknesses

- Lines 203-205. As a baseline, the authors should include experiments for scenarios where downstream tasks (single, or 2-3 combined) are trained using the "feature extractors". It's true that "a single downstream task cannot adequately reflect...", but it would be nice to have some estimates (performance numbers) to know for sure how much this difference really is.
- Lines 322-322: How about adding noise in intermediate layers (or inputs/outputs), or dropout?
- Table 2: Performance difference between $f_s$ for some cases does not seem to be too far from $f_i$
questions: # Minor comments

- Line 34: Isn't 'dataset inference' the threat model? Like membership inference or property inference?
- Figure 1 is too complicated- either increase the size (and de-clutter) or simplify the flow/structure.
- Line 342: "implies" -> "suggests"
- Line 347: "defense" -> "detection". At the point in time where the method is launched, the adversary has reportedly already stolen the model, and thus it is merely the question of detecting (or confirming) whether that is true, not preventing the adversary from stealing the victim's model.
- Tables 2,3 seem to have overfill issues.
limitations: Addressed in Section 6
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper generalizes the dataset inference defend originally proposed for supervised learning to self-supervised learning. Focusing on the model stealing black-box attack,  the authors utilize GMM to capture the density discrepancy between training and testing datasets of an adversary model. Furthermore, the authors propose new evaluation metrics based on mutual information and cosine similarity, and demonstrate the effectiveness and robustness through extensive experiments. Overall, it is an interesting paper.
strengths_and_weaknesses: - Strengths:
  - The motivation is clearly explained in the introduction to demonstrate that adaptation of data inference defend for SSL models is non-trivial.
  - Extensive experiments are provided to show the effectiveness of the proposed defending towards model stealing attack.
  - The code is provided in the supplementary. I appreciate the authors to do that.

- Weakness:

  - The proposed method heavily relies on the assumption that the stolen model shares similar distribution in the feature space on the private training dataset with the victim model. If so, I'm curious whether the proposed method can only work when the stolen model is well-trained, which is to some extent proved by Table 3 (e.g., in the bottom part, when the number of queries is 5K, the p-values is 1.23e-1 > 0.05).
  - I think a common problem for density estimation related algorithms to work in reality is the generality of the confidence threshold. In your case, this is the p-value threshold which has been set as 0.05 according to line 193. Although it works in your experiments, we can see in Table 3, the range of p-values might vary a lot under different datasets.
  - I wonder how the robustness of dataset inference to obfuscations comes, especially for shuffle, which will totally disrupt the high-dimensional space and GMM. Could you provide further explanation about this property?

  - It would be stronger to show the effectiveness of the proposed mutual information score and cosine similarity score as evaluation metrics for defending if they are applicable for different defending methods instead of only the proposed data inference defending, like the watermark defense mentioned in line 95-103.

  - Writing:
    - In Section 2 (e.g., line 95-103), it seems that the terminology "encoder" has been viewed the same with "self-supervision pre-trained encoder" by default, which is not the case.
    - It would be better to put the theorem in the main paper with the proof remained in the appendix, like Theorem 1.1 in line 266.
questions: - See the Weakness part.
limitations: - See the Weakness part.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a new defense against model stealing of self-supervised models, using the recently proposed dataset inference framework. Similar to original dataset inference for supervised models, the proposed defense detects stealing based on a hypothesis that stolen models exhibit different behavior on private and test data. In this case, this is quantified by using auxilliary density estimation models and observing the log likelihood of data from the two sources. Further, the paper proposes two metrics to measure the quality of stealing: mutual information score and cosine similarity score.
strengths_and_weaknesses: The paper is a solid contribution to an active field and is a good follow-up to [14] which posit that dataset inference is significantly harder in the case of SSL. While the results still are less convincing than in the supervised case, the work clearly demonstrates that successfully adapting dataset inference to SSL is possible if more tailored approaches are used. The paper is very well written and structured. 

I believe the work has no fundamental weaknesses. However, I have several relatively minor concerns (raised below in Questions), which I hope the authors can sufficiently address in their response; I am willing to reassess the work after the discussion.
questions: - The motivation for the additional contribution (two scores) is somewhat unclear. Can the authors make a stronger point for why linear evaluation on several tasks (as in Table 3 in [14]) is insufficient/unnecessary; what is the reason for not including such evaluation alongside proposed metrics? In a similar vein, both scores (MI and cosine) to me seem analogous to fidelity, and not accuracy as claimed in the paper. Can the authors elaborate on this claim further? Making this part more cohesive with the rest of the paper would make the contribution stronger.
- Some results seem to be missing in Table 1, e.g. ImageNet as $D_S$ for first two choices of $D_P$. While the conclusions certainly hold in almost all cases, the test does not pass for CIFAR10 as $D_P$ and SVHN as $D_S$, making it more necessary to provide a complete and convincing evaluation.
- Further, 4.3 claims that the paper focuses on both MSE and InfoNCE stealing losses, but 5.1 seems to be imply only InfoNCE is used in the evaluation. Can the authors comment on this mismatch? The evaluation would be stronger if MSE and SoftNN were added.
- (Nit) It would be interesting to see if different density estimation methods would be able to further boost the performance, did the authors experiment with this?
limitations: I find the discussion of societal impact sufficient. Regarding limitations, it would be good to more explicitly acknowledge the strength of results compared to the supervised case and reflect on statements given in [14]. More importantly, to avoid the cat-and-mouse nature of attack/defense work in the field (similar to adversarial examples literature until recently), I find it necessary to consider the prospect of adaptive adversaries, i.e. the robustness of the defense against attackers aware of the proposed defense. I acknowledge that designing an adaptive attacker is often a highly non-trivial contribution in itself; a short explicit discussion would still benefit the paper.
review_title: null

Meta-review: metareview: This paper joins an interesting area that tackles the use of models in the real world that are accessible publicly via APIs. In these cases, there may be adversaries that  attempt to steal the model. This can be done by accessing information about the model from particular queries. One of the approaches used to tackle such scenarios is to develop defenses that can detect when such models are being stolen. Much of this area is focused on looking at supervised models; the authors introduce a similar approach for encoders.

In the encoder case, techniques that involve the decision boundary, suitable for supervised models, no longer apply. The authors provide some technical innovations to get around this issue. Essentially they look for evidence of memorization by using a metamodel.

Overall, the problem is well-motivated and the tools the authors develop are interesting; the results also appear convincing. The reviewers reached a near-consensus that the paper is worth accepting, and I agree. Nothing here is extremely groundbreaking, but it's a well-crafted approach to handle a case of an important problem that hasn't been addressed yet. 

The authors generally responded to all of the questions the reviewers had, and I furthermore found the answers convincing. Ultimately I think it's worth accepting.
confidence: Less certain
Decision: Accept