Dataset Inference for Self-Supervised Models
Authors: Adam Dziedzic
Haonan Duan
Muhammad Ahmad Kaleem
Nikita Dhawan
Jonas Guan
Yannis Cattan
Franziska Boenisch
Nicolas Papernot
Venue: NeurIPS 2022 Accept (2022)
Keywords: self-supervised models
model stealing
defenses

Abstract: Self-supervised models are increasingly prevalent in machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used as a service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due to the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations is higher on the victim's training data than on test data if it is stolen from the victim, but not if it is independently trained. We compute this log-likelihood using density estimation models. As part of our evaluation, we also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results in the vision domain demonstrate that dataset inference is a promising direction for defending self-supervised models against model stealing.

Dataset Inference for Self-Supervised Models
Adam Dziedzic" Haonan Duan', Muhammad Ahmad Kaleem', Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, Nicolas Papernot University of Toronto and Vector Institute
Abstract
Self-supervised models are increasingly prevalent machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce new dataset inference defense, which uses the private training set of the victim encoder model to attrihute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations higher on the victim's 's training data than test data if it is stolen from the victim, but not if independently trained. We compute this log-likel liikeiihoodd using density estimation models. As part of evaluation, also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results the vision domain demonstrate that dataset inference promising direction for defending self-supervised models against model stealing.
1 Introduction
The self-supervised learning (SSL) paradigm enables pre-training models with unlabeled data to learn generally useful domain knowledge and then transfer the knowledge to solve specific downstream tasks. The ability to learn from unlabeled data alleviates the high costs of labeling large datasets (24). and the transfer learning setup reduces the computational costs of retraining. These advantages have
made SSL increasingly popular 200 in domains like vision [5] language [2]. and bioinformatics
Recently, commercial service providers like Cohere || and OpenAl (2] began offering paid query access to trained SSL encoders over public APIs. This exposes the encoders to black-box extraction attacks, i.e., model stealing. In model stealing attack, attacker aims to train approximate copy of a victim model by submitting carefully chosen queries and observing the victim's 's outputs. The high costs of data collection, preprocessing and model training make encoders valuable targets for stealing. For example, the training data of CL.PP includes 400 million image and text pairs (37), while computation costs of training large language model exceed million USD 41 The threat of model stealing in SSL real: researchers have demonstrated that encoders can be stolen at a fraction of the victim's training
Yet, most current defenses designed for supervised and cannot be directly applied to encoders 14!
14 39
models 15
2 35
Dataset inference 31 is sfate-of-the-an defense against model stealing in the supervised learning setting. The defense provides ownership resolution: it enables the model owner to make a strong statistical claim that given model is stolen copy of their model by showing that this model is derivative of their own private training data. Dataset inference does not require retraining or overfitting the model to any form of explicit watermark [4] and has been shown to resist attacks from
"Corresponding and leading author: adam, fiam.dziedzic @ttoronto.ca t Equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Training Inference
Test
Data D
p(hpp.) > p(hon)
how hpps
Dp1
Stolen f
Density Estimator &
Dp2
p phhppp) = p(hpg) Independent f
Private Train
Royz
Data Dp
Ownership Resolution for Encoders. First, an arbitrator trains density estimator E: divide Dp into non-overlapping partitions P1 and D2, and train density estimator using the representations lup of I on D22. Next, the arbitrator performs dataset inference: apply & on the representations of Dp1 and Dy of the encoder For stolen encoder, the log-likelihood of the representations hipp is significantly higher than hpx- while, for an independent encoder, the log- likelihoods of the representations are significantly different. adaptive adversaries B1. These properties make the defense particularly attractive for SSL, as large encoders can be expensive retrain, and performance is paramount because carries over to all downstream tasks. However, the original dataset inference algorithm from []] cannot be applied to encoders, because it relies on computing distances between data points and decision boundaries. These decision boundaries do not exist in SSL encoders since they are trained on unlabeled data.
Figure 1:
We introduce new dataset inference method (Figure D defend against model stealing for encoders. Our algorithm is suitable for the high- gigh-dimensional outputs of SSL encoders and does not rely on labeled data or decision boundaries. Instead, it relies solely on the private training data of the victim encoder as signature. Moreover, our algorithm retains the advantages of dataset inference for supervised models 3I namely, it does not require retraining or overfitting the SSL encoder.
Our key intuition is to identify stolen encoders by characterizing differences between an encoder's representations on its training data such as stolen copies, exhibit different behavior on the victim's private training data than on test data; independently trained encoders do not. These differences exist because encoders overfit to training data 18 30 Although for well-trained encoders the effect is minimal on any given data point, we show that when aggregated over many training points provides statistically strong signal. To identify the differences, we train a Gaussian Mixture Model (GMM), as an efficient general approximation, model the distribution of an encoder's data representations from its training domain. We then use the GMMs to predict the log- liglliiklhood of the encoders' representations of the victim's 's training set and a test set: derivatives of the victim encoder will have higher log- likelihood on the training set than on the test set. We perform experiments on five datasets from the vision domain and show that we are able to distinguish between stolen and independent encoders even in cases when adversaries obfuscate the representations from the stolen encoders to hide the theft (e.g., by shuffling the elements the representation vectors or applying to them some form of a linear transformation).
unseen test data. The victim encoder and its derivatives,
As part of our evaluation, we also introduce new metrics to measure the fidelity of stolen encoders without involving downstream tasks and quantify the effectiveness of theft detection. We compute scores directly on the representations using tools from information theory and distance metrics. These methods work well because losses used for stealing encoders directly minimize distances between representations of victim and stolen encoders. Our mutual information score to assess the quality of the stolen encoders robust against obfuscations that an adversary might apply the representations returned by a stolen encoder. Without any obfuscation, our cosine similarity score shows a clearer distinction between stolen and independent encoders. Finally, using these metrics, we observe that the higher the quality of the stolen encoders, the more confident our dataset inference defense becomes.
Our main contributions are as follows:
We propose a new defense against model stealing attacks on encoders, by combining dataset inference with density estimation models for ownership resolution on unlabeled data.
We are the first to design new metrics that quantify the quality of stolen encoders, which are derived from the mutual information and distances between representations.
We evaluate our defense using five datasets from the computer vision domain and show that our defense can successfully identify stolen encoders with strong statistical significance.

---

2 Related Work
In model stealing, an adversary queries the victim model, obtains outputs, and uses them to recreate copy of the victim 22 This is most commonly performed with black-box access, egg, via a public API. When stealing encoders in SSL, the goal of an adversary is to extract high-quality embeddings either to train a stolen copy that achieves high performance on downstream tasks, or to obtain faithful replicas of the victim's 'ss embeddings on the same inputs. Stolen encoders might be further used for model reselling, backdoor attacks, or membership inference [9]
While most past research model stealing and defenses focuses on classifiers trained via supervised
learning, recent work constructed new attacks that target encoders |44 39).
The main differences between the attacks in these settings are that the outputs of encoders leak more information due to their higher dimensionalities
39 and the attacks require different loss functions. Inspired by
contrastive learning. Cont-Steal
39 provides a method of stealing encoders using a loss function framework for losses including mean squared error, InfoNCE, Soft Nearest Neighbor,
based on InfoNCE [5]. SSL extraction [4-- general Seamese-network-bbee
stealing pecoderrs-leeerane and Wasserstein distance. The authors empirically show that an adversary can steal an ImageNet victim encoder in less than fifth of the queries required for training.
Proof of Learning (PoL) 22 is a reactive defense that involves the defender claiming ownership of a model by showing incremental updates of the model training. It complementary method to dataset inference, which instead identifies a stolen model. PoL. could be applied directly to SSL encoders, however, it requires an expensive verification process, where the verifier needs perform model updates and the prover needs to save intermediate weights of the model, which is more expensive than dataset inference with GMMs. Unfortunately, other current defenses against model stealing for supervised learning are inadequate for defending encoders, and adjusting them to the specificities of encoders is non-trivial [4 One line of approach watermarking p 14, where the defender embeds a secret trigger into the victim encoder during training to determine ownership at test time. However, watermarking based defenses have two significant disadvantages. First, researchers have repeatedly shown that adaptive attackers can remove watermarks without severely affecting model performance
46 e.g., through pruning. tne-ttunig, rounding or performing backdoor removal 4.. Second, the watermark must be embedded during training; if a model is already trained, or if watermark defense needs to be updated, the model must be retrained. This is not practical for large encoders.
Another state-of-the-art defense against model stealing in the supervised setting is dataset infer- ence [31 which addresses these disadvantages. However, the adaptation of dataset inference to encoders difficult, because (1) the algorithm [31] relies on decision boundaries, which do not exist for encoders; and (2) encoders are less prone to overfitting, which provides the signal for dataset inference [4 31. Therefore, naive approaches like computing the loss of representations or distances between train and test sets are ineffective [14]. To overcome these issues, we extract more signals from the representations by estimating their densities for train and test sets, as described in Section 3
When it comes to comparing signals within representations, prior work has considered measuring the similarity between different representations. This has led the proposal of various similarity metrics including canonical correlation analysis (CCA) (33,, centered kernel alignment (CKA) 225, and the orthogonal Procrustes distance [3) which use methods from linear regression, principal component analysis (PCA), and singular value decomposition (SVD). However, these metrics are very general and complex. They have also been shown to disagree in some cases
[ Since we can only access the final embeddings from encoders, we design metrics more closely related to our setting.
3 Defense Method
Dataset inference serves as a defense against model stealing. It enables the model owner or third-party arbitrator to attribute the ownership of a model in the event that it is stolen. The idea is to take advantage of the effects of knowledge from the victim's training set and to use that as a signature for attributing ownership. Given a well-trained encoder, the effects are small on any single data point; however, when aggregated over many points in the training set, they collectively provide a strong statistical signal for dataset inference. As depicted in Figure 2 for a victim that leverages the private data Dp during training stolen copy, identify a difference between
3

---

distributions of the train data's representations hD, and the test data's representations h px while the distributions of these representations from an independent encoder cannot be distin- guished. We use this signal to determine whether model is derivative of the victim's s training data, i.e., either directly trained on the data or stolen from the victim. To capture the sig- nal, we first partition the victim encoder's 's training data into two subsets and use one to train Gaussian Mixture Models (GMMs) with the aim of modeling the distribution of representations of data from the encoder's 's domain. Then, we apply the GMMs to perform dataset inference by measuring the oog-likeldood of the encoder's 's representations the remaining training data some test data (see Figure D. For the victim encoder or its stolen copies, the log-likelihood of training data representations is significantly higher than the one of test data. We use this to
Now
Row
kay
Stolen f Victim f.
Independent
Figure 2: Dataset Inference In- tuition. Dp and DN come from the same distribution. For inddpen- dent encoders, their representations hDw and hpp are i..d. while for victim/stolen encoders, IDD and hop induce different distributions.
construct statistical t-tests to determine whether model is stolen, and present our empirical results in Section 5
3.1 Threat Model
As described in Figure 7. we consider a victim encoder fo trained on private training dataset D An adversary with black-box access to fe trains a stolen encoder fs by querying the victim with data points from a dataset Dş to obtain representations. These representations are then used part of the training objective for the stolen encoder. During the dataset inference, we assume the presence of a third-pau partt-rrtyy arbitrator, such as law enforcement, with white-box access to the victim's 's training data, as well as all encoders. Additionally, the arbitrator requires the test dataset Dy from the same distribution as Dp to perform dataset inference. An independent encoder ʃa is trained with no access to the victim's 's private training set Dr and without any queries to the victim's 's encoder. It is used as a baseline for the ownership resolution in the dataset inference.
3.2 Density Estimation of Representations
Representations from encoders contain rich features for given inputs. We analyze the inputs that come from the training and test sets through their representations. For the training inputs, we compute their representations and model their densities [16]. To this end, we leverage GMMs as universal approximators of densities. We give representations a probabilistic interpretation such that they have a smooth enough density which can be ippeed by any specific nonzero amount of error using GMM with enough components. Each component has separately parameterized mean f and covariance Σ. In some cases, we observe that GMMs can overfit to their training data when no constraints are applied to the covariance matrix, hence we limit the covariance matrix for each component to be diagonal. Moreover, this constraint makes training more computationally efficient since it avoids storing and inverting full high-dimensional covariance matrices.
3.3
Data Flow
The full flow of our dataset inference for encoders consists of the following four main steps (which are also visualized in Figure in Appendix):
1. Victim Training. The victim's encoder f. is trained using the whole private training dataset Dp-
2. Encoder Stealing. To steal the victim encoder fu, adversary queries fo with data points from Ds to obtain representations ling R hD2 = fe (Ds). With these representations, the adversary trains the stolen encoder ʃa in contrastive manner.
3. Training Estimators. To perform ownership resolution, an arbitrator trains three density estima- tors E,,, &g and & for the victim fo. stolen fs. and independent encoder fa as follows:
a) Dr (where Dp is not necessarily the whole private training dataset) is divided into two non-overlapping subsets Dri and D22 While D22 serves as the base for training the density estimators, Dr1 is used to evaluate density estimates of the private training data vs part of the test data DN.

---

b) For a given encoder f E fer the arbitrator generates representations h E R" hppz = p22) on dataset D2. Training the density estimators on the respective represen- tations yields the final density estimators E,, and &
Estimating Densities. The arbitrator generates representations of Dp1 and (a subset of) D with each encoder f E {fe. f.). Applying the respective density estimator & E {Ex, Eg. E on the representations yields the log- ligeeiil of each data point x in the respective dataset: Vr E D p(z) E(f(z)).
3.4 Ownership Resolution
For an encoder f. we compute the log- segllikelihood on Dpi and DN as: up [Ppp]] EZEDpI E(f())) and ux := I Ezeny {(f(z). The density estimator ε measures the similarity between the distributions over the victim's representations of the training Dp vs test data DN The intuition behind the setup is that if an encoder was trained on Dp, representations of Dp1 are much more similar representations of Dr,, because the whole dataset Dp was used for training the encoder, however, representations of Dy differ from representation of Dp2 since DN was not used to train the encoder. For victim f. and a stolen encoder fa. up is significantly larger than uN, whereas, for an independent encoder fú, the values do not differ significantly. Finally, we carry out a hypothesis test with the null hypothesis being: Hy := up S uN. If the null-hypothesis can be rejected (p-value < 0.05), i.e., when the log- ligellkldoodd for the training set Dp1 is higher than that for the test set DN we can conclude that the tested model was stolen. On the other hand, if the null hypothesis cannot be rejected then the test is inconclusive and we cannot determine if a tested encoder was stolen or not.
4 Encoder Similarity Scores
Measuring the quality of stolen encoders allows supervised learning, the quality of a stolen model is evaluated using two main objectives, namely task accuracy, which is the model's performance on the test set, and fidelity, which is the agreement in the predictions for a given task between the stolen and the victim model 19 One of the approaches to measure the quality of extracted encoder is use its outputs train downstream task and compute the accuracy of that task or fidelity (with respect to the outputs of the downstream task trained on the victim encoder). However, single downstream task cannot adequately reflect the degree of similarity between encoders since it reduces their high dimensional embeddings to single label representations, which are confounded by choices of downstream data and training protocol. Instead, we propose two new metrics. Our first metric is an cotomatiionthheoree
assess attacks and defenses. In standard
score based on mutual information I10 px Our second metric a cosine similarity score based on the representations returned by different encoders. These metrics correspond to the fidelity metric in supervised learning. The behavior of the two metries differs certain cases, for example, when used on obfuscated representations (e.g., with shuffled elements) or with independent models, however, we find that the overall trend similar. Moreover, the mutual information score is based on an approximation while the cosine similarity score is calculated exactly given representation vectors. Often the effectiveness of defenses may be underestimated against low-quality stolen copies that haven't successfully stolen victim behavior. Our metrics help disentangle such effects and enable faithful evaluation of defenses.
4.1 Mutual Information Score
Our first approach to assessing the quality of a stolen encoder uses a score based on mutual information. We sample N data points from the victim's 's private training dataset Dp and pass them through the encoders fe, fa. and f: to generate the respective representations. Per standard practice, we recenter and normalize the representations [ We denote the entropy by Il and compute according to Algorithm 2 which takes fu, Dr, and N as input. For the joint entropy H(fu, f.(D), we generate representations from the two encoders (in this case victim fo and stolen fa) and concatenate them, which increases the dimensionality of the final representation to 2d, while other steps remain unchanged. detailed algorithm for computing the joint entropy can be found in the Appendix as Algorithm 3 We compute an approximate score that is based on the definition of mutual information [s between the victim encoder fo and the stolen copy fs as well as the analogous mutual information [(fv. f() between the victim encoder fo and the independently trained encoder f..
5

---

We rely on approximations since we measure mutual information using finite data. Yet, in practice,
such approximations have proven useful 32
We define our mutual information score as follows:
T(So, = ((A.) H(AIDr)) - H(,,,
(1)
A higher value of the mutual information I [(fv f. Dp) indicates a higher information leakage incurred by the stolen encoder. Expectedly, mutual information is higher between the victim and the stolen encoder than between the victim and independent encoders [(fe: fs Dr) >> I(fe, Dp) We can normalize mutual information into a score (between 0 and 1) by setting the lower bound as the mutual information between the victim fu and a randomly initialized model fc: Imin = I(J Sr Dp) and the upper bound as the mutual information between the victim and itself: Imaz = 1 1s So Dp) For the current mutual information score I., the normalized score is defined as S == 1. L--main
Imme Amin
4.2 Cosine Similarity Score
The second score to assess the quality of a stolen encoder is based the cosine similarity between its representations and the victim's representations. More specifically, we first compute representations for the two encoders
set of N randomly selected data points from the dataset Dp. Again as per standard practice [3], we recenter and normalize these representations, For each of the N inputs, we then compute the cosine similarity between the corresponding representations from both encoders where the cosine similarity sim (a, b) ||||a||b|la
a"b for representation vectors a and b.
We show (in Section maximize the cosine similarity between representations from victim and stolen encoders. We thus propose to use the cosine similarity score C as a metric, which we define as: C = |sim b) (2). The score yields values in the range [0, 1], with higher score indicating closer representations. To calculate a per-encoder cosine similarity score, we average the cosine similarity scores over all inputs. We find that the cosine similarity score is well-calibrated across encoders. Namely, an independent encoder, expected to have representations unrelated to the victim encoder, has an average cosine similarity concentrated around 0 [ while stolen encoder exhibits significantly higher scores. The cosine similarity score is also easy to compute since it only requires the corresponding representations of the two models and their dot product.
that the loss functions, which are used for stcaling encoders, directly
4.3 Analysis
There are various ways in which attacker may steal encoder. To simplify analysis of the cosine similarity score, we consider the two best pest-performing loss functions used for stealing [4] : the first where the attacker minimizes the non-contrastive MSE (Mean Squared Error) loss between its representations and the victim encoder's 's representations to train the stolen encoder, and the second where the attacker uses contrastive loss function, such the InfoNCE loss 4 which is used in SimCLR (5)
Stealing with MSE loss. In the case where the MSE loss is used, let I; be query made by an attacker and let (z;) = he. (z.) = ha, E R" be the corresponding representations of the victim and stolen encoders, respectively. The MSE loss between these two representations is 1 (her, hh.,)) = he, - 11. It follows directly that minimizing the MSE loss also minimizes the l2 distance between representations and equivalently maximizes the cosine similarity between representations: Theorem ||a 6|12 = 2 - sim(a, 6)), ||al|2 = ||b]| = 1 (see G.1
Stealing with a contrastive loss. When an attacker uses a contrastive loss function for stealing, minimizing the loss corresponds maximizing the sum of the cosine similarities between positive " /T)) The InfoNCE loss, or contrastive losses in general, also increase
pairs, f.e., >== the mutual information score R We therefore expect that stolen encoders will have larger similarity scores w.r.t. the victim encoder than independent encoders. We refer the reader to Appendix + for more detailed discussion of the loss functions and their relationship with the similarity scores.
mm (sim(h.,,

---

5 Empirical Evaluation
We evaluate our defense against encoder extraction attacks using five different vision datasets (CIFAR1O, CIFARIO0 23 SVHN [4 STL10 R and ImageNet (II).. Table I shows that our dataset inference method is able to differentiate between the stolen copies of the vietim encoder and independently trained encoders by using the victim's 's private training data as the signature. We also show that our defense works in the scenario where the adversary modifies the representations to render them inconspicuous, e.g, by shuffling the order of elements in the representation vectors. To assess the quality of the stolen encoders and the performance of our defense, we measure the mutual information and cosine similarity scores between encoders and present our results in Tables 2 and
5.1 Training Victim, Stolen and Independent Encoders
Victim. We use victim encoders trained on the ImageNet, CIFARIO, and SVHN datatsets. For the
ImageNet victim encoder, we use a model released by the authors of SimSiam + To train CIFARIC 10 and SVHN victim encoders, we use an open-source PyTorch implementation of SimCLR For SVHN, we merge the original training and test splits, and use the mandomly-selecterr 80% as the training set and the rest 20% as the test set. This is necessary because the original training and test splits for SVHN are not i.i.d [6) which violates the assumption for dataset inference (see Section B). The ImageNet victim has an output representation dimension of 2048, while the CIFARIO and SVHN victim encoders have 512 312-dimensional representations.
Stolen. When stealing from the victim encoders, we evaluate different numbers of queries from various datasets, including CIFAR10, 10, SVHN, ImageNet, and STL10. Stolen encoders are trained in a similar contrastive way as the victim and use the InfoNCE loss, where the positive pairs consist of representations from the victim and stolen encoder for a given input. Algorithm summarizes the stealing approach used by an adversary.
Independent. For each victim encoder, we train independent encoders using datasets different from the victim's private training dataset Dr. The encoders are trained with the SimCLR approach, similar to the way the victim encoders were trained. In the case where the dataset used to train the independent model had different image dimensions from the victim's ' training dataset, the dataset was resized to be of the same size.
More details on the training and stealing of encoders can be found in Section D.3 of the Appendix.
5.2 Dataset Inference on Encoders
Setup. We train GMMs with 10 components for SVHN and CIFAR1O, and 50 components for ImageNet. In general, we observe that the larger number of components for GMMs, the better the defense is. For ImageNet, we restrict the covariance matrix to be diagonal for efficiency. For CIFARIO and SVHN, we use the full covariance matrix. For SVHN and CIFARIO, we use 50% of the training set to train GMMs, and the remaining for evaluation. For ImageNet, we use 100K images from the training set to train GMMs, and another 100K of the training set as evaluation set. We normalize representations by l2 norm for training GMM. For ImageNet, we also standardize representations (subtract mean and divide by standard deviation) before normalization. We do not use augmentations in dataset inference. For each setting, the hyperparameters are tuned on the victim model.
model and
Evaluation of our Defense. The empirical results in Table [ demonstrate that we are able to differentiate between stolen and independent encoders from the difference in fog-liselihoods. We observe that the stolen encoders have significantly larger Âμ than the independent encoders. The p-values further show that for stolen encoders the null hypothesis is rejected while for independent encoders, the test is inconclusive. Similar to dataset inference for supervised learning 3I, the victim model typically has the largest Áμ and the smallest p- p-vauues. We also observe that our method is better at detecting encoders that are stolen using queries from the victim's 's training set.
Number of Stolen Queries. Table 3 shows that as the attacker steals with more queries, the p- value from our defense becomes lower. This is consistent with the finding in (31) that dataset inference works better with stronger stolen encoders. We also find that our defense is able to detect stolen
7

---

Table Dataset inference via density estimation of representations. We detect if given encoder was stolen. fo denotes the victim encoder trained on data D, f. is the stolen encoder extracted using queries from a given stealing dataset D. and Sa is an independent encoder trained on data (different than the victim's 's private training data). Each value is an average of 3 trials. A/ is the effect size from the statistical t-test. Obfuscations: the representation can be modified by an attacker in the following ways: (1) Shuffle the elements in the representation vectors, (2) Pad with zeros or add zeros at random
positions, and (3) apply a linear Transform. The first row below denotes the victim's ' private data Dp.
Victimns 's private data:
CIFARIO
SVHN
ImageNet
Encoder
Obfuscate
D
p-value
D
p-valoc
Ap
D
p-value
Ap
f. N/A
CIFARIO
5.61e-82
18.92
SVHN
2.75e-125
23.88
ImageNet
6.23e-14
7.09
SVHN
3.97e-2
3.04
SVIN
6.35e-41
13.36
SVHN
3.33e.4
f.
N/A
CIFARIO
8,73e-7
5.09
CTFARIO
2.38c-4
4.61
CIFARIO
1.47e44
6.21
STLI0
1.04o-2
3.42
STL10
1.230-5
5.22
STLI0
1.09c4
5.87
ImageNet
6.34e-3
3.47
ImageNet
9.81e-3
3.74
ImageNet
3.14e-5
7.32
Shuffee
CIFARIO
6.72e-6
4.98
CIFARIO
7.32c-4
4.77
CIFARIO
6.72e-4
5.21
f.
Pad
CIFARIO
3.44c-6
4.84
CIFARIO
2,51e-3
3.08
CIFARIO
2.31e-3
4,23
Transform
CIFARIO
6.81e:7
5.11
CIFARIO
6.45e.3
CIFARIO
8.45e-3
CI
3.67e-1
-0.37
CIFARI00
6.21e-1
0.52
CIFAR100
7,53e-2
f.
N/A
SVHN
2.96c-1
0.98
CIFARIO
4,820-1
0.56
SVHN
5,420-1
0.69
encoders even if the attacker only steals from a small number of queries. For example, in Table 3 we able to claim ownership when only 50K 100k queries are used for stealing ImageNet victims (around 4% of its training set).
Robustness of Dataset Inference to Obfuscations. The attacker obfuscate the stolen encoder representations by, for instance, applying shuffling (changing the order of elements), padding (adding zeros), linear transformations (e.g., scaling or adding constant). These obfuscations have little impact on the downstream performance [17] but may pose challenges to the defenses of the victim. The results in Table show that the p-values for the stolen encoders after attackers' obfuscations remain low, which implies that our method is robust to these types of obfuscations.
5.3 Measuring Quality of Stolen Encoders
Setup. To measure the quality of stolen encoders, we lect a random subset of N = 20K unaugmented images from the private training dataset Dr and compute their representations from stolen and victim encoders. We then centralize (subtract the mean for each dimension) and nor- malize the representations (divide by the f2 norm). For the mutual information score, we first estimate the entropies (fo)), H (fa), (fv. fz), which are then added and nor-
Stolen Independent
15 10
Density
malized as in Section 4.1] The score is capped to be in the
range [0, 1]. To compute the cosine similarity score, we
find the absolute value of the dot product of corresponding representations for the two encoders (Equation 2). These dot products are then averaged over all representations.
0.0 0.2
0.4
0.6
0.8
Cosine Similarity Score
Evaluation of Metrics. To evaluate the mutual informa- tion and the cosine similarity scores, we conduct two sets of experiments to verify if: (1) the scores are higher for stolen than independent encoders, and (2) the scores increase as more queries are used to steal encoders, which suggests higher quality of the stolen copies [14. In Table we observe that both our scores assign higher values to the stolen encoders than the independent encoders. Table 3 that our mutual information and cosine similarity scores generally increase while the p-values from our dataset inference decrease with respect to the number of queries used to steal an encoder. This implies that the performance of our defense is consistent with the similarity metrics and becomes more effective as the quality of the stolen encoder improves. We also plot a histogram of the cosine similarity scores for the stolen and independent encoders in Figure 3 for an SVHN victim encoder, a stolen encoder from (using CIFAR10 10 training data for queries), and an independent encoder (trained on CIFAR100). There is pronounced difference between the two distributions with the
Figure 3: Distribution of cosine similar- ity scores.
8

---

Table 2:
Encoder similarity scores. We compare encoders via the encoder quality metrics using the same setting as in Table We compute the score S fu. based on the mutual information between a given encoder (in a row) and the victim encoder So. Analogously, we compute the cosine similarity
score So).
Vicrim''s private date:
CIFARIO
SVIN
ImugeNNer
Encoder
Ohfascate
D
S(.
C(s f0)
D
S(- fu)
C(.
D
fw)
C(- fw)
fo
NIA
CIEARIO
1.0
1.0
SVHN
1.0
1.0
ImageNett
1.0
1.0
NIA
SVHN
0.75
0,504
SVHN
0.96
0.1
SVHN
0.86
0.39
J.
NIA
CIFARIO
0.8
0.95
CIFARIO
0:34
0.69
CIFARIO
0.85
0.43
NIA
SEL1O
0.89
SILIO
0.55
0.89
ImageNet
0.78
Shugfle
SVHN
0.74
0.002
CIFARIO
0.34
0,005
SVHN
0.86
0.005
Puf
SVHN
CIFARIO
0:35
0.015
SVHN
0.003
Transform
SVHN
0.504
CIFARIO
0.3
SYHN
NIA
CIFARIC0
CIFARIOD
CIFARIDO
0.0015
NIA
SVHN
0.12
CIFARIO
0.90
0.009
SVHN
0.002
Table 3: Encoder similarity scores and p-values from dataset inference vs the number of queries. The quality of the stolen encoders increases with more stealing queries, which is reflected by the rise in the mutual information and cosine similarity scores as well as the better performance of our defense as indicated by the decreasing p p-values. Dp is the private dataset used to train the victim and Dy is the dataset used for stealing.
Dp
Dg
Score
Nomber of Queries
3K
10K
20K
30K
40K
50K
100K
200K
250K
8(. ( C(s fu)
0.62
0.79
0.79
0.81
0.82
0.84
0.5
0.85
0.86
ImageNet
SVINN
0.25
0.32
0.33
0,36
0.35
0.38
0.38
0.40
0.39
p-talues
1.25e-1
791e-2
6.3e2
8.88-2
1.10622
2.1ee33
1.1633
3.33e-4
7K
30K
50K
86. fo)
0.55
0.60
0.62
0,75
0.58
0.64
0.87
0.82
0.88
ImageNet
CIFARIO
Cl. p-oouues
0,40
8.88e-2
7.13e-2
8.33-11
4.14e-1
3.46-33
8.51e-21
9.23e1
7:36-2
1476-4
7K
10K
30K
50K
86. (e)
0.75
0.72
0.89
0,92
ImageNet
SELIO
C( fu)
0.29
0.36
0.38
0.37
0,44
0.52
0.58
pooducs
9.63e-1
8:21e-1
7.32p-1
5.446-1
1.26611
5.98-22
8.100-2
6.28o-2
1 1000-4
20K
100K
250K
SC. (c)
0.23
ImageNet
Image:Ntt
C(:.6))
0.29
0.48
0.49
0.51
0.46
0.38
0.52
0.76
0.78
p-values
9.88e 1
3.21e-1
5.32e-1
1.08e 1
3,61e-3
3.976-
5.3e-1
8.72e-4
3.1
cosine similarity scores for the independent encoder being close to 0 and the scores for the stolen encoder being much higher than 0.
Robustness of Metrics to Obfuscations. We also consider the effect of obfuscations on these metrics. Without any obfuscation of the representations from stolen encoders, the cosine similarity score shows a clearer distinction between stolen and independent encoders than the mutual information score: in Table the cosine similarity scores for all independent encoders are close to zero, but the mutual information scores can be quite high (such as 0.9 for the independent encoders of SVHN, which is likely because of the mutual information score being based on an approximation). However, the mutual information score is robust to the obfuscations of the attackers while cosine similarity is not: in Table g the cosine similarity score for the stolen encoders after shuffling and padding drops close to zero. Mutual information, as more general metric based on the information measurement instead of the brittle structure of the representation vectors, performs better and is oblivious to the obfuscations that attackers might introduce.
5.4 Limitations
If the t-test run as part of dataset inference inconclusive for an extracted encoder, we cannot state whether the encoder was stolen. Similarly, for an independent encoder, there is the possibility of it being incorrectly classified as stolen. Previous work 226 38 has shown that self-supervised encoders trained using heavy augmentations and contrastive learning generalize better than their supervised counterparts, which makes it harder for the dataset inference to differentiate between train and test representations in SSL than in the SL setting [14] The loss values of projected individual

---

representations are insufficient for dataset inference 14. We build on top of this observation to enable dataset inference for encoders and use GMMs to distinguish between train and test representations.
6
Conclusions
New public APIs expose self-supervised encoder models which return high-dimennionall embeddings for provided inputs. Adversaries can use these embeddings steal the encoders. We present novel method based on dataset inference for defending against such stealing attacks along with metrics to assess the quality of the stolen encoders and quantify the effectiveness of our defense. We observe that knowledge contained in the private training set is transferred from the victim encoder to its stolen copy. Thus, the private data acts signature of the victim encoder. By leveraging density estimation on the respective encoders' representations, we obtain signal allowing to differentiate between the encoder's 's training and test data. This difference is detectable in both the victim encoder and its stolen copy but not in independent encoders which are legitimately trained on different data than the victim's private training data. Thus, we are able to flag the stolen copy of the victim encoder while not accusing creators of legitimately trained encoders of theft. We show the high effectiveness of our defense on vision encoders. Future work may explore additional applications defense and metrics beyond model stcaling and ownership verification, well their in other domains such as natural language processing (NLP). In particular, our method may help enforce the ethical usage of sensitive online data, such images social media, in accordance with privacy regulations by auditing if given provider's encoder contains knowledge of these sensitive data.
our proposed
Acknowledgments
We would like to acknowledge our sponsors, who support our research with financial and in-kind contributions: CIFAR through the Canada CIFAR Al Chair program, DARPA through the GARD program, Intel, Meta, NFRF through an Exploration grant, and NSERC through the Discovery Grant and COHESA Strategic Alliance. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute. We would like to thank members the CleverHans Lab for their feedback.
References
[1] Cohere, https//ccohrre.ai URL https:////cc ai/
[2] Openai, https:/openai.ccm. URL https://openai: com/
[3] Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views, 2019. URL http::tt////xxi..rr///////// org//b/996/
00910
[4] Franziska Boenisch. A systematic review on model watermarking for neural networks. Frontiers in Big Data, 4, nov 2021. doi: 10.3389/fdata.202.. 2021.72663.
[5] Ting Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. International Conference on Machine Learning, 2020.
[6] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning, 2020.
[7] Xinyun Chen, Wenxiao Wang, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, and Dawn Song. Refit: unified watermark removal framework for deep learning systems with limited data. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, pages 321-335, 2021.
[8] Adam Coates, Andrew Ng, and Honglak Lee. An Analysis of Single Layer Networks in Unsupervised Feature Learning. In AISTATS, 2011. https://cs.
stanford....... http:///sssstasfrdd.... aidu/"acoates"
papprss/coatesilenng..iic aistats tatistttt__olt...._aeee11e111ttt 2011.pddf pdf
[9] Tianshuo Cong, Xinlei He, and Yang Zhang. Sslguard: A watermarking scheme for self- supervised learning pre-trained encoders. CoRR, abs/2201. 1692, 2022. URL. https: :tttps:///rxiyy org /
10

---

[10] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory ( (Wiley Series in Telecom- munications and Signal Processing). Wiley-Intersceie
USA, 2006. ISBN 0471241954.
[11] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hicrarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages
leee, 2009.
[12] Jacob Devlin, Ming-Woi Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 04805, 2018.
[13] Frances Ding, lean-Stanislas Denain, and Jacob Steinhardt. Grounding representation sim- ilarity through statistical testing. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wort- man Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreviev..eettt wwtp://ww.eeeooommmeeeeeeee. _wjjdd
[14] Adam Dziedzic, Nikita Dhawan, Muhammad Ahmad Kaleem, Jonas Guan, and Nicolas Pa- pernot. On the difficulty of defending self-supervised learning against model extraction. In International Conference on Machine Learning. 2022.
[15] Adam Dziedzic, Muhammad Ahmad Kaleem, Yu Shen Lu, and Nicolas Papernot. Increasing the cost of model extraction with calibrated proof of work. In International Conference Learning Representations, 2022. URI https:/ttts////xii wwwp://gg/abaa..///o// 092433
[16] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http://ww..depplerrninnggg
org
[17] Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self- supervised deep learning with spectral contrastive loss. Advances in Neural Information Processing Systems, 34, 2021.
[18] Xinlei He and Yang Zhang. Quantifying and mitigating privacy risks of contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS 21, page 845 863, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384544 doi: 10. 10.1145/3460120.348888 34.4557.. URL https::/tooo org /2///10.11 1145/ 3460120 3484571
[19] Matthew Jagielski, N. Carlini, D. Berthelot, A. Kurakin, and N. Papernot. High accuracy and high fidelity extraction of neural networks. USENIX Security Symposium, 2020.
[20] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. A survey on contrastive self-supervised learning. Technologies, 9(1,, 2021. ISSN 2227-7080.
doi: 1033390ccchnooogeee000000. URL https: /twp::://wwww..e mdpi cww www.0m//222770002 7080/
9/1/2
[21] Hengrui Jia, C. A. Choquette-Chov Chandrasckaran, and N. Papernot. Entangled watermarks defense against model extraction. USENIX Security Symposium, 2021.
[22] Hengrui Jia, Mohammad Yaghini, Christopher A Choquettc-Choo, Natalie Dullerud, Anvith Thudi, Varun Chandrasekaran, and Nicolas Papernot. Proof-offleamnnn:: Definitions and practice. arXiv preprint 210.x::2103.5550 2021.
[23] Mika Juuti, Sebastian Szyller, Samuel Marchal, and Asokan. Prada: protecting against dnn model stealing attacks. In 2019 IEEE European Symposium Security and Privacy (EuroS&P), pages 512-527. IEEE, 2019.
[24] Ashish Kapoor, Eric Horvitz, and Sumit Basu. Selective supervision: Guiding supervised learning with decision-theorede active learning. In IJCAI, volume 7, pages 877-882, 2007.
[25] Simon Korbblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited, 2019. URL https: arxiv. org/ httpa///////99////// 00414

---

[26] Klemen Kotar, Gabriel Ilharco, Ludwig Schmidt, Kiana Ehsani, and Roozbeh Mottaghi. Con- trasting contrastive self-supervised representation learning pipelines. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages 2021.
October
[27] L. F. Kozachenko and N. N. Leonenko. Sample estimate of the entropy of a random vector. Probl. Peredachi Inf., 16,9-16, 1987. URL http://www http:///www.attnee..mm mathnet. ru/links/
pdf
[28] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009,
[29] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So. and Jaewoo Kang. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4)) 1240, 09 2019. ISSN 1367-4803, doi: 10. (093/ioiinfommaiic//z666.. URL pattpss:/ddot wrg/10...11.
[30] Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong. Encodermi: Membership inference against pre-trained encoders in contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS '21, New York, NY, USA, 2021, Association for Computing Machinery. ISBN 9781450384544. URL https 3484749
//doi org/ 10.
[31] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. In Proceedings of ICLR 2021: 9th International Conference on Learning Representationss, 2021.
[32] David McAllester and Karl Stratos. Formal limitations the measurement of mutual in- formation. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Pro- ceedings of Machine Learning Research, pages 875-884, PMLR, 26-28 Aug 2020. URL attps://proceecdnngs .ml.iiinnn.mm........mm.m..m...m mlr
[33] Ari S. Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neural networks with canonical correlation, 2018.
[34] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. URL http::/ufffd std aeu.hhousemmmerr//ninnno http:///mwmmmmmmemmnn/aaampmamme hww.oo/ooouuuuueeeeeeoeoao pdd
[35] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction poisoning: Towards defenses against dnn model stealing attacks. In International Conference on Learning Representations, 2020. /RRlitttps://opereviee http:////opppereeeeeeeeeeeroorre mett/forummid-Syer/kaaa
[36] Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems, 32, 2019.
[37] Alec Radford, Jong Wook Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. Arxiv, abs/2103.00 0000000, 2021.
[38] Mert Bulent Sariyildiz, Yannis Kalantidis, Diane Larlus, and Karteek Alahari. Concept gen- cralization in visual representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9629-9639, October 2021.
[39] Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang Can't steal? cont-steal! contrastive stealing attacks against image encoders. 2022. URL URL https://arxivv.rgg
[40] Masoumeh Shafieinejad, Nils Lukas, Jiaqi Wang, Xinda Li, and Florian Kerschbaum. On the robustness of backdoor-based watermarking in deep neural networks. In Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security, pages 177-188, 2021.
12

---

[41] Or Sharir, Barak Peleg, and Yoav Shoham. The cost of training NLP models: A concise overview. CoRR, abs/2004.08900 2020. URL
[42] Florian Tramèr, F. Zhang, A. Juels, M. Reiter, and T. Ristenpart. Stealing machine learning models via prediction apis. USENIX Security Symposium, 2016.
[43] Aäron den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. ArXiv, abs/ .0744, 2018.
[44] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707-723, IEEE, 2019.
[45] Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, and Qi Zhu. Non-transferable learning: A new approach for model ownership verification and applicability authorization. In International Conference on Learning Representations, 2022. URL id=tYRrDdSnVUy
het.rerrrrrieeeeettorr net/forum?
13


Reviews:
[{"review_id": "fv4Rt-Xf8-I", "reviewer": "Reviewer_Ay7D", "summary": "The paper proposes a new defense against model stealing of self-supervised models, using the recently proposed dataset inference framework. Similar to original dataset inference for supervised models, the proposed defense detects stealing based on a hypothesis that stolen models exhibit different behavior on private and test data. In this case, this is quantified by using auxilliary density estimation models and observing the log likelihood of data from the two sources. Further, the paper proposes two metrics to measure the quality of stealing: mutual information score and cosine similarity score.", "questions": "- The motivation for the additional contribution (two scores) is somewhat unclear. Can the authors make a stronger point for why linear evaluation on several tasks (as in Table 3 in [14]) is insufficient/unnecessary; what is the reason for not including such evaluation alongside proposed metrics? In a similar vein, both scores (MI and cosine) to me seem analogous to fidelity, and not accuracy as claimed in the paper. Can the authors elaborate on this claim further? Making this part more cohesive with the rest of the paper would make the contribution stronger.\n- Some results seem to be missing in Table 1, e.g. ImageNet as $D_S$ for first two choices of $D_P$. While the conclusions certainly hold in almost all cases, the test does not pass for CIFAR10 as $D_P$ and SVHN as $D_S$, making it more necessary to provide a complete and convincing evaluation.\n- Further, 4.3 claims that the paper focuses on both MSE and InfoNCE stealing losses, but 5.1 seems to be imply only InfoNCE is used in the evaluation. Can the authors comment on this mismatch? The evaluation would be stronger if MSE and SoftNN were added.\n- (Nit) It would be interesting to see if different density estimation methods would be able to further boost the performance, did the authors experiment with this?", "limitations": "I find the discussion of societal impact sufficient. Regarding limitations, it would be good to more explicitly acknowledge the strength of results compared to the supervised case and reflect on statements given in [14]. More importantly, to avoid the cat-and-mouse nature of attack/defense work in the field (similar to adversarial examples literature until recently), I find it necessary to consider the prospect of adaptive adversaries, i.e. the robustness of the defense against attackers aware of the proposed defense. I acknowledge that designing an adaptive attacker is often a highly non-trivial contribution in itself; a short explicit discussion would still benefit the paper. ", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is a solid contribution to an active field and is a good follow-up to [14] which posit that dataset inference is significantly harder in the case of SSL. While the results still are less convincing than in the supervised case, the work clearly demonstrates that successfully adapting dataset inference to SSL is possible if more tailored approaches are used. The paper is very well written and structured. \n\nI believe the work has no fundamental weaknesses. However, I have several relatively minor concerns (raised below in Questions), which I hope the authors can sufficiently address in their response; I am willing to reassess the work after the discussion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "WNlitlmLZn", "reviewer": "Reviewer_z6xt", "summary": "This paper generalizes the dataset inference defend originally proposed for supervised learning to self-supervised learning. Focusing on the model stealing black-box attack,  the authors utilize GMM to capture the density discrepancy between training and testing datasets of an adversary model. Furthermore, the authors propose new evaluation metrics based on mutual information and cosine similarity, and demonstrate the effectiveness and robustness through extensive experiments. Overall, it is an interesting paper.", "questions": "- See the Weakness part.", "limitations": "- See the Weakness part.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Strengths:\n  - The motivation is clearly explained in the introduction to demonstrate that adaptation of data inference defend for SSL models is non-trivial.\n  - Extensive experiments are provided to show the effectiveness of the proposed defending towards model stealing attack.\n  - The code is provided in the supplementary. I appreciate the authors to do that.\n\n- Weakness:\n\n  - The proposed method heavily relies on the assumption that the stolen model shares similar distribution in the feature space on the private training dataset with the victim model. If so, I'm curious whether the proposed method can only work when the stolen model is well-trained, which is to some extent proved by Table 3 (e.g., in the bottom part, when the number of queries is 5K, the p-values is 1.23e-1 > 0.05).\n  - I think a common problem for density estimation related algorithms to work in reality is the generality of the confidence threshold. In your case, this is the p-value threshold which has been set as 0.05 according to line 193. Although it works in your experiments, we can see in Table 3, the range of p-values might vary a lot under different datasets.\n  - I wonder how the robustness of dataset inference to obfuscations comes, especially for shuffle, which will totally disrupt the high-dimensional space and GMM. Could you provide further explanation about this property?\n\n  - It would be stronger to show the effectiveness of the proposed mutual information score and cosine similarity score as evaluation metrics for defending if they are applicable for different defending methods instead of only the proposed data inference defending, like the watermark defense mentioned in line 95-103.\n\n  - Writing:\n    - In Section 2 (e.g., line 95-103), it seems that the terminology \"encoder\" has been viewed the same with \"self-supervision pre-trained encoder\" by default, which is not the case.\n    - It would be better to put the theorem in the main paper with the proof remained in the appendix, like Theorem 1.1 in line 266.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "V3ybhIP9d-p", "reviewer": "Reviewer_g8VY", "summary": "This paper investigates dataset inference for encoders. Dataset inference is a defense against model stealing, which allows the defender to identify whether a suspected stolen model has private knowledge from the original model’s training data. Originally formulated for classifiers, the authors adapt dataset inference to encoders by testing for differences in encoder representations on the defender’s training data vs. held-out data. Specifically, they fit a GMM to estimate the distribution of the suspected stolen encoder’s representations on training data. They then use the GMM to evaluate the likelihood of the encoder’s representations: if the encoder is stolen then the likelihood of the training data is expected to be higher than the likelihood of the held-out data. Experiments in the vision domain demonstrate that the method successfully identifiers stolen encoders with high confidence. The confidence is shown to improve when the stolen encoder’s representations are closer to the original, as measured by mutual information and cosine similarity.", "questions": "Is it possible to make this approach more practical given the issues raised above?\n\nCan this approach handle more subtle forms of stealing? E.g., where an encoder is initialized by model extraction, then refined on independent training data. Such cases are apparently handled in the original formulation of data inference by Maini et al. (2020).\n\nIt would be good to elaborate on what makes GMMs suited for this task. As I understand, GMMs are not universal approximators when the number of components is fixed. In practice, how would one choose the number of components? Could other density estimators be used?", "limitations": "Limitations are discussed in appendices, however a summary should be included in the main paper.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Quality*\n\nThe idea of comparing encoding representations on private training data vs held-out data makes sense. However some aspects of the approach are not practical:\n- The defender is required to reveal private training/test data to a trusted arbitrator. This is not required in the original formulation for classifiers (Maini et al., 2021), where the defender is able to run dataset inference themselves given query access to the stolen model.\n- In order to run dataset inference, the defender’s private train/test data sets must be passed through the adversary’s encoder (in their entirety). This is problematic if it is necessary to query the adversary’s encoder via a public API under their control. The adversary could simply steal the defender’s private data. One way around this would be to compel the adversary to share their model with the trusted arbitrator. However this may not be possible from a legal perspective. Note that this is not an issue for the original formulation by Maini et al. (2021), where it is only necessary to pass a small fraction of the private training set through the adversary’s model.\n \n*Originality and Significance*\n\nWhile dataset inference is not a new idea, it has only been applied to classifiers to date. Extensions to other models with more complex outputs, such as encoders, are well-motivated. There is some prior work on membership inference for encoders (Liu et al., 2021), which ought to be discussed in the paper. Given the close connection between membership inference and dataset inference, Liu et al.’s approach could be an alternative to the approach proposed in this paper. Another related work that ought to be cited is by (Jia et al., 2021). They discuss an alternative reactive defense for model stealing called proof-of-learning. \n\n*Clarity*\n\nThe paper is clear overall. I wonder if Figure 1 could be simplified, as it seems many of the quantities do not need to be computed to run the test against a single potentially stolen encoder.\n\n- Maini, Pratyush, Mohammad Yaghini, and Nicolas Papernot. \"Dataset Inference: Ownership Resolution in Machine Learning.\" International Conference on Learning Representations. 2020.\n- Liu, Hongbin, et al. \"EncoderMI: Membership inference against pre-trained encoders in contrastive learning.\" Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021.\n- Jia, Hengrui, et al. \"Proof-of-learning: Definitions and practice.\" 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2gxLjDEV13C", "reviewer": "Reviewer_FsB3", "summary": "The authors propose a detection method for dataset inference for the unsupervised setting of models. Unlike prior work on models meant for classification, the authors in this work focus on models trained to generate feature representations (SSL). The detection technique relies on log-likelihood statistics using density estimation models, along with measures of distributions like MI. The proposed method has promising results, taking a step in the direction of making dataset inference broader and more useful in the real world.", "questions": "# Minor comments\n\n- Line 34: Isn't 'dataset inference' the threat model? Like membership inference or property inference?\n- Figure 1 is too complicated- either increase the size (and de-clutter) or simplify the flow/structure.\n- Line 342: \"implies\" -> \"suggests\"\n- Line 347: \"defense\" -> \"detection\". At the point in time where the method is launched, the adversary has reportedly already stolen the model, and thus it is merely the question of detecting (or confirming) whether that is true, not preventing the adversary from stealing the victim's model.\n- Tables 2,3 seem to have overfill issues.", "limitations": "Addressed in Section 6", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n- The paper is well written and sets up the threat model, as well as the proposed method(s) and the rationale behind them, quite well.\n- Empirical evaluation is thorough, with datasets ranging from CIFAR to ImageNet. \n\n# Weaknesses\n\n- Lines 203-205. As a baseline, the authors should include experiments for scenarios where downstream tasks (single, or 2-3 combined) are trained using the \"feature extractors\". It's true that \"a single downstream task cannot adequately reflect...\", but it would be nice to have some estimates (performance numbers) to know for sure how much this difference really is.\n- Lines 322-322: How about adding noise in intermediate layers (or inputs/outputs), or dropout?\n- Table 2: Performance difference between $f_s$ for some cases does not seem to be too far from $f_i$", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept