Dataset Ownership Verification in Contrastive Pre-trained Models
Authors: Yuechen Xie
Jie Song
Mengqi Xue
Haofei Zhang
Xingen Wang
Bingde Hu
Genlang Chen
Mingli Song
Venue: ICLR 2025 Poster (2025)
Keywords: Dataset Ownership Verification
Data Protection
Contrastive Learning
Pre-trained Models
Self-supervised Learning
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: The first dataset ownership verification method specifically designed for contrastive pre-trained models.

Abstract: High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concurrently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges as a crucial method in this domain, but existing approaches are often limited to supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a $p$-value markedly below $0.05$, surpassing all previous methodologies. Our code is available at https://github.com/xieyc99/DOV4CL.

Published as a conference paper at ICLR 2025
DATASET OWNERSHIP VERIFICATION IN CONTRASTIVE PRE- TRAINED MODELS
Yuechen Xie', .ie ; Menggi Xue², Haofei Zhang', Xingen Wang! -Bingde Hu! Genlang Chen', Mingli Song 1.566
'Zhejiang University, "Hangzhou City University, Bangsheng Technology Co., Ltd *Ningbo University. "State Key Laboratory of Blockchain and Security, Zhejiang University HHangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security {xyuechen, sjie, haofeizhang, newroot, tonyhu, egl, brooksong) @zju.edu.cn, mexxue@zcu.edu.c
ABSTRACT
High-quality open-source datasets, which necessitate substantial efforts for curation, has become the primary catalyst for the swift progress of deep learning. Concur- rently, protecting these datasets is paramount for the well-being of the data owner. Dataset ownership verification emerges crucial method in this domain, but existing approaches are often limited supervised models and cannot be directly extended to increasingly popular unsupervised pre-trained models. In this work, we propose the first dataset ownership verification method tailored specifically for self-supervised pre-trained models by contrastive learning. Its primary objective is to ascertain whether a suspicious black-box backbone has been pre-trained on a specific unlabeled dataset, aiding dataset owners in upholding their rights. The proposed approach is motivated by our empirical insights that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space exhibit significant variations compared to models trained without the target dataset. We validate the efficacy of this approach across multiple contrastive pre-trained models including SimCLR, BYOL, SimSiam, MOCO v3, and DINO. The results demonstrate that our method rejects the null hypothesis with a p-value markedly below 0.05, surpassing all previous methodologies. Our code is available at https:t/tttttateeee com wttmm/ieyy99/oo/CCC0
INTRODUCTION
The success of deep learning is greatly dependent on the the availability of high-quality open-source datasets, which empower researchers and developers train and test their models and algorithms. Presently, the majority of public datasets Deng et al. 2009): Krizhevsky et al. (2009): Netzer et al. prohibited without
(201T) are designated exclusively for academic purposes, with commercial explicit permission. Therefore, preventing the stealing of public datasets holds significant importance for the benefit of the data owners.
Numerous traditional techniques exist for data security, including encryption Boneh & Franklin (200T); Khamitkar differential privacy Dwork 2006);]Abaal et (2016), and digital watermark ing Cox et al.
PPodilchuk & Delp (2001): Kadian et al. (2021) However, these methods fall short in protecting the copyrights of open-source datasets, as they either impede dataset accessibility or necessitate the knowledge of the training process of potentially suspicious models. Recently, dataset ownership verification (DOV)
al. (223); Li al. (2022 20235) emerges as a novel defense measure to deter dataset theft. It allows defenders, i.e., dataset owners, to demonstrate whether suspects have infringed upon their rights by ascertaining whether a suspicious black-box backbone has been pre-trained their datasets. However, most existing DOV techniques designed solely for supervised models where verification relies on distances between data points and decision boundaries Li al. 2018) Karimi et al. 2019); Karimi & Tang 2020), they not directly
"Corresponding author

---

Published as a conference paper at ICLR 2025
Augmented Training
Augmensed Traizing
Dena's Repeesentations
Training
Pairwise Similarit
with Small Varistion
Contrsiive Carl
Closely Distributed
Truck1
Truck]
Observation
Observation (itnary Relationsspi
Augmemttd Testing
(Uhary Relattossiipp
Textin Parts
Augmested lesting
Dan''s Represenaatioss
Lomoll Distriunod
Painwise Similurit with Large Variation
Catl
Truck2
Cal
Trick?
A cat's augrennatiinss
trak'' sugmentations
Some oilerr cata! sugnestationn
Some otlerr inucki augicttations
The overview of the two key observations. The representations are visualized using t-SNE, The encoder is a ResNet18 pre-trained on CIFARIO 10 with BYOL Grill et al. (2020).
Figure I:
applicable recently increasing popular self-supervised pre-trained models Chen et al. (2020); Chen & He (022); Chen et al. (202 due to the absence of the well-defined decision boundaries.
In this work, we present, to the best of knowledge, the first DOV method for contrastive pre-trained models. It aids defenders in validating whether suspicious models have been illicitly pre-trained on their public datasets. Given third-party suspicious model that might be pre-trained dataset without authorization, we focus on the black-box setting where defenders have no information about other training configurations (e.g., loss function and model architecture) of the model and can only access model via Encoder as a Service (FaaS) [hha et al., (2023): ); Liu et al. (2022). It means defenders can only retrieve feature vectors via model API. The proposed approach formulated upon two key observations, as shown in Figure 1 (1) Unary relationship: encoders pre-trained through contrastive learning generate remarkably more similar representations for augmentations of the same seen samples at the training phase than the unseen samples. (2) Binary relationship: the pairwise similarity between the seen samples doesn't significant change after data augmentations.
the protected
We define the differences in unary and binary relationships between seen and unseen samples as the contrastive relationship gap of the suspicious model. Defenders can endeavor to activate this gap in the suspicious encoder by employing their own public datasets, in order to ascertain whether the suspect's s encoder was pre-trained on their data. More specifically, as illustrated in Figure [ the proposed DOV technique comprises three steps: (1) pre-training a shadow encoder devoid of the public dataset of the defender; (2) utilizing multi-scale augmentation to compute the contrastive relationship gaps of the suspect encoder and the shadow encoder; (3) conducting hypothesis testing on the contrastive relationship gaps of the two encoders to determine whether the suspect encoder has been pre-trained on the defender's public dataset.
In summary, the principal contributions of this paper are threefold: (1) we discern that when models are trained with the target dataset, the unary and binary instance relationships within the embedding space demonstrate noteworthy disparities in comparison to models trained without the target dataset; (2) we introduce the concept of the contrastive relationship gap, which, to the best of our knowledge, represents the first DOV technique for contrastive pre-trained models; (3) comprehensive experiments showcase that our approach refutes the null hypothesis with a p-value significantly below 0.05, surpassing all preceding studies.
RELATED WORK
Data Protection. Dataset ownership verification is an emerging field in data security. Typically,
involves embedding watermarks into the original dataset (Guo et al. 2023 Li et al. 2022 2023b
Zhang et al. 2024 Xie et al. 2025). Models trained on the watermarked dataset will incorporate a pre-designed backdoon allowing defenders to verify data ownership simply by triggering the model's backdoor. However, current DOV methods primarily target supervised models and require altering the original dataset's distribution to inject watermarks, which makes it susceptible various watermark
2

---

Published as a conference paper at ICLR 2025
removal mechanisms (Chen et al. 2021b Liu et al. 2021b; Sun et al. 2023 Kwon 2021 Hayase jet al. 2021). The proposed method demonstrates that, for contrastive learning, dataset ownership can be efficienlly verified without modifying the original dataset.
Dataset inference Maini ct al. (202I) is a state-of-the-an defense against model stealing (Sha et al. 2023 Sanyal al.j [022 [Shen et al. 2022). It does not require retraining the model or embedding watermarks within the dataset, which reduces the time cost significantly while preserving the original
distribution of the data. The latest dataset inference method Dziedzic et al. (2022) has expanded its application to self-supervised learning. Although it's primarily aimed at encoder theft, it can also be directly used for dataset ownership verification. However, it necessitates inferring the entire training set to model the oulput features of all data from both the training and testing sets. It is prohibitively cine-conu for large datasets, such as ImageNet (Deng et al. 2009). In contrast, our method achieves accurate verification using only small fraction of the dataset. For instance, on ImageNet, we use only 0.1% % of the training set for verification.
Membership inference Shokri al. (2017); [Chouettee et al. (2021); Carlini al. (0022)
Hu et al.] (2022) aims to determine whether an input was part of the model's training dataset. EncoderMI Liu et al. (2021a) is a powerful method specifically designed for membership inference on encoders pre-trained via contrastive learning, which takes advantage of the overfitting tendencies of the image encoder. However. it directly trains the inferencer high -igheniinnii representations that contain a large amount of redundant information, which leads to a heavy computational cost and increased training difficulty. In contrast, our method extracts the most critical information for verification from the representations, namely contrastive relationship gap, achieving effective verification without the need train an inferencer.
Inspired by Proof of Learning (PoL) Jia et al. (2021); Fang et al. 20233) : Zhao et al. (2024), Proof of Training Data (PoTD) Choi et al. (2024) is proposed assist third- third-party auditor in validating which data were used to train models. It helps develop practical and robust tools for accountability in the large-scale development of artificial intelligence models. However, it entails substantial verification costs, as the model trainer (suspect) is required to disclose detailed training records to the verifier, including training data, training code, and intermediate checkpoints. In practical scenarios, if the models trained by the suspect possess significant commercial value, the suspect is seldom willing to comply with such disclosures. Our setup is more reflective real-world scenarios, where the model is black box, and the defender can only access its API.
Contrastive Learning. The central idea of contrastive learning Chen et al. 20200; Chen He (202T): Albelwi (2022): He et al. (2020) is to enable the encoder to produce similar feature vectors for a pair of augmentations derived from the same input image (positive samples), and distinct feature vectors for augmentations derived from different input images (negative samples). Classical approaches like SimCLR Chen et al.] (020)). MoCo He et al. (2020). SwAV Caron et al. (2020), utilize both positive samples (for feature alignment) and negative samples (for feature uniformity). Surprisingly, researchers notice that contrastive learning can also work well by only aligning positive samples, such as BYOL Grill al. (2020) and DINO Caron et al. (02T). We follow some litera- tures Albelwi 2022); Gao et al. (2022) to coin these methods as a special type of contrastive learning, or contrastive learning without negatives. We make no strict distinction between these concepts here due to the clear context in this work. Our method is designed to protect the unlabeled datasets used in contrastive learning, thereby securing and fostering healthy development in this field.
THE PROPOSED METHOD
3.1 PROBLEM FORMULATION
In this study, we focus on the dataset ownership verification task in black-box scenarios. The problem involves two key player: the defender and the suspect. The defender, assuming the role of the dataset provider, endeavors to ascertain whether the suspect model, Mauer has been unlawfully trained on his public dataset Dpub. MMuus can be classified into four scenarios based on its training datasets: ! Maue is exclusively trained on the public dataset Dput of the defender, indicating the occurrence of dataset misappropriaation; 2 Maus is trained on dataset that encompasses the designated public dataset D pmtb along with an additional dataset Date. signifying dataset misappropriaatio, albeit posing more challenging DOV task than case ① due the presence of Dati 3 Maus is trained on an
3

---

Published as a conference paper at ICLR 2025
unrelated dataset Dunre outside the scope of the defender's public dataset, indicating the innocence of the suspect; ( Maus is trained on an alternative dataset Date that bears significant resemblance yet doesn't 't overlap with the public dataset D puhh suggesting the innocence of the suspect, albeit posing a more arduous DOV challenge than case ( These four scenarios encompass nearly every conceivable real-world circumstance.
3.2
CONTRASTIVE RELATIONSHIP GAP
3.2.
OBSERVATIONS AND DEFINITIONS
In contrastive learning, a pivotal training objective for encoders is to maximize the similarity between the representations of positive samples, which are different augmentations of the same training image. This training approach leverages the neural network's memory capacity, prompting the encoder to retain the features of the training data, As result, we derive the following two significant insights:
Observation 1 (Unary Relationship) Contrastive pre-trained encoders can produce representations for the same seen samples' augmentations during pre-training than unseen samples.
alike
Observation 2 (Binary Relationship) The pairwise similarity between the seen samples' represen- tations hardly change after augmentations, unlike with unseen samples during pre-training. We characterize the disparity between familiar and unfamiliar data encountered during the training phase as the encoder's contrastive relationship gap, metric that can aid defenders in discerning whether the queried encoder has been pre-tras pre-trained on their dataset. The precise definition is follows:
Definition (Contrastive Relationship Gap) Given a contrustive pre-trained encoder M and dataset D, the contrastive relationship gap of M is defined as:
d(D d(D.D,M,T) - -./e [1. ES(D. 8, S((.
(I)
where D is dataset that M has not been pre-trained on. T(-) denotes an augmentation function. -.,',)) is a similarity set. |S] is the total number of samples S(D, M)D)
larger mean contrastive relationship gap suggests that M is more likely to have been pre-trained on D. According Observation and Observation 2 S consists of unary relationship similarity set Sy and binary relationship similarity set SB.
3.2.2 THE CALCULATION OF Sy AND SB
Random cropping is a commonly used data augmentation technique in contrastive learning Chen al. ) Chen & He (202T): Chen et al. (2011a), which enhance the model's generalization ability significantly. In this paper, we use multi-scale random cropping to capture both global and local features of objects. Specifically, we design the 7' in Eq. a multi-scale augmentation function 7 = (T', 7" J. hoping to activate the encoder's contrastive relationship gap from various dimensions. 799 is the global augmentation function responsible for larger regions, while T' the local augmentation function focusing on smaller regions. Through T we calculate Sy and S3 at multi-scale. Their definitions are as follows:
Definition (Unary Relationship Similarity Set) Given an encoder M and a dataset D, the unary relationship similarity set is defined as:
Su
= (S S sal
(2)
where Sig. SH, and Sgi respectively denote the unary relationship similarity between global and global views, local and local views, and global and local views.
The specific formulas are as follows:
Σ DD
Σ
Sg0 DM
sim
(3)
m=
n=
4

---

Published as a conference paper at ICLR 2025
SU
2
D
m=1 n=m+1 sim (M((M (x)T) (2+)) ( M((T(())))
N
(4)
[DIN
1)
Sy DDMM
sim (M(T)()))) M(TH
(5)
n=1
where z; E DD and [D] is the total number of samples in dataset D. and N are the execution number for T and T respectively. T% (z;) denotes the m-th augmentation of 2; by T similarly for T. (z,), Th (x)). (r,). sim (:,)) represents the cosine similarity function.
Definition (Binary Relationship Similarity Set) Similar Sv. given an encoder M and dataset D, the binary relationship similarity set defined as:
(p, M,,T")) = B S S
(6)
where SB. Stt. and Sq: the binary relationship similarity between global and global views, local and local views, and global and local views respectively.
We first introduce the binary relationship set G. It includes the pairwise similarity between the augmented images' representations, denoted as:
G(D, - (iimM(((a))
[1. 11.1 (t.111)
(7)
where sim (.))) denotes the cosine similarity function, with 24, E D, (D] is the total number of samples in dataset D. and T( represents the augmentation function. By substituting the augmentation functions 7 and T into Eq. 7). we obtain the binary relationship set Ga and g' at respective scales.
Below, we formally present the specific formulas for S
S and Sg.
2
M
M
f(g 9%)
M(M 1)
n=m+1
2
S/ = N(N 1)
f( Ith
m=1 n=m+1
sgi
1
N M Σ
(
(10)
MN
where / (---) is distance measurement function, which is implemented the mean absolute error in this paper. M and N are the execution number for T and T", respectively. G% represents the m-th binary relationship set based T similarly for gg. Gt, and G',-
Using unary relationship similarity set Sy and binary relationship similarity set Sg. we can determine the contrastive relationship gap of the encoder M as follows:
d= Σ. (S
Si)) S r(St > St), Σ. (S S1)) T(SB > $
(11)
where E (9g, 11,111, U(,)) , and S come from S(D, M, T') and S(D,M, in Eq. respectively. I(-) is the function returning if the input statement is true or returning 1 if the input statement is false. a hyperparameter with default value of 1. As a increases, the contrastive relationship gap of encoder M between D and D becomes larger.
3.2.3 THE COMPLETE PROCESS
We propose method of dataset ownership verification by contrastive relationship gap. Figure displays the entire process of our method, divided into three stages:
(1) pre-training shadow encoder Msdu: shadow dataset Daaio compare with Msusi
(2) performing samplings on Dpub and D Dpot (a defender's 's private dataset which isn't publicly
available, and Maus has not been trained on it), that represent D and D in E. respectively. The sampling sizes are kpub and kpet respectively, resulting in the subsets {D) pub. ... .D/K..) and put }. Then using these subsets calculate the contrastive relationship gaps d.us = {D) put: DK
dl U... ak and dydre di ddd U... dk of Maus and Madar respectively;

---

Published as a conference paper at ICLR 2025
1. Encoders Pre-training
2. Contrastive Relationship Gap Calculation
3. H Hypotheais Testing
Dart
Use
Pass
7Tm
Sp
Stat
Sispect
Tata Source
S
Multi-scale
Augmratation
S
Defender
Pyss
Duta Source
Dytt
D all
Use
T
OLOSP OIOO Mass
datu
dydoa
Sg
Notation
Security Boundary:
Dataset:
Pass
pyt
Dan
Enceder:
Figure 2: The overview of our method (best viewed under color conditions).
(3) one-tailed pair-wise T-test Hogg et al. (2013) conducted on daus and dsdw- The null hypothesis, Hu. posits that the mean of daus is less than equal to that of dadw. while the alternative hypothesis, denoted as Hi, i, posits that the mean of dana is greater than the mean of dadu- the p-value p is less than 0.05, can reject the null hypothesis and conclude that Dpub has been stolen. On the other hand, if the null hypothesis can't 't be rejected, we think the suspect is innocent.
EXPERIMENTS
We evaluate our method using six visual datasets (CIFARIO Krizhevsky al. (2009), CI- FAR 100 Krizhevsky et al. (200)), SVHN Netzer et al. (2011), ImageNette Howard (2019). Im- age Woof Howard (2019) and NetgeNet Deng et al. (2009)) and five contrastive learning algorithms (SimCLR, BYOL, SimSiam, MOCO v3, and DINO). ImageNette and ImageWoof are two non- overlapping subsets of ImageNet, each containing 10 classes.
The specific experimental setup is introduced in Section 4.1 results and analyses presented in the application of our method on the ImageNet pre-trained models are demonstrated in Section ablation studies are conducted in Section 4.4 Appendix A Specifically. Appendix presents the ablation study of sampling size, the ablation study of global and local augmentation number is shown in A Appendix
Section 4.2
A.8 and the ablation study of shadow dataset and hyperparameter are featured in Appendix A.9 The impact of shadow model's 's training hyperparameters is shown in Appendix [A.5] The unti-interference capability our method is conducted in Section 4.5) Appendix A. intrroducss the comparison with the method based on watermark. Appendix introduces the impact of early stopping. Appendix A.13 presents some visualization results of our method.
4.1 EXPERIMENTAL SETUP
For SimCLR, BYOL, SimSiam, and MoCo v3, we use VGG16 Simonyan & Zisserman (201)). and Resnet18 He al. (2016) as encoder architectures. Additionally, we use ViT-T, ViT-S, and ViT-B Dosovitskiy et al. (0200) for DINO. For Madw. we default using ResNet18 and SimCLR as its encoder architecture and training algorithm.
To simulate Datt. a dataset similar to Dpub but without overlapping data (as described in Section 3.1 we randomly divide a dataset into two subsets of equal size representing Dpus and Datt. respectively. For Dpet, set it as the testing set of the undivided dataset for convenience. Specific settings are as follows:
Experiment 1: Dpus is random half of CIFAR10 training set and Date is the other half. D Dunres Dedw and Dpet are SVHN, CIFARI00 and CIFARIO testing set respectively.
Experiment 2: D Drus is random half of ImageNette training set and Datt is the other half. Dunres Dsdw and D prot are ImageWoof, SVHN and ImageNette testing set respectively.
The settings for the remaining parameters provided in Appendix A.2 To simulate adversarial behavior, we pre- train Maus using D puh, Pub U Datts , D Dunres and D Pits respectively, which is corresponds to the four cases in Section 3.1

---

Published as a conference paper at ICLR 2025
Regarding evaluation metrics, in addition to using the p-value, also the sensitivity, specificity and AUROC. Sensitivity is the proportion of correctly predicted positive cases among all actual positive samples, and specificity is the proportion of correctly predicted negative cases among all actual negative samples. They reflect the ability to identify positive and negative samples, respectively.
When Maus pre-trained on Prub D Ppub U Dait, which means the suspect is illegal, p should be less than 0.05. When Maus pre-trained on Date or Dunres which means the suspect is legal. should be greater than 0.05. We compare our method with two representative methods, as detailed below:
DI4SSL Dziedzic et al. (022): This is the most recent method for dataset inference targeting self-supervised encoders. It also applies to dataset ownership verification. The principle behind DI4SSI. is that if the encoder pre-trained on Dpub, the representations outputs will have a higher log-likelihood on the defender's training data than on testing data. Conversely, if the encoder is not pre-trained on Ppub. this pattern will not be observed.
MI Liu al. (2021a): This is classic method which designed for member inference on contrastive pre-trate are-trained models. The fundamental mechanism of EncoderMI is that the encoder produces similar representations for different augmentation of the training data. We have adapted this method to suit dataset ownership verification better. Specifically, we augment images from pub and D put and input them into Maus and Madu- By comparing the distribution of the output representations' similarity, we can determine potential dataset stealing. If Mane is pre-trained on Dyub. the representations similarity of Maus will significantly exceed those of M Madw vice versa.
4.2 EXPERIMENTAL RESULTS
Our approach is proven effective illustrated in Figure (refer to Appendix and for specific y-values), which display the experimental results of baselines and our method on CIFAR10 and ImageNette. Note that when Daus is CIFAR10-1 (ImageNette-1) or CIFAR10 (ImageNette), DD Paus includes Puu (D is CIFARIO-1 and ImageNette-1 I in two cases respectively), ), which implies the suspect illegal, and p should be less than 0.05. However, when Dau. CIFARIO-2 (ImageNette-2) or SVHN (ImageWoof), the suspect did not use Duub and is legal, so should be greater than 0.05.
The two baselines struggle to accurately distinguish the legality of various scenarios. There are a large number of false positive or false negative samples in all cases. In contrast, our method consistently produces correct results in all cases. Unlike the baselines, which model high-dimensional representations containing a large amount of redundant information directly, our method refines the most valuable information from these representations.
This crucial information, contrastive relation- ship gap, is extracted based on the charac- teristics of contrastive learning. Therefore, our method is not constrained by the encoder architecture and training algorithm, achiev- ing desirable outcomes in various scenarios. As shown in Table [1 we calculate sensitivity, specificity and AUROC based on the experi- mental results on CIFARIO and ImageNette, which demonstrates the superiority of our method quantitatively.
Table 1: Sensitivity, specifieity, and AUROC of three methods on CIFARIO and ImageNette.
Dataset
Method
Sensitivity
Specificity
AUROC
DI4SSL
0.2
1.0
0.6
CIFARIO
EncoderMI
0.8
0.5
Ours
1.0
DI4SSL
0.3
1.0
0.775
ImageNette
EncoderMI
0.15
0.9
0.5
Ours
1.0
1.0
1.0
Sensitivity and specificity reflect the algorithm's 's ability to identify positive and negative samples.
4.3 THE APPLICATION OF OUR METHOD ON IMAGENET
To validate the efficacy of our method in real-world scenarios, we conduct dataset ownership ver- ification on ImageNet, large-scale visual dataset containing over 14 million images across 1000 classes, using ten pre-trained encoders. The architecture of these encoders includes CNN and ViT, and they are pre-trained using the six popular contrastive learning methods currently. Among these, the prantrd model for DINO is obtained from the official repository]] while the models for the
7

---

Published as a conference paper at ICLR 2025
DI4SSL
EncoderMI
Ours
1.00
1.00-
1.00-
0.75
FN
TN
FN
TN
FN
TT
0.50
0:50
0.50
R
0.25
0.25
0.25
0:05
0.05
0.05
0.00
TP
FP
0.00
TP
FP
0.00
TP
FP
Tigaal
Magal
Migg (CRAMUO-)) (LIFARES) (EPARRI (19I8)
Blgal
Legl
Lead
Logal
(CRA)EI-1) (LIFAMEE)) (LDARRE-- I5VENO
(CHAAII (CHARII))) (INIIS))
Secnario
Scenario
Secaario
DI4SSL
EncoderMI
Ours
1.00
1.00
0.75
0.75
0.75
FN
TN
FN
TN
FN
TN
0.50
0.25
0.25
0.25
p= 0.05
0.05
p= 0.05
0.00
TP
FP
0.00
TP
FP
0.90
TP
FP
Hapd (bapphoatt
Eepil (Eeegcherrs (iapphete (beageWoot)
Tlggll
Hage (Imagchete-1)
IRgal (tmighhotes theaghhete- (brageWve) Scenario
Hapl
Lege
Lape
(maggheee-l) (brighheee (Ireghhete-) (lengewee)
Sccnario
Scenario
SinCLR-VVG16
BYOL-VGGI6
SimSism- MunSiammResNettiee
13-V0066
*
DINO-VIT-TM
Rinsl.R-Resaa118
Net1
MoCo v5-ResNet1X
DINO-V
Figure 3: Experimental results of three methods CIFARIO (the first line) and ImageNette (the second line). Each value is an average of 3 trials. Each pattern represents a suspicious model trained using a specific architecture, contrastive learning method, and dataset. VGG16 trained using SimCLR, and the rest follows similarly. 'CIFARIO-1' and 'CIFAR10-2" are the two oon-orelag random halves of CIFARIO training set, similarly for 'ImageNette-1 I and 'ImageNette-2", D pub is CIFARI0-1 and ImageNette-1 1 in two cases respectively. We consider illegal/legal behavior as positive/negative cases and classify each situation based on p-value. The datasets in parentheses on the x-axis are D
represents
other contrastive learning methods are sourced from MMSelfSup our experiments, we designate Dpet the validation set of ImageNet and Dedu SVHN. The architecture and training algorithm of Madw are ResNet18 and SimCLR, respectively. Parameter settings are provided in Appendix A.2 As shown in Table 2 the experimental outcomes demonstrate that method is well-suited for pre-trained models on ImageNet, even when using only 0. 11% of ImageNet data for dataset ownership verification. Conversely, the performances of baselines unsatisfactory.
4.4
ABLATION STUDIES
4.4.1 THE IMPACT OF MULTI-SCALE AUGMENTATION IN UNARY AND BINARY RELATIONSHIP
We use pre-trained models ImageNet verify the effectiveness and robustness of unary and binary relationship's multi-scale augmentations. Specifically, the models are ResNet50 and ViT-B/16 pre-trained by DINO. Both Ppub and D.ue are ImageNet. As shown in Table The combined use of unary and binary relationship's s multi-scale augmentations outperform other choices. This superiority attributed its attempts to activate the encoder's contrastive relationship gap from various angles, thereby endowing it with strong generalization capabilities to adapt to different encoders.
4.4.2 THE IMPACT OF SAMPLE NUMBER OF Ppub AND Dolt
We study the impact of the sample number of Dpub and Datt on our method. Specifically, we denote the proportion of D Dpub in Dyas Dalt as r. And Dpub Dute always CIFAR10. For example. when r = 0.1, D is 10% of the CIFARIO training set randomly sampled, while Date consists of the remaining 90%. Similarly, when r = 0.2, Dpub 20% of the CIFARI0 training set randomly sampled, and DDtt is the remaining 80%.
http:://mwmmiiiiaaaaaaaacooooooc zoo.html
8

---

Published as a conference paper at ICLR 2025
Table 2: The results (p-values) baselines and our method applied ImageNet. "Daus" is the dataset used to pre-train Maus. Each value is average of 3 trials. Dass and D pub are both ImageNet. Note that this scenario, sus includes D Puub. making the suspect's behavior illegal, and the p-values should be less than 0.05.
Method
Model
DI4SSL
EncoderMI
Ours
SimCLR
0.15
10-3
BYOL
0.91
10-3
ResNet50
SimSiam SwAV
0.56
10
0.88
10-4
ResNe150
0.51
10-3
MoCo v3
ViT-S/16
0.99
10-159
10-1
ViT-B/16
0.99
10-158
10-4
ResNet50
0.99
10-1
DINO
ViT-S/16
0.99
10-3
VIT-B/16
0.99
10-3
Table 3: The impact of multi-scale augmentation unary and binary relationship. Both pub and Daus are ImageNet. 'DINO-ResNer represents ResNet50 trained using DINO, with "DINO- ViT- B/16' being similar. Note that the suspect illegal this case, and the p-values should be less than 0.05. Bold and underline respectively represent the best and second best results.
Study Subject
50
0
Se
Sti
DINO-ResNeL50
DINO-VIT-B/16
/
0.02
0.01
Unary/Binary Relationship
4
3.1 10
2.4 10 0.02
0.06
3.3 x 10
2.5 x 10-3
3.9 x 10-3
1.4 x 10
Global/Local View
8.2 10
2.5 x 10 3
5.0 10-4
10
1.8 10-3
1.0 10
Ours
4.2 10-1
1.1 10-3
Then we use R pre-trained using SimCLR to obverse the performance changes of our method under different values. In Figure each point represents the p-value tro-ffraaffrrm)d of the model trained on the corresponding dataset. shows our method demonstrates good robustness to the sample number of Ppub and Dalt-
0=0.55
4.5
THE ANTI-INTERFERENCE CAPABILITY OF OUR METHOD
0.1 0.2 0.3 044 0L5 06 0.7 08 09
4.5.1
THE IMPACT OF PRIVACY TRAINING METHOD
Figure 4: The impact of the ratio of DD, Dpub to Dpub U Date on our method. Each point the p-value fog-traasformed) of the model trained on the cor- responding dataset.
Private training methods Abadi al. (2016); Papernot al. 2018 are typically used to protect private. non non-open-sooure datasets. In our scenario, the suspect might employ private training methods to obscure their illegal activities and interfere with the defender's ss dataset ownership verification, even if reduces the encoder's nor- mal performance. Therefore, we chose the classic private training method DP-SGD Abadi et al. (016)) and conducted the following experiments. Specifically, we trained the suspicious encoder ImageNette using DP-SGD not. The for DP-SGD is 50, and the maximum norm for gradient clipping is 1.2. The results are shown in Table and Table [ indicating that method remains effective in this more arduous scenario.

---

Published as a conference paper at ICLR 2025
Table 4: Results (p-values) on SimCLR.
Table 5: Results (p-values) on SimSiam.
Model
w/o DP-SGD
w/ DP-SGD
Model
wo DP-SGD
w/ DP-SGD
VGG16
10 10-27
10 10-14
VGG16
0.01 10-4
0.01 10-3
Re
10-14
10-13
ResNet18
4.5.2 THE APPLICATION OF OUR METHOD ON FINE-TUNED ENCODERS
We also challenge the scenario where Maus is applied to downstream tasks. Specifically, we train the entire classifier on CIFARIO and CIFAR100 respectively, whose backbone is a ResNet50 pre-trained on ImageNet using SimCLR. Similarly, in the black-box, environment, we can only use the predicted probability vectors of the input samples. The results are shown in the dataset of downstream tasks. 'Acc' represents the accuracy on downstream tasks. Daus and puuh are both ImageNet. Note that in this scenario, Daus includes Ppub. making the suspect's behavior illegal, and the p-values should be less than 0.05. Morcover, we set the hyperparameter a to 5 to amplify the contrastive relationship gap. Excitingly, even after fine-tuning, are still able to identify the suspect's s theft. For details on fine-tuning, please refer to Appendix A.6
Table [ "D downnstream
Table 6: Fine-tuning Results on CIFAR-10.
Table 7: Fine-tuning Results CIFAR-100.
D Ddownstream
Epoch
p(L)
Acc
D Doownstream
Epoch
p(L)
Acc
50
10
0.87
50
10 6
0,44
100
10-3
0.88
100
10-4
0.50
CIFARIO
CIFAR100
150
10-8
0.88
150
10-5 5
0.63
200
10-4
0.89
200
10-3
0.66
4.6 THE TIME COST OF OUR METHOD
Table 8: The time required for method
We calculated the time required for our method and DI4SSL perform single verification on ImageNet. The experiments were conducted using an NVIDIA GeForce RTX 4090. The encoder is a ResNet50 pre-trained on ImageNet using SimCLR. As shown in Table 8 the time consumption of our method is significantly less than that of DI4SSL This is because our method only requires inferring on a small subset of ImageNet (depending on
and DI4SSL to execute once on ImageNet.
Method DI4SSL
Time Consumption p(L)
10014s
Ours
293s
10-3
kpub, kiput, M and N), whereas DI4SSL needs to infer the entire dataset. Additionally, our method was properly validated (p < 0.05), further demonstrating its superiority.
4.7 LIMITATIONS
Not all encoders are pre-trained using contrastive learning. Masked Image Modeling (MIM) Girdhar jt al. (2023): He et al. (2022) is also a significant method for pre-training encoders. However, as shown in Appendix A. 12 our method doesn't 't effectively apply to encoders pre-trained via MIM. This is because that the representations learned through MIM are harder to distinguish compared to those from contrastive learning Zhou et al. (0022), although MIM-base pre pre-training methods demonstrate superior performance in downstream tasks. This results in less pronounced unary and binary relational gaps in the representations. We plan to refine this aspect in our future work.
CONCLUSION
High-quality open-source datasets are essential for the rapid development of deep learning. We propose method for verifying dataset ownership in contrastive learning to protect the legitimate right of dataset owners. Specifically, we propose the concept of contrastive relationship gap based on the unary and binary relationship of contrastive pre-trained models. The experiment proves that can effectively verify dataset ownership. Promising future work includes (1) extending our method to other self-supervisee learning approaches; (2) adapting our method to protect other types of data (e.g., text); (3) exploring other privacy risks associated with encoders.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGEMENTS
This work was partially supported by the Pioneer R&D Program of Zhejiang (No.2024C01021) and Zhejiang Provincial Natural Science Foundation of China (LQ24F020020, LD24F020011).
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Saleh Albelwi. Survey on self-supervised learning: auxiliary pretext tasks and contrastive learning methods in imaging. Entropy, 24(4):551, 2022.
Dan Boneh and Matt Franklin. Identity- identity-based encryption from the weil pairing, In Annual interna- tional cryptology conference, pp. 213-229. Springer, 2001.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privucy (SP), pp. 1897-1914. IEEE, 2022.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912-9922 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 15750-15758, 2021.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9640-9649, 2021a.
Xinyun Chen, Wenxiao Wang, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, and Dawn Song. Refit: a unified watermark removal framework for deep learning systems with limited data. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, pp. 321-335, 2021b.
Dami Choi, Yonadav Shavit, and David Duvenaud. Tools for verifying neural models' training data. Advances in Neural Information Processing Systems, 36, 2024.
Christopher A Choquette-Chov Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International conference on machine learning, pp. 1964-1974. PMLR, 2021.
Ingemar Cox, Matthew Jelfrry Bloom, and Chris Honsinger. Digital watermarking. Journal of Electronic Imaging, 11(3):414441, 2002.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn. Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2020.
11

---

Published as a conference paper at ICLR 2025
Cynthia Dwork. Differential privacy. In International colloquium automata, languages, and programming, pp. 1-12. Springer, 2006.
Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kaleem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot. Dataset inference for sulf-vupe models. Advances in Neural Information Processing Systems, 35: 1ww111208-1111 12070, 2022.
Congyu Fang, Hengrui Jia, Anvith Thudi, Mohammad Yaghini, Christopher A Choquette-Chou Natalie Dullerud, Varun Chandrasckaran, and Nicolas Paperno. Proof-of-learning is currently more broken than you think. In 2023 IEEE 8th European Symposium on Security and Privacy (FuroS&P P). pp. 797-816. IEEE, 2023.
Yuting Gao, Jia-Xin Zhuang, Shaohui Lin, Hao Cheng, Xing Sun, Ke Li, and Chunhua Shen. Disco: Remedying self supervupe learning lightweight models with distilled contrastive learning, In European Conference on Computer Vision, pp. 237-253. Springer, 2022.
Rohit Girdhar, Alaacldin El-Nouby, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Omnimae: Single model masked pretraining on images and videos. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10406-10417, 2023.
nann-Batieen Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271-21255 2020.
Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. Domain watermark: Effective and harmless dataset copyright protection is closed at hand. Advances in Neural Information Processing Systems, 36, 2023.
Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. Spectre: Defending against backdoor attacks using robust statistics. International Conference on Machine Learning, pp- 4129-4139. PMLR, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEESCVF conference on computer vision and pallern recognition, pp. 9729-9738, 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on compuler vision and pattern recognition, pp. 16000-16009, 2022.
Robert V Hogg, Joseph W McKean, Allen T Craig, al. Introduction mathematical statistics. Pearson Education India, 2013.
Jeremy Howard. A smaller subset of 10 easily classified classes from imagenet, and a little more 2019.
french. URL http::///iiubb.cooo//sss///a////
Hongsheng Hu, Zoran Saleie, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Member- ship inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54(11s): 1-37, 2022.
Hengrui Jia, Mohammad Yaghini, Christopher Choquette- Chho, Natalie Dullerud, Anvith Thudi, Varun Chandrasekaran. and Nicolas Papernot. Proof-of-leaming: Definitions and practice. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 1039-1056. IEEE, 2021.
Poonam Kadian, Shiafali M Arora, and Nidhi Arora. Robust digital watermarking techniques for copyright protection of digital data: A survey. Wireless Personal Communications, 118:3225-3244 2021.
12

---

Published as a conference paper at ICLR 2025
Hamid Karimi and Jiliang Tang. Decision boundary of deep neural networks: Challenges and opportunities. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 919-920. 2020.
Hamid Karimi, Tyler Derr, and Jiliang Tang. Characterizing the decision boundary of deep neural networks. arXiv preprini arXiv: /912. 1460, 2019.
Siddhi Khamitkar. A survey on fully homomorphic encryption. IOSR Journal of Computer Engineer- ing (IOSR-JCE) e-ISSN, pp. 2278-0661.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Hyun Kwon. Defending deep neural networks against backdoor attack by using de-trigger autoencoder. IEEE Access, 2021.
Changjiang Li, Ren Pang, Zhaohan Xi, Tianyu Du, Shouling Ji, Yuan Yao, and Ting Wang. An em- barrassingly simple backdoor attack on self-supervised learning. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 4367-4378, 2023a.
Yiming Li, Yang Bai, Yong Jiang, Yong Yang, Shu-Tao Xia, and Bo Li. Untargeted backdoor watermark: Towards harmless and stealthy dataset copyright protection. Advances in Neural Information Processing Systems, 35:13238-13250 2022.
Yiming Li, Mingyan Zhu. Xue Yang, Yong Jiang. Tao Wei, and Shu-Tao Xia. Black-box dataset ownership verification via backdoor watermarking, IEEE Transactions on Information Forensics and Security, 2023b.
Yu Li, Lizhong Ding, and Xin Gao. On the decision boundary of deep neural networks. arXiv preprint arXiv: /808. 05385, 2018.
Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong, Encodermi: Membership inference against pre-trained encoders in contrastive learning. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2091-2095, 2021a.
Yang Liu, Zhen Zhu, and Xiang Bai. Wdnet: Vatermark-decomposiie network for visible watermark removal. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 3685-3693, 2021b.
Yupei Liu, Jinyuan Jia, Hongbin Liu, and Neil Zhenqiang Gong. Stolenencoder: stealing pre-trained encoders self-supervised learning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pp. 2115-2128, 2022.
Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. arXiv preprint arXiv:2104.107000 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, Spain, 2011.
Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with pate. arXiv preprint arXiv: 08908, 2018.
Christine Podilchuk and Edward Delp. Digital watermarking: algorithms and applications. IEEE
signal processing Magazine, 18(4):33 46, 2001.
Sunandini Sanyal, Sravanti Addepalli, and R Venkatesh Babu. Towards data-free model stealing in a hard label setting. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15284-15293, 2022.
Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can't steal? cont-steal! contrastive stealing attacks against image encoders. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16373-16383, 2023.
13

---

Published as a conference paper at ICLR 2025
Yun Shen, Xinlei He, Yufei Han, and Yang Zhang. Model stealing attacks against inductive graph neural networks. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1175-1192. IEEE, 2022.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.155: 2014.
Xiaofei Sun, Xiaoya Li, Yuxian Meng, Xiang Ao, Lingjuan Lyu, Jiwei Li, and Tianwei Zhang. Defending against backdoor attacks in natural language generation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 5257-5265, 2023.
Yuechen Xie, Jie Song, Huiqiong Wang, and Mingli Song. Training data provenance verification: Did your model use synthetic data from my generative model for training? arXiv preprint 022 2025.
Jinghuai Zhang, Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. Data poisoning based backdoor attacks to contrastive learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24357-24366, 2024.
Zishuo Zhao, Zhixuan Fang, Xuechao Wang, and Yuan Zhou. Proof-of-learning with incentive security. arXiv preprint 20Xvv22404009005, 2024.
Qiang Zhou, Chaohui Yu, Hao Luo, Zhibin Wang, and Hao Li. Mimeo: Masked image modeling pre-training with contrastive teacher. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 4487-4495 2022.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A THE DETAILS AND ADDITIONAL SUPPLEMENTS OF EXPERIMENTS
A.1 DATASETS USED
20099: The CIFARIO dataset consists of 32x32 colored images with 10 classes. There are 50000 training images and 10000 test images.
CIFARIO Krizhevsky et al.
2009): The CIFARI00 dataset consists of 32x32 coloured images with 100 classes. There are 50000 training images and 10000 test images.
CIFAR100 Krizhevsky et al.
SVHN Netzer al. 2011 The SVHN dataset contains 32x32 coloured images with 10 classes. There are roughly 73000 training images, 26000 test images and 530000 "extra" images.
ImageNette Howard (209)): ImageNette is subset of 10 easily classified classes from Imagenet. It includes the following categories: tench, English springer, cassette player, chain saw, church, French horn, garbage truck, gas pump, golf ball and parachute. There roughly 10000 training images and 4000 test images.
Image Howard 2019): ImageWoof is subset of 10 classes from Imagenet that aren't It easy to classify. It includes the following categories: Australian terrier, Border terrier, Samoyed, Beagle, Shih-Tzu, English foxhound, Rhodesian ridgeback, Dingo Golden retriever, Old English sheepdog. There are approximately 9000 training images and 4000 test images.
ImageNet Deng et al.) (2009): Larger sized coloured images with 1000 classes. There are approxi- mately 1 million training images and 50000 test images. As is commonly done, we resize all images to be of size 224x224.
A.2 EXPERIMENTAL DETAILS
The 18 18 trained on CIFAR 1O/CIFARIO uses a convolutional kernel size of 3x3 with stride of 1, instead the default 7x7, and doesn't use a max pooling layer.
On CIFAR RO/CIOR VHO,, we pre-train the encoder for 800 epochs with batch size of 512. On mateeNetee/Imaee Woof, the encoder with non- Von-VV architecture is pre-trained for 800 epochs, while ViT-S/16 architecture is pre-trained for 2000 epochs with a batch size of 64. The initial learning rate for all pre-training sessions is set 0.06 and adjusted using a Cosine Annealing scheduler. The optimizer is SGD, with momentum of 0.9 and a weight decay of 5 x 10-1, All experiments are conducted on four NVIDIA RTX A6000s and one NVIDIA GeForce RTX 4090. In all experiments, set M = 2 and N 6. Both T and T' are composed of random cropping, color jitter, random flipping. and random grayscale, with respective cropping ranges of (0.4, 1.0) and (0.05, 0.4).
Furthermore, the settings for other parameters are follows:
Experiment on CIFARIO. We set kipub = 256, kipnt = 128, K = 30 and a 10000.
Experiment on ImageNette. We set kiput = liput 32, K = 50 and a 0.1.
Experiment on ImageNet. We set kipub = lipet = 32 and K 50 and a
A.3 DETAILED EXPERIMENTAL RESULTS ON CIFAR10
This section presents the experimental results (p-values) of several baselines and our method on CIFARIO. Prub CIFARIO-1 10-1, The results are shown in Table Table and Table 12 respectively. "D.us' is the dataset used to pre-train Maus- 'CIFAR10- and "CIFAR10-2" are the two non- overlapping halves of CIFAR 10 training set after a random split. Each value an average of 3 trials. Note that when D... is CIFARIO- 10-1 CIFARIO, 10, the suspect used Dpub. and this scenario is illegal, so p should be less than 0.05. However, when D Deus is CIFAR1O-2 or SVHN, the suspect did not use Dpub, and this scenario is legal, p should be greater than 0.05. The illegal (p should be less than 0.05) and legal (p should be greater than 0.05) scenarios correspond to the pink and green areas in Table 11 and Table [2] respectively.
Table 10
15

---

Published as a conference paper at ICLR 2025
A.4 DETAILED EXPERIMENTAL RESULTS ON IMAGENETTE
This section presents the experimental results (p-values) of several baselines and our method on ImageNette. pub is ImageNette-l. The results are shown in Table 13 Table 14 and Table [5] respectively. 'ImageNette-1 and "ImageNette-2 are the two non-overlapping random halves of ImageNette training set. Each value is an average of 3 trials. Note that when D is ImageNette- or ImageNette, the suspect is illegal, so p should be less than 0.05. However, when Daus is ImageNette-2 or SVHN, the suspect is legal,
should be greater than 0.05. The illegal (p should be less than 0.05) and legal (p should be greater than 0.05) scenarios correspond to the pink and green areas in
Table 13
Table 14 and Table 15 respectively.
A.5
THE IMPACT OF SHADOW MODEL'S S TRAINING HYPERPARAMETER
In the real world, the training hyperparameters of shadow models and suspicious models are often different. We analyzed whether these differences would affect our method. Specifically, we set different batch size (32 for the shadow model and 64 for the suspicious model), learning rate (0.01 for shadow model and 0.06 for the suspicious model), and weight decay (Ie-4 for the shadow model and 5e-4 for the suspicious model) for the shadow model compared the suspicious model. As shown in Table [ our method demonstrates good robustness to the training hyperparameter settings of the shadow model.
Table The impact of shadow model's training hyperparameters on our method. Model is ResNet18. Both Puu and D are ImageNette. Dedw is SVHN.
Method
All Same
Different Batch Size
Different Learning Rate
Different Weight Decay
SimCLR
10 10-11
10 6
10 10
10
BYOL
10-10
10-4
10-3
10-
SimSiam
10-5
10
10-3
10
A.6 THE DETAILS OF FINE-TUNING THE PRE-TRAINED MODEL
We fine-tuned the encoder on CIFARIO/CIFARI using a learning rate of 0.001, a batch size of 512, weight decay of 5e-4, and the SGD optimizer with momentum of 0.9. The pre-trained ResNet50 on ImageNet is sourced from MMSelfSup
http::/mmmmmmsssssaaaaoddeeeeeet zoo.html
16

---

Published as a conference paper at ICLR 2025
Table 10: p-values for each scenario of DI4SSL on CIFARIO.
D
Alg
Method
Model
CIFARIO-1
CIFARIO
CIFARIO-2
SVHN
VGG16
0.01
0.43
0.54
0.42
SimCLR
ResNet18
0.64
0.97
0.71
0.62
VGG16
10
10-16
0.42
0.60
BYOL
ResNet18
0.99
10-5
0.44
0.73
VGG16
0.99
0.99
0.28
0.50
DI4SSL
SimSiam
ResNet18
0.79
0.93
0,41
0.89
VGG16
0.99
0.27
0.06
MoCo v3
ResNet18
1
0,46
0.82
ViT-T/4
0.90
0.52
0.75
0.55
DINO
ViT-S/4
0.85
0.17
0.67
0.94
Table p-values for each scenario of EncoderMI on CIFAR10.
D
Alg
Method
Model
CIFARIO-1
CIFARIO
CIFARIO-2
SVHN
VGG16
SimCLR
ResNet18
0
VGG16
0
0
0
BYOL
ResNet18
0
0
10-43
VGG16
0
0
EncoderMI
SimSiam
ResNet18
0
10
VGG16
0
0
0
MoCo v3
ResNet18
0
0
10
ViT-T/4
0.99
0.94
1
0.80
DINO
ViT-S/4
0.99
0.99
1
Table 12: p-values for each scenario of method CIFAR10.
Daus
Alg
Method
Model
CIFARI0-1
CIFARIO
CIFARIO-2
SVHN
VGG16
10-16
10-12
0.75
0.24
SimCLR
ResNet18
10-12
10-8
0.29
0.32
VGG16
10-20
10-18
0.91
0.64
BYOL
ResNet18
10-17
10-11
0.55
0.33
VGG16
10-4
10-6
0.99
0.99
Ours
SimSiam
ResNet18
10-11
10
0.47
0.87
VGG16
10-11
10-14
0.91
0.76
MoCo v3
ResNet18
10-4
10-3
0.84
0.76
VIT-T74
0.03
0.01
0.63
0.67
DINO
VIT-S/4
10-7
10
0.43
0.62
17

---

Published as a conference paper at ICLR 2025
Table 13: p-values for each scenario of DI4SSL on ImageNette.
D Deus
Alg
Method
Model
ImageNette-1
ImageNette
ImageNette-2
Image Woof
VGG16
0.01
0.43
0.54
0.42
SimCLR
ResNet18
0.64
0.97
0.71
0.62
VGGI6
10-4
10-16
0.42
0.60
BYOL
ResNet18
0.99
10-5
0.44
0.73
VGG16
0.99
0.99
0.28
0.50
DI4SSL
SimSiam
ResNet18
0.79
0.93
0.41
0.89
VGG16
0.99
0.27
0.06
MoCo v3
ResNet18
0.46
0.82
ViT-T/4
0.90
0.52
0.75
0.55
DINO
ViT-S/4
0.85
0.17
0.67
0.94
Table 14: p-values for each scenario of EncoderMI on ImageNette.
D sus
Alg
Method
Model
ImageNette-1
ImageNette
ImageNette-2
Image
VGG16
0
SimCLR
ResNet18
VGG16
0
0
BYOL
ResNet18
0
0
0
10
43
VGG16
0
0
0
EncoderMI SimSiam
ResNet18
0
0
10
13
VGG16
0
0
MoCo v3
ResNet18
0
0
0
10 74
ViT-T/4
0.99
0.94
0.80
DINO
ViT-S/4
0.99
0.99
Table 15: p-values for each scenario of our method on ImageNette.
D Dous
Alg
Method
Model
ImageNette-1
ImageNette
ImageNette-2
Imaf
VGG16
10 10-16 16
10-12
0.75
0.24
SimCLR
ResNet18
10 12
10-8
0.29
0.32
VGGI6
10 20
10-18
0.91
0.64
BYOL
ResNet18
10-17
10-11
0.55
0.33
VGG16
10-4
10-6
0.99
0.99
Ours
SimSiam
ResNet18
10-11
10-5
0.47
0.87
VGG16
10-11
10-14
0.91
0.76
MoCo v3
ResNet18
10
10
0.84
0.76
ViT-T/4
0.03
0.01
0.63
0.67
DINO
ViT-S/4
10-7
10-7
0.43
0.62
18

---

Published as a conference paper at ICLR 2025
A.7 THE IMPACT OF SAMPLING SIZE
We also evaluate the performance of our method using different amounts of data. Specifically, conduct verification by selecting different sampling size kipub and kpot- We conduct experiments using pre-trained encoders on ImageNet, with the encoder architecture being ResNet50. Figure [ shows that, across various contrastive learning methods, the effectiveness of our method improves as kpub and kppt increase. This is because larger sampling size better represent the distribution of the dataset, making the contrastive relationship gap of the encoder more pronounced. Note that in this scenario, D Deus includes Dym pub are both ImageNet, and D Deus includes Dpub. so the p-values should be less than 0.05.
0.30
64-0.04
10
10-3
100 10-9
64
64-0.04
0.04
0.01
10
10-3 10"4
0.4
0.25
32 0.12
0.10
10
10
10-3 1
32
0.15
0.13
10-3
0.02
10 10
0.3
0.20
0.30
10
0.01
0.01
0.01 10-3
16
0.34
10
0.06
0.02
0.09 0.03
kpw
0.15
kpvt
0.2
0.25
0.05
0.01
10-3 10-3 0.02
8 0.44
0.07
10
0.03
0.02 0.04
0.10
0.28
10
0.04
0.10
0.08
0.02
0.28
0.05
0.08
0.15
0.09
0.02
0.1
0.05
2.0.9
0.06
0.09
0.22
10-3
0.28
2 2.1.4
0.04
1.19
0.3
10-3
0.30
4
8
16
32
64
16
32
64
Kpub
Kpub
(a)
(b)
0.4
64
0.07 0.03
0.03
0.02
10-+ 10
64
0.07
0.02
0.01
0.02 10-3
0.3
32 0.16
0.14
0.01
0.02
10 10-3
32 0.15
0.22
0.05
0.02
10-3 10
0.3
16
0.34
10
0.13
0.03
0.09 0.02
16
0.33
10
0.01
0.03
0.03 0.02
kpmt
0.2
kpot
0.2
0.35
0.07
0.01
0.04
0.03 0.04
0.40
0.08
0.01
0.03 0.03 0.02
0.31
0.13
0.08
0.19
0.13
0.03
0.1
0.27
0.15
0.16
0.16
0.15
0.05
0.1
2 0.144
0.05
0.11
0.37
0.02
0.34
2 0.15
0.04
0.
0.30
0.02
0.25
2
4
8
16
32
64
16
32
64
Kpub
Kpub
(c)
(d)
Figure 5: The p-values obtained using pre-trained ResNet50 on ImageNet with different kpub and kpot values.
Each heatmap corresponds to the results of different training algorithms. Figure 5a SimCLR, Figure 5b BYOL, Figure 5ct SimSiam, and Figure 5d MoCo v3.
19

---

Published as a conference paper at ICLR 2025
A.8 THE IMPACT OF GLOBAL AND LOCAL AUGMENTATION NUMBER
We evaluate the effectiveness
method under different numbers global augmentations M and local augmentations N. We conduct experiments using pre-trained ResNer50 on ImageNet. Figure shows that, the performance of our method improves as M and N increase. This because greater number of augmentations provides more information to the encoder, thereby amplifying the contrastive relationship gap. Note that in this scenario, 2 Dacce includes D pub are both ImageNet, and Daus includes D pub, so the p-values should be less than 0.05.
0.02
10
10
10
10-5
10
10
10
10
10-3
0.06
0.02
4
0.03
10-3
10-4
10-4
10-4
4
0.07
10
10-4
10-4
10-3
0.04
0.01
0.02
2
10
10-3
10
10
10-5
0.02
0.04
10
10
10
4
8
10
2
8
10
N
(a)
(b)
0.06
6
0.01
10
0.01
10
10-4
0.02
10
0.01
10
10
0.04
0.04
0.06
10 4
10
10-3
10-4
0.06
10 3
10
10
10-4
0.02
0.02
2
0.02
0.03
10
10-4
10-3
2
0.01
0.04
10
10-4
10-3
4
8
10
8
(c)
(d)
Figure 6: The p-values obtained using pre-trained ResNet50 on ImageNet with different M and N values. Each heatmap corresponds to the results of different training algorithms. Figure 6a SimCLR, Figure 6b BYOL, Figure bct SimSiam, and Figure 6d MoCo v3.
20

---

Published as a conference paper at ICLR 2025
A.9 THE IMPACT OF SHADOW DATASET AND HYPERPARAMETER
We investigate the impact of the shadow dataset D adw and the hyperparameter on our method. As shown in Figure the setting of affects the validation results of our method. This effect is related the distributions of Duu and Ddw and is not fixed. This indicates that the defender need set appropriate based on his actual situation.
p= 0.05
p=0.05
Dap CIEATIOU
Dip ImugeNotic
Dhp in Image Woof
1gp
Igp
-6
Da - SYHV CIFAR10
7
Das CIFAR100
-2
Iga
Iga
(a)
(b)
Figure The impact of shadow dataset and hyperparameter a on our method. The left figure represent the cases where D Puub is ImageNet and Maus is pre-trained ResNet50 using SimCLR. and in the right figure, D Dpub and Maus are CIFARIO and pre-trained ResNet18 using SimCLR, respectively.
A.10 THE IMPACT OF EARLY STOPPING ON OUR METHOD
The early stopping technique can terminate model training prematurely, which may result in less pronounced contrastive relationship gap. To investigate the impact of early stopping on our method, we specifically set the patience of early stopping (the maximum number of epochs allowed continue training when the K-Nearest Neighbors accuracy on the validation set does not improve significantly over multiple consecutive epochs) 15 and 30, respectively. We then calculated the p-values of the trained models using the same method, shown in Table 16 Both the datasets of defender and suspect are CIFARIO, 10, meaning p-value should be less than 0.05. Self-supervieed method is SimCLR. The shadow model is ResNet18 pre-trained on Image Woof using SimCLR. The results demonstrate that our method remains effective even under early stopping conditions.
The results (p-values) of our method on suspicious models that used early stopping. The datasets of defender and suspect are both CIFAR10 10, making the suspect's behavior illegal, so the p-values should be less than 0.05.
Table 16:
Model
wo Early Stopping
w/ Early Stopping (patience=15)
w/ Early Stopping (patience=30)
ResNet18
10-12
0.01
10
VGG16
10
10
10
21

---

Published as a conference paper at ICLR 2025
A.11 THE COMPARISON OF OUR METHOD WITH THE WATERMARK-BAN -BSSED METHOD
dataset ownership verification method for pre-trained encoders, so we adapt CTRL Li et al. (2023a), the current state-of- of-the-art backdoor attack for self self-supervised encoders, into watermark- baase dataset ownership verification method. Specifically, prior the release of public dataset, we inject the the CTRL trigger watermark into small subset of the data. During the verification phase, we input both watermarked and non-watermarked images into the suspicious encoder. If the representations of the watermarked images significantly more similar to each other than those the non watermarked images, we can conclude that the suspicious encoder was pre-trained using the public dataset. Table 17 indicates that although methods based on watermark can accurately identify cases where public datasets have been stolen, they also wrong the innocent suspect.
Currently, there is no
Table 17:
The comparison of our method with watermark- method (Dpub is CIFARIO). "Daus' is the dataset used to pre-train Msus. 'Alg' is the dataset ownership verification method. Each value an average of trials. The illegal (p should less than 0.05) and legal (p should be greater than 0.05) scenarios correspond to the pink and green areas.
D sus
Alg
Method
Model
CIFARIO
SVHN
ImageNette
Image Woof
VGG16
10 317
10-4
10
10
SimCLR
ResNet18
10-287
10-6
0.04
0.19
VGG16
0.02
0.57
0.47
DOV-CTRL
BYOL
ResNet18
0.29
0.52
0.04
VGG16
10-19
0.17
10-8
10-4
SimSiam
ResNet18
10-290
0.25
0.18
0.31
VGG16
10-10 10
0.45
0.67
0.99
SimCLR
ResNet18
10-7
0.41
0.19
0.16
VGG16
10
0.84
0.85
0.90
Ours
BYOL
ResNet18
10-9
0.61
0.59
0.59
VGG16
10
0.99
0.98
0.98
SimSiam
ResNet18
10-4
0.88
0.36
0.37
A.12 THE PERFORMANCE OF OUR METHOD ON MAE
We also conduct experiments using encoders pre-traime trained with methods other than contrastive learning. We select Masked Autoencoder (MAE) He et al. (2022) for experimentation, which is representative method Masked Image Modeling (MIM). Specifically, we use pre-trained models on ImageNet from the official MAE repository Fl with encoder architectures VIT-B/16 and ViT-L/16. Additionally, to better adapt the encoders pre-trained with MIM, we incorporate random masking into our multi- scale augmentation. According the experimental results presented in Table I8 our method still didn't 't perform well despite targeted improvement. We will address the DOV issue of MIM pre-trained models in our future work.
Table 18:
The results (p-values) of our method on MAE (Dpus is ImageNet). "Dsus' the dataset used to pre-train Msus- Each value is an average of 3 trials. Note that in this scenario, Daus and D puu are both ImageNet and Daus includes Dpus, making the suspect's behavior illegal, so the p-values should be less than 0.05.
Method
Model
Ours + Random Masking
VIT-B/16
0.75
MAE
ViT-L/16
0.44
22

---

Published as a conference paper at ICLR 2025
A.13 VISUALIZATION RESULTS
We present the visualization results of our method on ImageNette. Specifically, pab set as ImageNette, and the shadow model is ResNet18 trained on SVHN using SimCLR. We calculated the contrastive relationship gap d of the shadow model and suspicious models trained on different datasets and visualized the comparison. When the suspicious model is pre-trained on D Pab. it is considered illegal, and the contrastive relationship gap d should be significantly higher than that of the shadow model. Conversely, if the suspicious model is legitimate, the two contrastive relationship gaps should be similar. shown Figure 8 when the suspicious model illegal, its contrastive relationship gap is significantly higher than that of the shadow model. When the suspicious model is legitimate, the two contrastive relationship gaps are close. This observation aligns with our previous findings.
18
Illeag] Mac
Illeag Mac
Shadow Model
Shadow Model
Frequency
10
Frequency
0.
40.00
-0.05
0.00
0.05
0.10
0.15
-0.05
0.00
0.05
0.10
(a)
(h)
Lengl M...
Leugl Mar
Shadow Model
Shadow Model
Frequency
Frequancy
10
10
5
-0.10
-0.05
0.00
0.05
0.10
-0.075
-0.050
-0.025
0.000
0.025
0.050
0.075
(c)
(d)
Figure 8: The contrastive relationship gap d of the shadow model and suspicious models trained on different datasets. Each subplot corresponds to a different suspicious model. Figure suspicious model is a ResNet18 trained on ImageNette using SimCLR, Figure 8b suspicious model is ResNet18 8 trained on ImageNette using SimSiam, Figure &c] suspicious model is ResNet18 trained on Image Woof using SimCLR, and Figure 8d suspicious model is ResNet18 trained on Image Woof using SimSiam.
23


Reviews:
[{"review_id": "rqDW5zl1fR", "reviewer": "Reviewer_Vzgv", "summary": "This paper introduces a novel method for verifying dataset ownership in self-supervised contrastive learning models, protecting curated datasets from unauthorized use. The proposed approach leverages the \"contrastive relationship gap,\" capturing distinct similarities in representations for models trained on a defender's dataset versus unrelated data. Through a three-step process, the method identifies unauthorized usage efficiently and with high accuracy. Experimental results on several datasets and models show that this approach outperforms existing methods in accuracy, computational efficiency, and robustness, even under privacy-preserving settings like DP-SGD, making it a promising tool for dataset security in machine learning.", "questions": "- How effective is the approach when the suspect model undergoes adaptation, such as fine-tuning on a different dataset or with altered weights? Is the method still robust under such conditions, or are there specific scenarios where fine-tuning could mask the original dataset’s influence on the model?\n- How sensitive is the proposed DOV method to variations in the suspect model’s hyperparameters or pre-training dataset characteristics (e.g., domain-specific data)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper presents a unique dataset ownership verification (DOV) method specifically tailored for self-supervised contrastive learning models. This is a valuable addition to the field, as existing DOV methods are generally focused on supervised or non-contrastive learning models, leaving a gap that this paper addresses.\n-The authors conduct extensive experiments across multiple datasets (CIFAR10, CIFAR100, SVHN, ImageNet variants) and contrastive learning architectures (SimCLR, BYOL, MoCo, DINO), demonstrating the method's effectiveness and generalizability. This robust experimental setup strengthens the validity of the proposed approach.\n- By requiring only a small subset of the defender’s data for verification, the proposed method is more computationally efficient than baseline methods like D4SSL, which require access to the entire dataset, making it suitable for large-scale applications.", "weaknesses": "- While the proposed method offers a novel approach to dataset ownership verification, its applicability is limited to contrastive learning models. Many self-supervised learning models use objectives other than contrastive learning, so expanding the method’s scope could enhance its impact. However, this limitation is relatively minor.\n- In line 488, the authors state that \"the private training method does not affect our verification results,\" but this claim is based on experiments using only DP-SGD with a high privacy budget (epsilon=50). To support this claim, it would be beneficial to test the method under stronger privacy settings or with alternative privacy-preserving techniques, such as differentially private generative models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EDgBQbAJrD", "reviewer": "Reviewer_RYsp", "summary": "The paper proposes a novel method for dataset ownership verification (DOV) specifically tailored for contrastive pre-trained models in self-supervised learning. The method utilizes two observations about contrastive learning: the unary and binary relationships in the embedding space of models trained on a specific dataset. These observations are exploited through a contrastive relationship gap metric, calculated between the suspected model and a shadow model pre-trained without the dataset in question. Comprehensive experiments across datasets and models demonstrate that the method effectively detects unauthorized dataset usage, outperforming baseline techniques.", "questions": "1. Could early stopping or other techniques be leveraged to reduce the effectiveness of this detection method? A brief section on possible attack approaches in the appendix may be considered.\n\nThe paper is quite extensive very thorough! :)", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Innovative Approach:** The method uniquely applies to self-supervised models by leveraging characteristics of contrastive learning, filling a gap in current DOV methods that primarily target supervised learning.\n2. **Black-box Applicability:** The approach is suitable for black-box scenarios, which is practical and aligned with real-world applications where full model access is unavailable. The approach demonstrates robust performance across different datasets (e.g., CIFAR, ImageNet) and architectures, indicating generalizability.\n3. **Effective Performance:** Results show high sensitivity, specificity, and AUROC scores, suggesting that the proposed metric reliably distinguishes between legitimate and unauthorized dataset use.\n4. **Efficiency:** Compared to alternatives, the method is computationally efficient, which enhances its applicability to large datasets like ImageNet.\n5. **Thorough evaluation:** The paper is very comprehensive, they make specific claims and justify them with solid experiments and results.", "weaknesses": "1. **Dependency on Feature Representation Access:** The method requires access to feature representations, which might not be feasible in all practical scenarios, as many services limit this access for security reasons.\n\n2. **Limited Application to Non-Contrastive Pre-Trained Models:** The method’s effectiveness is constrained to contrastive learning. Other prevalent pre-training strategies, such as masked image modeling (MIM), are not effectively addressed, potentially limiting applicability. However, the authors make a clear claim and explain this as a limitation, thus I think the weaknesses are ok given this is one of the earlier works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jlnda6C9NY", "reviewer": "Reviewer_F2ST", "summary": "This paper proposes a method for protecting datasets from infringement in contrastive learning scenarios by introducing two relationships: unary and binary. The unary relationship assesses the clustering ability of representations for augmentations of a known sample, while the binary relationship evaluates the separation between representations of two known samples. Using these properties, the authors define distance metrics to evaluate protected training data (known to both the suspect and defender) and secret data (unseen by the suspect but known to the defender). Since suspect models are overfitted to training data and have not seen the secret data, they exhibit different behavior for these two data types, which the method uses to identify suspect models.", "questions": "See above", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper addresses an important and novel problem—dataset copyright protection in contrastive learning. The authors provide a comprehensive range of experiments, and the proposed method consistently demonstrates outstanding results across all tested settings.", "weaknesses": "I have several concerns: \n\n\n1. The proposed unary and binary relationships align with the goals of contrastive learning, which promotes close representations for variants of the same sample and separation for different samples. The authors rely on overfitting to training data for verification, but as contrastive learning improves, this approach may be less effective. Enhanced contrastive learning might eventually generalize representations, clustering representations from single-sample into a single point, and representations obtained from $N$ distinct samples into $N$ separate points, even for unseen data. Though this is an idealized scenario, this aligns with the direction of contrastive learning research, so verification should be robust to and able to coexist with advances in contrastive learning.\n\n\n2. The proposed method is similar to verification approaches in supervised learning that use differences in confidence scores or accuracies between training and test data. As Guo et al. [1] demonstrated, task performance differences between seen and unseen data are well-documented, but they are not commonly used for verification due to two primary reasons:\n1) Future research is expected to reduce overfitting and improve generality, as noted in Comment 1.\n2) An adversary could argue that similar samples exist in their training data by chance, complicating proof that observed low p-values are due to protected data.\n\nTo address these, verification metrics should be designed independently of task performance, ensuring that evidence cannot naturally occur by chance. Given this, the proposed method may lack admissible evidence of dataset infringement.\n\n[1] Chuan Guo et al. On Calibration of Modern Neural Networks\n\n3. Given the challenges noted above, many dataset protection methods in supervised learning use backdoor attacks or data poisoning. There are also backdoor attack studies specific to contrastive learning, such as Zhang et al. [2] and Carlini et al. [3]. The authors, however, only compare their method to a model protection technique and a unary-only method (e.g., EncoderMI). I suggest adding comparisons with established backdoor and data poisoning methods for contrastive learning.\n\n[2] Zhang et al. Data Poisoning-based Backdoor Attacks to Contrastive Learning\n\n[3] Carlini et al. Poisoning and Backdooring Contrastive Learning\n\n\n4. Section 4.5.2 is critical, as contrastive learning is often used as a pretraining method, and adversaries are more likely to release fine-tuned models. Thus, verification post-fine-tuning is essential. However, this section only states that experiments were conducted, without presenting results in the main text. It references \"Table 7 in Appendix A.6,\" which are outside the main manuscript. Ideally, essential content should be included in the main text, with the appendix for supplementary details. Additionally, details on the experiments are missing from the appendix, and Table 7 should include downstream performance results, as small learning rates could affect the reported outcomes. Additionally, Figure 3 occupies too much space; it would be better to reduce its size and include more analysis results directly in the main manuscript.\n\n\n5. The authors state that they \"focus on the black-box setting where defenders have no information about other training configurations (e.g., loss function and model architecture) and can only access the model via Encoder as a Service (EaaS)\" and that \"defenders can only retrieve feature vectors via the model API.\" However, Section 4.5.2 notes, \"we can only use the predicted probability vectors of the input samples,\" which seems inconsistent. In a true black-box setting, I would expect only the predicted class ID, not output logits or probability vectors, to be available.\n\n\n\n6. The analysis related to the amount of $D_{alt}$ in Figure 4 is essential but lacks explanation in Section 4.4.2. There is no clarification on how the authors control the ratio, whether by increasing $D_{alt}$ or reducing $D_{pub}$, or on what each point in Figure 4 represents. Since the change in log(p) for $D_{pub}$ suggests a controlled amount of $D_{pub}$, it may not be appropriate. With a fixed $D_{pub}$, only the amount of $D_{alt}$ should be adjusted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X0M2L0ImHa", "reviewer": "Reviewer_KUSb", "summary": "This work proposes the first dataset ownership verification method specifically for self-supervised pre-trained models using contrastive learning. The paper identifies significant variations in unary and binary instance relationships within embedding spaces when models are trained with specific datasets, compared to those trained without them. It introduces the concept of \"contrastive relationship gap,\" a novel technique for verifying dataset ownership in contrastive pre-trained models. Extensive experiments demonstrate the approach's effectiveness, with a p-value significantly below 0.05, surpassing previous methods.", "questions": "1. Could the authors provide both theoretical and experimental insights into the role of comparative learning within your approach?\n2. Authors only compare the method with two baselines: DI4SSL and EncoderMI. Could the authors compare more recent related works?\n3. Table.5 is missing EncoderMI time cost. Could author afford more information about time cost of other baselines?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel method for dataset ownership verification (DOV) specifically tailored for contrastive pre-trained models, addressing a critical need in data rights protection.\n2. The paper introduces the concept of \"contrastive relationship gap,\" providing a clear technical approach to differentiate the model's performance on training and non-training datasets.\n3. The method has been validated across multiple contrastive pre-trained models, including SimCLR, BYOL, SimSiam, MoCo v3, and DINO, demonstrating its broad applicability.\n4. Experimental results show that the method can significantly outperform previous methodologies with a high probability of rejecting the null hypothesis (p-value well below 0.05).", "weaknesses": "1. As paper illustrated in limitations and conclusion, the method is primarily effective for encoders pre-trained with contrastive learning and may not perform well with other self-supervised learning pre-training methods like Masked Image Modeling (MIM).\n2. The method lacks comparisons with enough baselines in the experimental section to clearly show its superiority.\n3. CONTRASTIVE RELATIONSHIP GAP Part is difficult and too mathematical to understand. The authors could improve the writing to make it easier for the reader to understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WiO6dSB7kQ", "reviewer": "Reviewer_DVkE", "summary": "In this paper, the authors investigate the challenge of protecting high-quality open-source datasets from misuse by individuals who lack ownership verification. To address this issue, they propose a method that exploits a natural property of contrastive learning: the distinct distances between seen and unseen examples. Specifically, they demonstrate that these distances often appear significantly larger for unseen examples compared to their seen counterparts. The authors conduct an extensive set of experiments with various models and datasets to evaluate the efficacy of their proposed approach, which yields substantial improvements over existing baselines.", "questions": "1. Does the proposed method work with the CLIP model, which also utilizes contrastive learning for pre-training? \n2. Are the findings influenced by the number of training classes and examples? For instance, if Msus has 10k classes and Msdw has only 10, do the proposed methods work as expected?\n3. How sensitive is our method to the choice of augmentations? Specifically, we consider scenarios where new augmentation techniques are introduced during evaluation but were not present during training of Msus. In such cases, the distances between outputs from Msus may increase, even when it used public data?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The research topic is important.\n2. The authors conduct many experiments.\n3. The performance is strong compared with baselines.", "weaknesses": "1. While the authors present distance metrics for dsus and dsdw, I believe it would be beneficial to include some visualizations.\n2. Contrastive learning is currently a hot research area in computer vision, but the proposed methods appear to be limited to it, which may restrict their broader applicability.\n3. The distances between examples are influenced by many factors beyond seen and unseen examples, including generalization capabilities and augmentations. I have concerns that the results presented may not fully support the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)