Stealing and Defending Transformer-based Encoders
Authors: Adam Dziedzic
Franziska Boenisch
Mingjian Jiang
Haonan Duan
Nicolas Papernot
Venue: Submitted to ICLR 2023 (2023)
Keywords: model stealing
model extraction
defenses against model extraction
transformers
encoders
self-supervised learning

Abstract: Self-supervised learning (SSL) has become the predominant approach to training on large amounts of unlabeled data. New real-world APIs offer services to generate high-dimensional representations for given inputs based on SSL encoders with transformer architectures. Recent efforts highlight that it is possible to steal high-quality SSL encoders trained on convolutional neural networks. In this work, we are the first to extend this line of work to stealing and defending transformer-based encoders in both language and vision domains. We show that it is possible to steal transformer-based sentence embedding models solely using their returned representations and with 40x fewer queries than the number of victim's training data points.  We also decrease the number of required stealing queries for the vision encoders by leveraging semi-supervised learning. Finally, to defend vision transformers against stealing attacks, we propose a defense technique that combines watermarking with dataset inference. Our method creates a unique encoder signature based on a private data subset that acts as a secret seed during training. By applying dataset inference on the seed, we can then successfully identify stolen transformers.

Under review as a conference paper at ICLR 2023
STEALING AND DEFENDING TRANSFORMER-BASBD ENCODERS
Anonymous authors Paper under double-bling review
ABSTRACT
Self-supervised learning (SSL) has become the predominant approach training on large amounts of unlabeled data. New real-world APIs offer services to generate nigh-dimensional representations for given inputs based on SSL encoders with transformer architectures. Recent efforts highlight that it is possible to steal high- quality SSL encoders trained on convolutional neural networks. In this work, we are the first to extend this line of work to stealing and defending transformer-based encoders in both language and vision domains. We show that it is possible to steal transformer- beaaoeeer--eeddddde sentence embedding encoders solely using their returned representations and with 40x fewer queries than the number the victim's training data points. We further decrease the number of required stealing queries for language encoders by reusing extracted representations on semantically similar sentences and for the vision encoders by leveraging semi-supervised learning. Finally, to defend transformers against stealing attacks, we develop new scheme watermark the language encoders, where for the last training iterations, we alternate between optimizing for the standard sentence embedding and a chosen downstream task. For the vision domain, we design a defense technique that combines watermarking with dataset inference. Our method creates a unique encoder signature based on a private data subset that acts as secret seed during training. By applying dataset inference on the seed, we can then successfully identify stolen transformers.
INTRODUCTION
The success of self-supervised learning (SSL) motivates the emergence of large-scale services offering API access to encoders which return high-dimensional representations for given inputs. These representations serve to train diverse set of downstream tasks with small amount of labeled data. Latest APIs (Clarifai 2022, Cohere 2022 OpenAl 2022) use transformer-bases encoders (Devlin et al., 2018 Dosovitskiy et al. (2220) to generate representations. Such encoders have a high number of parameters (e.g.. the toite--o-theaatt RoBERTa-Large language encoder (Liu ct al. 2019) has roughly 355M parameters) and are trained on datasets consisting of millions of data points-yyelding
highly expensive training procedure (Sharir et al. 2020). Therefore, these encoders are lucrative targets for stealing attacks (Tramèr et a (0016) where an adversary extracts a victim encoder by submitting queries and using the outputs train local stolen copy, often at fraction of the victim's 's training cost (Sha et al. 2022, Dziedzic et al. (2022a). The stolen encoder can then be used for inferences without the owner's permission, violating their intellectual property right and causing financial loss.
While stealing and defending supervised models has been heavily studied (Tramèr al. 2016 Juuti et al.) 2019 Orekondy et al.) 2020). research on the topic of stealing and defending transformer based encoders is limited. Despite the immediate practical importance of this problem, the best of our knowledge, all previous works on model stealing attacks and defenses against SSL encoders are conducted offline in contrived experimental settings (Cong et al. 2022 Sha et al. 2022 Dziedzic et al. 2022a b). focusing the vision domain with convolutional neural network (CNN)-based architectures, and do not attack the popular transformer (Vaswani et al. 2017) architecture, which currently holds sfte-of--he-ant results many vision (Caron al. 2021 and natural language processing (NLP) tasks (Gao et al.) (02T). Since transformers are trained differently than CNNs and represent larger and more complex architectures, stealing them is more challenging.

---

Under review as a conference paper at ICLR 2023
Downstraam
Public Trakiing Data
P(PL) P(P Soaten
Stilen
-0
0,90, Indsperdar.
POR x
GMM
Independent
Encoder
Privete Sood
Encoder
Watermarknng
Verification
(a) DataSeed Inference (DSI) for Vision Encoders.
(b) Watermark for Language Encoders.
Figure 1: Defenses against Stealing Transformer-based Encoders. (a) Our DSI relies on private dataset, randomly partitioned into three subsets D1, D2, Drest- D2 U D2 is injected into the encoder's public training data secret seed. For ownership resolution, Gaussian Mixture Model (GMM) is trained on the representations of D₁- DSI marks an encoder as stolen if the average likelihood from the GMM on D2 significantly higher than the likelihood on Dest- (b) To imbed our watermark for sentence embedding encoders, training alternates between the main task and a secretly chosen downstream task during the last iterations. For verification, the fully-connected (FC) layer is attached to potentially stolen copy and agreement to the victim's 's output on the downstream task is measured.
We show how stealing attacks (Sha et al. 2022 Dziedzic et al. 2022a) can be successfully applied to extract transformer encoders through their returned representations. In NLP, we are able to successfully steal sentence encoders using only a small number of queries; up to 40x fewer stealing queries than the number of the original training data points. We also show that this number can be further reduced by re-using the representations obtained from the victim encoder for semantically similar sentences of the stealing queries. For vision encoders, we decrease the number of queries against victim encoders by adapting semi-supervision based on MixMatch (Berthelot et al. 2019). which has, so far, only been applied to attacks in the supervised setting (Jagielski et al. 2020).
The successful applicability of encoder stealing to transformer-
architectures in public API settings motivates the urgent need for defenses. We first propose DataSeed Inference (DSI), a combination between watermarking (Uchida al.] 2017 Jia at. 2021 Adi et al., 2018) and Dataset Inference (DI) (Dziedzic et al., 2022b) as a successful defense for vision encoders. While standard DI operates the assumption that encoder is uniquely characterized by its whole training data and exploits that the unique data signature of the victim encoder is transferred to stolen copies, DSI adds private data subset as a unique seed the original training data and uses this seed to identify stolen copies. This is necessary adaption of DI to transformer- based encoders since these are trained on a large amount of public data data-possibly scrapped from the Internet (Radford al. (02T), and, thereby not necessarily unique to single encoder. We note that the signal from the private seed is transferred during stealing. Additionally, our defense does not harm the performance of the defended encoder on downstream tasks, which distinguishes it from watermarks in supervised settings.
Since we find that DSI is not successful in defending language encoders, we propose a new wa- termarking scheme to protect them from theft. Our watermark relies on alternating between the actual sentence embedding task and secretly chosen downstream task during the last iterations of training. This transforms the representations that they preserve their high performance on sentence embedding tasks while increasing their accuracy on the downstream task. To embed the watermark task, we append a fully-connected layer to the encoder. The additional layer acts verify whether a given encoder is a stolen copy by attaching that layer and checking the agreement to the vietim encoder's output for the watermark downstream task. Victim and independent encoders have significantly different outputs whereas victim and stolen copies return similar outputs.
secret key. We
To summarize, we make the following contributions:
We successfully steal NLP and vision transformer-basse encoders in real-world API setting. assuming access to representations only and using up to 40x fewer queries than the number of samples in the respective encoder's training dataset. Our stolen encoders achieve comparable performance to the respective victims on standard benchmarks.
We further reduce the number of stealing queries by using semantically similar sentences for language encoders and semi-supervised methods for vision encoders.
2

---

Under review as a conference paper at ICLR 2023
For the vision domain, propose DSI as defense which adds private dataset as a secret seed to the defender's 's original training data and uses this seed for ownership resolution.
For NLP sentence embedding transformers, we propose a method to watermark their representations by alternating between the actual sentence embedding and a secretly chosen downstream task during the last iterations of training.
BACKGROUND AND RELATED WORK
Model extraction attacks. The goal of the model extraction attacks is to replicate functionality of a victim model fo trained on a dataset Dp. An attacker has a black box access to the victim model and uses stealing dataset D. = {g:, fu(g:) } consisting of queries yi and the corresponding predictions or representations returned by the victim model, to train a stolen model fs- Model extraction attacks have been shown against various types of models including classification (Tramer
ct al. 2016 Jagielski et al. 2020) and representation models (Sha et al. 2022 Dziedzic et al. 2022a).
Language Encoder. We use SimCSE (Gao et al.) 2021) learn sentence representations since outperforms other methods and is exposed via public APIs (Clarifai 2022). The SimCSE frame- work by (Gao al., (22T)) proposes unsupervised and supervised approaches to generate sentence embeddings. It starts from pre-trained checkpoint of a BERT-
encoder, e.g., RoBERTa, and takes the representation for the classification token ([CLS) as the sentence embedding. In this work, we rely on their supervised approach leveraging pairs of sentences from natural language inference (NLI) datasets within a contrastive learning framework. It uses the entailment pairs as positives and contradictions as hard negatives.
Vision Encoder. In our work, we use DINO (Caron et al. 2021 to train vision transformers (ViT) since it returns high-quality representations that achieve SoTA performance downstream tasks when assessed by training a lincar classifier directly on representations. DINO trains student and teacher encoders, both with the same architecture but different parameters, where the teacher is updated with an (exponential moving) average of the student. Different random transformations of the same image generated and passed through both the student and teacher. The student is provided with smaller crops of the inputs than the teacher, which forces the student to generate representations that restore parts of the initial image. The training objective is minimize the cross-entropy loss between representations from teacher and student.
Stealing Encoders. Thus far, methods for stealing encoders through representations have been shown in the computer vision domain and only for CNNs (Sha et al. 2022; Dziedzic et al. (022a).
Previous work in the NL.P domain focuses on classification tasks and performs stealing against
fine-tuned models through labels based on a given pre-trained language encoder (Krishna et al. 2020
He et al. 2021), specifically for sentiment classification and machine translation tasks. The setup of previous work differs from ours which is concerned with stealing through representations instead of low-dimensional outputs, such as labels. This is motivated by the fact that these representations are exactly what new public APIs expose (Cohere 2022; Clarifai 2022). Distillation methods used in the NL. domain (iao et al. 2019) which could, in principle, be applied to stealing encoders, usually require white box access to the original model, for example, to the attention layers (Jiao et al. 2019). Therefore, distillation cannot be applied to stealing in public API-access scenarios.
Kanella-Beguelin et al.] 202T; He et al. 202T). Model extraction against NLP APIs are shown by (Xu
Defending Encoders. Dataset inference (DI) (Maini et al. 1021) is a defense against model stealing attacks. uses the victim's 'ss training dataset as unique signature, leveraging the following observation: for a victim encoder trained on its private data as well as for its stolen copies, the distribution of the representations generated from the victim's training data differs from the distribution of the representations generated on the test data. In contrast, for an independently trained encoder, these two distributions cannot be distinguished, allowing the detection of stolen copies (Dziedzic et al., 2022b). For modeling the distributions, Gaussian Mixture Models (GMMs) are trained on fraction of the private training data and applied to disjoint fraction of the training data and the test data. An encoder is identified as victim or stolen copy if the log-likelihood on private representations is significantly higher than on the test representations. Recently, watermarking (Uchida et al. 2017 Jia et al. 202 Adi al. 2018) methods have been proposed for encoders (Dziedzic et al. [002a Cong al. 2022; Wu et al., 2022). The main difference between previous work and our DSI lies in verification.
Previous watermarking techniques use downstream tasks to detect watermark while we resolve
3

---

Under review as a conference paper at ICLR 2023
ownership based the representations directly. For more detailed overview watermarking for encoders and a more thorough overview of related work, see Appendix A
3 STEALING TRANSFORMER-BASE ENCODERS
We aim at stealing BERT-based transformers, fine-tuned to return sentence embeddings in the language domain, and general embeddings for images in the vision domain. Our stealing operates in public API setting where the adversary can query the models through a pre-defined interface to obtain high-dimensional representations for their inputs. Stealing is then performed following previous work (Dziedzic et al.) 2022a): (i) The adversary sends N raw or augmented inputs to the victim encoder. These inputs can, in principle, be taken from any data distribution of the target domain, using open-source data. (ii) With the obtained representations, the adversary trains a stolen copy of the victim. The goal of this training is to maximize the similarity of the stolen copy's output and the representations output by the victim. Therefore, the adversary either imitates a supervupe training using a contrastive loss function, e.g. InfoNCE Chen al. (2020) SoftNN Frosst et al. (201), or directly matches both models' representations via the Mean Squared Error (MSE) loss.
Stealing Transformer-Bart formers which are first pre-traine trained on large corpus of text data to return per-token representations and then fine-tuned to return high- -igh-nimensoonll embeddings for a given full--tett input, e.g., a sentence. In the vision domain, APIs such as (Clarifai (022) expose encoders trained from scratch on large amounts of image data return per-image representations. We find that public APIs (Clarifai) (022) provide metadata about exposed encoders, which can contain information about datasets used for pre-training as well as the encoder architecture. Thus, we can instantiate our stolen encoders with the victim encoder's s architecture. We also follow the API setting and initialize the stolen copies of language encoders with publicly available pre-trained transformers The stolen copies of vision encoders initialized with random weights.
Encoders. For language, public APIs (Cohere 2022 expose trans-
Stealing with Semii-uuperviion (Vision). semi-supervised learning which has been used to improve stealing in the supervised setting (Jagielski et al. 2020). The approach relies on semi remi-supervision based on Mix.a (Berthelot et al.) 2219) and helps to leverage a large pool of unlabeled data while having access only small fraction of labeled data (Sohn et al. 2020 Assran et al.) 2021I. The inspiration for applying MixMatch-bast
To reduce the number of stealing querie we apply
methods to stealing encoders comes from DINO (see Section 2. which normalizes representations with a temperature softmax and uses the cross-entropy loss to minimize distances between outputs from student and teacher encoders. Similarly, during stealing. we pass the victim and stolen representations through the softmax layer whose outputs act as our new labels for MixMatch. Note that this method not applicable to the fine-tuned sentence embedding encoders since these not rely on data augmentations, which is necessary for MixMatch.
Re-using Representations (NLP). For stealing sentence encoders, we reduce the number stealing queries by re-using representations over semantically similar sentences. This is possible since sentence encoders are required to return similar representations to such semantically similar sentences. Hence, when stealing dataset holds such similar sentences (e.g., in the form of positive pairs, such as all the datasets used in this work), we only have to query one of these sentences, and assign the same representation all semantically similar sentences to augment our fine-tuning dataset for the stolen-copy. Our experimental evaluation in Section 5.1 shows the effectiveness of this approach.
4 DEFENSES AGAINST STEALING TRANSFORMERS
Given the high training costs of transformer-based encoders, defending them from stealing is an urgent need. Yet, their representations' high dimensionality and complexity make defenses challenging.
I use transformers from Hugging Face (https: //huggi facc... co/
This is relevant in public API settings since costs usually increase linearly with the number of queries and since the number of representations that be obtained given time-unit often limited.
4

---

Under review as a conference paper at ICLR 2023
4.1 DATASEED INFERENCE FOR VISION TRANSFORMERS
Prior work (Dziedzic et al. 2022b) has successfully applied dataset inference (DI) (Maini et al. 202T) to identify stolen copies of encoders with -NN-trrhitectures. To verify ownership, DI assumes the training data to be private in order to act as a unique model signature. However, transformers usually trained millions of public data points, including noisy and uncurated data scrapes from the Internet (Radford et al. [022]. Thus, large transformers are trained on overlapping datasets which makes DI non-applicable this setting. To overcome this difficulty and leverage transformers' training signatures for ownership resolution, we propose DataSeed Inference (DSI). This defense combines DI with watermarking (Uchida et al.] 2017) by including private data as a secret seed into the original training data and training the defended transformer on the combined datasets. The private seed then serves to uniquely identify encoders and their stolen copies. Even if an adversary aware of our defense, they still require knowledge about the private seed to avoid extracting features contained in it. After stealing an encoder with an arbitrary dataset from the same data domain, we use the private data as well as its corresponding private test data for DSI. We visualize our approach in Figure Ta An advantage of DSI is that the private subset does not harm the performance of the defended encoder. We observe that by adding more diverse training data, the performance can be improved for downstream tasks that share features with the private subset. The observation of increased performance is contrary to standard watermarking in supervised settings. An intuition to this behavior is that our defense does not directly influence the downstream task but only transforms the representations. In contrast, in supervised learning, watermarking directly impacts the task that the model is trained for, yielding performance loss.
To provide the defended encoder with stronger unique signal for DSI, we train differently on the private seed than on the original training data. Specifically, we apply weaker augmentations to the private data points. This enables the defended encoder to overfit more easily to the private seed. To this end, we modify the training procedure of the DINO framework (Caron et al.) 2021). Concretely, we change the standard DINO's data pre-procs and increase the size of crops for images passed through the student and teacher encoders. This is applied only to data points from the private subset and not to the original training points. Instead of using the student's s crops that cover only small areas (less than 50%) of input image, increase the size of crops to the range between 70% and 90%. Furthermore, we also increase the size of the crops for the teacher encoder from greater than 50% to greater than 90%. DSI leverages this property and detects different behavior of encoders on their private training data versus unseen test data. Next, in our empirical evaluation we show that the unique signature from the private data seed is transferred to stolen encoder copies, making DSI applicable to perform ownership resolution. To further amplify the signal from the training signature, we add the victim's projection head on top of the verified encoders.
4.2 WATERMARKED SEVTEECC-EEMEEDDIN ENCODERS
We find that DSI (as well standard DI) do not yield significant results in detecting ownership for stolen language encoders. This happens because of multiple reasons. First, all victims, stolen copies, and independent encoders the same underlying pre-trained transformers. This similarity overlies the individual signals from datasets used for fine-tuning to sentence embeddings. Second, victim encoders are usually fine-tuned over small number of epochs (10 for Tiny and for BERT and RoBERTa). We observe that with longer fine-tuning, the results returned by DI on victim encoders become more confident. However, the signal on the stolen encoders remains insignificant.
As an alternative approach, we develop a new watermarking- based defense against stealing sentence embedding encoders. We embed the watermark starting from an already fine-tuned encoder. This is a realistic scenario where the model owner would like to add a watermark post hoc at a low cost. To embed the watermark, we perform a few iterations of training, where we always alternate between one iteration of the original sentence embedding training (with SimCSE), and then one iteration of training for a downstream task. During training of the downstream task, we add an additional fully-connected layer, which serves as our secret key during verification. Our watermarking approach is visualized in Figure lb In this work, we select SST2 (binary classification for sentiment analysis) as the watermark downstream task. Note, however, that defender can select from many possible downstream tasks, reshuffle or flip the labels, or use their own private downstream task, which makes the detection of the watermark much harder.
5

---

Under review as a conference paper at ICLR 2023
To resolve ownership, verifier simply attaches the fully-connected layer (secret key) to the output of an encoder suspected to be a stolen copy. Then, agreement between the outputs of the victim encoder (plus fully-connected layer), and the outputs of the potentially stolen copy (plus fully-connected layer) on the secret downstream task is measured as the percentage of labels where both outputs agree. We resolve that an encoder was stolen if the agreement is above the threshold of 95%, otherwise, the encoder is marked as independent. We assume that the adversary does not obfuscate the output representations as described in Dziedzic et al. 2022b
EMPIRICAL EVALUATION
We evaluate our methods for stealing and defending transformer-based encoders trained on different vision and NLP datasets.
5.1 STEALING TRANSFORMERS
Vision. For ImageNet victims, we use the ViT's checkpoints released by the original DINO paper (Caron et al. 1021). For CIFARIO, train Tiny ViTs from scratch. All training procedures follow (Caron et al. 202T) unless otherwise specified, using 300 epochs, a batch size of 256, and the learning rate set to 5e-4 with cosine annealing scheduler. For stealing, we experiment with different numbers of queries from various datasets, including CIFARIO, 10, CIFAR 100, SVHN, STL10, and ImageNet. Details on the datasets are included in Appendix B.I while further insights on the training can be found in Appendix [.22 We also use ViT' to instantiate stolen encoders and experiment with different loss functions, such as MSE, InfoNCE (Chen et al. 2020). or SoftNN (Frosst et al. 2019). Unless otherwise specified, our stealing procedure follows Dziedzic et al. (2022a).
We assess the performance of victim and stolen encoders using standard linear evaluation (Chen let al. (2020) on four downstream tasks: CIFARIO. CIFAR100, SVHN, and STL10 (see Table D observe that stealing with the victim's training distribution yields the closest performance of the stolen copy to the victim. The MSE loss performs better for the lower query regime (<100k) while contrastive losses like InfoNCE yield higher accuracy scores for the larger number of queries. When stealing with more complex datasets (e.g., CIFARIO 10 vs SVHN) we obtain a better generalization of the stolen copy, which is shown by higher accuracy on more downstream tasks. We also apply MixMatch for the case when an attacker has limited access the API and wants use as few queries as possible. Thus, in this case, we operate in the regime with a number of queries ranging from 4k to 10k, Table Our results show that Mixhatch outperforms all other methods, even when provided with fewer labeled queries, and retains high similarity to the victim encoder when measured with the cosine similarity score between the representations from the victim encoder and a stolen copy.
Language. We steal from BERT-based sentence embedding encoders fine-tuned on nli-for- SimCSE (Gao al. 202T) ("nli"), QQP (Iyer et al.) ("qqp"), and Flickr30k (Young et al. 2014) ("ffickr"). ). For more details on the datasets and our pre- processing, see Appendix B.3 For Tiny BERT. we fine-tune the victims using SimCSE (Gao al. 2021), for Bert Basq ("BERT"), and RoBERTa Large ("RoBERTaa) we use models from Hugging Face as victim encoders. We fine-tune indepen- dent models using SimCSE and initialize our stolen copies with pretrained transformers of the victim architecture. Following SimCSE, we replace the original BERT pooling layer of our stolen models with custom pooler that returns the [CLS] representation and add a randomly initialized MLP layer on top before stcaling For stealing, we issue 60K queries, using sentences from our three datasets against the victim encoders and fine-tune our stolen copies with the resulting representations for 20 (TinyBERT and BERT), 5 (RoBERTa) epochs, respectively, using MSE loss. Further details on our training and stealing methods are provided in Appendix B.4
Table depicts the performance of victim and stolen encoders, evaluated tasks from the SentEval benchmark. We observe that across all base encoders (TinyBERT, BERT, and RoBERTa), the performance of the stolen copies is comparable to their respective victim encoders over most benchmark tasks. This holds true even for the large qqp dataset (~2.66 training samples) and stolen
http://wiiinnne.aeoooeoeeeeeeeee aaaaace.e.. co/ http:prrinnncnnnn/eeeeeoe wwt.upppsmmssss... roberta- 'arge
Wee also experiment with stolen copies initialized without the MI.P layer and obtained similar performance in the benchmarks.

---

Under review as a conference paper at ICLR 2023
Table 1: Performance of Vision Transtormers. We follow SimCLR (Chen et al. 2020) and do linear evaluation of the encoders (denoted as EN) on downstream tasks. fv and I are victim encoders trained on data D.. fa the stolen encoder extracted using queries from a given stealing dataset Da. MM is the type of the underlying encoder, where T is ViT Tiny. S is VIT Small, and B is ViT Base. Victim encoder with an asterisk (*) is pre-trained encoder provided by Clarifai (022)). Other encoders are trained using the DINO code-base (https: g thub. com / fa ceboook. CE denotes the Cross Cross-Entropy loss and MSE is Mean Squared Error. For a given stealing dataset, we test loss functions: CE, MSE, InfoNCE, and SoftNN and report results for the best performing onc.
EN
M
Loss
OF QUERIES
Do
D.
CIFARIO
CIFARI00
SVHN
STL.10
f0
SVIN CIFARIO+SVIIN
73.3
49.03
90.4
36.2
59.6
87.9
59.97
78.4
63.3
MSF
SVHN
62.6
38.1
88.8
33.1
J
T
MSE
50K
SVHN
CIFARIO
72.7
49.3
56.8
37.2
f
CIFAR
f.
SVHN
60.1
35.0
67.2
31.3
fx
T
MSE
50K
CIFARIO
CIFARIO
86,9
58.7
64.7
52.7
f
MSE
CIFAR100
86.2
59.1
64.6
53.6
f0
S
IMAGENET
93.06
73.86
77.15
96.14
f
S
MSE
50K
IMAGENET
CIFARIO
88.75
58.4
60.3
42.5
f
MSE
100K
IMAGENET
SVHN
62.04
35.0
77.1
38.8
fo
S
CIFARIO
87.92
57.14
72.28
76.41
F0
S
CIFAR 10+S IO
88.07
57.49
78.07
76.28
f0
87.6
56.62
69.97
74.39
J0
CUPARIO-PASHOONNNIIT
88.05
57.19
71.74
76.83
Fo
IMAGENET
67.1
5.
CIFARIO
CIFARIO
85.49
54.6
Table 2: MixMatch for Stealing from VIT Tiny. We compare the MixMatch semi-supervised method of training a stolen encoder with previous approaches. We report the accuracy for CIFARIO (and for SVHN in Table[7]. We tune the temperature parameter for sharpening in MixMatch and report the highest results for a downstream task per given query number. We use the same number of unlabeled examples for Mix Match as the number of stealing queries. The stealing methods use different loss functions. CE denotes Cross-Entropa MSE is the Mean Squared Error, SoftNN is Soft-Nearest Neighbor, and we also use the InfoNCE loss. C(,, fo) (Cosine Similarity Score) is
between the victim model and the stolen copy, following Dziedzic al. 2022b
queries
Surlling
Stealing
5K
JOK
Method
Data
CIFARIO
SVHN
C(:
CIAARIO
SVHN
C(:fu)
CUARRI
SVHN
(v.)vv)
CE
65.93
40.77
0.90
68.95
46.23
0.92
76.11
51.01
0.97
SofNNN InfoNCE
CIPARIO
CIPARIO
64.05
43.52
0.54
66.83
47.02
0.86
74.14
48.95
0.92
66.16
41.37
0.57
68.51
48.96
0.89
75.59
48.15
0.94
MicMatch
72.86
50.31
75.04
50.9
copies obtained with only 60K queries, i.e., 400 fewer queries than training samples. In general, the performance of encoders stolen with nli and qqp is higher than the one of encoders stolen with flickr. We suspect this is due to the low semantic diversity in flickr which consists only of 30K images with five semantically equal captions each, leading semantic overlap within the 60K stealing-queries.
We further explore the impact of the number of stealing queries on the performance of the stolen copies. Our results in Table inn in Appendix C highlight performance decrease when reducing the number of stealing queries. The performance drop is most significant between 10k and 20k queries. This motivates an evaluation of the effectiveness of our method re-use representations for semantically similar sentences in this setup. We query the stolen model copy with 10k sentences from the nli dataset and assign the obtained representation also to the semantically equal positive partner of each sentence. This results in 20k fine-tuning samples for the stolen copy. Our results in Table higghlggh that the performance of the stolen copy with only 10k queries (augmented to 20k data points) similar the original stealing with 20k sentences.
7

---

Under review as a conference paper at ICLR 2023
Table 3: Performance of NLP Transtormers. We follow SimCSE (Gao al., 2021) and use the SentEval benchmark. fo denotes the victim encoder trained on data Dv. fs is the stolen encoder extracted using queries from a given stealing dataset D,- For stealing, we use 60,000 queries to the victim encoder and fine-tune our stolen copy with the resulting outputs for 20 (TinyBERT (T) and BERT (B)), and 5 (RoBERTa (R)) epochs. Victim encoders with an asterisk (*) are pre- trained encoders from Hugging Face obtained by the SimCSE code-base (https: / / thub com/ pr wniiiicttoo on- nlp/S /S//S/L SimCSE), while other encoders are trained using the SimCSE mode-bbsce
FN
D2
CR
MPQA
MRPC
SURII
ATS-STS
Avg-Toa
SICKR
STSR
nli
74.69
82,17
68,51
71.93
72,02
87.99
59.74
76.64
73,86
73667
79,61
10
eqp
72.27
65.93
08.97
57.17
56.15
76.22
fhecr
71.87
79.39
06.56
71.52
69.51
56.89
55.25
72.83
71.55
06.63
79.03
nii
73.56
79.67
67.16
71.06
71.33
87.28
56.53
73.48
72.37
69.56
77,40
nii
vip
73.45
79,58
67.22
70.78
71.9
57.23
56.51
71.08
72.42
65.74
76.41
Blickr
71.53
78.12
66.91
70.02
70.64
87.12
57.45
71.71
71.73
65.98
77.44
nli
71.42
78.08
66,88
70.14
70.64
86.62
50.24
64.97
70.57
55.18
74.75
T
44P
qup
72.2
71.67
56.83
52.97
71,43
54,06
74,06
Bickr
71:95
77.78
67.14
71.61
71.67
70:93
56.37
74.09
nli
70.51
78.53
85.7
70.53
63.43
77.63
Mickr
qup
71.05
66.63
71.49
70.87
$6.46
68.57
71.22
60.42
76. 73
Blickr
70.66
77.19
66.74
71.12
71.22
$6.13
53.76
70,92
70.97
63.82
78.03
Fu
B+
nli
89.20
89.67
82.88
73.51
87.31
94.81
88,40
81.57
86.54
80.39
84.26
nli flckr
89,05
88.95
$0.49
74.98
86.35
93.39
66.73
81.45
82.85
79.84
83.07
B
nli
88.25
89.18
79.74
75.20
$6.58
94.19
69.108
79.93
83.16
77.24
82.62
82.01
63.41
79.37
81.50
fx
R*
92.37
90.52
$8.04
76.64
92.31
95.13
91,20
83.76
89.46
81.95
86,70
nli 92.00
90.72
86.36
76.41
91.76
94.19
$6.00
82.15
88.21
81.03
85.21
8
quP
85.84
fickrr
91.74
90.14
35.13
74.72
90.39
93.13
83.20
79.61
86.92
79.06
82.55
Table 4: Re-using Representations. We assign the same extracted representation to a given query and its semantically similar sentences. # Samples denotes the final number of sentences used to fine-tune the stolen encoder.
EN
Do
D, A Queries
* Samples
CR
MPQA
MR
MRPC
8ST2
SUB]
TREC
Avg/inm
SICKR
STSD
10000
51.38
57.64
71.88
73.09
83.94
93.56
39.25
81.75
30.63
61.87
fx
B
al
sli 10000
20000
86.89
88.59
78.56
76.42
85.44
93.29
68.62
68.35
N2.54
65.30
71.40
20000
20000
87.22
89.11
79.33
75.66
86.93
93.45
70.14
69.35
85.12
66.51
73.18
5.2 DEFENDING TRANSFORMERS
DataSeed Inference. We design DataSeed Inference to incorporate private seed into the victim's 's training set. Concretely, we insert randomly-selected 10k images from the SVHN and GTSRB training splits into the CIFAR10 training sets. We train ViT Tiny and ViT Small on the combined datasets as victims. We use weaker augmentations for the private set, which results in more similar inputs during training and evaluation and provides a stronger signal for DSI. Otherwise, the training procedures for the victim and stolen encoders are the same as before. Table 12 shows that DSI is more effective when the projection head in training DINO uses shallower MLPs. This result meets our intuition: since the training loss is minimized directly with the projection head, some information about the training signature is lost when the head is removed. Therefore, to make the training signature easier to detect, we attach the victim's projection head on top of the verified models. We use 20k randomly-selectee images from the rest of the training splits as the validation set in DSL. We train GMMs with 10 components across all settings. For more details, see Appendix B.2 The results in Table 5 demonstrate that DSI is able to differentiate the stolen transformers of CIFARIO by injecting a private subset of SVHN and GTSRB, even when the stealing datasets are different from training. Similar to DI for supervised learning (Maini et al.) 2021). the victim encoder typically has the largest Âp and the smallest p-values. We also find that increasing the size of the private set makes this defense more effective. Finally, our comparisons with the independent models ensure that we avoid false positives when detecting stolen models (see last two rows in Table We also assess the performance of the encoders trained with CIFAR10 and a private data seed. We observe that these encoders achieve similar performance to the victim trained only on CIFARIO 10 and can outperform when assessed on the downstream tasks that share features with the private subset, see Table
Watermarking Language Encoders. We present the performance of the watermarked encoders as We use the initial fine-tuned sentence embedding
well as the success of our watermarking in Table 6
8

---

Under review as a conference paper at ICLR 2023
Table 5: DataSeed Inference for Vision Transformers. We train ViT-Tiny (left two columns) and Vit-Small (right two columns) using DINO. We add 10k data points from SVHN and GTSRB into the CIFARIO training set. fr is the victim transformer trained on data D. fa is a transformer stolen with queries from dataset D, and fi is an independent encoder trained on data D (different than the victim's 's private training data). Each value an average over trials. Au is the effect size from the statistical t-test. An encoder is marked as stolen if the p-value is smaller than a threshold of 0.05.
Thn
Small
Vietion's dete
CVEARTO-SVRN
CIFARID-GTSRR Ay
CIFARIO-SYIN
CIFARIO+GTSRB
Encoder
p-value
Ap
p-taine
p-value
Ap
p-value
fu
CHARRID+
SVHN
8.346-63
16.85
CHARRIO+
GTSRR
6.31e-19
8.81
CIFARIO+
SVHN
1.47e-262
35.29
CHEARI++
GTSRB
4.95p-77
18.64
CIFAR100
5.69p-12
6.79
CIFARIO0
1.8lc4
3.56
CI
5.31e-
3.27
CIFAR100
4.61e-3
2.06
STLI0
2.5
1.95
STLIO
2.79c-13
7.21
SILID
2.27e-2
1,93
STLIO
3,332-5
3,87
ImageNet
6.80e-4
2447
ImageNet
1.M4c-5
4.14
ImageNet
3.246
1.91
ImageNet
1.76e:3
2.92
CIFARIO
0.63
-0.35
CIFARIO
0.18
0.88
CIFARIO
0.57
-0.19
CIFARIO
0.18
0.89
CIFARID+
CIFARIO+
CIFARIO-
CIFARIO+
FablioMNNIS
0.88
-1.17
FashionMNIST
0.42
0.20
FashioMMNIST
0.95
-1.64
FishionMNIST
0.94
-1.61
Table 6: Watermarking Sentence Embedding Encoders. We embed the watermark into an encoder and present the performance of the downstream task, the underlying encoder, and the comparison between the victim fu. stolen far and independent fi encoders. Steps denotes the number of fine- tuning steps for watermarking. Agreement is denoted by Agr and accuracy on the watermark downstream task by Acc, Both are given in %. Test loss is denoted as L, p-value as p, and effect size
as Ap.
Steps
(gr 1o) Agr(Ja.)) 1.) Ace(fo) L(fs)
SIS
SICKR
SISB
Acc(/a)
L(fa)
pl (a) Au(Jo. 16) pl
Ap(fo.
100
97.59
88.3
57
0.58
0.77
0.74
0.79
56.42
0.68
0.79
0.27
0.0016
3.15
200
96.44
67.54
65.37
0.65
0.76
0.73
0.79
64.79
0.64
0.47
1.74E-14
97.71
72.59
0.53
0.76
0.73
65.14
0.62
9.71
400
96.55
39.98
69.95
0.51
0.76
0.73
0.79
70.18
0.51
0.49
0.69
1.438/27
11.07
500
96.35
60.66
71.1
0.59
0.75
0.71
0.79
71.1
0.6
0.46
0.74
2/678-28
11.24
96.79
62.73
0.59
0.74
0.69
72002
0.44
4.686-2
62.61
71.44
0.58
0.72
0.67
0.77
72:25
0.58
0.45
0.76
2,309,35
12,69
800
96.79
56.31
73.85
0.56
0.71
0.66
0.76
72.71
0.57
0.45
0.76
2.18-42
14
900
96.34
59.75
23.39
0.55
0.71
0.65
0.76
72.13
0.56
0.42
0.8
3.028-45
14.53
encoder as the independent encoder. This can be considered the worst-case evaluation because the representations returned from this encoder are expected to be most similar to its watermarked derivation. We show that even in this worst-case, the independent model is never incorrectly resolved as being stolen copy. To compute the p-values, leverage the confidence scores (softmax outputs) for the correct labels from the downstream task and use the t-test. The p-values indicate that there significant difference between the distribution of the confidence scores from independent vs victim encoders (p-value < 5%). In contrast, the difference is not significant between the victim and stolen encoders. Our results highlight that a relatively small number of fine-tuning steps (e.g., 200 alternations between the original task and downstream task) are sufficient to successfully embed the watermark into the encoder while preserving the high performance of the defended encoder on other unrelated and general downstream tasks.
6 CONCLUSIONS AND FUTURE WORK
Modern APIs offer access to high-value transformer-based encoders for generating representations of given input text or images. We demonstrate how to steal these transformers in the language and vision domains by using only representations. Our stealing requires up to 40x fewer queries than the number of training data points used to train the victim and it yields stolen copies with comparable performance on standard benchmarks. We further decrease the number of stealing queries by using semantically similar sentences in the language domain and semi-supervised learning in the vision domain. We propose a new defense for vision transformers, where a private data seed acts as the training signature of the victim encoder. We find that this type of defense is unable to protect sentence embedding encoders since the signal of the private data subset used during fine-tuning is overlaid with the data used during pre- pre-training of the base encoder. To overcome this obstacle, propose a method for embedding a watermark into the encoders by fine-tuning a defended encoder with a specific downstream task.

---

Under review as a conference paper at ICLR 2023
7 ETHICS STATEMENT
Even though we place our work in a public access setting and implement our stealing based on representations such as the ones returned by public APIs, we do not actually steal the models from real-world APIs. All the models stolen in this work are either trained by ourselves or obtained from open-source platforms, such as Hugging Face. However, we note that several public APIs expose access to the same or similar models. Hence, our attacks could potentially be applied against such APIs to steal their models. Given that the models attacked in this work are publicly available anyways, their extraction from an API would not cause additional harm to the API provider. We still decided to keep the code implementing our model stcaling confidential to minimize the risk of exposure through work for such providers. Instead, we submit only the code implementing our defenses.
REPRODUCIBILITY STATEMENT
Due to the reasons elaborated in the ethics statement, we do not publicly submit the code of our stealing procedure. Instead, for reproducibility, we describe our approach in detail-documenii the frameworks we used, all hyperparameters for training, and the architectures. We submit the code for defense in the supplementary material. In the README. file provide the main commands needed to run our code. We also describe the pointers to the crucial parts of the code that directly reflect the implementation described in the main part of the submission. We also describe the defenses used in our experiments thoroughly.
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1615-1631, 2018.
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Ballas, and Michael G. Rabbat. Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8423- 8432, 2021.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference Learning Representations, 2022. URL htttpsss///peeeeeiiiee
net htte//orumm22 Socuum?iice Pi- BhZS25904
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2015.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9650-9660, October 2021.
Chen Chen, Xuanli He, Lingjuan Lyu, and Fangzhao Wu. Killing bird with two stones: Model extraction and attribute inference attacks against bert-based apis. 2021. URL httttt::////x:: org/ htg//bb//a 25. 10909
Ting Chen, S. Komblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. International Conference Machine Learning, 2020.
10

---

Under review as a conference paper at ICLR 2023
Clarifai. https://clarffai.com, 2022. URL httlttta//laaraaaai...
Adam Coates, Andrew Ng. and Honglak Lee. An Analysis of Single Layer Networks in Unsupervised Feature Learning. In AISTATS, 2011.
coatesleeng__isttee 2011 pdf
Cohere. https://cohere.aai ai, 2022. URL bttts::/cohere:
Tianshuo Cong, Xinlei He, and Yang Zhang. Ssiguard: A watermarking scheme for self-supervised learning pre-trained encoders. CoRR, abs/2201.11692, 2022. URL https::/axxxii..
abs 12201.11692
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):441-14,, 2012.
Jacob Devlin, Ming-Wa, Wing-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 18X0:v:8100040000 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 11929. 2020.
Adam Dziedzic, Nikita Dhawan, Muhammad Ahmad Kaleem, Jonas Guan, and Nicolas Papernot. On the difficulty of defending self self-super learning against model extraction. In International Conference on Machine Learning, 2022a.
Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kaleem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot. Dataset inference for self-supervised models. arXiv, 2022b. StRLhhts: www.ivi.vv.v..///aa./.aa./aaaavl www.iiiiiooiooooooooooooooo abs 220.2229222. 022.209.0900.44
Nicholas Frosst, Nicolas Papernot, and Geoffrey Hinton. Analyzing and improving representations with the soft nearest neighbor loss. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2012-2020. PMLR, 09-15 Jun 2019. URL https:// proceedings www.m..pp....////eemmm /
9a htm
Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence
embeddings. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pp. 6894-6910, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 1/22223/1222211... //aclanthology. wrt.:/.rr////0
main. URLhttps
main. 552
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000-16009, 2022.
Xuanli He, Lingjuan Lyu, Qiongkai Xu, and Lichao Sun. Model extraction and adversarial transfer- ability, your BERT is vulnerable! arXiv preprint 1rXiv2213..00113 2021.
Felix Hill, Kyunghyun Cho, and Anna Korhonen. Learning distributed representations of sentences from unlabelled data. In Proceedings of the 2016 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies, pp. 1367-1377, San Diego, California, June 2016. Association for Computational Linguistics. doi: 10.18653/v1/
N16- URL https: httpa/caannooolooooooooooooooooa http:hhhooooooooggggoogo 6-
Shankar Iyer,
Nikhil Dandekar, and Kornel Csemni.. release: Question pairs.
First quora dataset
URL https::/quozzaataa. www.aaucorrrooo..o..o com/
F ratt-Quora-Daaaseee
Mamoor--Daaseee Seteeasse-uest... P
11

---

Under review as a conference paper at ICLR 2023
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papemot. High accuracy and high fidelity extraction of neural networks. In 29th USENIX Security Symposium (USENIX Security 20), pp. 1345-1362, 2020.
Hengrui Jia, Christopher A Choquette Choo, Varun Chandrasekaran, and Nicolas Papernot. Entangled watermarks as defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 2, pp. 1937-1954, 2021.
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang. and Qun Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint 2019.
Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan. Prada: protecting against dnn model stealing attacks. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P), pp- 512-527. IEEE, 2019.
Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran As- sociates, Inc., 2015. URL htt.::ttt////t htt.ttps://prooeeednnge nwwrips.erppp..ee httprr/pps.cpppppet///////// pdf
Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit lyyer. Thieves on sesame street! model extraction of bert-based apis. In International Conference on Learning Representations, 2020. URL htthhtps::/// https/:://pppeeeeeeeeee
net/ wow.:ooidmmnnllllaaeea 55REFDr
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009,
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1rXiv: 2019.
Lajanugen Logeswaran and Honglak Lee. An efficient framework for learning sentence repre- sentations. In International Conference on Learning Representations, 2018. URL https
//openreview ealleewneeteee net/ www.:orddduuuuda2220
Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. In Proceedings of ICLR 2021: 9th International Conference on Learning Representationsn, 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. URL http: http://fidd..sttfford...... stanford.edrrr
hooseennberrs//pp220111 httpoloiuoouuuuuuuuerruu pdf
OpenAl. https://openai.com, 2022. URL
com/
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction poisoning: Towards defenses against dnn model stealing attacks. In International Conference on Learning Representations. 2020. URL / http///pppp////////////tpp/ net wwweeee/oooommoodeeeeeeeeee
Namuk Park and Songkuk Kim. How do vision transformers work? arXiv preprint arXiv:2202.006709 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763. PMLR, 18 24 Jul 2021. URL
https: fttts::/proceest wtt.:nni.sss.es/e////eeep
ngs. m www.sr/s/eaeec http://......aaa////////oooooooa
html
Mujahid Al Rafi, Yuan Feng, and Hyeran Jeon. Revealing secrets from pre-trained models. 2022.
URL httphhtsss///ai...ttct org.//iioooooooeoooe.o. wtw. 09539
12

---

Under review as a conference paper at ICLR 2023
Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can't steal? cont-steal!
contrastive stealing attacks against image encoders. 2022. URL/attp::/axxxv..o
Or Sharir, Barak Peleg, and Yoav Shoham. The cost of training NLP models: A concise overview. CoRR, abs/2004.08900 009000 2020. URL httpt::tttaaattt
08900
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Do- gus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi- coni-supervised learning with consistency and confidence. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33. pp. 596-608. Cur- Associates, Inc., 2020. URL httppsss/pproceeeinngs http://wwweiiin......e.eeeeeeeee www.ipeurpppp.... cc wttppcppppppccc...//////p. http://555.eeeeddccccccceeeeeee.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé
Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. 25th {USENIX) Security Symposium ((USENIX) Security 16), pp. 601-618, 2016.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and iiin' Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp. 269-277, 2017.
Aäron van den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. ArXiv, abs//807. ht5/1807.03748 003748, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long Papers), pp. 1112-1122. Association for Computational Linguisties, 2018. URL http:ttt/acc//eee
org/ anthology/Nii 1101
Yutong Wu, Han Qiu, Tianwei Zhang, Lin Jiwei, and Meikang Qiu. Watermarking pre-trained encoders in contrastive learning. ArXiv, htts/2201.082/// 17,17,200 2022.
Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari. Beyond model extraction: Imitation attack for black-box NLP APIs. arXiv preprint arXiv:2 13873, 2021.
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67-78, 2014.
Santiago
Shruti Tople, Andrew Paverd, and Boris Köpf. Grey-box extraction of natural language models. In International Conference on Machine Learning, pp. 12278-12286 PMLR, 2021.
13

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL RELATED WORK
Self-Supervised Learning. In computer vision, one of the most popular self-supervised algorithms
contrastive learning (Chen et al. 2020 van den Oord et al. 2018), where representations that come from differently transformed views of the same image are brought closer to each other and the representations from views of different inputs are repelled. In NLP, a popular self-supervised
pre-training approach is to mask selected words in the input sequence and train the model to predict that masked words Devlin et al. (2018).
Transformers. Transformer (Vaswani et al. 2017 becoming a ubiquitous architecture in NLP and computer vision. While the onginal transformer consists of an encoder and decoder component, our work only studies the encoder part for representation learning. Transformers are composed of several identical layers, namely a multi-head attention sublayer followed by feed forward sublayer. The multi-head attention sublayer utilizes the self-attention mechanism to learn the pairwise relationships between all tokens. Self-attention is the key to the success of transformers, which makes parallel training and learning long-range dependency between tokens easier.
NLP transformers. In this paper, we investigate BERT-based models (Devlin al., 2018), pre- trained bidirectional transformers (Vaswani et al. 2017). In addition to BERT Base, we also use TinyBERT (Jiao et al., 22019), obtained by distilling BERT to a smaller transformer architecture, and RoBERTa Large (Liu al. 2011),, which is pre-trained on a larger dataset than BERT. Then we analyze the task of learning highly generic sentence representation, a fundamental problem in
NLP (Kiros al. 2015 Hill al. 2016
Logeswaran & Lee 2018).
Vision Transformers. Vision transformers (Dosovitskiy et al. 2020: Touvron et al.) 2021 Park
Kim 2022), inspired by transformer models from the NLP domain, achieve state-of -art
performance for many downstream tasks, such as standard image classification (He et al. 2022) or
segmentation (Caron et al., 2021). They are based on either self-distillation with no labels (Caron
eet al. 102T) or masked autoencoders (MAE) (Devlin et al., 2018 Dosovitskiy et al. 2020). MAE is a masked autoencoder for the supf-suue training of vision transformers. The approach is based on training an asymmetric encoder-decoder architccture that learns to reconstruct randomly masked patches of an input image. BEiT (Bao et al., 2022) is another similar approach to MAF that also masks patches of an image and learns to recover the original inputs.
Watermarking Encoders The watermark scheme proposed by Dziedzic et al. 2022a) trains the defended encoder simultancously with an augmentation predictor. As the watermarking task, the defended encoder is trained to predict the rotation range of an input image. For watermark verification, an encoder's 's representations are evaluated on the augmentation predictor to obtain the accuracy on the watermark task and when the accuracy is significantly higher than 50% the encoder is marked as stolen. SSLGuard (Cong et al. 2022) is another watermarking-based defense. First, it generates secret key, a verification dataset, and a decoder. watermarked encoder and its stolen copies map samples in the verification dataset to secret representations that in turn can be transformed into the secret key through the decoder. For independently trained encoders, the decoder transforms the representations generated from the verification dataset into random vectors. SSLGuard assumes a specific attack to which the trigger is transferred, which makes it vulnerable to new types of attacks. On the other hand, our DSI uses private seed as signature and does not rely on any assumptions about a type of stealing attack. [Wu et al. (2022) manipulates the verification samples to obtain for them unique feature representations from protected and stolen encoders, which leads to unique predicted labels for any downstream task. However, it was shown that representations can be easily obfuscated (Dziedzic et al. 2022b) remove their similarity to a stolen encoder.
Stealing from NLP Classifiers The stealing methods proposed in Chen et al. (2021) and Rañ et al. (2022) are for downstream classification tasks while we steal representations from self-supervised encoders in both language as well as vision domains. For the vision encoders, we consider a more challenging task and start stealing from randomly initialized encoders instead of pre-trained ones. Apart from improving the stealing process by leveraging semantically similar sentences, we also propose a defense method against stealing sentence embedding encoders using a watermark- based approach. Chen al. (2021) steal fine-tuned classification models, while our extraction targets
14

---

Under review as a conference paper at ICLR 2023
encoders, hence, it is more general. Rafi al. (2022) extract architectures and weights, while attack method extracts representations. Additionally, Rafi et al. (2022) require adversaries to have more knowledge about the victim than is assumed in our work. For example, architecture extractions require access to the target machine.
B DETAILS ON THE EXPERIMENTAL SETUP
B. VISION DATASETS
MNIST Deng 2012) The MNIST database of handwritten digits has training of 60,000 examples, and a test set of 10,000 examples. The digits have been normalore and centered in a fixed-size image. It consists of 28x28 black and white images with 10 classes.
CIFAR10 Krrihhvskky 2009): The CIFAR1O dataset consists of 32x32 colored images with 10 classes. There are 50000 training images and 10000 test images.
The CIFAR 10 dataset consists of 32x32 coloured images with 100 classes. There 50000 training images and 10000 test images.
CIFAR100 (Krizhessky
2009):
10 The SVHN dataset contains 32x32 coloured images with 10 classes. There roughly 73000 training images, 26000 test images and 530000 "extra" images.
SVHN Netzer et al.
Image et al. 2009): Larger sized coloured images with 1000 classes. As is commonly done, we resize all images to be of size 224x224. There are approximately I million training images and 50000 test images.
STL10 (Coates et al., (01TT): The STL10 dataset contains 96x96 coloured images with 10 classes. There are 5000 training images, 8000 test images, and 100000 unlabeled images.
B.2 EXPERIMENTAL SETUP FOR STEALING VISION TRANSFORMERS
For vision, ViT Tiny, Small architectures used for the CIFARIO and SVHN victim encoders. We use DINO (Caron et al. 2021) to train ViT with patch size 16 a self-supervised way. We resize the images to 224 x 224 so that we can the hyperparameters provided in the DINO paper. We use 3-layer MLP and then apply an L2 normalization, and a weight normalized fully-connected layer as the DINO paper by default, and do experiments with the different number of MLP layers in ??.
We train all vietim models for 300 epochs. To train the stolen encoders, we used 150 epochs. We use AdamW optimizer and Cosine Annealing scheduler when training victim and stolen models. When training the victim model, the initial learning rate is 4c-5 while for training stolen models, the initial learning rate le-3. A batch size of 128 or 256 was used for training the models. The temperature we use for InfoNCE is 0.07, and the temperature for soft nearest neighbors loss is 1000.
For the stealing experiments, we use the last 4 block outputs of dummy representations, in the same way as in DINO (Caron et al.) (021). Thus, the dimensions of output representations for ViT-tiny and ViT-small are 768 and 1536, respectively.
Victim We train tiny-ViT's on CIFAR10. All training procedures follow (Caron et al. 2021 unless otherwise specified. The number of epochs is 300, the bs=256 and I==5e-4 with a cosine annealing scheduler. The summary of the victim encoder's performance can be found in Table [
Stolen When stealing from the victim encoders, we experiment with different numbers of queries from various datasets, including CIFAR10. CIFAR 100, SVHN, and STL10. We also use both ResNet and TinyVil as the architectures for stolen encoders. Stolen encoders are trained by minimizing different loss functions, such as MSE, InfoNSE (Chen al. 2020), or SoftNN (Frosst et al. 2019). Unless otherwise specified, the stealing procedure follows Dziedzic et al. (202aa).
Independent We ViTs trained other datasets than the victim's 's training set as independent encoders.
Private Data Subsets For DSI, we insert 5k, 10k SVHN, and 10k MNIST images into the CIFAR10 training sets respectively. The training procedures for the victim and stolen encoders are the same as before.
15

---

Under review as a conference paper at ICLR 2023
B.3 NLP DATASETS AND PROCESSING
nli. We (Gao al. 2021 is nli-forSSimSCE dataset, consisting of 275,602 data rows from SNLI (Bowman et al., 2015) and MNLI (Williams et al. 2018). Each row holds three sentences, an original sentence, a positive entailment, and a contradiction. In training, the contradiction acts as hard-negative.
qqp. We use the merve/qqp dataset from Hugging Face. The train split consists of 2,607,949 data rows, each holding two semantically equal questions. We use this data for training positive pairs.
flickr. The flickr dataset consists of images, each annotated with five human-writter captions. Following (Gao et al. 202T). we consider any two captions of the same image positive pair. We split the training and test set 90%, 10%, making sure that all caption pairs related to one image end up in the same set. This yields 286,050 positive-pair training examples. When using flickr as dataset for stealing, we drop the duplicates arising from generating all possible caption pairs before sampling the stealing-queries. Thereby, mitigate too small diversity over the stealing due to repeated queries.
B.4 EXPERIMENTAL SETUP FOR STEALING LANGUAGE TRANSFORMERS
Victim. As victim encoders, we use TinyBERT-based encoders and fine-tune them for the sentence embedding task on nli, qqp, and flickr by using SimSCE (Gao et al.) 2021). For more details on data pre-processing and the datasets, see Appendix B.3 We fine-tune our encoders for 10 epochs, with batch size (bs)=128, learning rate (Ir)=5e-5, and remperature=0.05. Additionally, we use transformers fine-tuned with nli from BERT [ and RoBERTa from Hugging Face as victim encoders. For fine-tuning and stealing over all encoders, we set the maximal input sequence length to 32 and use truncation and padding. The performance of our victim encoders can found in Table 3
Stolen. We initialize our stolen encoders with pre-trained transformers from Huggingface /bbertii/,,,, bert-base-uncased, and roberta-large), in accordance with the respective victim encoder. During stealing, TinyBERT and BERT use Ir=1e-5, bs=256, and linear Ir-scheduling with patience 200 iterations and factor 0.5. For RoBERTa, we use the same setup, however with bs=64, Ir-patience 600 iterations, and Ir=5e-6 when stealing with nli or flickr. As in SimCSE, evaluate our stolen encoders on the SentEval benchmark.
Independent. We fine-tune the independent TinyBERT-base encoders nli, qqp. and flickr in the same setup as the victim encoders. To obtain independent encoders based on BERT and RoBERTa, we fine-tune the respective base encoders nli, qqp, and flickr for using SimCSE. We keep Ir=5e-5, temperature=0005, bs= 18 and bx=32 for BERT and RoBERTa, respectively, but following (Gao et al. 2021). we fine-tune only for epochs.
DSL. To evaluate DSI, we generate training datasets with a private dataset as secret seeds. We generate three such sets: 50K flickr data points the qqp dataset, 2) the full flickr, i.e., ~30K into qqp, and 3) 50K qqp into flickr. We fine-tune the-tune sentence embedding encoders based on TinyBERT using the same procedure as for the victim and independent encoders. When analyzing the performance of encoders trained with the data subsets in Table 7 and note that for most tasks the flickr+qqp encoder overpasses the performance the encoder trained only on flickr, while the performance of qqp+flickr increases only for the STS benchmarks.
Table Performance of NLP Transformers with Mixed Dataset. We follow SimCSE (Gao et al. 2021) and use the SentEval benchmark. fu denotes the victim encoder trained on data D.. fa is the stolen encoder extracted using queries from a given stealing dataset D..
EN
CR
MPQA
MR
MRPC
SST2
SURJ
TRBC
Avg.sTS
SICKR
STSB
70.67
76.71
64.74
70.51
67.2
83.96
55.15
70.13
09.35
62.26
78.01
spptflickt
(ickr-gg<50k)
66.54
71.17
67.02
79.1
https://hugginggface
roberta- large
16

---

Under review as a conference paper at ICLR 2023
Table 8: Cosine Similarity Scores. Scores calculated over 20K data points between a victim encoder for copies fa stolen with data D,, and independent encoders fi trained on D, Results reported with standard deviation. In case fu and fi use the same data, we train two independent encoders with the same architecture, data, and hyperparameters (gray). Results with a double asterisk (**) indicate that we compare two independently trained BERT or RoBERTa-based encoders, instead of comparing the public Hugging Face encoder (*) with an independently trained encoder. This serves for better comparability to results obtained with TinyBERT. The notation follows Table For encoders trained with the additional private subset, stolen copies should be compared with independent encoders trained on the same public data without the inserted subset (blue).
stolen with
trained
fu on D,
nli
44p
flickr
q4p
llickr
nli
0.94
0.87:10.05
0.86 12
0.99:10.00
0.58.10.09
0.73.1..09
qap
0.75=00.08
0.83::0.04
0.66:0.11
0.61=00.08
0.95:10001
0.52::0.08
flickr
0.91=0.04
0.81=0.07
0.94:0.03
0.77::0.06
0.46:t0009
0.98:00.0
T
flickr
0.81
0.37=0.09
0.84
0.5::0.09
0.694 0.08
0.38
fgp+fickk(full))
0.79=0.05
0,3:0.09
0.84=0.05
0.62:+0.08
0.6 0.09
0,59+
llicktqupp 0.8:
0.89:+0.03
0.45=00111
0.74::0.06
0,52::0.09
0.74::0.06
B
ali
0.87:10.04
0.68.10.09
0.7.10.21
0.99.10.0*
0.66110.07*
0.7.0.1
R
nli
0.1120006
0.68:10.08
0.62:10:22
1.010.000*
0.3
0.4100.11
C ADDITIONAL EXPERIMENTAL RESULTS
C.I ANALYSIS OF THE NLP ENCODERS AND THEIR SIMILARITIES
As an alternative to the DSI, which is ineffective in protecting our sentence embedding encoders, we evaluate the similarity between the victim, stolen, and independent encoders, we use the cosine similarity score, in (Dziedzic) ct al. 2022b). Note that the cosine similarity score should not be used for the ownership resolution. This is a very fragile metric and any deviation in the training of the stolen copy from the victim increases the distances between their representations. As shown in Dziedzic al. (2022b), most obfuscations of stolen representations leave a stolen copy undetectable by the cosine similarity score. We track the cosine similarity between representations of the stolen and victim encoder over the course of stealing on 50 representations from the test dataset (Figure 2 and observe that similarity increases from the very beginning of stealing providing evidence of the highly accurate stealing process. Table depicts the final cosine similarity scores between language encoders, stolen copies, and independent models after stealing completed. We find that the stolen copies have higher scores than independent encoders trained on different datasets than the victim's 's data.
1903
2000
100
Figure 2: Cosine Similarity Score over Stealing.
In Table g we also evaluate the cosine similarity of our encoders trained with private seed against their stolen copies and independent encoders. We evaluate the similarity of our encoders trained with private subsets against their stolen copies and independent encoders. We evaluate qqp and flickr in both combinations as public and private data and do not consider nli. This is because the nli dataset contains SNLI, which, in turn, contains data from flickr which might corrupt our results. Experimenting with the full flickr dataset inserted into qqp serves a fair comparison in terms of the fraction of subset data, given that qqp holds roughly 10x more data than flickr (2.6M vs. 300k). Our results highlight that inserting a large enough subset of flickr increases the cosine similarity between the qqp+flickr encoder an independent flickr encoder while it decreases the similarity to an independent flickr encoder. The same holds for the qqp-subset. By inserting larger fractions of flickr, the qqp+flickt encoder's 's representations also resemble more the ones of the nli encoder, which can be explained by nli containing some sentences from flickr. For meaningful ownership resolution, the similarity between the victim and its stolen copies has to be compared to the similarity of the victim and an independent encoder trained on the same public dataset but without the private subset (color-code in blue in Table 8) Our results show that if a victim is stolen with the private data subset independent dataset, the similarity to its stolen copies is higher than to the independent
encoder. However, the information on the private data subset is not transferred to the stolen copies when the stealing is done with the victim's 's original training data distribution. In public setting, this
17

---

Under review as a conference paper at ICLR 2023
is problematic since APIs reveal information about types of data used to pre-train and fine-tune their encoders.
Note that for the pre-trained encoders, BERT* and RoBERTa* from Hugging Face, the cosine similarity scores to our independently trained encoders are close to zero. This might be caused by different factors, such modifications to the code base differences in the execution environment. Moreover, (Dziedzic et a 2022b) showed that even a small obfuscation of the representations decreases the cosine similarity between IWO encoders substantially. Note that despite some inconsis- tencies in representations between BERT and RoBERTa from Hugging Face and our independently trained encoders, we are able to steal the encoders from Hugging Face. Our stolen copies do not only achieve high performance on benchmarks (Table 3 but also produce representations with high cosine similarity scores to the pre-trained encoders (Table This highlights that while language transformers can be stolen effectively, the signal in their representations does not easily benefit ownership resolution.
Therefore, to evaluate a worst-case scenario (the exact data and training procedure of the victim are public and the independent encoder is trained in exactly the same way), we train BERT and RoBERTa-based encoders nli twice and report the cosine similarity scores between these two encoders, instead. The encoders trained independently on the same data and with the same procedure yield similarity score close to one (color-coded in gray))hight copies. This indicates that the scores cannot be used for ownership resolution since an honest party who has trained encoder using the same data and procedure as the victim would be falsely accused of theft. These results are expected. In fact, standard DI cannot distinguish between a victim encoder and an independent encoder trained with the same procedure using the same data. Therefore, DI assumes that the victim's training data is private, which, as we motivated above might not be a realistic assumption for transformers trained on public data. This motivates our novel DSI.
than the victim and its stolen
Our experiments with DI, DSI and cosine similarity for language transformers, yield some additional interesting insights that need be taken into account when designing future defenses against stealing NLP encoders. We observe that if the training and test sets of an encoder consist of semantically equal or similar sentences, DI is not even enable to mark a victim encoder. This is caused since the encoders are trained to output similar/same embeddings for semantically equal sentences. For example, when splitting the flickr dataset such that captions of the same image are distributed over the train and test sets, the distributions of the resulting representations are similar that DI cannot extract any signal. Therefore, when attempting to identify stolen encoder copies in NLP, auditors need to be very careful not accuse honest third parties of theft, when they hold semantically equal training data for their encoders than a victim encoder. Furthermore, we observe that the data pre-processing and tokenization in NLP plays a significant role for DI. While DI is able to mark victim encoders if their input data is batched and tokenized in exactly the same way, it is unable to do so when the data is tokenized differently (e.g., with different padding, truncation, batching). This observation is intuitive since DI relies on an exact training signature. In ownership resolution, this, furthermore, does not represent a limitation since the owner of the victim knows the exact pre-processing method used during their training / fine-tuning. We also experimented with DI using only sentences above or below a certain length for ownership resolution at inference time. This implements the intuition that some very short or long sentences might represent more unique training signature, however, neither DI nor DSI were able to successfully mark stolen copies. Finally, we observed that in nli (in contrast to qqp and flickr), DI was not even always able to mark the victim model. Results varied broadly, depending on the data sampled from the training and test set to fit the GMMs and estimate likelihood. We assume that this is due to nli containing data from many different domains (the underlying MNLI contains text from, among others, fiction, telephone speech, and letters). Subsampling might, hence, lead to having data from completely different distributions in training and test data and, thereby, prevent DI.
C.2 NLP: KNOWN ARCHITECTURE BUT UNKNOWN CHECKPOINT
We observe that when stealing is started from a different checkpoint for the underlying encoder than the one used by the victim, the performance changes depending on the difference between the pre- pre-trained checkpoints (first rows in Table TO). For example, while the difference between using bert-base-uncased and bert-base-cased the starting point for stealing is relatively small when the
18

---

Under review as a conference paper at ICLR 2023
Table 9: Number of Stealing Queries and Impact on Model Performance. We follow Sim- CSE (Gao et al. 2021) and use the SentEval banchmark. fo denotes the victim encoder trained on data De. J. the stolen encoder extracted using queries from a given stealing dataset D.. For stealing, we use a different number of queries to the victim encoder and fine-tune our stolen copy with the resulting outputs for 20 epochs. We use the BERT-based model pre-trained and taken from Hugging Face obtained by the SimCSE code-base, marked with an asterisks * (ht : /t///ttttuu com/ wwwppciinnnoooooooo on nipannnpp/aa SSMCSE)) and steal using the nli dataset.
EN
De. D2
A Queries
CR
MPQA
MR
MRPC
SST2
SUBJ
TREC
ATESTS
Avg/Tsa
SICKR
STSB
61k
89.05
88.05
80..4
74.98
86.35
93.39
66.73
81.45
82.85
79.84
K3,077
50%
85.04
88.68
78.96
75.74
86.01
92.76
64.49
80.98
82.1
78.96
82.99
4fk
82.36
fa
B"
ali
20K
30k
87.18
86.41
89.16
80.13
78.26
78.45
75.44
76.4
84.4
84.4
92.98
71.37
74.39
79:09
82.60
81.93
71.93
76.9
76.84
81.29
10k
83.81
87.52
78.08
74. 30
84,06
93.28
73.27
60.03
82,06
53,86
66.2
80.15
83.09
77.42
70.68
84.75
93.0
68.12
79.67
53.14
37:75
Table 10: Base Checkpoints and Influence on Model Performance. We follow SimCSE (Gao
et al. 2021) and the SentEval banchmark. Su denotes the victim encoder trained data D,. J. is the stolen encoder extracted using queries from a given stealing dataset D,. For stealing, we 60,000 stealing queries the victim encoder and fine-tune our stolen copy with the resulting outputs for 20 epochs. We use the BERT-based model pre-trained and taken from Hugging Face obtained by the SimCSE code-base, marked with asterisks * (ht=ps: / gttptbtttuuuuuutg/.g.oo//gg.b.ht com cww/proioiinocececccc wwom/prinneetonnnn m...p SimCSE) and steal using the nli dataset. While the original BERT was initialized with best-base- uncased, we vary the weight initialization of the checkpoint used for stealing. We also check performance after 1 epoch of fine-tuning (1 epoch). We use two different tokenizers. The API uses their origin original tokenizer (bert-base-uncarr potentially unknown to attacker), while the attacker uses the tokenizer that corresponds to the checkpoint from which they load the weights.
Stealing Initialicationn
MPQA
MR
MRPC
SUBI
Avg-sTs
Aug-Tra Trax
SICKR
STSB
heri-hrsee-un -ancased (luseline spacks)
89J05
88.95
80.49
74,98
N6.35
03.39
66.73
81.45
82.85
79.84
83.07
hart-hase-cased (20 opochs)
77.94
76.35
92.04
66.45
77.88
82.08
75.84
79.91
bert-base- muhilingual cased (20 spochs)
70.36
09:73
60009
09.72
64.91
79.14
49.5
58.62
66.20
57.95
59.31
ber: base ancased (1 spoch)
75.06
$1.68
73.12
71.37
83,94
91.95
62.51
48.67
77.09
52.9
44.85
fx
D
nli
nli
bert base cased (1 spoch)
69.28
$1.9
61.96
70.07
R2.34
90.18
52.2
49,77
72.99
51.42
48.13
bert best-base mutt.lingual cased (1 spoch)
09.18
54.11
58.94
30.78
43.08
44.75
41.41
original model used pert-base-unassed the performance difference when using bert- bettbbae--muliiingual case is larger.
Next, we show that the underlying encoder leaves distinct traces the fine-tuned sentence embedding model. We analyze the performance of different checkpoints only after a single epoch of fine- tuning during which we use the stolen embeddings (last rows in Table [O). This computationally inexpensive step allows attackers to quickly identify the persfreeforming checkpoint that they have access to and continue the tne-tuning only for the best checkpoint. Note that evaluating different model checkpoint initializations does not require obtaining additional representations from the victim model. Instead, the obtained representations can be reused over all checkpoints.
C.3 DATASET INFERENCE VS DISTRIBUTION INFERENCE IN VISION
Table II show that DI is a robust detection method. We train two ResNet models on the equal partitions of the CIFARIO train set. The result suggests that we are able to differentiate between two such encoders as being trained independently. We also run the same experiments for the original version of dataset models, where we train two classifiers on the equal splits of the CIFARIO train set. We observe that both methods are robust in this setting. To apply GMMs to supervised models, we discard the last classification layer and use the representations from the last but one layer to distinguish between these two classifiers using GMMs. This shows that the dataset inference is not only a distribution inference but the intended dataset inference. We can use the method based on GMMs when the public API exposes access to the softmax values for the supervised models.
19

---

Under review as a conference paper at ICLR 2023
Table 11: Dataset Inference. We divide the CIFARIO train into two equal splits. Then, we train on each split. We test if such two encoders are marked as independent. Each value is an average of 3 trials. Sp is the effect size from the statistical t-test. DI denotes the original Dataset Inference proposed by Maini et al. (202I). For Blind Walk (rand) the results fluctuate due to the randomness coming from the selected data points and the embedding generation method.
DATASET FOR
DATASKT FOR TRAINING MODELS
DATASRT INFERENCE
1ST HALF
CIFARTD
HALF HALF CIFAR
P-VALUB
Al
P-VALUB
Ap
1ST HALF OF CIFARIO
7.40-112
683
0.56
0.57
GMMS ON ENCODERS
2ND BALF CIFARIO
182
2.15E-7
1ST HALF OF CIFARTO
9.610-25
1.99
0.63
0.01
(MAINI
2021
2ND HALF OF CIFARIO
0.98
-014
3.68B-23
3,99
DI MAINI BT AL.
2021 RAND 100 POINTS
1ST
CIFARIO
2.926-149
1.99
0,43
0.31
2ND HALF OF CIFARIO
0.30
0.32
2. 2:246-203
199
111 MAINI ET AL.
2021
RAND 1000 POINTS
2ND HALF CIFARIO
1ST HALF OF CIFARIO
2554
0.44
0.21
1.57
7,32F 203
0.53
0.14
188
1ST HALF OF CIFARIO 2
2.35E-12
1.92
0.999
-182
DI MAINI AL.
2021 MINGD 10 POINTS
2ND HALF CIFARIO
8.14F.8
1ST HALF OF CIFARIO
1666-44
1884
0.048
0.20
MAINI BT AL.
2021 MINGD POINTS
2ND HALF OF CIFARIO
0.34
0.27
4.82n-50
1884
D1 (MAINI FT Al
2021 MINGD 1000 POINTS
1ST HALF CIFARIO
1699.100 104
1.98
0.999
2ND HALF CIFARIO
0.74
0.12
3.21E-
99
1ST HALF OF CIFARIO
3.178.63 63
16.86
0.43
0.79
GMMS ON SUPERVISED SOFTMAX
2ND HALF CIFARIO
0.64
0.67
7.956224
10.08
IST HALF OF CIFARTO
1969-178
28.85
0.44
0.77
GMMS SUPERVISGD-LOGITS
2ND HALF OF CIFARIO
0.18
0.85
1.02g- 105
22.01
GMMS ON
1ST
CIFARIO
4.476-189
29.75
0.69
0.48
2ND HALF OF CIFARIO
0.33
-0.96
1.945-133
24.83
Table 12: Effects of Projection Heads on DI. We perform dataset inference for DINO trained on CIFARIO with different projection heads. DINO uses 3-layer MLP followed by & normalization and a linear layer as the projection head during training, which is then discarded for evaluations. The first column denotes the number of layers for MI.P in the projection head. The second column denotes whether the projection head is included in DI. For each set-up, we also vary the number of components for GMM. The bolded number indicates that DI successfully detects the victim.
NUMBER OMM COMPONENTS
MLP LAYERS
INCLUDED
5
30
40
50
0
No
4.72k-2
6.09-4
3.67E-5
7.82x-8
9.23g-11
1.73g-15
5.43g-20
5,630-2
3.31E-4
7.43E-6
6.34g-7
8.92E-8
7.328-8
5.608-9
2
No
0.71
0.53
0.38
0.11
8,57F.3
2.34g-4
1.21R-4
No
0.52
0.53
0.65
0.68
0.14
0.25
0.36
ONLY-1-LAVER MLP
0.42
0.57
0.28
0.33
0.64
0.13
0.26
3
ONLY-2-LAYER MLP
0.73
0.82
0.57
0.39
0.26
3,91E-2
4.628-2
ONLY LAYER MLP
0.83
0.12
1.126-2
1.918-6
2.43g-12
4.568-16
8.34F-21
YES (ALL)
5.92F-6
3.758-3
5.63F-14
5.01E-9
1.496-11
1.288-22
1.67--66
C.4 INFLUENCE OF DIFFERENT PROJECTION HEAD SETTINGS
We investigate the effect of the depth and the final-layer dimension of DINO's projection head on the model's performance on downstream tasks. Table shows that having a deeper MLP is crucial to the model's quality.
C.5 MORE EXPERIMENTS ON STEALING VIT SMALL
Table 14 shows the performance of stolen models using different datasets, query numbers and loss functions. The result suggests that using the same dataset as the pretrained dataset gives the best result.
C.6 NUMBER OF PARAMETERS
We count the number of parameters for the base encoders and their corresponding heads and show the results in Table [3] The ViT-tiny has much fewer parameters than even the single projection head. VmT-small has smaller number of parameters than the full projection head used in DINO. Only the
20

---

Under review as a conference paper at ICLR 2023
Table 13: Influence of Projection Heads. We train Vit Tiny from scratch using DINO on CI- FAR10 with different configurations of projection head with respect to number of layers and output dimension(denoted as od), and evaluate them on different downstream tasks: CIFAR1O, SVHN.
Model
Pretrained Dataset
Method
CIFARI0
SVHN
ViT-tiny
CIFARIO
DINO(Head 3+1)
88.3
67.2
ViT-tiny
CIFARI0
DINO(Head 1+1)
86.7
58.2
ViT-tiny
CIFARIO
DINO(Head 1+1, od192)
85.1
59.2
ViT-tiny
CIFARIO
DINO(Head 1+1, od576)
83.9
58.8
ViT-tiny
CIFARIO
DINO(Head 1+1, od3840)
84.4
58.6
ViT-tiny
CIFARIO
DINO(Head 0+1)
86.7
58.8
ViT-tiny
CIFARIO
DINO(Head 0+1, od192)
86.2
58.5
ViT-tiny
CIFARIO
DINO(Head 0+1, od4096)
84.6
58.5
Table 14: Stealing VIT Small. The victim encoder is VIT Small pretrained on ImageNet with the DINO training approach. We assess the encoder on CIFARIO.
Stealing Dataset
Loss
# of Queries
CIFARIO
Victim Model
N/A
N/A
91.91
CIFARIO
MSE
50000
88.75
CIFARIO
InfoNCE
50000
87.15
CIFAR100
MSE
50000
85.65
SVHN
MSE
100000
62.04
ImageNet
InforNCE
100000
84.91
ImageNet
MSE
100000
71.87
ImageNet
InfoNCE
200000
87.65
ImageNet
InfoNCE
250000
88,86
ImageNet
MSE
250000
80.30
ImageNet
SoftNN
250000
79.69
ViT-base has larger number of parameters than the full projection head from DINO. The number of parameters in the projection heads is different for various encoder types due to their different dimensionality of output representations.
C.7 FINE-TUNING
we present the results after fine-tuning with the SVHN dataset on ViT Small/16 pre-trained on ImageNet.
In Table Table 16
C.8 MIXHATCH FOR SVHN
We present the results applying the MixMatch semi-supervised learning to the SVHN dataset Table
21

---

Under review as a conference paper at ICLR 2023
Table 15: Number of Parameters in base encoders (backbones) and projection heads.
Encoder Type
Base Encoder
Head-1
Head-1-2
Head-1-2-3
Tiny ViT
5.524.416
16.826.624
17.697.024
21.893.376
Small VIT
21.665.66
16.875.776
18.090.240
22.286.592
Base VIT
85.798.656
16.974.080
18.876.672
23.073.024
Table 16: Dataset Inference for Fine-tuning on Vision Transformers. We fine-tune using the SVHN dataset on ViT Small/16 pre-trained on ImageNet. Each value an average over 3 trials. Aμ is the effect size from the statistical t-test. An encoder is marked as stolen if the p-value is smaller than threshold of 0.05.
ImageNNet
SVHN
p-value
1.28c
11.54
0.43
0.27
10
1.54c-13
1:29
0.08
1.38
15
1.11e 12
7,02
0.17
0.95
20
1.57e
6.29
0.12
1.16
2.6e-8
30 7.96c
3.65
0.25
0.65
35
1816-8
3.51
0.07
1.44
7.266/6
50 6.46-77
4.84
0.08
1.40
8
3
3.41
2.9e-3
2775
2
5.04
1.36e
7.666-8
1.5663
90
1.46-7
5.12
9.00:3
233
100
1076-7
5.19
7.77c-4
3.16
Table 17: MixMatch for Stealing from VIT Tiny for SVHN. We follow the same notation as in Table
Number queries
Stealing
Vietim
Stealing
4K
SK
10K
CIFARIO
CIFARID
C(:fu)
C(-, fv)
MSE
45.74
$9.27
0.89
17.82
68.42
0.91
19.61
78.17
0.95
SomNN
SVHN
SVHN
44.6
29.92
(1778
46.9
60.58
0.2
52.68
73.38
0.90
InfoNCE
51.47
52.39
76.18
22


Reviews:
[{"review_id": "IjeVkYpF5KY", "reviewer": "Reviewer_zM28", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents results on stealing a transformer based encoder model (past work has mostly focused on stealing CNN based models). The authors show that both vision and language encoders can easily be stolen with 40x fewer train queries than used to train the model. The authors also present DataSeed Inference (DSI), which is a method based on Dataset Inference, to protect vision transformers against stealing. For DSI, private data is added to the dataset and is trained with augmentations and the probability of private training data and test data is compared for stolen model and independently trained models (for stolen models, the probability of the private training data will be much larger). ", "summary_of_the_review": "This paper claims to be the first work on stealing transformer based encoders. The empirical evaluation is strong and results with different models/datasets are presented for the stated setting. There is not much novelty in the method; a simple and straightforward method is used to steal the transformer encoders. Although the method is not complicated or novel, this work could as a baseline for future work in this area. ", "strength_and_weaknesses": "The authors present many experimental results and back up their claim that vision and language encoders can be easily stolen. The experimental results are obtained on multiple datasets and models. Additionally, the authors make use of different training sets to steal the encoder and show that the closer the training set is to the training set used to train the victim encoder, the easier it is to steal the victim encoder. The authors also show that DSI can be used to identify stolen vision transformers. \n\nHere are a few additional questions for the authors. In the case of language encoders, if the architecture is known but the pre-trained checkpoint used to initialize the model is unknown, how does the stealing performance change? With vision encoders, is it important to know what training strategies (crops, data augmentations, etc.) are used in order to steal the encoder? If a lot of the training data is private or pre-processed in an undisclosed manner, can the encoder trained with that data still be stolen? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GDYT9Y40w8A", "reviewer": "Reviewer_rUX6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the model stealing attacks on Transformer-based Encoders and shows that model stealing attacks are very effective for both vision and NLP transformer models In addition, the authors also propose DataSeed Inference, which combines dataset inference and watermarking to defend against model stealing attacks.", "summary_of_the_review": "The paper is well motivated, clearly written, and technically sound. However, due to the limited novelty discussed above, I would rate the paper slightly below the acceptance threshold.", "strength_and_weaknesses": "Strength: This paper is well motivated as it investigates model stealing attacks on state-of-the-art Transformer-based encoders for vision and language tasks, while most of the previous art has focused on attacking CNNs. The empirical evaluation of this work is also very solid in that it demonstrates the effectiveness of applying existing model stealing attacks on Transformer-based encoders. Furthermore, the proposed DataSeed Inference addresses the problem that Dataset Inference cannot be directly applied to transformer-based encoders, which are likely to be pre-trained from a mixture of private and public data.\n\nWeaknesses: The main weakness of this paper is that the model stealing attack techniques are adopted from previous work. For example, the authors make the observation that Transformer-based encoders are pre-trained with both private and public data. However, this paper does not study the attacks under this threat model. Moreover, Transformer-based encoders are often pre-trained using very large dataset. A recent paper [1] points out that the existing large language models requires even more data to achieve compute-optimal model. As a result, requiring a 40x fewer queries than the victim data points may still be prohibitively expensive. Is it possible to further reduce the number of queries? \n\n[1] Training Compute-Optimal Large Language Models, arxiv'22", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DsWUYwzprc", "reviewer": "Reviewer_e9r9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper evaluates a set of known model-theft attacks to vision and NLP transformers. For vision, adaptation of MixMatch [Berthelot et al., 2019] to semi-supervised scenarios is the key novelty. Then the authors adapt the Dataset Inference (DI) [Dziedzic et. al. 2022b] technique for transformer models that get trained on publicly available data. As opposed to having a fully private dataset as in DI, the authors add a private counter-part data source to the publicly available training data; the expectation is that behavior on this data acts as a watermark to detect model theft. The authors show that this idea works for transformers in the vision domain and fails for the transformers in the NLP domain. The success and failure analysis are based on intuitions that are not well tested.\n", "summary_of_the_review": "The paper lacks a strong motivation, a clearly defined threat-model, analysis of proposed methods and has limited novelty.", "strength_and_weaknesses": "### Things I liked\n\n- The paper is well-written making it easy to understand.\n\n- The authors do run a set of experiments that shows that some of the findings hold for NLP model variants like BERT and RoBERTa.\n\n### Things that need clarification / improvement\n\n- There are recent works on stealing transformer-based models [1, 2]. The authors should compare their methods to these works. Given the paper is purely empirical, relevant baselines cannot be ignored.\n\n- Novelty is limited to applying known methods (or its simple adaptations) to a particular settings. On the attack side, they use known model theft attacks (Sha et al., 2022; Dziedzic et al., 2022a) in the context of transformer encoders. For vision scenarios, they adapt MixMatch (Berthelot et al., 2019), which has been shown to work in supervised settings to a semi-supervised setting. On the defense side, they adapt the dataset inference idea (Dziedzic et. al. 2022b) and add watermarking (Uchida et al., 2017; Jia et al., 2021; Adi et al., 2018) to it. It turns out to be effective for the vision domain and doesn't work for the NLP domain. Reading Section 3 & 4 confirms this belief; the authors don't even need to describe the methods (or the modifications they make to it) in detail.\n\n- While the modification of this know defense is shown to work for vision domains, analysis on its security is missing. For example, when clubbed with public data, what are the chances that a classifier classifies as expected on the private data given it is really well trained with the public data? How should one measure the distribution of the public and the private data to ensure that classifiers trained on public data doesn't falsely give the incorrect signal on the private data (and ensure there is no overlap between them given the public training data comes form a huge web corpus)? Can an adversary obtain leak in information about private data via interaction when trying to steal the model? (Given many of these questions don't have promising answers for NLP, which is sort-of obvious from the first-order experimental analysis, further study is necessary for the vision domain to call DSI a defense.)\n\n- The motivation isn't strong and the security threat-model is unclear. For models tried in the paper, all are made available publicly (as per the current trend in vision and NLP research). Hence, showcasing theft in these models is not a strong argument. Second, in the paper the authors know the architecture of the model they are stealing. What if a new architecture comes down the road and there is no information in the public domain about them. How do you know these are transformer-based? In turn, how would you design the architecture of the thief and ensure you can steal it properly?\n\n- The authors make a bunch of statements that need further (experimental or theoretical) support. A few of them are:\n  - \"When stealing with more complex datasets (e.g., CIFAR10 vs SVHN) we can obtain a better generalization of the stolen copy, which is shown by higher accuracy on more downstream tasks. However, we are unable to steal the exact representations from vision transformers. For this, additional knowledge about the victim’s training data or access to a pre-trained encoders are needed. This involves more compute as well.\"\n    - Several assumptions made here. Exact representations can't be stolen without training data or access to pre-trained encoders. Why is more compute needed to obtain victim's training data or pre-trained encoders? Given more compute, how would you steal these?\n  - \"In this work, we rely on their supervised approach leveraging pairs of sentences from natural language inference (NLI) datasets within a contrastive learning framework. It uses the entailment pairs as positives and contradictions as hard negatives.\"\n    - Why? Why is normal BERT/RoBERTa not considered?\n  - [Appendix] \"Our experiments with DI, DSI and cosine similarity for language transformers, yield some additional interesting insights that need to be taken into account when designing future defenses against stealing NLP encoders.\"\n    - I encourage the authors to use these insights to design a defense for NLP transformers. That would definitely make the submission stronger. Some of the statements such as \"We observe that if the training and test sets of an encoder consist of semantically equal or similar sentences, DI is not even enable to mark a victim encoder. This is caused since the encoders are trained to output similar/same embeddings for semantically equal sentences\" are more relevant for the SimCSE bert models maybe? Do these hold for any NLP transformer?  \n\n[1] Lyu, Lingjuan, Xuanli He, Fangzhao Wu, and Lichao Sun. \"Killing two birds with one stone: Stealing model and inferring attribute from bert-based apis.\" arXiv preprint arXiv:2105.10909 (2021).\n\n[2] Rafi, Mujahid Al, Yuan Feng, and Hyeran Jeon. \"Revealing Secrets From Pre-trained Models.\" arXiv preprint arXiv:2207.09539 (2022).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9Xo2aYZyXsH", "reviewer": "Reviewer_MeVg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method to steal and defend transformer-based SSL encoders in both language and vision domains. The stealing can be completed using the returned representations with 40x fewer queries for the languages-related tasks. And the number of queries can be decreased further for vision encoders by utilizing the semi-supervised learning. And the authors also design the corresponding defense technique, creating a unique encoder signature based on a private data subset.", "summary_of_the_review": "This paper's experimental results are sufficient but there are also some results need to be explained. And the authors should demonstrate clearly the novelty of their proposed strategies.", "strength_and_weaknesses": "Strength:\nThe proposed method can successfully steal NLP and vision transformer-based encoder in a real-world API setting. And the proposed DSI can alleviate the problem of stealing transformer-based encoders.\n\n\nWeakness:\n1. For the vision tasks, the authors only proved the success of stealing on the classification models. In the real-world API, the models can be designed for other tasks, like segmentation and detection. In the proposed method is able to steal the models of other tasks?\n\n2. What if the attackers have known the private seed, or some of the private seed? \n\n3. In Table 1, when $D_v$ is SVHN, why the best performance of CIFAR10 is obtained when $D_s$ is CIFAR100?\n\n4. In the Table 3, the best performance should be bold, similar to the results of Table 1.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject