Original: S TEALING AND D EFENDING T RANSFORMER - BASED E NCODERS
Revised: S TEALING AND D EFENDING T RANSFORMER - BASED E NCODERS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Self-supervised learning (SSL) has become the predominant approach to training on large amounts of unlabeled data.
Revised: Self-supervised learning (SSL) has become the predominant approach to training on large amounts of unlabeled data.

Original: New real-world APIs offer services to generate high-dimensional representations for given inputs based on SSL encoders with transformer architectures.
Revised: New real-world APIs offer services to generate high-dimensional representations for given inputs based on SSL encoders with transformer architectures.

Original: Recent efforts highlight that it is possible to steal highquality SSL encoders trained on convolutional neural networks.
Revised: Recent efforts highlight that it is possible to steal highquality SSL encoders trained on convolutional neural networks.

Original: In this work, we are the first to extend this line of work to stealing and defending transformer-based encoders in both language and vision domains.
Revised: In this work, we are the first to extend this line of work to stealing and defending transformer-based encoders in both language and vision domains.

Original: We show that it is possible to steal transformer-based sentence embedding models solely using their returned representations and with 40x fewer queries than the number of  victim’s training data points.
Revised: We show that it is possible to steal transformer-based sentence embedding encoders solely using their returned representations and with 40x fewer queries than the number of the victim’s training data points.

Original: We also decrease the number of required stealing queries for  the vision encoders by leveraging semi-supervised learning.
Revised: We further decrease the number of required stealing queries for language encoders by reusing extracted representations on semantically similar sentences and for the vision encoders by leveraging semi-supervised learning.

Original: Finally, to defendvision transformers against stealing attacks, we propose a defense technique that combines watermarking with dataset inference.
Revised: Finally, to defend transformers against stealing attacks, we develop a new scheme to watermark the language encoders, where for the last training iterations, we alternate between optimizing for the standard sentence embedding and a chosen downstream task. For the vision domain, we design a defense technique that combines watermarking with dataset inference.

Original: Our method creates a unique encoder signature based on a private data subset that acts as a secret seed during training.
Revised: Our method creates a unique encoder signature based on a private data subset that acts as a secret seed during training.

Original: By applying dataset inference on the seed, we can then successfully identify stolen transformers.
Revised: By applying dataset inference on the seed, we can then successfully identify stolen transformers.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The success of self-supervised learning (SSL) motivates the emergence of large-scale services offering API access to encoders which return high-dimensional representations for given input data.
Revised: The success of self-supervised learning (SSL) motivates the emergence of large-scale services offering API access to encoders which return high-dimensional representations for given inputs.

Original: These representations serve to train a diverse set of downstream tasks with a small amount of labeled data.
Revised: These representations serve to train a diverse set of downstream tasks with a small amount of labeled data.

Original: Latest APIs (Clarifai, 2022; Cohere, 2022; OpenAI, 2022) use transformer-based encoders (Devlin et al., 2018; Dosovitskiy et al., 2020) to generate representations.
Revised: Latest APIs (Clarifai, 2022; Cohere, 2022; OpenAI, 2022) use transformer-based encoders (Devlin et al., 2018; Dosovitskiy et al., 2020) to generate representations.

Original: Such encoders have a high number of parameters ( e.g., the state-of-the-art RoBERTa-Large language encoder (Liu et al., 2019) has roughly 355M parameters) and are trained on datasets consisting of millions of data points—yielding a highly expensive training procedure (Sharir et al., 2020).
Revised: Such encoders have a high number of parameters ( e.g., the state-of-the-art RoBERTa-Large language encoder (Liu et al., 2019) has roughly 355M parameters) and are trained on datasets consisting of millions of data points—yielding a highly expensive training procedure (Sharir et al., 2020).

Original: Therefore, these encoders are lucrative targets for stealing attacks (Tramèr et al., 2016). In such attacks, an adversary extracts a victim encoder by submitting queries and using the outputs to train a local stolen copy, often at a fraction of the victim’s training cost (Sha et al., 2022; Dziedzic et al., 2022a).
Revised: Therefore, these encoders are lucrative targets for stealing attacks (Tramèr et al., 2016) where an adversary extracts a victim encoder by submitting queries and using the outputs to train a local stolen copy, often at a fraction of the victim’s training cost (Sha et al., 2022; Dziedzic et al., 2022a).

Original: The stolen encoder can then be used for inferences without the owner’s permission, violating their intellectual property right and causing financial loss.
Revised: The stolen encoder can then be used for inferences without the owner’s permission, violating their intellectual property right and causing financial loss.

Original: While stealing and defending supervised models has been heavily studied (Tramèr et al., 2016; Juuti et al., 2019; Orekondy et al., 2020), research on the topic of stealing and defending transformer-based encoders is limited.
Revised: While stealing and defending supervised models has been heavily studied (Tramèr et al., 2016; Juuti et al., 2019; Orekondy et al., 2020), research on the topic of stealing and defending transformer-based encoders is limited.

Original: Despite the immediate practical importance of this problem, to the best of our knowledge, all previous works on model stealing attacks and defenses against SSL encoders are conducted offline in contrived experimental settings (Cong et al., 2022; Sha et al., 2022; Dziedzic et al., 2022a;b), focusing on the vision domain with convolutional neural network (CNN)-based architectures, and do not attack the popular transformer (Vaswani et al., 2017) architecture, which currently holds state-of-the-art results on many vision (Caron et al., 2021) and natural language processing (NLP) tasks (Gao et al., 2021).
Revised: Despite the immediate practical importance of this problem, to the best of our knowledge, all previous works on model stealing attacks and defenses against SSL encoders are conducted offline in contrived experimental settings (Cong et al., 2022; Sha et al., 2022; Dziedzic et al., 2022a;b), focusing on the vision domain with convolutional neural network (CNN)-based architectures, and do not attack the popular transformer (Vaswani et al., 2017) architecture, which currently holds state-of-the-art results on many vision (Caron et al., 2021) and natural language processing (NLP) tasks (Gao et al., 2021).

Original: Since transformers are trained differently than CNNs and represent larger and more complex architectures, stealing them is more challenging. 
Revised: Since transformers are trained differently than CNNs and represent larger and more complex architectures, stealing them is more challenging. Independent

Revised: Public Training Data 𝑃 𝐷 2 >𝑃(𝐷 𝑇𝑒𝑠𝑡 ) 𝑃 𝐷 2 ≤ 𝑃(𝐷 𝑇𝑒𝑠𝑡 )

Revised: GMM

Revised: Stolen

Revised: Private Seed

Original: We show how stealing attacks (Sha et al., 2022; Dziedzic et al., 2022a) can be successfully applied to extract transformer encoders through their returned representations.
Revised: D𝐷𝐷 𝑇𝑒𝑠𝑡 𝐷 1 Encoder (a) [Figure] Inference ( [Figure]

Revised: ) for Vision Encoders.

Revised: 𝑶 ?

Revised: ≈

Revised: 𝑶 𝒗 𝑶 ?

Original: In NLP, we are able to successfully steal sentence encoders using only a small number of queries; up to 40x fewer stealing queries than the number of the original training data points.
Revised: ≉𝑶 𝒗 Main DataDownstream Data Watermarking Verification ?

Original: For vision encoders, we decrease the number of
Revised: 𝑶 𝒗 𝑶 ? Stolen Independent Encoder

Original: [Figure]
Revised: FC-Layer (b) Watermark for Language Encoders. We show how stealing attacks (Sha et al., 2022; Dziedzic et al., 2022a) can be successfully applied to extract transformer encoders through their returned representations. In NLP, we are able to successfully steal sentence encoders using only a small number of queries; up to 40x fewer stealing queries than the number of the original training data points. We also show that this number can be further reduced by re-using the representations obtained from the victim encoder for semantically similar sentences of the stealing queries.

Original:  queries against victim encoders by adapting semi-supervision based on MixMatch (Berthelot et al., 2019), which has, so far, only been applied to attacks in the supervised setting (Jagielski et al., 2020).
Revised: For vision encoders, we decrease the number of queries against victim encoders by adapting semi-supervision based on MixMatch (Berthelot et al , 2019), which has, so far, only been applied to attacks in the supervised setting (Jagielski et al., 2020).

Original: The successful applicability of encoder stealing to transformer-based architectures in public API settings motivates the urgent need for defenses.
Revised: The successful applicability of encoder stealing to transformer-based architectures in public API settings motivates the urgent need for defenses.

Original: Yet, we find that existing defenses for encoders, such as dataset inference (DI) (Dziedzic et al., 2022b), are not directly applicable to real-world transformers since DI requires the whole training dataset for protected encoders to be private data . DI then operates on the assumption that an encoder is uniquely characterized by its whole training data and exploits that the unique data signature of the victim encoder is transferred to stolen copies. However, transformers behind APIs are trained on a large amount of public data —possibly scrapped from the Internet (Radford et al., 2021), and, thereby not even necessarily unique to a single encoder. To leverage the idea of DI to transformer-based vision encoders, we combine DI with watermarking (Uchida et al., 2017;
Revised: We first propose DataSeed Inference (DSI) , a combination between watermarking (Uchida et al., 2017;

Original: Jia et al., 2021; Adi et al., 2018).
Revised: Jia et al., 2021; Adi et al., 2018) and Dataset

Revised: Inference (DI) (Dziedzic et al., 2022b) as a successful defense for vision encoders.

Original: Our method adds a private data subset as a unique seed to the original training data. The resulting transformer is then uniquely characterized by its private seed, which can be used to identify stolen copies.We call our method
Revised: While standard DI operates on the assumption that an encoder is uniquely characterized by its whole training data and exploits that the unique data signature of the victim encoder is transferred to stolen copies, DSI adds a private data subset as a unique seed to the original training data and uses this seed to identify stolen copies.

Original: DataSeed Inference (DSI) and visualize the concept in Figure 1. To provide the defended encoder with a stronger signal for DSI, our key insight is  to train differently on the private seed than on  the original training data. Specifically, we apply weaker augmentations to the private data points.
Revised: This is a necessary adaption of DI to transformer-based encoders since these are trained on a large amount of public data —possibly scrapped from the Internet (Radford et al., 2021), and, thereby not necessarily unique to a single encoder.

Original: We note that the signal from the private seed is transferred during stealing.
Revised: We note that the signal from the private seed is transferred during stealing.

Original: If an adversary is aware of DSI, then knowledge about the private seed is required to avoid extracting features contained in the seed.

Original: Additionally, our defense does not harm the performance of the defended encoder, which distinguishes it from watermarks in supervised settings.
Revised: Additionally, our defense does not harm the performance of the defended encoder on downstream tasks, which distinguishes it from watermarks in supervised settings.

Original: The intuition to this observation is that while the space of optimal solutions in supervised learning is more limited, in the high-dimensional space of the representations, there exist many more multiple equally performant optima.
Revised: Since we find that DSI is not successful in defending language encoders, we propose a new watermarking scheme to protect them from theft.

Original: Hence, seeding the training of  thedefended encoder with private data can change representations without influencing the performance of downstream tasks. However, for language transformers, we find that such training-signature-based defenses are not applicable.
Revised: Our watermark relies on alternating between the actual sentence embedding task and a secretly chosen downstream task during the last iterations of training. This transforms the representations so that they preserve their high performance on sentence embedding tasks while increasing their accuracy on the downstream task.

Original: Next, we hypothesize why. This is because, first, the sentence embedding encoders do not augment their inputs.
Revised: To embed the watermark task, we append a fully-connected layer to the encoder. The additional layer acts as our secret key.

Original: Second, their underlying transformers are usually initialized from the same or similar pre-trained transformers and fine-tuned only for a few epochs, resulting in a non-significant signal for DI and DSI. This leaves the high-value language encoders unprotected against practical stealing attacks.
Revised: We verify whether a given encoder is a stolen copy by attaching that layer and checking the agreement to the victim encoder’s output for the watermark downstream task. Victim and independent encoders have significantly different outputs whereas victim and stolen copies return similar outputs.

Original: To summarize, we make the following contributions:
Revised: To summarize, we make the following contributions:

Original: • We successfully steal NLP and vision transformer-based encoders in a real-world API setting, assuming access to representations only and using up to 40x fewer queries than the number of samples in the respective encoder’s training dataset.
Revised: • We successfully steal NLP and vision transformer-based encoders in a real-world API setting, assuming access to representations only and using up to 40x fewer queries than the number of samples in the respective encoder’s training dataset.

Original: Our stolen encoders achieve comparable performance to the respective victims on standard benchmarks.
Revised: Our stolen encoders achieve comparable performance to the respective victims on standard benchmarks.

Original: •  For the vision domain, we propose DSI as a defense to alleviate the problem of stealing transformerbased encoders. Our method adds a private dataset as a secret seed to the defender’s original training data and uses this seed for ownership resolution.
Revised: • We further reduce the number of stealing queries by using semantically similar sentences for language encoders and semi-supervised methods for vision encoders. • For the vision domain, we propose DSI as a defense which adds a private dataset as a secret seed to the defender’s original training data and uses this seed for ownership resolution.

Original: • For NLP sentence embedding transformers, we show that while these are easily extracted, the dataset inference is not effective and other defenses, like watermarking, are not applicable.
Revised: • For NLP sentence embedding transformers, we propose a method to watermark their representations by alternating between the actual sentence embedding and a secretly chosen downstream task during the last iterations of training.

Original:  B ACKGROUND AND R ELATED W ORK
Revised:  B ACKGROUND AND R ELATED W ORK

Original: Model extraction attacks.
Revised: Model extraction attacks.

Original: The goal of the model extraction attacks is to replicate a functionality of a victim model f v trained on a dataset D P .
Revised: The goal of the model extraction attacks is to replicate a functionality of a victim model f v trained on a dataset D P .

Original: An attacker has a black box access to the victim model and uses a stealing dataset D s = { q i , f v ( q i ) } ni =1 , consisting of queries q i and the corresponding predictions or representations returned by the victim model, to train a stolen model f s .
Revised: An attacker has a black box access to the victim model and uses a stealing dataset D s = { q i , f v ( q i ) } ni =1 , consisting of queries q i and the corresponding predictions or representations returned by the victim model, to train a stolen model f s .

Original: Model extraction attacks have been shown against various types of models including classification (Tramèr et al., 2016; Jagielski et al., 2020) and representation models (Sha et al., 2022; Dziedzic et al., 2022a).
Revised: Model extraction attacks have been shown against various types of models including classification (Tramèr et al., 2016; Jagielski et al., 2020) and representation models (Sha et al., 2022; Dziedzic et al., 2022a).

Original: Language Encoder.
Revised: Language Encoder.

Original: We use SimCSE (Gao et al., 2021) to learn sentence representations since it outperforms other methods and is exposed via public APIs (Clarifai, 2022).
Revised: We use SimCSE (Gao et al., 2021) to learn sentence representations since it outperforms other methods and is exposed via public APIs (Clarifai, 2022).

Original: The SimCSE framework by (Gao et al., 2021) proposes unsupervised and supervised approaches to generate sentence embeddings.
Revised: The SimCSE framework by (Gao et al., 2021) proposes unsupervised and supervised approaches to generate sentence embeddings.

Original: It starts from a pre-trained checkpoint of a BERT-based encoder, e.g., RoBERTa, and takes the representation for the classification token ([CLS]) as the sentence embedding.
Revised: It starts from a pre-trained checkpoint of a BERT-based encoder, e.g., RoBERTa, and takes the representation for the classification token ([CLS]) as the sentence embedding.

Original: In this work, we rely on their supervised approach leveraging pairs of sentences from natural language inference (NLI) datasets within a contrastive learning framework.
Revised: In this work, we rely on their supervised approach leveraging pairs of sentences from natural language inference (NLI) datasets within a contrastive learning framework.

Original: It uses the entailment pairs as positives and contradictions as hard negatives.
Revised: It uses the entailment pairs as positives and contradictions as hard negatives.

Original: Vision Encoder.
Revised: Vision Encoder.

Original: In our work, we use DINO (Caron et al., 2021) to train vision transformers (ViT) since it returns high-quality representations that achieve SoTA performance on downstream tasks when assessed by training a linear classifier directly on representations.
Revised: In our work, we use DINO (Caron et al., 2021) to train vision transformers (ViT) since it returns high-quality representations that achieve SoTA performance on downstream tasks when assessed by training a linear classifier directly on representations.

Original: DINO trains student and teacher encoders, both with the same architecture but different parameters, where the teacher is updated with an (exponential moving) average of the student.
Revised: DINO trains student and teacher encoders, both with the same architecture but different parameters, where the teacher is updated with an (exponential moving) average of the student.

Original: Different random transformations of the same image are generated and passed through both the student and teacher.
Revised: Different random transformations of the same image are generated and passed through both the student and teacher.

Original: The student is provided with smaller crops of the inputs than the teacher, which forces the student to generatethe representations that restore parts of the initial image.
Revised: The student is provided with smaller crops of the inputs than the teacher, which forces the student to generate representations that restore parts of the initial image.

Original: The training objective is to minimize the cross-entropy loss between representations from teacher and student.
Revised: The training objective is to minimize the cross-entropy loss between representations from teacher and student.

Original: Stealing Encoders.
Revised: Stealing Encoders.

Original: Thus far, methods for stealing encoders through representations have been shown in the computer vision domain and only for CNNs (Sha et al., 2022; Dziedzic et al., 2022a).
Revised: Thus far, methods for stealing encoders through representations have been shown in the computer vision domain and only for CNNs (Sha et al., 2022; Dziedzic et al., 2022a).

Original: Previous work in the NLP domain focuses on classification tasks and performs stealing against fine-tuned models through labels based on a given pre-trained language encoder (Krishna et al., 2020;
Revised: Previous work in the NLP domain focuses on classification tasks and performs stealing against fine-tuned models through labels based on a given pre-trained language encoder (Krishna et al., 2020;

Original: Zanella-Beguelin et al., 2021;
Revised: Zanella-Beguelin et al., 2021;

Original: He et al., 2021).
Revised: He et al., 2021).

Original: Model extraction against NLP APIs are shown by (Xu et al., 2021), specifically for sentiment classification and machine translation tasks.
Revised: Model extraction against NLP APIs are shown by (Xu et al., 2021), specifically for sentiment classification and machine translation tasks.

Original: The setup of previous work differs from ours which is concerned with stealing through representations instead of low-dimensional outputs, such as labels.
Revised: The setup of previous work differs from ours which is concerned with stealing through representations instead of low-dimensional outputs, such as labels.

Original: This is motivated by the fact that these representations are exactly what new public APIs expose (Cohere, 2022; Clarifai, 2022).
Revised: This is motivated by the fact that these representations are exactly what new public APIs expose (Cohere, 2022; Clarifai, 2022).

Original: Distillation methods used in the NLP domain (Jiao et al., 2019) which could, in principle, be applied to stealing encoders, usually requires white box access to the original model, for example, to the attention layers (Jiao et al., 2019).
Revised: Distillation methods used in the NLP domain (Jiao et al., 2019) which could, in principle, be applied to stealing encoders, usually require white box access to the original model, for example, to the attention layers (Jiao et al., 2019).

Original: Therefore, distillation cannot be applied for stealing in public API-access scenarios.
Revised: Therefore, distillation cannot be applied to stealing in public API-access scenarios.

Original: Defending Encoders.
Revised: Defending Encoders.

Original: Dataset inference (DI) (Maini et al., 2021) is a defense against model stealing attacks.
Revised: Dataset inference (DI) (Maini et al., 2021) is a defense against model stealing attacks.

Original: It enables a model owner or a third-party arbitrator to attribute the ownership of a potentially stolen model.

Original: Therefore, it uses the victim’s training dataset as a unique signature, leveraging the following observation: for a victim encoder trained on its private data as well as for its stolen copies, the distribution of the representations generated from the victim’s training data differs from the distribution of the representations generated on the test data.
Revised: It uses the victim’s training dataset as a unique signature, leveraging the following observation: for a victim encoder trained on its private data as well as for its stolen copies, the distribution of the representations generated from the victim’s training data differs from the distribution of the representations generated on the test data.

Original: In contrast, for an independently trained encoder, these two distributions cannot be distinguished, allowing the detection of stolen copies (Dziedzic et al., 2022b).
Revised: In contrast, for an independently trained encoder, these two distributions cannot be distinguished, allowing the detection of stolen copies (Dziedzic et al., 2022b).

Original: For modeling the distributions, Gaussian Mixture Models (GMMs) are trained on a fraction of the private training data and applied to a disjoint fraction of the training data and the test data.
Revised: For modeling the distributions, Gaussian Mixture Models (GMMs) are trained on a fraction of the private training data and applied to a disjoint fraction of the training data and the test data.

Original: An encoder is identified as a victim or stolen copy if the loglikelihood on private representations is significantly higher than on the test representations.
Revised: An encoder is identified as a victim or stolen copy if the log-likelihood on private representations is significantly higher than on the test representations.

Original: Recently, watermarking (Uchida et al., 2017; Jia et al., 2021; Adi et al., 2018) methods have been proposed for encoders (Dziedzic et al., 2022a; Cong et al., 2022; Wu et al., 2022).
Revised: Recently, watermarking (Uchida et al., 2017; Jia et al., 2021; Adi et al., 2018) methods have been proposed for encoders (Dziedzic et al., 2022a; Cong et al., 2022; Wu et al., 2022).

Original: The main difference betweenour DSI and previous work  lies inwatermark verification.
Revised: The main difference between previous work and our DSI lies in verification.

Original: Previous watermarking techniques use downstream tasks to detect a watermark while we resolve ownership via DI on the private data seed.
Revised: Previous watermarking techniques use downstream tasks to detect a watermark while we resolve ownership based on the representations directly.

Original: For a more detailed overview on watermarking for encoders and a more thorough overview of related work, see Appendix A.
Revised: For a more detailed overview on watermarking for encoders and a more thorough overview of related work, see Appendix A.

Original:  S TEALING T RANSFORMER -B ASED E NCODERS
Revised:  S TEALING T RANSFORMER -B ASED E NCODERS

Original: We aim at stealing BERT-based transformers, fine-tuned to return sentence embeddings in the language domain, and general embeddings for images in the vision domain.
Revised: We aim at stealing BERT-based transformers, fine-tuned to return sentence embeddings in the language domain, and general embeddings for images in the vision domain.

Original: Our stealing operates in a public API setting where the adversary can query the models through a pre-defined interface to obtain high-dimensional representations for their inputs.
Revised: Our stealing operates in a public API setting where the adversary can query the models through a pre-defined interface to obtain high-dimensional representations for their inputs.

Original: Stealing is then performed following previous work (Dziedzic et al., 2022a): (i)
Revised: Stealing is then performed following previous work (Dziedzic et al., 2022a): (i)

Original: The adversary sends N raw or augmented inputs to the victim encoder.
Revised: The adversary sends N raw or augmented inputs to the victim encoder.

Original: These inputs can, in principle, be taken from any data distribution of the target domain, using open-source data.
Revised: These inputs can, in principle, be taken from any data distribution of the target domain, using open-source data.

Original: (ii) With the obtained representations, the adversary trains a stolen copy of the victim.
Revised: (ii) With the obtained representations, the adversary trains a stolen copy of the victim.

Original: The goal of this training is to maximize the similarity of the stolen copy’s output and the representations output by the victim.
Revised: The goal of this training is to maximize the similarity of the stolen copy’s output and the representations output by the victim.

Original: Therefore, the adversary either imitates a self-supervised training using a contrastive loss function, e.g., InfoNCE Chen et al.
Revised: Therefore, the adversary either imitates a self-supervised training using a contrastive loss function, e.g., InfoNCE Chen et al.

Original: or SoftNN Frosst et al.
Revised: or SoftNN Frosst et al.

Original: (2019), or directly matches both models’ representations via the Mean Squared Error (MSE) loss.
Revised: (2019), or directly matches both models’ representations via the Mean Squared Error (MSE) loss.

Original: Stealing Transformer-Based Encoders.
Revised: Stealing Transformer-Based Encoders.

Original: For language, public APIs (Cohere, 2022) expose transformers which are first pre-trained on a large corpus of text data to return per-token representations and then fine-tuned to return high-dimensional embeddings for a given full-text input, e.g., a sentence.
Revised: For language, public APIs (Cohere, 2022) expose transformers which are first pre-trained on a large corpus of text data to return per-token representations and then fine-tuned to return high-dimensional embeddings for a given full-text input, e.g., a sentence.

Original: In the vision domain, APIs such as (Clarifai, 2022) expose encoders trained from scratch on large amounts of image data to return per-image representations.
Revised: In the vision domain, APIs such as (Clarifai, 2022) expose encoders trained from scratch on large amounts of image data to return per-image representations.

Original: We find that public APIs (Clarifai, 2022) provide metadata about exposed encoders, which can contain information about datasets used for pre-training as well as the encoder architecture.
Revised: We find that public APIs (Clarifai, 2022) provide metadata about exposed encoders, which can contain information about datasets used for pre-training as well as the encoder architecture.

Original: Thus, we can instantiate our stolen encoders with the victim encoder’s architecture.
Revised: Thus, we can instantiate our stolen encoders with the victim encoder’s architecture.

Original: We also follow the API setting and initialize the stolen copies of language encoders with publicly available pre-trained transformers.
Revised: We also follow the API setting and initialize the stolen copies of language encoders with publicly available pre-trained transformers.

Original: 1 The stolen copies of vision encoders are initialized with random weights.
Revised: 1 The stolen copies of vision encoders are initialized with random weights.

Original: Stealing with Semi-Supervision.
Revised: Stealing with Semi-Supervision (Vision).

Original: To reduce the number of stealing queries 2 , we apply semisupervised learning which has been used to improve stealing in the supervised setting (Jagielski et al., 2020).
Revised: To reduce the number of stealing queries 2 , we apply semi-supervised learning which has been used to improve stealing in the supervised setting (Jagielski et al., 2020).

Original: The approach relies on semi-supervision based on MixMatch (Berthelot et al., 2019) and helps to leverage a large pool of unlabeled data while having access to only a small fraction of labeled data (Sohn et al., 2020; Assran et al., 2021).
Revised: The approach relies on semi-supervision based on MixMatch (Berthelot et al., 2019) and helps to leverage a large pool of unlabeled data while having access to only a small fraction of labeled data (Sohn et al., 2020; Assran et al., 2021).

Original: The inspiration for applying MixMatch-based methods to stealing encoders comes from DINO (see Section 2), which normalizes representations with a temperature softmax and uses the cross-entropy loss to minimize distances between outputs from student and teacher encoders.
Revised: The inspiration for applying MixMatch-based methods to stealing encoders comes from DINO (see Section 2), which normalizes representations with a temperature softmax and uses the cross-entropy loss to minimize distances between outputs from student and teacher encoders.

Original: Similarly, during stealing, we pass the victim and stolen representations through the softmax layer whose outputs act as our new labels for MixMatch.
Revised: Similarly, during stealing, we pass the victim and stolen representations through the softmax layer whose outputs act as our new labels for MixMatch.

Original: Note that this method is not applicable to the fine-tuned sentence embedding encoders since these do not rely on data augmentations, which is necessary for MixMatch.
Revised: Note that this method is not applicable to the fine-tuned sentence embedding encoders since these do not rely on data augmentations, which is necessary for MixMatch.

Revised: Re-using Representations (NLP).

Original: In our experimental evaluation (Section 5.1), weshow, however, that language encoders do not require additional improvements to reduce the number of stealing queries  since they can already be stolen successfully with a small number of queries.
Revised: For stealing sentence encoders, we reduce the number of stealing queries by re-using representations over semantically similar sentences. This is possible since sentence encoders are required to return similar representations to such semantically similar sentences. Hence, when a stealing dataset holds such similar sentences ( e.g., in the form of positive pairs, such as all the datasets used in this work), we only have to query one of these sentences, and assign the same representation to all semantically similar sentences to augment our fine-tuning dataset for the stolen-copy. Our experimental evaluation in Section 5.1 shows the effectiveness of this approach.

Original:  D ATA S EED I NFERENCE AS A D EFENSE FOR T RANSFORMERS
Revised:  D EFENSES A GAINST S TEALING T RANSFORMERS

Original: Given the high training costs of transformer-based encoders, defending them from stealingattacks is an urgent need.
Revised: Given the high training costs of transformer-based encoders, defending them from stealing is an urgent need.

Original: Yet, the high dimensionality and complexityof their representations make defenses challenging.
Revised: Yet, their representations’ high dimensionality and complexity make defenses challenging.

Revised: is relevant in public API settings since costs usually increase linearly with the number of queries and since the number of representations that can be obtained in a given time-unit is often limited.

Revised: 4. D ATA S EED I NFERENCE FOR V ISION T RANSFORMERS

Original: Prior work (Dziedzic et al., 2022b) has  applied dataset inference (DI) (Maini et al., 2021) to identify stolen copies of encoders with CNN-architectures.
Revised: Prior work (Dziedzic et al., 2022b) has successfully applied dataset inference (DI) (Maini et al., 2021) to identify stolen copies of encoders with CNN-architectures.

Original: To verify ownership, DI assumes the training data to be private in order to act as a unique model signature.
Revised: To verify ownership, DI assumes the training data to be private in order to act as a unique model signature.

Original: However, transformers are usually trained on millions of public data points, including noisy and uncurated Internet scrapes  (Radford et al., 2021).
Revised: However, transformers are usually trained on millions of public data points, including noisy and uncurated data scrapes from the Internet (Radford et al., 2021).

Original: Thus, large transformers are trained on overlapping datasets which makes DI non-applicable in this setting.
Revised: Thus, large transformers are trained on overlapping datasets which makes DI non-applicable in this setting.

Original: Additionally, the training methods for vision transformers (Caron et al., 2021) require multi-layer projection heads on top of the base encoder during training.

Original: These heads are discarded after training to remove specific features of the training data points for better generalization which degrades the performance of DI.

Original: To overcome this difficulty and leverage transformers’ training signatures for ownership resolution, we propose DataSeed Inference (DSI) .
Revised: To overcome this difficulty and leverage transformers’ training signatures for ownership resolution, we propose DataSeed Inference (DSI) .

Original: This defense combines DI with watermarking (Uchida et al., 2017) by includinga private data as a secret seed into the original training data and training the defended transformer on the combined datasets.
Revised: This defense combines DI with watermarking (Uchida et al., 2017) by including private data as a secret seed into the original training data and training the defended transformer on the combined datasets.

Original: The private seed then serves to uniquely identify encoders and their stolen copies.
Revised: The private seed then serves to uniquely identify encoders and their stolen copies.

Revised: Even if an adversary is aware of our defense, they still require knowledge about the private seed to avoid extracting features contained in it.

Original: More precisely, after stealing an encoder with an arbitrary dataset from the same data domain, we use the private data as well as its corresponding private test data for DSI.
Revised: After stealing an encoder with an arbitrary dataset from the same data domain, we use the private data as well as its corresponding private test data for DSI.

Original: We visualize our approach in Figure 1.
Revised: We visualize our approach in Figure 1a.

Original: An advantage of DSI is that the private subset does not harm the performance of the defended encoder.
Revised: An advantage of DSI is that the private subset does not harm the performance of the defended encoder.

Original: We observe that by adding more diverse training data, the performance can be improved for downstream tasks that share features with the private subset.
Revised: We observe that by adding more diverse training data, the performance can be improved for downstream tasks that share features with the private subset.

Original: The observation of increased performance is contrary to standard watermarking in supervised settings.
Revised: The observation of increased performance is contrary to standard watermarking in supervised settings.

Original: An intuition to this behavior is that our defense does not directly influence the downstream task but only transforms the representations.
Revised: An intuition to this behavior is that our defense does not directly influence the downstream task but only transforms the representations.

Original: In contrast, in supervised learning, watermarking directly impacts the task that the model is trained for, yielding performance loss.
Revised: In contrast, in supervised learning, watermarking directly impacts the task that the model is trained for, yielding performance loss.

Original: To provide the defended encoder with a stronger unique signal for DSI, we train differently on the private seed than on the original training data.
Revised: To provide the defended encoder with a stronger unique signal for DSI, we train differently on the private seed than on the original training data.

Revised: Specifically, we apply weaker augmentations to the private data points.

Original: This enables the defended encoder to overfit more easily to the private seed.
Revised: This enables the defended encoder to overfit more easily to the private seed.

Original: To this end, we modify the training procedure of the DINO framework (Caron et al., 2021).
Revised: To this end, we modify the training procedure of the DINO framework (Caron et al., 2021).

Original: Concretely, we change the standard DINO’s data pre-processing and increase the size of crops for images passed through the student and teacher encoders.
Revised: Concretely, we change the standard DINO’s data pre-processing and increase the size of crops for images passed through the student and teacher encoders.

Original: This is applied only to data points from the private subset and not to the original training points.
Revised: This is applied only to data points from the private subset and not to the original training points.

Original: Instead of using the student’s crops that cover only small areas (less than 50%) of an input image, we increase the size of crops to the range between 70% and 90%.
Revised: Instead of using the student’s crops that cover only small areas (less than 50%) of an input image, we increase the size of crops to the range between 70% and 90%.

Original: Furthermore, we also increase the size of the crops for the teacher encoder from greater than 50% to greater than 90%.
Revised: Furthermore, we also increase the size of the crops for the teacher encoder from greater than 50% to greater than 90%.

Original: DSI leverages this property and detects different behavior of encoders on their private training data versus unseen test data.
Revised: DSI leverages this property and detects different behavior of encoders on their private training data versus unseen test data.

Original: Next, in our empirical evaluation we show that the unique signature from the private data seed is transferred to stolen encoder copies, making DSI applicable to perform ownership resolution. 
Revised: Next, in our empirical evaluation we show that the unique signature from the private data seed is transferred to stolen encoder copies, making DSI applicable to perform ownership resolution. To further amplify the signal from the training signature, we add the victim’s projection head on top of the verified encoders. 4. W ATERMARKED S ENTENCE -E MBEDDING E NCODERS We find that DSI (as well as standard DI) do not yield significant results in detecting ownership for stolen language encoders.

Revised: This happens because of multiple reasons.

Revised: First, all victims, stolen copies, and independent encoders use the same underlying pre-trained transformers.

Original: E MPIRICAL E VALUATION
Revised: This similarity overlies the individual signals from datasets used for fine-tuning to sentence embeddings. Second, victim encoders are usually fine-tuned over a small number of epochs (10 for TinyBERT and 3 for BERT and RoBERTa). We observe that with longer fine-tuning, the results returned by DI on victim encoders become more confident.

Revised: However, the signal on the stolen encoders remains insignificant.

Revised: As an alternative approach, we develop a new watermarking-based defense against stealing sentence embedding encoders.

Revised: We embed the watermark starting from an already fine-tuned encoder.

Revised: This is a realistic scenario where the model owner would like to add a watermark post hoc at a low cost.

Revised: To embed the watermark, we perform a few iterations of training, where we always alternate between one iteration of the original sentence embedding training (with SimCSE), and then one iteration of training for a downstream task.

Revised: During training of the downstream task, we add an additional fully-connected layer, which serves as our secret key during verification.

Revised: Our watermarking approach is visualized in Figure 1b.

Revised: In this work, we select SST2 (binary classification for sentiment analysis) as the watermark downstream task.

Revised: Note, however, that a defender can select from many possible downstream tasks, reshuffle or flip the labels, or use their own private downstream task, which makes the detection of the watermark much harder.

Revised: To resolve ownership, a verifier simply attaches the fully-connected layer (secret key) to the output of an encoder suspected to be a stolen copy.

Revised: Then, agreement between the outputs of the victim encoder (plus fully-connected layer), and the outputs of the potentially stolen copy (plus fully-connected layer) on the secret downstream task is measured as the percentage of labels where both outputs agree.

Revised: We resolve that an encoder was stolen if the agreement is above the threshold of 95%, otherwise, the encoder is marked as independent.

Revised: We assume that the adversary does not obfuscate the output representations as described in Dziedzic et al.

Original:  We evaluate our methods for stealing and defense against transformer-based encoders trained on different vision and NLP datasets.
Revised: (2022b).  E MPIRICAL E VALUATION We evaluate our methods for stealing and defending transformer-based encoders trained on different vision and NLP datasets.

Original: 5. S TEALING T RANSFORMERS
Revised: 5. S TEALING T RANSFORMERS

Original: Vision.
Revised: Vision.

Original: For ImageNet victims, we use the small-ViT’s checkpoint released by the original DINO paper (Caron et al., 2021).
Revised: For ImageNet victims, we use the ViT’s checkpoints released by the original DINO paper (Caron et al., 2021).

Original: For CIFAR10, we train tiny-ViTs from scratch.
Revised: For CIFAR10, we train Tiny ViTs from scratch.

Original: All training procedures follow (Caron et al., 2021) unless otherwise specified, using 300 epochs, a batch size of 256, and the learning rate is set to 5e-4 with a cosine annealing scheduler.
Revised: All training procedures follow (Caron et al., 2021) unless otherwise specified, using 300 epochs, a batch size of 256, and the learning rate is set to 5e-4 with a cosine annealing scheduler.

Original: For stealing, we experiment with different numbers of queries from various datasets, including CIFAR10, CIFAR100, SVHN and STL10.
Revised: For stealing, we experiment with different numbers of queries from various datasets, including CIFAR10, CIFAR100, SVHN, STL10, and ImageNet.

Original: Details on the datasets are included in Appendix B.1, while further insights on the training can be found in Appendix B.2.
Revised: Details on the datasets are included in Appendix B.1, while further insights on the training can be found in Appendix B.2.

Original: We also use ViTs to instantiate stolen encoders and experiment with different loss functions, such as MSE, InfoNCE (Chen et al., 2020), or SoftNN (Frosst et al., 2019).
Revised: We also use ViTs to instantiate stolen encoders and experiment with different loss functions, such as MSE, InfoNCE (Chen et al., 2020), or SoftNN (Frosst et al., 2019).

Original: Unless otherwise specified, our stealing procedure follows Dziedzic et al.
Revised: Unless otherwise specified, our stealing procedure follows Dziedzic et al.

Original: (2022a).
Revised: (2022a).

Original: We assess the performance of victim and stolen encoders using standard linear evaluation (Chen et al., 2020) on four downstream tasks: CIFAR10, CIFAR100, SVHN, and STL10 (see Table 1).
Revised: We assess the performance of victim and stolen encoders using standard linear evaluation (Chen et al., 2020) on four downstream tasks: CIFAR10, CIFAR100, SVHN, and STL10 (see Table 1).

Original: We observe that stealing with the victim’s training distribution yields the closest performance of the stolen copy to the victim.
Revised: We observe that stealing with the victim’s training distribution yields the closest performance of the stolen copy to the victim.

Original: The MSE loss performs better for the lower query regime (<100K) whilefor the larger number of queries, contrastive losses like InfoNCE yield higher accuracy scores.
Revised: The MSE loss performs better for the lower query regime (<100k) while contrastive losses like InfoNCE yield higher accuracy scores for the larger number of queries.

Original: When stealing with more complex datasets ( e.g., CIFAR10 vs SVHN) we can obtain a better generalization of the stolen copy, which is shown by higher accuracy on more downstream tasks.
Revised: When stealing with more complex datasets ( e.g., CIFAR10 vs SVHN) we can obtain a better generalization of the stolen copy, which is shown by higher accuracy on more downstream tasks.

Original: However, we are unable to steal the exact representations from vision transformers. For this, additional knowledge about the victim’s training data or access to a pre-trained encoders are needed. This involves more compute as well.
Revised: We also apply

Original: We also apply MixMatch for the case when an attacker has limited access to the API and wants to use as few queries as possible.
Revised: MixMatch for the case when an attacker has limited access to the API and wants to use as few queries as possible.

Original: Thus, in this case, we operate in the regime with a number of queries ranging from 4K to 10K, see Table 2.
Revised: Thus, in this case, we operate in the regime with a number of queries ranging from 4k to 10k, see Table 2.

Original: Our results show that MixMatch outperforms all other methods, even when provided with fewer labeled queries. [Table] [Table] [Table]
Revised: Our results show that MixMatch outperforms all other methods, even when provided with fewer labeled queries, and retains high similarity to the victim encoder when measured with the cosine similarity score between the representations from the victim encoder and a stolen copy.

Original: Language.
Revised: Language.

Original: We steal from BERT-based sentence embedding encoders fine-tuned on nli-forSimCSE
Revised: We steal from BERT-based sentence embedding encoders fine-tuned on nli-forSimCSE

Original: (Gao et al., 2021) ("nli"), QQP (Iyer et al.)
Revised: (Gao et al., 2021) ("nli"), QQP (Iyer et al.)

Original: ("qqp"), and Flickr30k (Young et al., 2014) ("flickr").
Revised: ("qqp"), and Flickr30k (Young et al., 2014) ("flickr").

Original: For more details on the datasets and our pre-processing, see Appendix B.3.
Revised: For more details on the datasets and our pre-processing, see Appendix B.3.

Original: For TinyBERT, we fine-tune the victims using SimCSE (Gao et al., 2021), for Bert Base 3 ("BERT"), and RoBERTa
Revised: For TinyBERT, we fine-tune the victims using SimCSE (Gao et al., 2021), for Bert Base 3 ("BERT"), and RoBERTa

Original: Large 4 ("RoBERTa") we use models from Hugging Face as victim encoders.
Revised: Large 4 ("RoBERTa") we use models from Hugging Face as victim encoders.

Original: We fine-tuneall the independent models using SimCSE. Our stolen copiesare initialized with pretrained transformers of the victim architecture.
Revised: We fine-tune independent models using SimCSE and initialize our stolen copies with pretrained transformers of the victim architecture.

Original: Following SimCSE, we replace the original BERT pooling layer of our stolen models with a custom pooler that returns the [CLS] representation and add a randomly initialized MLP layer on top before stealing.
Revised: Following SimCSE, we replace the original BERT pooling layer of our stolen models with a custom pooler that returns the [CLS] representation and add a randomly initialized MLP layer on top before stealing.

Original: 5 For stealing, we issue 60K queries, using sentences from our three datasets against the victim encoders and fine-tune our stolen copies with the resulting representations for 20 (TinyBERT and BERT), or 5 (RoBERTa) epochs, respectively, using MSE loss.
Revised: 5 For stealing, we issue 60K queries, using sentences from our three datasets against the victim encoders and fine-tune our stolen copies with the resulting representations for(TinyBERT and BERT), or 5 (RoBERTa) epochs, respectively, using MSE loss.

Original: Further details on  trainingthe victim andindependent encoders, as well as on our stealing methods are provided in Appendix B.4.
Revised: Further details on our training and stealing methods are provided in Appendix B.4.

Original: To evaluate DSI, we generate training datasets with a private dataset as secret seeds. 3 https [Table] 84 flickr+qqp(50k) -
Revised: Table 3 depicts the performance of our victim and stolen encoders, evaluated on tasks from the

Original: [Table] [Table] [Table] [Table]
Revised: SentEval benchmark.

Original: [Table]

Original: [Table]

Original: [Table]

Original: f s

Original: T nli nli [Table] [Table] [Table]
Revised: We observe that across all base encoders (TinyBERT, BERT, and RoBERTa), the performance of the stolen copies is comparable to their respective victim encoders over most benchmark tasks.

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: nli [Table] [Table] [Table]
Revised: This holds true even for the large qqp dataset ( ∼ 2.6M training samples) and stolen 3 https://huggingface.co/princeton-nlp/sup-simcse-bert-base-uncasedhttps://huggingface.co/princeton-nlp/sup-simcse-roberta-large

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: flickr nli

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: f v * B* nli -

Original: [Table]

Original: [Table]

Original: We generate three such sets: 1) 50K flickr data points into the qqp dataset, 2) the full flickr, i.e., ∼ 300K into qqp, and 3) 50K qqp into flickr.

Original: We fine-tune sentence embedding encoders based on TinyBERT using the same procedure as for the victim and independent encoders. Table 3 depicts the performance of our victim and stolen encoders, evaluated on tasks from the SentEval benchmark.
Revised: et al.

Original: We observe that across all base encoders (TinyBERT, BERT, and RoBERTa), the performance of the stolen copies is comparable to their respective victim encoders over most benchmark tasks.
Revised: (2022b

Original: This holds true even for the large qqp dataset ( ∼ 2.6M training samples) and stolen copies obtained with only 60K queries, i.e., ∼ 40x fewer queries than training samples. In general, the performance of encoders stolen with nli and qqp is higher than the one of encoders stolen with flickr.
Revised: [Table] copies obtained with only 60K queries, i

Revised: [Table]

Revised: e., ∼ 40x fewer queries than training samples.

Revised: In general, the performance of encoders stolen with nli and qqp is higher than the one of encoders stolen with flickr.

Original: We suspect this is due to the low semantic diversity in flickr which consists only of 30K images, leading to semantic overlap within the 60K stealing-queries.
Revised: We suspect this is due to the low semantic diversity in flickr which consists only of 30K images with five semantically equal captions each, leading to semantic overlap within the 60K stealing-queries.

Original: When analyzing the performance ofencoders trained with the data subsets, we note that for most tasks the flickr+qqp encoder overpasses the performance of the encoder trained only onflickr, while the performance ofqqp+flickr increases only for the STS benchmarks.
Revised: We further explore the impact of the number of stealing queries on the performance of the stolen copies. Our results in Table 9 in Appendix C.1 highlight a performance decrease when reducing the number of stealing queries. The performance drop is most significant between 10k and 20k queries.

Original: 5. D EFENDING T RANSFORMERS
Revised: This motivates an evaluation of the effectiveness of our method to re-use representations for semantically similar sentences in this setup.

Original: Dataset Inference. We perform standard DI for the ViTs trained on CIFAR10. In Table 4, we show that DI is not able to differentiate between victim and independent transformers from the differences in their log-likelihoods.
Revised: We query the stolen model copy with 10k sentences from the nli dataset and assign the obtained representation also to the semantically equal positive partner of each sentence. This results in 20k fine-tuning samples for the stolen copy.

Original: We conduct experiments to further investigate why DI fails inthis scenario. During training, DINO uses a three-layer MLP followed by an ℓ 2 normalization and a linear layer as the projection head on top of the original encoder. This head is discarded after training. We find that the projection head plays an important role in the successful detection of victim encoders by standard DI.
Revised: Our results in

Original: Table5 shows that DI is more effective when the projection head uses shallower MLPs. In the case of three-layer MLPs, DI can detect the victim only when the projection head is also used during evaluation. This result meets our intuition: since the training loss is minimized directly with the projection head, some information about the training signature is lost when the head is removed. The deeper the head is, the higher the information loss incurred.
Revised: Table

Original: Therefore defending transformers calls for more effective methods than DI.

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: 5. D EFENDING T RANSFORMERS

Original: DataSeed Inference.
Revised: DataSeed Inference.

Original: We design DataSeed Inference to incorporate a private seed into the victim’s training set.
Revised: We design DataSeed Inference to incorporate a private seed into the victim’s training set.

Original: Concretely, we insert 5K SVHN, 10K SVHN, and 10K MNIST images  into the CIFAR training sets, respectively.
Revised: Concretely, we insert randomly-selected 10k images from the SVHN and GTSRB training splits into the CIFAR10 training sets.

Revised: We train ViT Tiny and ViT Small on the combined datasets as victims.

Original: We use weaker augmentations for the private set, which results in more similar inputs during training and evaluation and provides a stronger signal for DSI.
Revised: We use weaker augmentations for the private set, which results in more similar inputs during training and evaluation and provides a stronger signal for DSI.

Original: Otherwise, the training procedures for the victim and stolen encoders are the same as before.
Revised: Otherwise, the training procedures for the victim and stolen encoders are the same as before.

Original: For more details, see Appendix B.2. The last three columns in Table 4 demonstrate that DSI is able to differentiate the stolen transformers of CIFAR10 by injecting a private subset of SVHN and MNIST, while the original DI fails to do so.
Revised: Table 12 shows that DSI is more effective when the projection head in training DINO uses shallower MLPs. This result meets our intuition: since the training loss is minimized directly with the projection head, some information about the training signature is lost when the head is removed.

Original:  We also assess the performance of our encoders trained with CIFAR10 and a private data seed.
Revised: Therefore, to make the training signature easier to detect, we attach the victim’s projection head on top of the verified models. We use 20k randomly-selected images from the rest of the training splits as the validation set in DSI. We train GMMs with 10 components across all settings.

Original: We observe that these encoders achieve similar performance to the victim trained only on CIFAR10 and can outperform it when assessed on the downstream tasks that share features with the private subset, see  Table 1. Moreover, we note thatcompared with DI, DSI provides us the flexibility to select a private set, potentially with a stronger signature.
Revised: For more details, see Appendix B.2. The results in Table 5 demonstrate that DSI is able to differentiate the stolen transformers of CIFAR by injecting a private subset of SVHN and GTSRB, even when the stealing datasets are different from training.

Original: Similar to DI for supervised learning (Maini et al., 2021), the victim encoder typically has the largest ∆ µ and the smallest p-values.
Revised: Similar to DI for supervised learning (Maini et al., 2021), the victim encoder typically has the largest ∆ µ and the smallest p-values.

Original: We also find that increasing the size of the private set makes this defense more effective.
Revised: We also find that increasing the size of the private set makes this defense more effective.

Original: Finally, our comparison with the independent models ensures that we avoid false positives when detecting stolen models (see last two rows in Table 1). [Figure]
Revised: Finally, our comparisons with the independent models ensure that we avoid false positives when detecting stolen models (see last two rows in Table 5).

Original: Language. We find that both DI and DSI do not yield significant results in detecting ownership for stolen language encoders. This happens because of multiple reasons.
Revised: We also assess the performance of the encoders trained with CIFAR10 and a private data seed.

Original: First in language, all victims, stolen copies, and independent encoders use the same underlying pretrained transformers. This similarity overlies theindividual signals from datasets used for fine-tuning. Second, victim encoders are usually fine-tuned over a small number of epochs (10 for TinyBERT and 3 for BERT and RoBERTa).
Revised: We observe that these encoders achieve similar performance to the victim trained only on CIFAR10 and can outperform it when assessed on the downstream tasks that share features with the private subset, see Table 1. Watermarking Language Encoders.

Original: We observe that with longer finetuning, the results returned by DI on victim encoders become more confident.
Revised: We present the performance of the watermarked encoders as well as the success of our watermarking in Table 6. We use the initial fine-tuned sentence embedding

Original: However, the signal on the stolen encoders remains insignificant. [Table] 
Revised: [Table] [Table] encoder as the independent encoder.

Original: As an alternative metric, we use the cosine similarity score, as in (Dziedzic et al., 2022b), to compare the encoders. However, this metric should not be used for ownership resolution because obfuscations of stolen representations can leave the extracted encoders undetectable.
Revised: This can be considered the worst-case evaluation because the representations returned from this encoder are expected to be most similar to its watermarked derivation. We show that even in this worst-case, the independent model is never incorrectly resolved as being a stolen copy.

Original: We track the cosine similarity between representations of the stolen and victim encoder over the course of stealing on 50 representations from the test dataset (Figure 2) and observe that similarity increases from the very beginning of stealing providing evidence of the highly accurate stealing process. Table 6 depicts the final cosine similarity scores between language encoders, stolen copies, and independent models are after stealing is completed. We find that the stolen copies have higher scores than independent encoders trained on different datasets than the victim’s data. We also evaluate the similarity of our encoders trained with private seed against their stolen copies and independent encoders.
Revised: To compute the p-values, we leverage the confidence scores (softmax outputs) for the correct labels from the downstream task and use the t-test.

Original: Our results show that if a victim isstolen with the private data seed or an independent dataset, the similarity to its stolen copies is higher than to the independent encoder. However, the information on the private data subset is not transferred to the stolen copies when stealing with the victim’s original training data distribution. In a public setting, this is problematic since APIs reveal information about the types of data used to pre-train and fine-tune their encoders. Note that despite some inconsistencies in representations between BERT and RoBERTa from Hugging
Revised: The p-values indicate that there is a significant difference between the distribution of the confidence scores from independent vs victim encoders (p-value < 5%).

Original: Face and our independently trained encoders, we are able to steal the encoders from Hugging Face.
Revised: In contrast, the difference is not significant between the victim and stolen encoders.

Original: Our stolen copies do not only achieve high performance on benchmarks (Table 3) but also produce representations with high cosine similarity scores to  the pre-trained encoders (Table 6). This highlights that while language transformers can be stolen effectively, the signal in their representations does not easily benefit ownership resolution. We present an extended analysis of the NLP encoders their representation similarities in Appendix C.1.
Revised: Our results highlight that a relatively small number of fine-tuning steps ( e.g.,alternations between the original task and downstream task) are sufficient to successfully embed the watermark into the encoder while preserving the high performance of the defended encoder on other unrelated and general downstream tasks.

Original:  C ONCLUSIONS AND F UTURE W ORK
Revised:  C ONCLUSIONS AND F UTURE W ORK

Original: Modern APIs offer access to high-value transformer-based encoders for generating representations of given input text or images.
Revised: Modern APIs offer access to high-value transformer-based encoders for generating representations of given input text or images.

Original: We demonstrate how to steal these transformers in the language and vision domains by using only representations.
Revised: We demonstrate how to steal these transformers in the language and vision domains by using only representations.

Original: Our stealing requires up to 40x fewer queries than the number of training data points used to train the victim and it yields stolen copies with comparable performance on standard benchmarks.
Revised: Our stealing requires up to 40x fewer queries than the number of training data points used to train the victim and it yields stolen copies with comparable performance on standard benchmarks.

Revised: We further decrease the number of stealing queries by using semantically similar sentences in the language domain and semi-supervised learning in the vision domain.

Original: We propose a new defense for vision transformers, where a private data seed acts as the training signature of the victim encoder.
Revised: We propose a new defense for vision transformers, where a private data seed acts as the training signature of the victim encoder.

Original: We find that this type of defense is unable to protect sentence embedding encoders since the signal of the private data subset used during fine-tuning is overlaid with the data used during pre-training of the base encoder.
Revised: We find that this type of defense is unable to protect sentence embedding encoders since the signal of the private data subset used during fine-tuning is overlaid with the data used during pre-training of the base encoder.

Original: Our work highlights that, in particular, such sentence embedding  encoders are easily stolen but hard to defend, which is an important direction for future work.
Revised: To overcome this obstacle, we propose a method for embedding a watermark into the encoders by fine-tuning a defended encoder with a specific downstream task.

Original:  E THICS S TATEMENT
Revised:  E THICS S TATEMENT

Original: Even though we place our work in a public access setting and implement our stealing based on representations such as the ones returned by public APIs, we do not actually steal the models from real-world APIs.
Revised: Even though we place our work in a public access setting and implement our stealing based on representations such as the ones returned by public APIs, we do not actually steal the models from real-world APIs.

Original: All the models stolen in this work are either trained by ourselves or obtained from open-source platforms, such as Hugging Face.
Revised: All the models stolen in this work are either trained by ourselves or obtained from open-source platforms, such as Hugging Face.

Original: However, we note that several public APIs expose access to the same or similar models.
Revised: However, we note that several public APIs expose access to the same or similar models.

Original: Hence, our attacks could potentially be applied against such
Revised: Hence, our attacks could potentially be applied against such

Original: APIs to steal their models.
Revised: APIs to steal their models.

Original: Given that the models attacked in this work are publicly available anyways, their extraction from an API would not cause additional harm to the API provider.
Revised: Given that the models attacked in this work are publicly available anyways, their extraction from an API would not cause additional harm to the API provider.

Original: We still decided to keep the code implementing our model stealing confidential to minimize the risk of exposure through our work for such providers.
Revised: We still decided to keep the code implementing our model stealing confidential to minimize the risk of exposure through our work for such providers.

Original: Instead, we submit only the code implementing our defenses.
Revised: Instead, we submit only the code implementing our defenses.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: Due to the reasons elaborated in the ethics statement, we do not publicly submit the code of our stealing procedure.
Revised: Due to the reasons elaborated in the ethics statement, we do not publicly submit the code of our stealing procedure.

Original: Instead, for reproducibility, we describe our approach in detail—documenting the frameworks we used, all hyperparameters for training, and the architectures.
Revised: Instead, for reproducibility, we describe our approach in detail—documenting the frameworks we used, all hyperparameters for training, and the architectures.

Original: We submit the code for our defense in the supplementary material.
Revised: We submit the code for our defense in the supplementary material.

Original: In the README.md file we provide the main commands needed to run our code.
Revised: In the README.md file we provide the main commands needed to run our code.

Original: We also describe the pointers to the crucial parts of the code that directly reflect the implementation described in the main part of the submission.
Revised: We also describe the pointers to the crucial parts of the code that directly reflect the implementation described in the main part of the submission.

Original: We also describe the defenses used in our experiments thoroughly.
Revised: We also describe the defenses used in our experiments thoroughly.

Original: A A DDITIONAL R ELATED W ORK
Revised: A A DDITIONAL R ELATED W ORK

Original: Self-Supervised Learning.
Revised: Self-Supervised Learning.

Original: In computer vision, one of the most popular self-supervised algorithms is contrastive learning (Chen et al., 2020; van den Oord et al., 2018), where representations that come from differently transformed views of the same image are brought closer to each other and the representations from views of different inputs are repelled.
Revised: In computer vision, one of the most popular self-supervised algorithms is contrastive learning (Chen et al., 2020; van den Oord et al., 2018), where representations that come from differently transformed views of the same image are brought closer to each other and the representations from views of different inputs are repelled.

Original: In NLP, a popular self-supervised pre-training approach is to mask selected words in the input sequence and train the model to predict that masked words Devlin et al.
Revised: In NLP, a popular self-supervised pre-training approach is to mask selected words in the input sequence and train the model to predict that masked words Devlin et al.

Original: Transformers.
Revised: Transformers.

Original: Transformer (Vaswani et al., 2017) is becoming a ubiquitous architecture in NLP and computer vision.
Revised: Transformer (Vaswani et al., 2017) is becoming a ubiquitous architecture in NLP and computer vision.

Original: While the original transformer consists of an encoder and decoder component, our work only studies the encoder part for representation learning.
Revised: While the original transformer consists of an encoder and decoder component, our work only studies the encoder part for representation learning.

Original: Transformers are composed of several identical layers, namely a multi-head attention sublayer followed by a feed forward sublayer.
Revised: Transformers are composed of several identical layers, namely a multi-head attention sublayer followed by a feed forward sublayer.

Original: The multi-head attention sublayer utilizes the self-attention mechanism to learn the pairwise relationships between all tokens.
Revised: The multi-head attention sublayer utilizes the self-attention mechanism to learn the pairwise relationships between all tokens.

Original: Self-attention is the key to the success of transformers, which makes parallel training and learning long-range dependency between tokens easier.
Revised: Self-attention is the key to the success of transformers, which makes parallel training and learning long-range dependency between tokens easier.

Original: NLP transformers.
Revised: NLP transformers.

Original: In this paper, we investigate BERT-based models (Devlin et al., 2018), pretrained bidirectional transformers (Vaswani et al., 2017).
Revised: In this paper, we investigate BERT-based models (Devlin et al., 2018), pretrained bidirectional transformers (Vaswani et al., 2017).

Original: In addition to BERT Base, we also use
Revised: In addition to BERT Base, we also use

Original: TinyBERT (Jiao et al., 2019), obtained by distilling BERT to a smaller transformer architecture, and RoBERTa Large (Liu et al., 2019), which is pre-trained on a larger dataset than BERT.
Revised: TinyBERT (Jiao et al., 2019), obtained by distilling BERT to a smaller transformer architecture, and RoBERTa Large (Liu et al., 2019), which is pre-trained on a larger dataset than BERT.

Original: Then we analyze the task of learning highly generic sentence representation, a fundamental problem in
Revised: Then we analyze the task of learning highly generic sentence representation, a fundamental problem in

Original: NLP (Kiros et al., 2015; Hill et al., 2016; Logeswaran & Lee, 2018).
Revised: NLP (Kiros et al., 2015; Hill et al., 2016; Logeswaran & Lee, 2018).

Original: Vision Transformers.
Revised: Vision Transformers.

Original: Vision transformers (Dosovitskiy et al., 2020; Touvron et al., 2021; Park & Kim, 2022), inspired by transformer models from the NLP domain, achieve state-of-the-art performance for many downstream tasks, such as standard image classification (He et al., 2022) or segmentation (Caron et al., 2021).
Revised: Vision transformers (Dosovitskiy et al., 2020; Touvron et al., 2021; Park & Kim, 2022), inspired by transformer models from the NLP domain, achieve state-of-the-art performance for many downstream tasks, such as standard image classification (He et al., 2022) or segmentation (Caron et al., 2021).

Original: They are based on either self-distillation with no labels (Caron et al., 2021) or masked autoencoders ( MAE )
Revised: They are based on either self-distillation with no labels (Caron et al., 2021) or masked autoencoders ( MAE )

Original: (Devlin et al., 2018; Dosovitskiy et al., 2020).
Revised: (Devlin et al., 2018; Dosovitskiy et al., 2020).

Original: MAE is a masked autoencoder for the self-supervised training of vision transformers.
Revised: MAE is a masked autoencoder for the self-supervised training of vision transformers.

Original: The approach is based on training an asymmetric encoder-decoder architecture that learns to reconstruct randomly masked patches of an input image.
Revised: The approach is based on training an asymmetric encoder-decoder architecture that learns to reconstruct randomly masked patches of an input image.

Original: BEiT (Bao et al., 2022) is another similar approach to MAE that also masks patches of an image and learns to recover the original inputs.
Revised: BEiT (Bao et al., 2022) is another similar approach to MAE that also masks patches of an image and learns to recover the original inputs.

Original: Watermarking Encoders
Revised: Watermarking Encoders

Original: The watermark scheme proposed by Dziedzic et al.
Revised: The watermark scheme proposed by Dziedzic et al.

Original: (2022a) trains the defended encoder simultaneously with an augmentation predictor.
Revised: (2022a) trains the defended encoder simultaneously with an augmentation predictor.

Original: As the watermarking task, the defended encoder is trained to predict the rotation range of an input image.
Revised: As the watermarking task, the defended encoder is trained to predict the rotation range of an input image.

Original: For watermark verification, an encoder’s representations are evaluated on the augmentation predictor to obtain the accuracy on the watermark task and when the accuracy is significantly higher than 50%, the encoder is marked as stolen.
Revised: For watermark verification, an encoder’s representations are evaluated on the augmentation predictor to obtain the accuracy on the watermark task and when the accuracy is significantly higher than 50%, the encoder is marked as stolen.

Original: SSLGuard (Cong et al., 2022) is another watermarking-based defense.
Revised: SSLGuard (Cong et al., 2022) is another watermarking-based defense.

Original: First, it generates a secret key, a verification dataset, and a decoder.
Revised: First, it generates a secret key, a verification dataset, and a decoder.

Original: A watermarked encoder and its stolen copies map samples in the verification dataset to secret representations that in turn can be transformed into the secret key through the decoder.
Revised: A watermarked encoder and its stolen copies map samples in the verification dataset to secret representations that in turn can be transformed into the secret key through the decoder.

Original: For independently trained encoders, the decoder transforms the representations generated from the verification dataset into random vectors.
Revised: For independently trained encoders, the decoder transforms the representations generated from the verification dataset into random vectors.

Original: SSLGuard assumes a specific attack to which the trigger is transferred, which makes it vulnerable to new types of attacks.
Revised: SSLGuard assumes a specific attack to which the trigger is transferred, which makes it vulnerable to new types of attacks.

Original: On the other hand, our DSI uses a private seed as a signature and does not rely on any assumptions about a type of stealing attack.
Revised: On the other hand, our DSI uses a private seed as a signature and does not rely on any assumptions about a type of stealing attack.

Original: Wu et al.
Revised: Wu et al.

Original: (2022) manipulates the verification samples to obtain for them unique feature representations from protected and stolen encoders, which leads to unique predicted labels for any downstream task.
Revised: (2022) manipulates the verification samples to obtain for them unique feature representations from protected and stolen encoders, which leads to unique predicted labels for any downstream task.

Original: However, it was shown that representations can be easily obfuscated (Dziedzic et al., 2022b) to remove their similarity to a stolen encoder. 
Revised: However, it was shown that representations can be easily obfuscated (Dziedzic et al., 2022b) to remove their similarity to a stolen encoder. Stealing from NLP Classifiers

Revised: The stealing methods proposed in Chen et al.

Revised: (2021) and Rafi et al.

Original: B D ETAILS ON THE E XPERIMENTAL S ETUP B. V ISION D ATASETS
Revised: (2022) are for downstream classification tasks while we steal representations from self-supervised encoders in both language as well as vision domains. For the vision encoders, we consider a more challenging task and start stealing from randomly initialized encoders instead of pre-trained ones. Apart from improving the stealing process by leveraging semantically similar sentences, we also propose a defense method against stealing sentence embedding encoders using a watermark-based approach. Chen et al.

Revised: (2021) steal fine-tuned classification models, while our extraction targets encoders, hence, it is more general.

Revised: Rafi et al.

Revised: (2022) extract architectures and weights, while our attack method extracts representations.

Revised: Additionally, Rafi et al.

Revised: (2022) require adversaries to have more knowledge about the victim than is assumed in our work.

Revised: For example, architecture extractions require access to the target machine.

Revised: B D ETAILS ON THE E XPERIMENTAL S ETUP B. V ISION D ATASETS

Revised: MNIST (Deng, 2012)

Revised: The MNIST database of handwritten digits has a training set of 60, examples, and a test set of 10,000 examples.

Revised: The digits have been size-normalized and centered in a fixed-size image.

Revised: It consists of 28x28 black and white images with 10 classes.

Original: CIFAR10 (Krizhevsky, 2009):
Revised: CIFAR10 (Krizhevsky, 2009):

Original: The CIFAR10 dataset consists of 32x32 colored images withclasses.
Revised: The CIFAR10 dataset consists of 32x32 colored images withclasses.

Original: There are 50000 training images and 10000 test images.
Revised: There are 50000 training images and 10000 test images.

Original: CIFAR100 (Krizhevsky, 2009):
Revised: CIFAR100 (Krizhevsky, 2009):

Original: The CIFAR100 dataset consists of 32x32 coloured images withclasses.
Revised: The CIFAR100 dataset consists of 32x32 coloured images withclasses.

Original: There are 50000 training images and 10000 test images.
Revised: There are 50000 training images and 10000 test images.

Original: SVHN (Netzer et al., 2011):
Revised: SVHN (Netzer et al., 2011):

Original: The SVHN dataset contains 32x32 coloured images with 10 classes.
Revised: The SVHN dataset contains 32x32 coloured images with 10 classes.

Original: There are roughly 73000 training images, 26000 test images and 530000 "extra" images.
Revised: There are roughly 73000 training images, 26000 test images and 530000 "extra" images.

Original: ImageNet (Deng et al., 2009): Larger sized coloured images with 1000 classes.
Revised: ImageNet (Deng et al., 2009): Larger sized coloured images with 1000 classes.

Original: As is commonly done, we resize all images to be of size 224x224.
Revised: As is commonly done, we resize all images to be of size 224x224.

Original: There are approximately 1 million training images and 50000 test images.
Revised: There are approximately 1 million training images and 50000 test images.

Original: STL10 (Coates et al., 2011):
Revised: STL10 (Coates et al., 2011):

Original: The STL10 dataset contains 96x96 coloured images with 10 classes.
Revised: The STL10 dataset contains 96x96 coloured images with 10 classes.

Original: There are 5000 training images, 8000 test images, and 100000 unlabeled images.
Revised: There are 5000 training images, 8000 test images, and 100000 unlabeled images.

Original: B. E XPERIMENTAL S ETUP FOR S TEALING V ISION T RANSFORMERS
Revised: B. E XPERIMENTAL S ETUP FOR S TEALING V ISION T RANSFORMERS

Original: For vision, ViT Tiny, Small architectures are used for the CIFAR10 and SVHN victim encoders.
Revised: For vision, ViT Tiny, Small architectures are used for the CIFAR10 and SVHN victim encoders.

Original: We use DINO (Caron et al., 2021) to train ViT with patch size 16 in a self-supervised way.
Revised: We use DINO (Caron et al., 2021) to train ViT with patch size 16 in a self-supervised way.

Original: We resize the images to 224 × 224 so that we can use the hyperparameters provided in the DINO paper.
Revised: We resize the images to 224 × 224 so that we can use the hyperparameters provided in the DINO paper.

Original: We use 3-layer MLP and then apply a L2 normalization, and a weight normalized fully-connected layer as the DINO paper by default, and do experiments with  different number of MLP layers in 5.
Revised: We use 3-layer MLP and then apply an L2 normalization, and a weight normalized fully-connected layer as the DINO paper by default, and do experiments with the different number of MLP layers in ?? .

Original: We train all victim models for 300 epochs.
Revised: We train all victim models for 300 epochs.

Original: To train the stolen encoders, we used 150 epochs.
Revised: To train the stolen encoders, we used 150 epochs.

Original: We use
Revised: We use

Original: AdamW optimizer and Cosine Annealing scheduler when training victim and stolen models.
Revised: AdamW optimizer and Cosine Annealing scheduler when training victim and stolen models.

Original: When training the victim model, the initial learning rate  4e-5 while for training stolen models, the initial learning rate is 1e-3.
Revised: When training the victim model, the initial learning rate is 4e-5 while for training stolen models, the initial learning rate is 1e-3.

Original: A batch size of 128 or 256 was used for training the models.
Revised: A batch size of 128 or 256 was used for training the models.

Original: The temperaturethat we used for InfoNCE is 0.07,  the temperature for soft nearest neighbors loss is 1000.
Revised: The temperature we use for InfoNCE is 0.07, and the temperature for soft nearest neighbors loss is 1000.

Original: For the stealing experiments, we use the last 4 blocks outputs of dummy representations, in the same way as in DINO (Caron et al., 2021).
Revised: For the stealing experiments, we use the last 4 block outputs of dummy representations, in the same way as in DINO (Caron et al., 2021).

Original: Thus, the dimensions of output representations for ViT-tiny and
Revised: Thus, the dimensions of output representations for ViT-tiny and

Original: ViT-small are 768 and 1536, respectively.
Revised: ViT-small are 768 and 1536, respectively.

Revised: Victim We train tiny-ViTs on CIFAR10.

Original: B. NLP D ATASETS AND P ROCESSING
Revised: All training procedures follow (Caron et al., 2021) unless otherwise specified. The number of epochs is 300, the bs =256 and lr =5e-4 with a cosine annealing scheduler. The summary of the victim encoder’s performance can be found in Table 1. Stolen When stealing from the victim encoders, we experiment with different numbers of queries from various datasets, including CIFAR10, CIFAR100, SVHN, and STL10.

Revised: We also use both ResNet and TinyViT as the architectures for stolen encoders.

Revised: Stolen encoders are trained by minimizing different loss functions, such as MSE, InfoNSE (Chen et al., 2020), or SoftNN (Frosst et al., 2019).

Revised: Unless otherwise specified, the stealing procedure follows Dziedzic et al.

Revised: (2022a).

Revised: Independent We use ViTs trained on other datasets than the victim’s training set as independent encoders.

Revised: Private Data Subsets For DSI, we insert 5k, 10k SVHN, and 10k

Revised: B. NLP D ATASETS AND P ROCESSING

Original: nli.
Revised: nli.

Original: We use (Gao et al., 2021)’s nli-for-SimSCE dataset, consisting of 275,602 data rows from
Revised: We use (Gao et al., 2021)’s nli-for-SimSCE dataset, consisting of 275,602 data rows from

Original: SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018).
Revised: SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018).

Original: Each row holds three sentences, an original sentence, a positive entailment and a contradiction.
Revised: Each row holds three sentences, an original sentence, a positive entailment, and a contradiction.

Original: In training, the contradiction acts as a hard-negative.
Revised: In training, the contradiction acts as a hard-negative.

Original: qqp.
Revised: qqp.

Original: We use the merve/qqp dataset from Hugging Face.
Revised: We use the merve/qqp dataset from Hugging Face.

Original: The train split consists of 2,607,949 data rows, each holding two semantically equal questions.
Revised: The train split consists of 2,607,949 data rows, each holding two semantically equal questions.

Original: We use this data for training as positive pairs.
Revised: We use this data for training as positive pairs.

Original: flickr.
Revised: flickr.

Original: The flickr dataset consists of images, each annotated with five human-written captions.
Revised: The flickr dataset consists of images, each annotated with five human-written captions.

Original: Following (Gao et al., 2021), we consider any two captions of the same image as a positive pair.
Revised: Following (Gao et al., 2021), we consider any two captions of the same image as a positive pair.

Original: We split the training and test set to 90%, 10%, making sure that all caption-pairs related to one image end up in the same set.
Revised: We split the training and test set to 90%, 10%, making sure that all caption pairs related to one image end up in the same set.

Original: This yields 286,050 positive-pair training examples.
Revised: This yields 286,050 positive-pair training examples.

Original: When using flickr as a dataset for stealing, we drop the duplicates arising from generating all possible caption pairs before sampling the stealing-queries.
Revised: When using flickr as a dataset for stealing, we drop the duplicates arising from generating all possible caption pairs before sampling the stealing-queries.

Original: Thereby, we mitigate a too small diversity over the stealing due to repeated queries.
Revised: Thereby, we mitigate a too small diversity over the stealing due to repeated queries.

Original: B. E XPERIMENTAL S ETUP FOR S TEALING L ANGUAGE T RANSFORMERS
Revised: B. E XPERIMENTAL S ETUP FOR S TEALING L ANGUAGE T RANSFORMERS

Original: Victim.
Revised: Victim.

Original: As victim encoders, we use TinyBERT-based encoders and fine-tune them for the sentence embedding task on nli, qqp, and flickr by using SimSCE (Gao et al., 2021).
Revised: As victim encoders, we use TinyBERT-based encoders and fine-tune them for the sentence embedding task on nli, qqp, and flickr by using SimSCE (Gao et al., 2021).

Original: For more details on data pre-processing and the datasets, see Appendix B.3.
Revised: For more details on data pre-processing and the datasets, see Appendix B.3.

Original: We fine-tune our encoders for 10 epochs, with batch size ( bs )=
Revised: We fine-tune our encoders for 10 epochs, with batch size ( bs )=

Original: 128, learning rate ( lr )=
Revised: 128, learning rate ( lr )=

Original: 5e-5, and temperature=0.05.
Revised: 5e-5, and temperature=0.05.

Original: Additionally, we use transformers fine-tuned with nli from BERT 6 , and RoBERTa 7 from Hugging Face as victim 
Revised: Additionally, we use transformers fine-tuned with nli from BERT 6 , and RoBERTa 7 from Hugging Face as victim encoders.

Revised: For fine-tuning and stealing over all encoders, we set the maximal input sequence length to 32 and use truncation and padding.

Revised: The performance of our victim encoders can be found in Table 3.

Original: Stolen.
Revised: Stolen.

Original: We initialize our stolen encoders with pre-trained transformers from Huggingface (prajjwal1/bert-tiny, bert-base-uncased, and roberta-large), in accordance with the respective victim encoder.
Revised: We initialize our stolen encoders with pre-trained transformers from Huggingface (prajjwal1/bert-tiny, bert-base-uncased, and roberta-large), in accordance with the respective victim encoder.

Original: During stealing, TinyBERT and BERT use an lr =1e-5, bs =256, and linear lr -scheduling with patience 200 iterations and factor 0.5.
Revised: During stealing, TinyBERT and BERT use an lr =1e-5, bs =256, and linear lr -scheduling with patience 200 iterations and factor 0.5.

Original: For RoBERTa, we use the same setup, however with bs=64, lr -patience 600 iterations, and lr =5e-6 when stealing with nli or flickr.
Revised: For RoBERTa, we use the same setup, however with bs=64, lr -patience 600 iterations, and lr =5e-6 when stealing with nli or flickr.

Original: As in SimCSE, we evaluate our stolen encoders on the SentEval benchmark.
Revised: As in SimCSE, we evaluate our stolen encoders on the SentEval benchmark.

Original: Independent.
Revised: Independent.

Original: We fine-tune the independent TinyBERT-based encoders on nli, qqp, and flickr in the same setup as the victim encoders.
Revised: We fine-tune the independent TinyBERT-based encoders on nli, qqp, and flickr in the same setup as the victim encoders.

Original: To obtain independent encoders based on BERT and RoBERTa, we fine-tune the respective base encoders on nli, qqp, and flickr for using SimCSE.
Revised: To obtain independent encoders based on BERT and RoBERTa, we fine-tune the respective base encoders on nli, qqp, and flickr for using SimCSE.

Original: We keep lr
Revised: We keep lr

Original: =5e-5, temperature=0.05, bs =128 and bs =32 for BERT and RoBERTa, respectively,
Revised: =5e-5, temperature=0.05, bs =128 and bs =32 for BERT and RoBERTa, respectively,

Original: but following (Gao et al., 2021), we fine-tune only for 3 epochs.
Revised: but following (Gao et al., 2021), we fine-tune only for 3 epochs.

Original: Victim (Vision).
Revised: DSI.

Original: We train tiny-ViTs on CIFAR10. All training procedures follow (Caron et al., 2021) unless otherwise specified.
Revised: To evaluate DSI, we generate training datasets with a private dataset as secret seeds.

Original: The number of epochs is 300, the bs =256 and lr =5e-4 with a cosine annealing scheduler. The summary of the victim encoder’s performance can be found in Table 1. Stolen (Vision).
Revised: We generate three such sets: 1) 50K flickr data points into the qqp dataset, 2) the full flickr, i.e., ∼ 300K into qqp, and 3) 50K qqp into flickr.

Original: When stealing from the victim encoders, we experiment with different numbers of queries from various datasets, including CIFAR10, CIFAR100, SVHN and STL10. We also use both ResNet and TinyViT as the architectures for stolen encoders. Stolen encodersare trained by minimizing different loss functions, such as MSE, InfoNSE (Chen et al., 2020), or SoftNN (Frosst et al., 2019).
Revised: We fine-tune sentence embedding encoders based on TinyBERT using the same procedure as for the victim and independent encoders. When analyzing the performance of encoders trained with the data subsets in Table 7 and note that for most tasks the flickr+qqp encoder overpasses the performance of the encoder trained only on flickr, while the performance of qqp+flickr increases only for the STS benchmarks.

Original: Unless otherwise specified, the stealing procedure follows Dziedzic et al. (2022a). Independent (Vision).
Revised: with Mixed Dataset. We follow SimCSE (Gao et al., 2021) and use the SentEval benchmark.

Original: We use ViTs trained on other datasets than the victim’s training set as independent encoders. Private Data Subsets (Vision).
Revised: f v denotes the victim encoder trained on data D v . f s is the stolen encoder extracted using queries from a given stealing dataset D s . [Table]

Original: For DSI, we insert 5k, 10k SVHN and 10k MNIST images and into the CIFAR10 training sets respectively.
Revised: 6 https://

Original: The training procedures for the victim and stolen encoders are the same as before.
Revised: -bert-base-uncased https:// -roberta-large

Original:  C A DDITIONAL E XPERIMENTAL R ESULTS C. A NALYSIS OF THE NLP E NCODERS AND THEIR S IMILARITIES
Revised: [Table] C A DDITIONAL E XPERIMENTAL R ESULTS C. A NALYSIS OF THE NLP E NCODERS AND THEIR S IMILARITIES

Original:  As an alternative to  evaluate  similarity between  victim, stolen, and independent encoders, we use the cosine similarity score, as in (Dziedzic et al., 2022b).
Revised: [Figure] As an alternative to the DSI, which is ineffective in protecting our sentence embedding encoders, we evaluate the similarity between the victim, stolen, and independent encoders, we use the cosine similarity score, as in (Dziedzic et al., 2022b).

Original: Note that the cosine similarity score should not be used for the ownership resolution.
Revised: Note that the cosine similarity score should not be used for the ownership resolution.

Original: This is a very fragile metric and any deviation in the training of the stolen copy from the victim increases the distances between their representations.
Revised: This is a very fragile metric and any deviation in the training of the stolen copy from the victim increases the distances between their representations.

Original: As shown in Dziedzic et al.
Revised: As shown in Dziedzic et al.

Original: (2022b)
Revised: (2022b)

Original: , most obfuscations of stolen representations leave a stolen copy undetectable by the cosine similarity score.
Revised: , most obfuscations of stolen representations leave a stolen copy undetectable by the cosine similarity score.

Original: We depict the cosine similarity scores between our language encoders in Table 6.

Original: Note that for the pre-trained encoders, BERT* and RoBERTa* from Hugging Face, the cosine similarity scores to our independently trained encoders are close to zero. This might be caused by different factors, such as modifications to the code base or differences in the execution environment. Moreover, (Dziedzic et al., 2022b) showed that even a small obfuscation of the representations decreases the cosine similarity between two encoders substantially.
Revised: We track the cosine similarity between representations of the stolen and victim encoder over the course of stealing on 50 representations from the test dataset (Figure 2) and observe that similarity increases from the very beginning of stealing providing evidence of the highly accurate stealing process.

Original: Therefore, to evaluate a worst-case scenario (the exact data and training procedure of the victim are public and the independent encoder is trained in exactly the same way), we train BERT and RoBERTa-based encoders on nli twice and report the  cosine similarity scores between these two encoders,instead. The encoders trained independently on the same data and with the same procedure yield a similarity score close to one (color-coded in gray)—higher than the victim and its stolen copies. This indicates that the scores cannot be used for ownership resolution since an honest party who has trained an encoder using the same data and procedure as the victim would be falsely accused of theft. These results are expected.
Revised: Table 8 depicts the final cosine similarity scores between language encoders, stolen copies, and independent models after stealing is completed.

Original: In fact, standard DI cannot distinguish between a victim encoder and an independent encoder trained with the same procedure using the same data. Therefore, DI assumes that the victim’s training data is private, which, as we motivated above might not be a realistic assumption for transformers trained on public data.
Revised: We find that the stolen copies have higher scores than independent encoders trained on different datasets than the victim’s data. In Table 8, we also evaluate the cosine similarity of our encoders trained with a private seed against their stolen copies and independent encoders.

Original: This motivates our novel DSI.

Original: We evaluate the similarity of our encoders trained with private subsets against their stolen copies and independent encoders.
Revised: We evaluate the similarity of our encoders trained with private subsets against their stolen copies and independent encoders.

Original: We evaluate qqp and flickr in both combinations as public and private data and do not consider nli.
Revised: We evaluate qqp and flickr in both combinations as public and private data and do not consider nli.

Original: This is because the nli dataset contains SNLI, which, in turn, contains data from flickr which might corrupt our results.
Revised: This is because the nli dataset contains SNLI, which, in turn, contains data from flickr which might corrupt our results.

Original: Experimenting with the full flickr dataset inserted into qqp serves a fair comparison in terms of the fraction of subset data, given that qqp holds roughly 10x more data than flickr (2.6M vs. 300k).
Revised: Experimenting with the full flickr dataset inserted into qqp serves a fair comparison in terms of the fraction of subset data, given that qqp holds roughly 10x more data than flickr (2.6M vs. 300k).

Original: Our resultsin Table 6 highlight that inserting a large enough subset of flickr increases the cosine similarity between the qqp+flickr encoder to an independent flickr encoder while it decreases the similarity to an independent flickr encoder.
Revised: Our results highlight that inserting a large enough subset of flickr increases the cosine similarity between the qqp+flickr encoder to an independent flickr encoder while it decreases the similarity to an independent flickr encoder.

Original: The same holds for the qqp-subset.
Revised: The same holds for the qqp-subset.

Original: By inserting larger fractions of flickr, the qqp+flickr encoder’s representations also resemble more the ones of the nli encoder, which can be explained by nli containing some sentences from flickr.
Revised: By inserting larger fractions of flickr, the qqp+flickr encoder’s representations also resemble more the ones of the nli encoder, which can be explained by nli containing some sentences from flickr.

Original: For meaningful ownership resolution, the similarity between the victim and its stolen copies has to be compared to the similarity of the victim and an independent encoder trained on the same public dataset but without the private subset (color-coded in blue in Table 6).
Revised: For meaningful ownership resolution, the similarity between the victim and its stolen copies has to be compared to the similarity of the victim and an independent encoder trained on the same public dataset but without the private subset (color-coded in blue in Table 8).

Original: Our results show that if a victim is stolen with the private data subset or an independent dataset, the similarity to its stolen copies is higher than to the independent encoder.
Revised: Our results show that if a victim is stolen with the private data subset or an independent dataset, the similarity to its stolen copies is higher than to the independent encoder.

Original: However, the information on the private data subset is not transferred to the stolen copies when the stealing is done with the victim’s original training data distribution.
Revised: However, the information on the private data subset is not transferred to the stolen copies when the stealing is done with the victim’s original training data distribution.

Original: In a public setting, this is problematic since APIs reveal information about types of data used to pre-train and fine-tune their encoders.
Revised: In a public setting, this is problematic since APIs reveal information about types of data used to pre-train and fine-tune their encoders.

Revised: Note that for the pre-trained encoders, BERT* and RoBERTa* from Hugging Face, the cosine similarity scores to our independently trained encoders are close to zero.

Revised: This might be caused by different factors, such as modifications to the code base or differences in the execution environment.

Original:  Our experiments with DI, DSI and cosine similarity for language transformers, yield some additional interesting insights that need to  be taken into account when designing future defenses against stealing
Revised: Moreover, (Dziedzic et al., 2022b) showed that even a small obfuscation of the representations decreases the cosine similarity between two encoders substantially. Note that despite some inconsistencies in representations between BERT and RoBERTa from Hugging Face and our independently trained encoders, we are able to steal the encoders from Hugging Face. Our stolen copies do not only achieve high performance on benchmarks (Table 3) but also produce representations with high cosine similarity scores to the pre-trained encoders (Table 8). This highlights that while language transformers can be stolen effectively, the signal in their representations does not easily benefit ownership resolution.

Revised: Therefore, to evaluate a worst-case scenario (the exact data and training procedure of the victim are public and the independent encoder is trained in exactly the same way), we train BERT and

Revised: RoBERTa-based encoders on nli twice and report the cosine similarity scores between these two encoders, instead.

Revised: The encoders trained independently on the same data and with the same procedure yield a similarity score close to one (color-coded in gray)—higher than the victim and its stolen copies.

Revised: This indicates that the scores cannot be used for ownership resolution since an honest party who has trained an encoder using the same data and procedure as the victim would be falsely accused of theft.

Revised: These results are expected.

Revised: In fact, standard DI cannot distinguish between a victim encoder and an independent encoder trained with the same procedure using the same data.

Revised: Therefore, DI assumes that the victim’s training data is private, which, as we motivated above might not be a realistic assumption for transformers trained on public data.

Revised: This motivates our novel DSI.

Revised: Our experiments with DI, DSI and cosine similarity for language transformers, yield some additional interesting insights that need to be taken into account when designing future defenses against stealing

Original: NLP encoders.
Revised: NLP encoders.

Original: We observe that if the training and test sets of an encoder consist of semantically equal or similar sentences, DI is not even enable to mark a victim encoder.
Revised: We observe that if the training and test sets of an encoder consist of semantically equal or similar sentences, DI is not even enable to mark a victim encoder.

Original: This is caused since the encoders are trained to output similar/same embeddings for semantically equal sentences.
Revised: This is caused since the encoders are trained to output similar/same embeddings for semantically equal sentences.

Original: For example, when splitting the flickr dataset such that captions of the same image are distributed over the train and test sets, the distributions of the resulting representations are so similar that DI cannot extract any signal.
Revised: For example, when splitting the flickr dataset such that captions of the same image are distributed over the train and test sets, the distributions of the resulting representations are so similar that DI cannot extract any signal.

Original: Therefore, when attempting to identify stolen encoder copies in NLP, auditors need to be very careful not to accuse honest third parties of theft, when they hold semantically equal training data for their encoders than a victim encoder.
Revised: Therefore, when attempting to identify stolen encoder copies in NLP, auditors need to be very careful not to accuse honest third parties of theft, when they hold semantically equal training data for their encoders than a victim encoder.

Original: Furthermore, we observe that the data pre-processing and tokenization in NLP plays a significant role for DI.
Revised: Furthermore, we observe that the data pre-processing and tokenization in NLP plays a significant role for DI.

Original: While DI is able to mark victim encoders if their input data is batched and tokenized in exactly the same way, it is unable to do so when the data is tokenized differently ( e.g., with different padding, truncation, batching).
Revised: While DI is able to mark victim encoders if their input data is batched and tokenized in exactly the same way, it is unable to do so when the data is tokenized differently ( e.g., with different padding, truncation, batching).

Original: This observation is intuitive since DI relies on an exact training signature.
Revised: This observation is intuitive since DI relies on an exact training signature.

Original: In ownership resolution, this, furthermore, does not represent a limitation since the owner of the victim knows the exact pre-processing method used during their training / fine-tuning.
Revised: In ownership resolution, this, furthermore, does not represent a limitation since the owner of the victim knows the exact pre-processing method used during their training / fine-tuning.

Original: We also experimented with DI using only sentences above or below a certain length for ownership resolution at inference time.
Revised: We also experimented with DI using only sentences above or below a certain length for ownership resolution at inference time.

Original: This implements the intuition that some very short or long sentences might represent a more unique training signature, however neither
Revised: This implements the intuition that some very short or long sentences might represent a more unique training signature, however, neither

Original: DI nor DSI were able to successfully mark stolen copies.
Revised: DI nor DSI were able to successfully mark stolen copies.

Original: Finally, we observed that in nli (in contrast to qqp and flickr), DI was not even always able to mark the victim model.
Revised: Finally, we observed that in nli (in contrast to qqp and flickr), DI was not even always able to mark the victim model.

Original: Results varied broadly, depending on the data sampled from the training and test set to fit the GMMs and estimate likelihood.
Revised: Results varied broadly, depending on the data sampled from the training and test set to fit the GMMs and estimate likelihood.

Original: We assume that this is due to nli containing data from many different domains (the underlying MNLI contains text from, among others, fiction, telephone speech, and letters).
Revised: We assume that this is due to nli containing data from many different domains (the underlying MNLI contains text from, among others, fiction, telephone speech, and letters).

Original: Subsampling might, hence, lead to having data from completely different distributions in training and test data and, thereby, prevent DI. 
Revised: Subsampling might, hence, lead to having data from completely different distributions in training and test data and, thereby, prevent DI. C.  NLP [Equation]

Original: C. D ATASET INFERENCE VS D ISTRIBUTION I NFERENCE IN V ISION
Revised: K NOWN A RCHITECTURE BUT U NKNOWN C HECKPOINT

Revised: We observe that when stealing is started from a different checkpoint for the underlying encoder than the one used by the victim, the performance changes depending on the difference between the pre-trained checkpoints (first 3 rows in Table 10). For example, while the difference between using bert-base-uncased and bert-base-cased as the starting point for stealing is relatively small when the [Table]

Revised: [Table]

Revised: original model used bert-base-uncased, the performance difference when using bert-base-multilingualcase is larger.

Revised: Next, we show that the underlying encoder leaves distinct traces on the fine-tuned sentence embedding model.

Revised: We analyze the performance of different checkpoints only after a single epoch of finetuning during which we use the stolen embeddings (last 3 rows in Table 10).

Revised: This computationally inexpensive step allows attackers to quickly identify the best-performing checkpoint that they have access to and continue the fine-tuning only for the best checkpoint.

Revised: Note that evaluating different model checkpoint initializations does not require obtaining additional representations from the victim model.

Revised: Instead, the obtained representations can be reused over all checkpoints.

Revised: C. D ATASET INFERENCE VS D ISTRIBUTION I NFERENCE IN V ISION

Original: Table 7 show that DI is a robust detection method.
Revised: Table 11 show that DI is a robust detection method.

Original: We train two ResNet models on the equal partitions of the CIFAR10 train set.
Revised: We train two ResNet models on the equal partitions of the CIFAR10 train set.

Original: The result suggests that we are able to differentiate between two such encoders as being trained independently.
Revised: The result suggests that we are able to differentiate between two such encoders as being trained independently.

Original: We also run the same experiments for the original version of dataset models, where we train two classifiers on the equal splits of the CIFAR10 train set.
Revised: We also run the same experiments for the original version of dataset models, where we train two classifiers on the equal splits of the CIFAR10 train set.

Original: We observe that both methods are robust in this setting.
Revised: We observe that both methods are robust in this setting.

Original: To apply GMMs to supervised models, we discard the last classification layer and use the representations from the last but one layer to distinguish between these two classifiers using GMMs.
Revised: To apply GMMs to supervised models, we discard the last classification layer and use the representations from the last but one layer to distinguish between these two classifiers using GMMs.

Original: This shows that the dataset inference is not only a distribution inference but the intended dataset inference.
Revised: This shows that the dataset inference is not only a distribution inference but the intended dataset inference.

Original: We can use the method based on
Revised: We can use the method based on

Original: GMMs when the public API exposes access to the softmax values for the supervised models.
Revised: GMMs when the public API exposes access to the softmax values for the supervised models.

Original: [Table]
Revised: [Table]

Original: (denoted as od), and evaluate them on different downstream tasks: CIFAR10, SVHN.

Original: [Table]
Revised: [Table]

Original: C. I NFLUENCE OF DIFFERENT PROJECTION HEAD SETTINGS
Revised: C. I NFLUENCE OF DIFFERENT PROJECTION HEAD SETTINGS

Original: We investigate the effect of the depth and the final-layer dimension of DINO’s projection head on the model’s performance on downstream tasks.
Revised: We investigate the effect of the depth and the final-layer dimension of DINO’s projection head on the model’s performance on downstream tasks.

Original: Table 8 shows that having a deeper MLP is crucial to the model’s quality
Revised: Table 13 shows that having a deeper MLP is crucial to the model’s quality.

Original: . C. M ORE EXPERIMENTS ON STEALING V I T S MALL
Revised: C. M ORE EXPERIMENTS ON STEALING V I T S MALL

Original: Table 9 shows the performance of stolen models using different datasets, query numbers and loss functions.
Revised: Table 14 shows the performance of stolen models using different datasets, query numbers and loss functions.

Original: The result suggests that using the same dataset as the pretrained dataset gives the best result.
Revised: The result suggests that using the same dataset as the pretrained dataset gives the best result . C. N UMBER OF P ARAMETERS

Revised: We count the number of parameters for the base encoders and their corresponding heads and show the results in Table 15.

Revised: The ViT-tiny has much fewer parameters than even the single projection head.

Revised: ViT-small has a smaller number of parameters than the full projection head used in DINO.

Revised: Only the

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: ViT-base has a larger number of parameters than the full projection head from DINO.

Revised: The number of parameters in the projection heads is different for various encoder types due to their different dimensionality of output representations.

Revised: C. F INE - TUNING

Revised: In Table Table 16, we present the results after fine-tuning with the SVHN dataset on ViT Small/ pre-trained on ImageNet.

Revised: C. M IX M ATCH FOR SVHN

Revised: We present the results on applying the MixMatch semi-supervised learning to the SVHN dataset in Table 17.

Revised: [Table]

Revised: [Table]

Revised: [Table]