Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders
Authors: Jan Dubiński
Stanisław Pawlak
Franziska Boenisch
Tomasz Trzcinski
Adam Dziedzic
Venue: NeurIPS 2023 poster (2023)
Keywords: model stealing
model defenses
self-supervised learning
TL;DR: We propose the first active defense against encoder extraction that transforms outputs without compromising representation quality for legitimate API users.

Abstract: Machine Learning as a Service (MLaaS) APIs provide ready-to-use and high-utility encoders that generate vector representations for given inputs. Since these encoders are very costly to train, they become lucrative targets for model stealing attacks during which an adversary leverages query access to the API to replicate the encoder locally at a fraction of the original training costs. We propose *Bucks for Buckets (B4B)*, the first *active defense* that prevents stealing while the attack is happening without degrading representation quality for legitimate API users. Our defense relies on the observation that the representations returned to adversaries who try to steal the encoder's functionality cover a significantly larger fraction of the embedding space than representations of legitimate users who utilize the encoder to solve a particular downstream task. B4B leverages this to adaptively adjust the utility of the returned representations according to a user's coverage of the embedding space. To prevent adaptive adversaries from eluding our defense by simply creating multiple user accounts (sybils), B4B also individually transforms each user's representations. This prevents the adversary from directly aggregating representations over multiple accounts to create their stolen encoder copy. Our active defense opens a new path towards securely sharing and democratizing encoders over public APIs.

Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders
Jan Dubiński 1,2+f
Stanislaw Pawlak
Franziska Boenisch 4t
Tomasz Trzciński 1,2,3 Adam Dziedzic 4+3
Warsaw University of Technology VDEEA NCBR *Tooploox
CISPA Helmholtz Center for Information Security
Abstract
Machine Learning as Service (MLaaS) APIs provide ready-to-use and high- utility encoders that generate vector representations for given imputs. Since these encoders are very costly to train, they become lucrative targets for model stealing attacks during which an adversary leverages query access to the API to replicate the encoder locally at fraction of the original training costs. We propose Bucks for Buckets (B4B), the first active defense that prevents stealing while the attack is happening without degrading representation quality for legitimate API users. Our defense relies on the observation that the representations returned to adversaries who try to steal the encoder's functionality cover significantly larger fraction the embedding space than representations of legitimate users who utilize the encoder solve particular downstream task. B4B leverages this to adaptively adjust the utility of the returned representations according to a user's 's coverage the embedding space. To prevent adaptive adversaries from eluding simply creating multiple user accounts (sybils), B4B also individually transforms each user's representations. This prevents the adversary from directly aggregating representations over multiple accounts to create their stolen encoder copy. Our active defense opens a new path towards securely sharing and democratizing encoders over public APIs 5
defense by
Introduction
In model stealing attacks, adversaries extract a machine learning model exposed via a public API by repeatedly querying it and updating their own stolen copy based on the obtained responses. Model stealing was shown to be one of the main threats to the security of machine learning models in prac- tice (38. Also in research, since the introduction of the first extraction attack against classifiers 40
lot of work on improving stealing (27 33 40 41, extending it to different model types [8] 37
has been put forward. With the recent shift in learning paradigms from supervised to self supervised learning (SSL), especially the need for new defenses becomes increasingly pressing. From an academic viewpoint, the urge arises because was shown that SSL models (encoders) are even more vulnerable to model stealing [16] 29 36 than their supervised counterparts. This is because whereas supervised models' output is low dimensional, e,g., per-class probabilities vectors that encode a larger amount of information and thereby facilitate stealing. In addition, from practical industry's 's viewpoint, defenses required since many popular API providers, such as Cohere, OpenAl, or Clarify [ already expose their high-value SSL encoders via APIs to a broad range of users.
and proposing adequate defenses 18
25
26
pure labels, SSL encoders output high-dimensional representation
Corresponding authors: jan.dubinskiddkkk@@....d. and walamddriexziceeiss....
I "quual contribution. Project Lead.
Code available http:///ss//tttto//////////////a
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

COST FUNCTION
TRANGFORMATIONG
DEFENSE FRANEWORK
Pad + Shuffle
DEFENSE INSTANTIATION
Affine Pad +Shuftle
Emitediding cooupled
X Binary
101
API call
LOCAL SENSITIVE HASHING
ADAPTIVE NOSING
TRANSFORMATONS
Figure I: Overview of B4B. In the upper part, we present our B4B framework that consists of three modular building blocks: (1) A coverage estimation to track the fraction of embedding space covered by the representations returned each user, (2) cost function that serves to map the coverage to concrete penalty to prevent stealing, and (3) per-user transformations that are applied to the returned representations prevent sybil attacks. In the lower part, we present concrete instantiation of B4B and the operation flow of our defense: The API calculates representations for the incoming queries. We instantiate the coverage estimation with local sensitive hashing and estimate the covered space as the fraction of hash buckets occupied. We calibrate the costs by adding noise to the representations according to the coverage. We apply a set of transformations on a per-user basis. The noised and transformed representations are returned to the user.
Most of the current defenses against encoder stealing are reactive, i.e., they do not actively prevent the stealing but rather aim at detecting it by adding watermarks to the encoder [14] or performing dataset inference identify stolen copies [7]. Since the point of detection, the damage of stealing has already been inflicted, we argue that reactive defenses intervene too late and we advocate for active defenses that prevent stealing while it happening. Yet. active defenses are challenging to implement because they not only need to prevent stealing but also should preserve the utility of representations for legitimate users. The only existing active defense against encoder stealing [29] falls short on this latter aspect since it significantly degrades the quality of representations for all users.
To close the gap between required and existing defenses, we propose Bucks for Buckets (B4B), the first active defense against encoder stealing that does harm utility for legitimate users. B4B leverages the observation that the representations returned to adversaries who try to steal the encoder's functionality cover a significantly larger fraction of the full embedding space than representations of legitimate users who utilize the encoder to solve a particular downstream task. To turn this observation into a practical defense, B4B is equipped with three modular building blocks: (1) The first building block is a tracking mechanism that continuously estimates the fraction of the embedding space covered by the representations returned to each user. The intuition why this relevant that by covering large fractions of the embedding space, the representations will suffice for an adversary to reproduce the encoder's functionality, i.e., successfully steal it. (2) B4B's second building block consists of cost function to translate the covered fraction of the embedding space into concrete penalty. We require this cost function to significantly penalize adversaries trying to steal the model while having only a minimal effect legitimate users. (3) The third building block contains transformations that can be applied to the representations on a per-user basis to prevent adaptive attackers from circumventing their queries over these accounts such that they minimize the overall cost. We present the different building blocks of B4B in Figure
defense by creating multiple user accounts (sybils) and distributing
While B4B's modularity enables different instantiations of the three building blocks, we propose a concrete end-to-ent instantiation to showcase the practicability of our approach. To implement tracking of the covered embedding space, we employ local sensitive hashing that maps any represen- tation returned to given user into a set of hash buckets. We base our cost function (i.e., the "bucks") on utility and make B4B add noise to the representations with a magnitude that increases with the number of buckets occupied by the given user. While the scale of noise added to legitimate users'

---

representations does not harm their downstream performance due to their small embedding space coverage, the representations returned to an adversary become increasingly degrading the performance of their stolen encoder. Finally, we rely on set of transformations (e.g. affine transformations, shuffling, padding) that preserve downstream utility [77. While, as a consequence, legitimate users remain unafeected by these transformations, adversaries cannot directly combine the representations obtained through different sybil accounts anymore to train their stolen copy of the encoder. Instead, they first have to remap all representations into the same embedding space, which we show causes both query and computation overhead and still reduces the performance of the stolen encoder.
mnisy-sinificantle
In summary, we make the following contributions:
1. We present B4B, the first active defense against encoder stealing that does not harm legitimate users' downstream performance. B4B's s three building blocks enable penalizing adversaries whose returned representations cover large fractions of the embedding space and prevent sybil attacks.
2. We propose a concrete instantiation of B4B that relies on local sensitive hashing and decreases the quality of representations returned user once their representations fill too many hash buckets.
3. We provide end-to-end evaluation of our defense to highlight its effectiveness in offering high utility representations for legitimate users and degrading the performance of stolen encoders in both the single and the sybil-accounts setup.
2 Related Work
Model Extraction Attacks. The goal of the model extraction attacks is to replicate the functionality of victim model fo trained on a dataset D,. An attacker has a black box access to the victim model and uses stealing dataset D, = {g:, fu [4.)] }= consisting of queries q: and the corresponding outputs (g:):) returned by the victim model, train stolen model fs. Model extraction attacks
have been shown against various types of models including classifiers 24 40
and encoders [66] 36
Self Supervised Learning and Encoders. SSL is increasingly popular machine learning paradigm. It trains encoder models to generate representations from complex inputs without relying on explicit labels. These representations encode useful features of a given input, enabling efficient learning for multiple downstream tasks. Many SSL frameworks have been proposed [9] 10 12 22
23 44
In our work, we focus the two popular SSL vision encoders, namely SimSiam [12 and
DINO which return high-qualit representations that achieve state-of-the- performance on downstream tasks when assessed by training a linear classifier directly on representations. SimSiam trains with two Siamese encoders with directly shared weights. A prediction MLP head is applied to one of the encoders f1, and the other encoder fz has a gradient, where both operations are used for avoiding collapsing solutions. In contrast, DINO shares only architecture (not weights) between a student fi and teacher model fz. also with the stop-gradien operation, but not the prediction head. While SimSiam uses convolutional neural networks (CNNs), DINO also employs vision transformers (ViTs). Both frameworks use a symmetrized loss of the form 5 (2(x)) +
(f (22), f2(1)) in their optimization objectives. where g(.,.) .) is negative cosine similarity for SimSiam and cross-entropy for DINO. SimSiam and DINO's 's similarities and differences demonstrate our method's broad applicability across SSL frameworks. More details can be found in Appendix E
Stealing Encoders. The stealing of SSL encoders was shown to be extremely effective [6 29
36
The goal of extracting encoders is to maximize the similarity of the outputs from the stolen local copy and the original representations output by the victim encoder. Therefore, while training the stolen copy, the adversary either imitates a self-supervised training using a contrastive loss function, e.g., InfoNCE [10) or SoftNN (21) or directly matches both models' representations via the Mean Squared Error (MSE) loss. To reduce the number of queries sent to the victim encoder, the attack proposed in (29 leverages the key observation that the victim encoder returns similar representations for any image and its augmented versions. Therefore, a given image can be sent to the victim while the stolen copy is trained using many augmentations of this image, where the representation of given augmented image is approximated as the one of the original image produced by the victim encoder.
methods have been proposed to detect stolen encoders [4 16 43). Many of these approaches use downstream tasks to check if watermark embedded into victim encoder is present in a suspect encoder. Dataset inference (30) is another type
Defending Encoders. Recently, watermarking 7 25 42
3

---

of encoder ownership resolution. It uses the victim's 's training dataset unique signature, leveraging the following observation: for victim encoder trained on its private data as well for its stolen copies, the distribution of the representations generated from the victim's training data differs from the distribution of the representations generated on the test data. In contrast, for an independently trained encoder, these two distributions cannot be distinguished, allowing the detection of stolen copies [17] However, all the previous methods are reactive and aim at detecting the stolen encoder instead of actively preventing the attack. The only preliminary active defenses for encoders were proposed by [16] 29). They either perturb or truncate the answers to poison the training objective of an attacker. These operations were shown to harm substantially the performance of legitimate users, which renders the defense impractical. In contrast, our B4B has negligible impact on the quality of representations returned to legitimate users.
3 Actively Defending against Model Stealing with B4B
B4B aims at actively preventing model stealing while preserving high-utility representations for legitimate users. Before introducing the three main building blocks of B4B, namely (1) the estimation of embedding space coverage, (2) the cost function, and (3) the transformation of representations (see Figure we detail our threat model and the observation on embedding space coverage that represents the intuition behind our approach.
3.1 Threat Model and Intuition
Our setup and the resulting threat model are inspired by public APIs, such as Cohere, OpenAI, or Clarify [1-3] that expose encoders to users through a pre-defined interface. These encoders are trained using SSL on large amounts of unlabeled data, often crawled from the internet, and therefore from diverse distributions. We notice that to provide rich representations to multiple users, the training dataset of the encoder needs to be significantly more diverse than the individual downstream tasks that the users query for representations. For instance, the encoder behind the API is trained on the ImageNet dataset, then the legitimate users are expected query the API for downstream tasks, such as CIFARIO 10 or SVHN. Similarly, if the encoder is trained on CIFAR1O, the expected downstream tasks are MNIST Fashion MNIST. Yet, in the design of our defense, consider adversaries who can query the encoder with arbitrary inputs to obtain high-dimensional representation vectors from the encoder. Our defense is independent of the protected encoder's architecture and does not rely on any assumption about the adversary's 's data and query strategy.
We argue that even in this restricted setup, our defense can distinguish between adversaries and legitimate users by ana- lyzing the distribution of representations returned to them. In Figure 2 by using PCA to project representations for different datasets to two-dimensional space, we visualize that repre- sentations for different downstream tasks cluster in disjoint and small sub-spaces of the full embedding space. The repre- sentations were obtained from a SimSiam encoder originally trained on ImageNet (we observe similar clustering for DINO shown in Appendix F). As a result, legitimate users can be characterized by their representations' small coverage of the embedding space. In contrast, the adversary does not aim at solving a particular downstream task. They instead would want to obtain representations that cover large fractions of the embedding space. This enables reproducing the overall functionality of the encoder (instead of only learning some local task- ssskispeiiiic behavior). Indeed, has been empirically shown by prior work, such as [16]. that stealing with multiple distributions, e.g., by relying on the complex ImageNet dataset, yields higher performance of the stolen encoder on various downstream tasks than stealing with downstream dataset, such as CI- FARIO. As result, intuitively, we can identify and penalize adversaries based on their coverage of the embedding space, which will be significantly larger than the coverage of legitimate users. We leverage this intuition to build our B4B defense and present our three main building blocks in the following sections.
0.10
0.05
0.00
-0.0
-0.05
0.00
0.05
0.10
Figure 2: Representations from Different Tasks Occupy Differ- ent Sub-Spaces of the Embedding Space. Presented for Fashion- MNIST, SVHN CIFARIO, and STL10.

---

3.2 Building Block 1: Coverage Estimation of the Embedding Space
The first building block of our B4B serves to estimate and continuously keep track of the fraction of the embedding space occupied by any given user. Let & denote our embedding space of dimension further, let U/ be a user with query dataset D 91: : In DD and let fo D - E be our protected victim encoder that maps data points from the input to the embedding space. Assume user U has, so far, queried a subset of their data points ạ1 ς1....; Dj with j S n to the encoder and obtained the representations ri.....- with each ra E R We aim to estimate the true fraction of the embedding space Ey that is covered by all returned representations ?......?? F to user U and denote our estimate by ey.
Local Sensitive Hashing. One of the methods to approximate the occupied space by representations returned to given user is via Local Sensitive Hashing (L.HH) [39]. We rely on this approach for the concrete instantiation of our B4B and use it to track per-user coverage of the embedding space. Standard (cryptographic) hash functions are characterized by high dispersion such that hash collisions are minimized. In contrast, LSH hashes similar data points into the same or proximal, so-called hash buckets. This functionality is desired when dealing with searches in high-dimensional spaces or with a large number of data points. Formally, an LSH function H is defined for a metric space M = (M, d). where d is a distance metric in space M, with a given threshold T > O, approximation factors f > 1, and probabilities P1 and Pz. where P1 > P2. H maps elements of the metric space to buckets b E B and satisfies the following conditions for any two points Qı 92 E M: (1) If d(gi 92) S T. then H(g)) = H(42) (i.e., Qi and d2 collide the same bucket b) with probability at least P1. (2) d(n,, (2) 2 JT, then H(g)) = H(22) with probability at most P2.
3.3 Building Block 2: Cost Function Design
Once we can estimate the coverage of an embedding space for a given user U as E we need to design a cost function C R+ + RR that maps from the estimated coverage to a cost. The cost function needs to be designed such that it does not significantly penalize legitimate users while imposing severe penalty on adversaries to effectively prevent the encoder from being stolen. The semantics of the cost function's 's range depend on the type of costs that the defender wants to enforce. We discuss a broad range of options in Appendix C These include monetary cost functions to adaptively charge users on a batch-query basis depending on their current coverage, costs in the form of additional computation that users need to perform in order to obtain their representations, similar to the proof of work in 18, costs in the form of delay in the interaction with the encoder behind the API 4 or
costs in form of disk space that needs to be reserved by the user (similar to proof of space [19] 20). Which type of cost function is most adequate depends on the defender's objective and setup.
Exponential Cost Functions to Adjust Utility of Representations. In the concrete instantiation of B4B that present in this work, we rely on costs in the form of the utility of the returned representations. We choose this concrete instantiation because it is intuitive, effective, and can be directly experimentally assessed. Moreover, is even suitable for public APIs where, for example, no monetary costs are applicable. We adjust utility by adding Gaussian noise with different standard deviation o the returned representations. Since do not want penalize legitimate users with small coverage but make stealing for adversaries with growing coverage increasingly prohibitive, we instantiate an exponential cost function that maps from the fraction of hash buckets occupied by the user to value for a. We choose the general form of this function as
(E)) = x (exp In
x=1xxxxx1111 -1)
xB
(I)
where x < 1 compresses the curve of the function to obtain low function values for small fractions of occupied buckets, and then we set a target penalty a for our cost function at a specified fraction of filled buckets B. For instance, if we want to enforce of 5 at 90% of filled buckets (i.e., for s0 = 0.9), we would need to set = 5 and B = 0.9. E/
3.4 Building Block 3: Per-User Representation Transformations against Sybil Attacks
Given that our defense discourages users from querying with wide variety of data points from different distributions, adversary could create multiple fake user accounts (sybils) and query
5

---

different data subsets with more uniform representations from each of these accounts. By combining all the obtained representations and using them to train stolen copy, the adversary could overcome the increased costs of stealing. To defend against such sybil attacks, we propose individually transforming the representations on a per-user level before returning them. As a result, the adversary would first have to map all the representations to one single unified space before being able to jointly leverage the representations from different accounts for their stolen copy.
Formally, for a given query Qi the protected victim encoder produces a representation r; (q)) which is transformed by a user-specific transformation Ty (r:) before being returned to the querying user U. For a new user U, the defense randomly selects the transformation Ty from all possible choices. Note that the randomness is also added on per- per-traasformaiion basis, instead of only on the level of selecting the transformations. For example, permutation of the elements in the output representations should be different for each user.
We formulate two concrete requirements for the transformations. First, they should preserve utility for legitimate users on their downstream tasks, and second, they should be costly reverse for an adversary.
Utility Preserving Transformations. As a concrete instantiation for our B4B, we propose a set of transformations that fulfill the above-mentioned two requirements: (1) Affine where we apply affine transformations to representations, (2) Pad where we pad representations with constant values, (3) Add where we add constant values at random positions within representations, (4) Shuffle where we shuffle the elements in the representation vectors, and (5) Binary where the original representations are mapped binary vectors relying on random partitioning of the representation space. To preserve the full amount of information contained in the original representations, in our binary transformations, we tune the length of binary representations. We visualize the operation of each of these transformations in Appendix C All these transformations can additionally be combined with each other, which further increases the possible set of transformations applied per user. This renders it impossible for an adversary to correctly guess and reverse the applied representation. Instead, the adversary has to remap the representations over all accounts into a single embedding space order to unify them and leverage them for training of their stolen encoder copy. We present an ex exhaustive list of strategies that adversaries can apply for the remapping in Appendix [ All the attack methods reduce to the minimum of remapping between representations of pair of users, i.e., they are at least as complex as mapping between two separate accounts. In the next section, we show that our defense already impedes stealing for an adversary with two accounts.
Empirical Evaluation
We first empirically evaluate our instantiation of B4B's three building blocks and show how calibrate each of them for our defense. Finally, we provide an end-to-end evaluation that highlights B4B's effectiveness in preserving downstream utility for legitimate users while successfully preventing the stealing by adversaries.
Experimental Setup. We conduct experiments on various kinds of downstream tasks and two popular SSL encoders. To test our defense, we FashionMNIST, SVHN, STL10, and CIFAR 10 as our downstream datasets, each with standard train and test splits. For stealing, we utilize training data from ImageNet and LAION-SB. We rely on encoder models from the SimSiam [22 and the DINO [9] SSL frameworks. As our victim encoders, we use the publicly available ResNet50 model from SimSiam trained for 100 epochs on ImageNet and the VIT Small DINO encoder trained for 800 epochs on ImageNet, each using batch size 256. The VIT architecture takes as input a grid of oveellerlg contiguous image patches of resolution NxN. In this paper, we typically use N = 16. The Simsiam encoder has an output representation dimension of 2048, while DINO returns 1536 dimensional representations. We examine the utility downstream classifiers using SimSiam's or DINO's 's representations obtained for the respective downstream datasets. To implement LSH, we rely on random projections 34 that we implement from scratch. For detailed description of our stcaling and downstream training parameters, we refer to Appendix F

---

100
FasHionMNiSS
Queries per bucket.
Buckots occupied
SVHN
10a
SVHN
CIFARIO
imageNet
102
ImageNet
40
20
10000
20000
30000
40000
50000
10
50
Queries
Buckets [%]
Fraction of Occupied Buckets.
(b) Number of Queries per Bucket.
Figure 3: Estimating Embedding Space Coverage through LSH SimSiam Encoder. We present the fraction buckets occupied by representations of different datasets as a function of the number of queries posed to the encoder (left). We observe that representations for the downstream datasets (FaShionMNIST, SVHN, CIFAR10) occupy a smaller fraction of buckets than representations from the complex ImageNet dataset. Our evaluation of the number of queries whose representations mapped to the same bucket (right) indicates that our total number of buckets (2 well calibrated for the estimation of covered representation space: i.e., queries whose representations are mapped to the same buckets. This indicates that our LSH is capable of representing similarities the representations.
all datasets, experience hash collisions,
4.1 Local Sensitive Hashing for Coverage Estimation
We first observe that the choice of the total number of hash buckets in the LSH influences the effectiveness of our method. In the extreme, we have a too large number of buckets, the number of buckets filled will correspond to the number of queries posed by user which fails to capture that similar representations cover similar sub-spaces of the embedding space, and hence does not serve to approximate the total fraction of the embedding space covered. However, if have too few buckets, even the representations for simple downstream tasks will fill large fractions of buckets, making impossible calibrate the cost function such that only penalizes adversaries. We experimentally find that for our evaluated encoders, 2 buckets represent a good trade-off. In Appendix F.6 we present an ablation study the effect of the number of total buckets.
Our evaluation of the LSH to track coverage of embedding space is presented in Figure 3 We observe that representations returned for standard downstream tasks (FashionMNIST,, SVHN, CIFARIO) occupy a significantly smaller fraction the total number of buckets than complex data from multiple distributions (ImageNet, LAION-5B). We present additional experimental results measuring the coverage of the representation space in Appendix F.3 Specifically, we show that our method of measuring the embedding space coverage has broad applicability across various encoders and datasets used for pretraining. We further observe that the fraction of buckets occupied by the representations saturates over time. These findings highlight that LSH is successful in capturing the differences between legitimate users and adversaries-eval in low-query regime. Finally, we note that total number of buckets (212) is well calibrated since, all datasets, it successfully maps multiple representations to the same hash bucket while still filling various fractions of the total number buckets.
4.2 Calibrating the Cost Function
We experiment with different sets hyperparameters to instantiate the cost function from Equation (I) in the previous section (3.3) As described there, we can calibrate the function (as shown in Figure 4 such that a desired penalty (in the form of specific σ) will be assigned at a certain fraction of buckets occupied. For B4B, we aim at penalizing high embedding space coverage severely. Therefore, we need to identify and optimize for two components: which value of a leads to significant performance drops, and 2) for what
Stdee: of added notse
Aeein
fexini
(eeutt))
044
0.0
20
Buckets [%)
Figure 4: Cost Function Calibration.
7

---

fraction of coverage do we want to impose this significant drop. We base both components on empirical observations. Our first observation is that for our four downstream tasks (FaShionMNIST. SVHN, STLIO, and CIFARIO) 10), performance drops to 10% (i.e., random guessing) at roughly = 0.5. we further see that with 50k queries, the downstream tasks occupy < 30% of the buckets. Ultimately, y, setting and | are design choices that an API provider needs to make in order to specify what type of query behavior they want to penalize. As very loose bounds (weak defense), based on our observation, we consider = as high penalty, which leads to = 1, and select 3 = 0.8. This 33 corresponds to considering 80% of buckets filled as a too-large coverage of the embedding space. We empirically observe that coverage of 80% of buckets occurs, for example, after around 100k of ImageNet queries. By choosing our target 3 so loose, i.e., significantly larger than the observed 30% for downstream tasks, offer flexibility for the API to also provide good representations for more complex downstream tasks. Finally, to obtain a flat cost curve close to the origin-which serves to map small fractions of covered buckets small costs-wa we find that we can set À 10-6, In the Appendix, we evaluate our defense end-to-end with differently parameterized cost functions.
In Figure 3
4.3 Assessing the Effect of Transformations
Transformations Do Not Harm Utility for Legitimate Users. We evaluate the downstream accuracy for transformed representations based on training a linear classifier on top of them. To separate the effect of the noise added by our defense from the effect of the transformations, perform the experiments in this subsection without adding noise to the returned representations. For example, the CIFARIO 10 dataset and a SimSiam encoder pre-trained on ImageNet, without any transformations applied, we obtain a downstream accuracy of 90.41 % (# 0.02), while, with transformations, we obtain 90.24% (+ 0.11) for Affine, 90.40% (# 0.05) for Shu+-huffle, 90.18% (#: 0.06) for SffnffPad+Shuffle, and 88. 78% (# 0.2) for Binary. This highlights that the transformations preserve utility for legitimate users. This holds over all datasets we evaluate as we show in Appendix F
Adversaries Cannot Perfectly Remap Representations over Multiple Sybil Accounts. To under-
stand the impact of our per-user account transformations sybil-attack based encoder stealing, we evaluate the difficulty of remapping representations between different sybil accounts. For simplicity, and since we established in Section 3.4 that multi-account attacks reduce two-account setup, assume an adversary who queries from two sybil accounts and aims at learning to map the transformed representations from account #2 to the representation space of account #1. Using more accounts for the adversary causes a larger query overhead and potentially more performance loss from remapping. Our evaluation here, hence, represents lower bound the overhead caused to the adversary through our transformations.
We learn the mapping between different accounts" representations by training linear model on overlapping representations between the accounts. We assess the fidelity of remapped represen- tations a function of the number of overlapping queries between the accounts. As fidelity metric for our remapping, we compare the cosine distance between representations (a and b de- a'b fined as: 1 - T. Once the remapping is trained, we evaluate by querying 10k data points from the test dataset through account #1 and then again through account #2. Then, we apply the learned remapping to the latter one and compute the pairwise cosine distances be- tween the representations from account #1 and their remapped counterparts from account #2. Our results are depicted in Figure [S] We show that the largest cosine distance is achieved with the binary trans- formations, making them the most protective against the adversary since they best prevent perfect remapping, even with an overlap of as many as 10k queries between both accounts. However, these binary transformations also in- cur the highest drop in accuracy for legitimate users. The defender has the possibility of selecting their preferred types of transformations between representations taking into account the trade-offs between the effectiveness of the defense and the negative impact on legitimate users.
ums-
Quarri
Figure 5: Quality of Remappings.
8

---

Table 1: Stealing and Using Encoders With and Without our Defense. The USER column represents the type of the APIs' user, where LEGIT denotes a legitimate user, ATTACKER stands for a standard single-account adversary, and SYBII. represents an adversary using two sybil accounts. We use InfoNCE loss for encoder extraction. # Queries stands for the number of queries used for stealing with ALL denoting that the entire downstream dataset was used. The TYPE column expresses how the dataset is used. We follow the stealing setup from [17.. In the first row, present the undefended victim encoder's performance as the accuracy for downstream tasks trained on the encoder's 's returned representations. In the following row, show downstream utility for legitimate users when the victim encoder is defended by our B4B. Finally, (in the remaining rows) we assess the performance of stolen encoders the downstream tasks. Our results highlight that while the performance of the encoder for legitimate users stays high, our B4B renders stealing inefficient with the stolen encoders obtaining significantly worse performance on downstream tasks.
USER
DEFENSE QUERIES DATASET
TYPE
CIFARIO
STL10
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.41 40.02
95.08=0.13
75.47
91.222t.11
LEGIT
B4B
ALL
TASK
QUERY
90.24.0.11
95.05
74.96=0.13
91.7=001
ATTACK
NONE
50K
IMGNET
STEAL
65.220.03
64.9
63.1
88.5 20.01
ATTACK
B4B
50K
IMGNET
STEAL
35.72
2..54
70.01 19.74
ATTACK
NONE
100K
IMGNET
STEAL
68.1 4:003
63.1 =0.01
61.5 40.01
89.0 40.07
ATTACK
B4B
100K
IMGNET
STEAL
12.01
13.94
19.96=0.03
69.63
ATTACK
NONE
100K
LAION
STEAL
64.92=0.03
62.51 10:03
59.0220.02
84.54
ATTACK
B4B
100K
LAION
STEAL
40.96 0.03
40.69 0.05
34.43=0.11
72.92=
SYBIL
B4B
2x50K
IMGNET
STEAL
39.5620.06
38.50=0.04
77.01 23.41
SYBIL
B4B
3x33.3K
IMGNET
STEAL
0.88
38.57
21.16=0001
72.95
SYBIL
B4B
4x25k
IMGNET
STEAL
33.98
34.52
70.7 21.21
0.05
SYBIL
B4B
5x20K
IMGNET
STEAL
32.65 0.05
32.45
29.63=0.01
70. 70.122:10 0.08
SYBIL
B4B
6x16.7K
IMGNET
STEAL
26.62: 0.04
26.85
24.32::.02
70.5 11.511:1 0.01
4.4 End-to-End Stealing of an Encoder under our Defense
We perform an end-to-end study to showcase how our B4B defense affects legitimate users vs adversaries. The hyperparameters for B4B are chosen according to the empirical evaluation of the previous sections with 21 as the number of buckets, a 1, = 0.8, A = 10-6 as the hyperparameter of the cost function, and different random affine transformations per-user account. Our main results are presented in Table 1 We observe that instantiating our framework with B4B has negligible impact on legitimate users while substantially lowering the performance of stolen encoders in the case of single-user and sybil attackers.
Legitimate Users. We compare the accuracy of downstream classifiers trained on top of unprotected vs defended encoders. The victim encoder achieves high accuracy on the downstream tasks when no defense is employed. With B4B in place, we observe that across all the downstream tasks, the drop in performance is below 1 1%. For example, there is only slight decrease in the accuracy of CIFARIO from 90.4 90.41 that their downstream representations cover a relatively small part of the representations space. This results in a very low amount of noise added to their representations which preserves performance.
to 90.24=0.11%% B4B's small effect on legitimate users stems from the fact
Adversaries. For adversaries who create a stolen copy of the victim encoder, we make two main observations. The most crucial one is that when our B4B in place, the performance of the stolen copies over all downstream tasks significantly drops in comparison to when the victim encoder unprotected (grey rows Table [p. This highlights that our B4B effectively prevents stealing. Our next key observation concerns the number of stealing queries used by the adversary: When no defense applied, the more queries are issued against the API (e.g.. 100k instead of 50k), the higher performance of the stolen encoder on downstream tasks (e.g.. CIFARIO or FashionMNiST).. In contrast, with B4B implemented as a defense, the performance decreases when using more stealing queries from a single account. This is because with more queries issued, the coverage of embedding space grows which renders the returned representations increasingly noisy and harms stcaling performance.

---

Moreover, we show that B4B can also prevent model stealing attacks with data from different distribution than the victim encoder's training set. We highlight this Table where we also use the LAION-SB dataset to steal an ImageNet pre-trained encoder. Our results highlight first that without any defense place, the LAION dataset highly effective to extract the ImageNet pre-trained encoder. Second, B4B effectively defends against such attacks, and yields significant drop in downstream accuracy (on average above 20%) of the stolen encoder.
We also show that this performance drop cannot be prevented by sybil attacks. Therefore, we first consider an adversary who queries from two sybil accounts with 50k queries issued per account and the first 10k queries of both accounts used to learn the remapping of representations between them. When the adversary trains their stolen encoder copy on all the remapped representations, they increase downstream performance over querying from a single account. Yet, their performance is still significantly smaller than the performance of the victim encoder for legitimate users, or the encoder stolen from an undefended victim. Moreover, using more than two sybil accounts further reduces the attack performance as remapping complications accumulate. With ten sybils, remapping leaves no more usable data for training the stolen encoder. This demonstrates increasing the number of sybil accounts makes encoder extraction impractical due to the growing remapping overhead. Overall, the results highlight that our B4B also successfully prevents sybil attacks.
method's advantage:
4.5 Baseline Comparison
Finally, we compare our B4B against the current state-of-the-art baseline defense, namely a static addition of noise to all the returned representations (as proposed in [16] (Section A.4), [2 36)). For direct comparability, we evaluate the defense using the our end-to-end experiment setup from the previous section. We present our results in Table 6 in Appendix F44
Confirming the findings from [66 our results also show that defenses that rely on static noise have the great disadvantage to harm legitimate users and attackers equally. When adding noise with small standard deviation of o = 0.1, we observe a negligible (<1%) performance drop for both attackers and legitimate users. Adding noise with a large standard deviation of, for example, a 10, we observe that both legitimate users' and attackers' performance drops between 1 15% and >40%. In summary. these defenses can either effectively defend stealing (but harm legitimate users), or keep utility for legitimate users high (but not defend well against stealing). In contrast, our B4B is able to provide high performance for legitimate users while effectively defending the encoder against stealing attacks.
Conclusions
We design B4B a new and modular active defense framework against stealing SSL encoders. All the previous approaches were either reactive, acting after the attack happened detect stolen encoders, or lowered the quality of outputs substantially also for legitimate users which rendered such mechanisms impractical. We show that B4B successfully distinguishes between legitimate users and adversaries by tracking the embedding space coverage of users' obtained representations. B4B then leverages this tracking to apply a cost function that penalizes users based on the current space coverage, for instance, by lowering the quality of their outputs. Finally, B4B prevents sybil attacks by implementing per-user transformations for the returned representations. Through our experimental evaluation, we show that our defense indeed renders encoder stealing inefficient while preserving downstream utility for legitimate users. Our B4B is therefore valuable contribution to a safer sharing and democratization of high-utility encoders over public APIs.
Acknowledgments and Disclosure of Funding
This research was supported by Warsaw University of Technology within the Excellence Ini- tiative Research University (IDUB) programme, National Science Centre, Poland grant no 2020039/0//ST//14478, grant no 2020/39/B/ST6/01511 grant no 2022/45/B/ST6/02811 part by PL-Grid Infrastructure grant nr PI.G/2022001603 The authors applied for CC BY license to any Author Accepted Manuscript (AAM) version arising from this submission, in accordance with the grants' open access conditions.
and in
10

---

References
[1] Clarifai, https//wwwwclarffi.ccom URL.
[2] Cohere, https://coher..ai URL (tttpttt//cceeeeecco
[3] Openai, htts://opemai.com. URL http://ww.....iiittpepe////opp..
[4] Poer2, hutp://hssllyyoorreee/////aaahel htttas//////t////////
[5] Google, http:///wwsaaaaaaaaa////////////
URL
[6] Facebook zstd, https://giihub.cco//acccc/////// URL http:///www.oaooooooo...o//oe.// http:::///gthhb..cc/aaaaaabbbbbe
zstd
[7] Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security 18), pages 1615-1631, 2018.
[8] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagiclski, Ariel Herbert- Voss, Kather- ine Lec, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In USENIX Security Symposium, volume 6, 2021.
[9] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages 9650-9660, October 2021.
[10] Ting Chen, S. Korbllith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. International Conference on Machine Learning, 2020.
[11] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, 2020.
[12] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. 2020.
[13] Adam Coates, Andrew Ng. and Honglak Lee. An Analysis of Single Layer Networks in Unsupervised Feature Learning. In AISTATS, 2011. https://cs. httpe///cssssaaordd....
papers/ccateeeleenn_eeisttt awwtaastttttasae 2011. .ddf
[14] Tianshuo Cong, Xinlei He, and Yang Zhang. Sslguard: watermarking scheme for self- supervised learning pre-trained encoders. CoRR, abs/2201.1 1692, 2022. Utt.htttss//arxiv org/ http/abb/2201..0 221..
[15] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255 Icee, 2009.
[16] Adam Dziedzie, Nikita Dhawan, Muhammad Ahmad Kaleem, Jonas Guan, and Nicolas Pa- pernot. On the difficulty of defending self-supervised learning against model extraction. In International Conference on Machine Learning, 2022.
[17] Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kalcem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot. Dataset inference for self-supervised models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL /topsrr//opeeevviee net/ forum?id=OCH_J9x...
[18] Adam Dziedzic, Muhammad Ahmad Kaleem, Yu Shen Lu, and Nicolas Papernot. Increasing the cost of model extraction with calibrated proof of work. International Conference on Learning Representations, 2022. http::ss: http:://wwxww...o/abbbbasse wwt.:or:///......... 09243.
[19] Stefan Dziembowski, Sehastian Faust, Vladimir Kolmogorov, and Krzysztof Pietzzak. Proofs of space. 2013. http::tttte:eeerittttiie wttpa/.iiii....r.///////9//// https://eprint... iacr .org/
2013/796

---

[20] Stefan Dziembowski, Sebastian Faust, Vladimir Kolmogorov, and Krzysztof Pietrzak. Proofs of space. In Advances in Cryptology- CRYPTO 2015: 35th Annual Cryptology Conference, Santa Barbara, CA, USA, August 16-20, 2015, Proceedings, Part II, pages 585-605. Springer, 2015.
[21] Nicholas Frosst, Nicolas Paperot, and Geoffrey Hinton. Analyzing and improving representa- tions with the soft nearest neighbor loss. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning. volume 97 of Proceedings of Machine Learning Research, pages 20200. PMLR, 09-15 Jun 2019. URL https://proceeedingg mlr ptw.sss/99/frrsstt1eee htt///ooosstt/a html
[22] Jean-Bastien Grill, F. Strub, Altché, C. Tallec, H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko. Bootstrap your own latent: A new approach to supervieed learning. Computer Vision and Pultern Recognition, 2020.
[23] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition: (CVPR), pages 16000-16009, June 2022.
[24] Matthew Jagielski, N. Carlini, D. Berthelot, Kurakin, and N. Papernot. High accuracy and high fidelity extraction of neural networks. USENIX Security Symposium, 2020.
[25] Hengrui Jia, Christopher Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot. En- tangled watermarks as defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 21), pages 1937-1954, 2021.
[26] Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan. Prada: protecting against dnn model stealing attacks. In 2019 IEEE European Symposium on Security and Privacy PP), pages 512-527. IEEE, 2019.
[27] Sanjay Kariyappa, Atul Prakash, and Moinuddin K Qureshi. Maze: Data-free model stealing attack using zeroth-order gradient estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13814-13823, 2021.
[28] Alex Krizhevsky. Learning multiple layers features from tiny images. Technical report, 2009.
[29] Yupei Liu, Jinyuan Jia, Hongbin Liu, and Neil Zhenqiang Gong. Stolenencoder: Stealing pre- pre-trained encoders in self self-supervised learning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS '22, page 2115-2128, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 97814 9781450394505. doi: 10.1 14443544806635665555555 3560586. URL https:: http:::/ddoi..rg///// 1145 3566606.3660686
URL
[30] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. Proceedings of ICLR 2021: 9th International Conference on Learning Representations,, 2021.
[31] Yuki Nagai, Y. Uchida, S. Sakazawa, and Shin' ichi Satoh. Digital watermarking for deep neural networks. International Journal of Multimedia Information Retrieval, 7:3-16, 2018.
[32] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. httpttpp///ffld stanffff pdf
stanford
oety/housemmmbers/nppp00000 hhwpsa/erseeeesoooooooooooeeeee
[33] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: of black-box models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1954-4963 2019.
Stealing functionality
[34] Loic Paulevé, Hervé Jégou, and Laurent Amsaleg. Locality sensitive hashing: A comparison of hash function types and querying mechanisms. Pattern recognition letters, 31(11):1348-1355 2010.
12

---

[35] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
[36] Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can't steal? cont-steal! contrastive stealing attacks against image encoders. 2022. URL httpp::/arxxvvorrg/ abs/
2201.07513
[37] Yun Shen, Xinlei He, Yufei Han, and Yang Zhang. Model stealing attacks against inductive graph neural networks. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1155-1992. IEEE, 2022.
[38] Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry perspectives. 2020 IEEE Security and Privacy Workshops (SPW), pages 69-75, 2020. doi: 10.1 200.S/PP...000.22220.000
[39] Malcolm Slaney and Michael Casey. Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal processing magazine, 25(2): -118 2008
[40] Florian Tramèr, F. Zhang, A. Juels, M. Reiter, and T. Ristenpart. Stealing machine learning models via prediction apis. USENIX Security Symposium, 2016.
[41] Jean-Baptiste Truong, Pratyush Maini, Robert J. Walls, and Nicolas Papernot. Data-free model extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021.
[42] Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM international conference on multimedia retrieval, pages 269-277, 2017.
[43] Yutong Wu, Han Qiu, Tianwei Zhang, Lin Jiwei, and Meikang Qiu. Watermarking pre- pre-trained encoders in contrastive learning. ArXiv, abs/2201 002217, 2022.
[44] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03230, 2021.
13

---

A Broader Impacts
The goal of our work is to actively defend self-supervised encoders against model stealing attacks. Since we are directly defending encoders, any negative societal impacts of our work are minimal. One potentially negative impact could be the degradation of performance for legitimate users. However, as shown in our experimental results, we are able to preserve high utility for standard users.
B Limitations
We show how our defense method tuned for SimSiam and DINO. There are more types of SSL encoders that can be tested with our method. The B4B defense method requires tuning the parameters, such as the number of occupied buckets that is allowed without any penalty for the cost function, or the selection of the transformations. These steps are rather difficult to automate but can be replaced with more data-driven approaches. For example, instead of designing a cost function from scratch, one could create an MI. model to obtain cost for a given occupation of the representation space. We explain more details the Appendix
C Alternative Building Blocks to Instantiate B4B
While we present reference implementation of B4B in our work that instantiates the three building blocks with (1) Local Sensitive Hashing, (2) Utility of the Representations, and (3) a set concrete transformations, there exists multitude of alternatives to concretely implement our B4B framework. In the following, we present these alternatives, grouped by building block.
C.1 Alternative Estimation of the Coverage of Embedding Space
We also explore alternative methods to measure the distances between representations for queries sent an API. One of them is to apply the cosine distance (where for two representations and is defined as: 1 - a 12-1 since it can be measured between individual data points in a pair-wise fashion. If the total pair-wise cosine distance between representations for given user is small, then the user queries presumably come from single downstream task distribution. Otherwise, user might be malicious and would like to cover large part of the representation space, then the total pair-wise cosine distance for the user's representations would be high. Note that in this case, the cosine distance can be replaced with any other distance measure, such the Minkowski distance. We opt for the LSH in our reference implementation, since much less expensive to compute than cosine distance. LSH requires only 2 = 4096 buckets that can be expressed as a binary table with the same number of elements, which requires in the worst case iterating over all of them to count how many are occupied. With more than 4096 queries sent by a given user, the computation the LSH is sublinear < O(n) with respect to the number of user queries. For the cosine distance approach, the required computation grows quadratically (n)) with the number of queries.
a"
C.2 Alternative Cost Functions
The cost functions can be designed from scratch manually or learned, for example, via an ML model, such neural network or SVM. In our initial version, the function was designed manually, where the underlying premise is that once a specified number of buckets is occupied, the cost should grow exponentially. Instead of defining such function or providing the high-level parameters for functions that we contributed, one could learn an ML model that for a given number of buckets occupied, should output an estimated cost, or even directly, the desired a (standard deviation) of the noise added to representations. This method requires a relatively large number of data points to be provided for training the model, however, lowers the burden on defender to either decide the specific function or adjust its parameters. Thus, it could be more user- ser--frennll, for example, not necessitating any mathematical background, but can be precise enough to obtain the desired behavior.
Note that instead of adding the calibrated noise (proportional to the estimated cost) to the represen- tations, we could rather require a given user to pay higher monetary cost for queries that cover large fraction of the representation space, or force a user to solve a puzzle form of the proof- wo-k [18), wait specified amount of time via proof-of-elapsed time (PoET) 4 or prove that
14

---

specified amount of disk space was reserved [19 20 For example, consider the approach with PoET. user sends queries to the API, which we cost based on their occupation of the embedding space. The user is sent waiting time. The users' resource (e.g., CPU) has to be occupied for this specific waiting time without performing any work. At the end of the specified amount of time, the user sends proof of the elapsed time, which can be easily verified by the server. PoET requires access to specialized hardware, for example, secure CPU instructions that are becoming widely available in consumer and enterprise processors. If a user does not own such hardware, proof of elapsed time could be produced using a service exposed by a cloud provider (e.g., Azure VMs that feature TEE 2). Note that if server sends the time based the calculated cost, the adversary might learn the cost function. Instead, the exact waiting time should be split in random subwaiting times and sent to the user one by one. Thus, server should rather have few rounds of exchange with the client to incur the additional cost.
C.3 Alternative to Transformations
Affine
Pad Shuffle
Affine Pad Shuffle
Binary
X
34
X
X
6.7
244
X2
0.7
0.7
4.2
X-1
0.5
X-1 0.5
-11
42
-111
0.5
0.5
0.4
-04
Figure 6: Overview on Transformations. We depict the inner-workings of the transformations
considered in this work.
As an alternative transformations used within this work (see Figure one could use a different set of transformations combinations thereof. The padding can be done with different constant values and combined with adding constant values within the representations. The padding and adding the constant values be followed by shuffling the elements within the representations. We can apply the affine binary transformations on top of the padding and shuffling. Additionally, we can also other pre-defined linear transformations like rotations shearing.
The representations could also be compressed smaller vectors and the compression rate would depend on the occupation of the representation space, for example, the higher the number of occupied buckets in our hash table, the more compressed the output representations could be. Such representa- tions could be compressed via FFT, a cosine transform, or standard compression techniques such as snappy [5). the information from the representations should not be lost, then the lossless com- pression techniques can be applied, for instance, zstd (6). The only requirement of the compression techniques is to ensure that they do not decrease the accuracy on downstream tasks for legitimate users.
Another alternative incorporate additional neural network layer for transforming the returned representations. The training of this supplementary layer should primarily focus on preserving the usability of the representations for legitimate users. This approach grants the API provider with additional capabilities, as it allows for the utilization of customized training objectives. For instance, if the API provider employs LSH (Locality-Sensitivee Hashing) to estimate the coverage of the representation space, they can leverage buckets and train the additional layer to maintain high-quality representations exclusively for frequently-used buckets and their surrounding areas, while not prioritizing the rest of the representation space. This approach safeguards legitimate users
15

---

from any adverse effects, as their coverage of the representation space is minimal. Simultaneously, it ensures that adversaries are unable to exploit representations from the entire representation space.
D Sybil Attacks
We consider adversary who generates sybil accounts to steal the encoder from the API. For each of the accounts, the representations are transformed a different way. Therefore, to replicate the victim model using all the obtained representations, the adversary has to map these representations into one single space. This can be done, for example, by training a neural network to perform the mapping.
We assume the adversary obtains {N1, N2...., } many representations from the victim for each of the sybil accounts. Without loss of generality, assume the adversary maps them back to the embedding space of the first sybil account. To learn the mapping, the adversary can apply different strategies.
D.1 Sybil Strategies
We present three potential approaches that Sybils might want to apply circumvent our defense. Con- sider three users: A, B, and C, with their respective datasets D DA, DB. and Dc. each with different distributions to maximize extraction effectiveness. First, user A is selected to unify representations from other users B and C. User A would have to query from at least two different datasets D B and Dc, while other users would act legitimately. Sybil attackers want deploy as many users as possible but with more fake accounts, user A incurs high coverage of the representation space, and this is prevented by our single-user defense. In all other cases neither of the sybil users act legitimately, thus they are already affected by the single-user defense. Second, user A would query from their own dataset D a and partially from dataset DB. Then user B would query from their own dataset n and partially from dataset De. and so on. This method is the most inconspicuous but requires a number of remappings that scales super-exponentially with the number of fake accounts, which is impractical. Finally, each user would query from their respective dataset, for example, user B would query from dataset DB and additionally from a remapping dataset, e.g., DA. Representations could be unified by mapping them to A's representations. The last approach as well as all other cases reduce to the minimum remapping between representations of a pair of users. We show that our defense cuts such attempts short by ensuring that the remapping between representations is prohibitive even for pair of users.
E
Additional Related Work
One of the main workhorse techniques used in the encoders is contrastive learning, where the representations are trained so that the positive pairs (two augmented versions of the same image) have similar representations while negative pairs (augmentations of two different images) have representations which are far apart.
SimSiam utilizes Siamese networks (two encoders with shared weights) but with simplified training process and architecture. In contrast to the previous frameworks, such as SimCLR [10, SimSiam's authors show that negative samples are unnecessary and collapsing solutions can be avoided by applying the projection head to of one of the encoders, and a stop- coo-graddentt operation to the other. SimSiam minimizes the negative cosine similarity between two randomly augmented views of the same image from the Siamese encoders, which expressed via a symmetrized loss (22). This creates simple yet highly effective representation learning method.
DINO is another popular representation learning framework. While SimSiam uses CNNs, DINO employs vision transformers (ViTs). It trains a student and teacher encoder with the same architec- ture, updating the teacher with an (exponential moving) average of the student. Different random transformations of the same image are passed through both encoders. The student receives smaller image crops, foreing to generate representations restoring parts of the original image. The training objective is minimizing cross-entropy loss between teacher and student representations.
16

---

Additional Experimental Results
F. Details Experimental Setup
The end-to-end experiments on stealing SimSiam and VIT DINO encoders were done using 3 A10 GPUs. Detailed experiments including mapping, transformations and the evaluation was performed using a single computer equipped with two Nvidia RTX 2080 Ti GPUs.
F.2 Datasets Used
CIFARIO 28 The CIFARIO dataset consists of 32x32 colored images with 10 classes. There 50000 training images and 10000 test images.
SVHN [32] The SVHN dataset contains 32x32 coloured images with 10 classes. There roughly 73000 training images, 26000 test images and 530000 "extra" images.
ImageNet [5]: Larger sized coloured images with 1000 classes. As is commonly done, we resize all images be of size 224x224. There are approximately million training images and 50000 test images.
The STL10 dataset contains 96x96 coloured images with 10 classes. There are 5000 training images, 8000 test images, and 100000 unlabeled images.
STL10 13:
LAION-5B (35) The LAION-5B dataset consists of 5,85 billion CLIP-filtere image-text pairs. The dataset was crawled from publically available internet.
More Results for the End2End Empirical Evaluation
We consider fine-tuning parameters 3, X, and for our cost function and the intuitive meaning behind these parameters. In general, our recommendation to adjust parameter 3 that specifies how many buckets are allowed to be filled by users' downstream tasks. On the other hand, when parameter À is increased, this causes higher amount of added noise before we reach the number of buckets specified by 8, which lowers the performance of given downstream task relatively early. For example, a higher value much earlier than for the target value of B. Finally, parameter controls the amount of noise once number of buckets specified by B is reached. Thus, in Figure 4 we set a = and the maximum standard deviation of the added Gaussian noise is 1.
Figure would cause an increase in the amount of added noise
Stdew, of addddd noise
1.0
fzz((n)
Stdev. of added noise
100n)
((-((n)
Talin)
((3-(n)
falnn)
0.6
f1
fuin)
0.4
0.4
0.2
0.2
0.0
60
Buckets [%)]
Buckets [%)
(a) Cost Function for different À parameter values.
(b) Cost Function for different parameter values.
Figure 7: Effects of À and parameters the Cost Function. We present the Cost Function for
a=1, 8=80 and different values of À (left) and = 10 10-6, f0800 and different values of a (right).
17

---

Table 2: Stealing and Using Encoders With and Without our Defense. The model used in the experiments is Simsiam, with the following parameters for the cost function 10 a 1, and 3 = 80%, and the number of buckets equal to 2 Due to the higher value of the parameter X, we observe lower performance on downstream tasks for the attackers since the magnitude of noise added to the representations is higher. However, for more complicated tasks than CIFARIO 10, this change might cause potential drop in accuracy for the legitimate users.
USER
DEFENSE # QUERIES DATASET
TYPE
CIFAR10
STLI0
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.41
95.08
91.22+0.11 75.47
LEGIT
B4B
ALL
TASK
QUERY
90.02 =0.1
94.88
10.17
74.72 20.13
91.76 10.09
ATTACK
NONE
50K
IMGNET STEAL
65.2 10.00
64.9 10.01
62.1 10.01
88.5 10.01
ATTACK
B4B
50K
IMGNET
STEAL
28.22
26.62 0.02
19.62 0:02
78.41 10.01
ATTACK
NONE
100K
IMGNET
STEAL
68.1 20.05
63.1 10.01
61.5 10:01
89.0 +0.07
ATTACK
B4B
100K
IMGNET STEAL
17.73 20.18
15.59
1061
19.53
55.11 0.05
SYBIL
B4B
50K+50K
IMGNET
STEAL
33.43
00.03
31.18
0012
22.91 = 0.01
75.35 0.05
Table 3: Stealing and Using Encoders With and Without our Defense. The model used in the experiments is Simsiam, with the following parameters for the cost function À = 10 a 1, and 50%, and the number of buckets equal to 212 This experiment corresponds to considering 50% of buckets filled as too-large coverage of the embedding space. This improves the defense but again might potentially harm the performance of more complicated tasks than CIFARIO since they could occupy more buckets than 50%.
USER
DEFENSE QUERIES DATASET
TYPE
CIFAR10
STL10
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.4110.02
95.0810.13
75.47:0004
91.22 10.11
LEGIT
B4B
ALL
TASK
QUERY
90.27 10.07
95.12 20.13
74.94
10.16
91.66
#0.05
ATTACK
NONE
50K
IMGNET
STEAL
65.2 20.05
64.9 10.01
62.1 10:01
88.5 10.01
ATTACK
B4B
50K
IMGNET
STEAL
15.52
2.37
12.57 2.23
19.53
23.17 0.00
ATTACK
NONE
100K
IMGNET
STEAL
68.1 20.05
63.1
10.01
61.5
20.01
89.0 20.07
ATTACK
B4B
100K
IMGNET
STEAL
16.27 0.04
13.93 0.35
19.54
0.02
54.69 0.02
SYBIL
B4B
50K+50K
IMGNET
STEAL
30.14
29.57
0.08
19.99
0.03
71.72
Table 4: Stealing and Using Encoders With and Without our Defense. The model used in the experiments is Simsiam, with the following parameters for the cost function À 10-6 a and 3 30%, and the number of buckets equal to 2 Since the value of parameter 3 is decreased substantially 30%, we observe drop in accuracy for legitimate users. For example, more than 1% for CIFAR10. In the next Table we show that by also decreasing the parameter a. we can attenuate this harmful effect and retain higher accuracy for legitimate users. In case of an attack, for 100k stealing queries, we observe much lower accuracy levels than for 3 = 50% shown in Table 3
USER DEFENSE QUERIES DATASET TYPE
CIFARIO
STL10
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.41
95.08
75.4740.04
91.22 40.11
LEGIT
B4B
ALL
TASK
QUERY
88.1 10:11
94.92 10111
74.37
20.02
91.67
20.07
ATTACK
NONE
50K
IMGNET STEAL
65.2 10.05
64.9 10.01
62.1 10.01
88.5 10.01
ATTACK
B4B
50K
IMGNET STEAL
30.82
0.00
26.37 0.00
21.87 0.03
66.0 1 0.00
ATTACK
NONE
100K
IMGNET
STEAL
68.1 +0.03
63.1 40.01
61.5 #0.01
89.0 +0.07
ATTACK
B4B
100k
IMGNET
STEAL
9.57 0.17
9.83 + 0.09
19.57 0.01
27.06 0.46
SYBIL
B4B
50K+50K
IMGNET
STEAL
29.15 0.02
28.67 0.06
19.98 0.03
70.62 0:03
18

---

Table 5: Stealing and Using Encoders With and Without Defense. The model used in the experiments is Simsiam, with the following parameters for the cost function À = 10-6, a = 0.1, and = 30%, and the number of buckets equal 212, Due the lower performance on downstream tasks observed in Table while keeping the parameter 3 fixed to 30% and À fixed to 10-6, we decrease the value of parameter a to 0.1, which increases the performance of legitimate users on their downstream tasks. In this experiment, we also carry out a sybil attack with more accounts (4 instead of 2). but observe that this modification does not improve the performance of the attacker. With more accounts, a sybil has to sacrifice more queries for the remappings between the representations from different accounts. Additionally, note that each account introduces a different remapping error by the dint of different transformations applied to each account by B4B.
USER
DEFENSE
QUERIES
DATASET
TYPE
CIFARIO
STLI0
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.411..02
95.08
91.22 75.47
LEGIT
B4B
50K
CIFARIO
QUERY
90.17 20.1
94.92
20.09
74.97
20.13
91.71
20.08
ATTACK
NONE
50K
IMGNET STEAL
65.2 20.03
64.9
20.01
62.1 =0.01
88.5 20.01
ATTACK
B4B
50K
IMGNET
STEAL
19.95 20.19
15,54
0.34
19.57
0.01
23.50 + 0.19
ATTACK
NONE
100K
IMGNET
STEAL
68.1 #0.03
63.1
#0.01
61.5
=0.01
89.0 #0.07
ATTACK
B4B
100K
IMGNET
STEAL
10.35
0.19
12.37
0.69
19.34
0.01
68.93 0.17
SYBIL
B4B
4x25k
IMGNET
STEAL
33.15 0.04
30.23
0.07
20.87
0.01
72.19
F4 B4B vs Static Noise Addition Defenses
We compare our B4B against the current state-of-the-art baseline defense, namely adding a static addition of noise to all the returned representations (as proposed in [16] (Section A.4), (3)). For the T we use the same setup as in Table [ (with an ImageNet pre-trained encoder).
Our results show the following insights:
1. If the amount of noise is small (a 0.1) then the performance drop is negligible but for both a legitimate user (row 2) and an adversary (row 6). this case, the defense does not affect the adversary at all (compare rows 5 & 6).
2. If the amount of noise is large (a = 10) then the performance drop is large for both a legitimate user (row 3) and an adversary (row 7). In this case, the encoder is worthless for legitimate users since the performance is too low.
Table 6: Stealing and Using Encoders with Static Noise Addition Defenses vs. Our B4B Defense. Adding small amount of noise results in negligible drop in performance for both legitimate user (row 2) and an adversary (row 6). Adding a large amount of noise defend stealing (row 7), but significantly harm legitimate users at the same time (row 3). Our B4B defense solves the above problem and provides high performance for legitimate users (row 4) while effectively defending the encoder against stealing attacks (row 8).
USER
DEFENSE
QUERIES DATASET TYPE
CIFAR10
STL10
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
90.41 10.02
95.0820.1
75.47
91.2249.11
LEGIT
NOISE a=0.1
ALL
TASK
QUERY
90.2010.05
95.1520.13
75.29
91.2410.2
LEGIT
NOISE 1=10
ALL
TASK
QUERY
65.1140.45
76.3720.14
33.23
65.8310.15
LEGIT
B4B
ALL
TASK
QUERY
90.2440.11
95.05
74.96=0.13
91.74001
ATTACK
NONE
50K
IMGNET
STEAL
65.2=0.03
64.9
63.1=0m1
88.5 40.0
ATTACK NOISE 0=0.1
50K
IMGNET
STEAL 64.92+a.04
64.61
62.35
88.414000
ATTACK NOISE 0=10
50K
IMGNET
STEAL
36.3210. 32.59
20.59
74.9410.02
ATTACK
B4B
50K
IMGNET
STEAL
35.7210.4
31.54
19.74
70.01 10.01
19

---

F,5 Additional Embedding Space Coverage experiments
We present additional experiments on measuring the coverage of the representation space.
First, we use the same set-up as from Table T SimSiam with ResNet50 pretrained on ImageNet. When querying the encoder with ImageNet- Full (includes all 1000 classes) and LAION-SB datasets, they both occupy a large fraction the representation space the victim encoder, as shown on Figure In contrast, CIFARIO covers the smallest portion of the representation space the simplest dataset tested. ImageNet-Dogs (with only 118 classes for dog breeds) falls in the middle, occupying more space than CIFARIO but less than ImageNet-Full and LAION-5B. Its intermediate coverage aligns with its mid-level difficulty compared to the other datasets, As indicated by representation space coverage, stealing the encoder is similarly effective with ImageNet-Full and LAION-SB datasets, as both datasets cover large fraction of the representation space. Overall, Figure demonstrates that: our B4B can successfully protect the encoder model even from attackers stealing with data that was not used to train the model (LAION-5B in this case) and 2) while providing clean representation for users querying from downstream tasks that are part more complicated datasets (ImageNet-Dogs)
8
Buckets occupiod [%6]
100
80
ImagaNot- Dogs
LAION
ImageNat
10000
20000
30000
10000
50000
Queries
Figure 8: Fraction of Occupied Buckets (Embedding Space Coverage) for the ImageNet encoder. Representations for the downstream datasets (CIFAR 10, ImageNet Dogs) occupy a smaller fraction of buckets than representations from the complex ImageNet or LAION-5B datasets. The underlying encoder SimSiam pre-trained on ImageNet with ResNet50.
Our method of measuring the embedding space coverage is not limited particular encoder dataset used for pretraining. We demonstrate this in Figure 9 showing the fraction of occupied buckets for SimCLR [II] Resnet34 encoder pretrained on CIFARIO.
Buckots occupiod [%6]
100
FeshionMNNIST
80
SVHN
STL10
CIFARI00
CIFAR10
10000
20000
30000
40000
50000
Queries
Figure 9: Fraction of Occupied Buckets (Embedding Space Coverage) for the CIFAR10 encoder. B4B can be applied an encoder trained on CIFAR1O. Representations for the downstream datasets (FashionMNIST.. SVHN) occupy smaller fraction of buckets than representations from CIFAR 10, CIFAR100 100, and STL10 datasets. The underlying encoder is SimCLR pre-trained on CIFAR1O with ResNet34.
20

---

F.6 Setting the number of buckets
We present our procedure find optimal number of buckets in Figure TO
100
100
MNNIS
FasaionMNIST
Buckstts occupled
Buckets occupied
CIPARIU ImageNNt
CIPARI0
60
60
ImssgeNet
20
10000
30000
40000
50000
10000
20000
50000
Queries
Queries
(a) Number of buckets = 2
(b) Number of buckets 2
100
100
MuSSicaMNIST
Resh SVHN
Buckets ocoupled
Buckets ocrupied
CIPARIO
CIEARTO
60
ImageNet
60
ImageNet
20
20-
20000
30000
40000
50000
10000
20000
30000
40000
50000
Queries
Queries
(c) Number buckets 212
(d) Number of buckets 2
Figure 10: Estimating Embedding Space Coverage through LSH the SimSiam Encoder. We extend the results from Figure 3| a) and present the fraction of buckets occupied by representations of different datasets number of buckets in the LSH table. We observe that 2 buckets is to small since queries from the ImageNet dataset saturate all the buckets after around 50k queries, while the number 2 of buckets is too large since it is never occupied more than 40%. Thus, the number 21 buckets is a good middle ground. Subfigure (c) corresponds Figure from the main paper. We also the same notation and carry out our experiments in the same way as in Figure 3
function of the number of queries posed to the encoder. We consider different
F.7 Results for DINO
We show that our defense is also applicable to the DINO encoder. The occupation of the representa- tions space is presented visually Figure 11 We also show that the number buckets 2 is optimal for DINO in Figure [2] The impact of transformation on the representations from DINO is shown Table g Finally, the end end experiment for DINO is presented in Table 7
400
SVHN
200
220
-200
295
<50
Figure 11: Representations from Different Tasks Occupy Different Sub-Spaces of the Embedding Space. Presented for FashionMNIST, SVHN, CIFAR10, and STL10. In this plot, used the DINO VIT Small encoder trained on ImageNet.
21

---

100
PashionMMNIST SVHN
Buckots occupied
CIFARIO
imageNet
40
20
10000
20000
30000
40000
50000
Queries
Figure 12: Estimating Embedding Space Coverage through LSH on the DINO Encoder. The number of buckets is set to 212, We also use the same notation and carry out our experiments in the same way as Figure
Table 7: Stealing and Using Encoders With and Without Defense. The model used in the experiments is DINO, with the following parameters for the cost function À = 10 10-6 a = 1000, and 3 60%, and the number of buckets equal to 2 We have increase the value of parameter by x 1000 since the norms of the DINO representations are also around 10 higher than for SimSiam. We observe that B4B performs similarly DINO for SimSiam.
USER DEFENSE # QUERIES DATASET
TYPE
CIFAR10
STLI0
SVHN
F-MNIST
LEGIT
NONE
ALL
TASK
QUERY
94.51
10.08
97.98
10.04
70.66
#0.16
89.98 +0.03
LEGIT
B4B
ALL
TASK
QUERY
94.25
10.11
98.05
10.04
69.66
20.14
89.68
10.01
ATTACK
NONE
50K
IMGNET STEAL
67.92
1.004
66.02
0.22
61.30
:: 0.01
89.46
0.01
ATTACK
B4B
50K
IMGNET STEAL
42.02:1005
38.91
19.94
13.3
ATTACK
NONE
100K
IMGNET STEAL
75.07 6.01
76.32 0.02
71.79
0.06
89.76
ATTACK
B4B
100K
IMGNET STEAL
19.27
21.244:0.0s
19.844a..1
71.01
SYBII.
B4B
50K+50K
IMGNET
STEAL
45.56+ 0.06
42.50400.2
24.25
78.01 + 0.08
F88
Additional evaluation transformations
Additionally, show the impact of transformations on the performance of legitimate users in Table (for both SimSiam and DINO).
GENERATE RD ENSTATTTN
TRANDIOOR REPRESENTIIIONS
TRAIN REFERENCE CLASSTIES
TRAU MARPERSS
ROMAP AGSIOO REPRRSSI
Reformnne
Mapper
Figure 13: Protocol to Evaluate the Mapping Between Representations. We present the protocol of evaluating remappings for two sybil accounts. API receives inputs from two sybil accounts and generates corresponding representations.
Representations are transformed on a per-user basis and returned. Adversary trains a reference classifier on representations from account one. Adversary trains linear model to find mapping from representations of account representations of account one. To check the quality of obtained mapping representations from test set of account two are mapped using the fixed mapper (from step 4) to representation space of account one. This enables the calculation of cosine distance between representations from account one and their counterparts from account two shown in Figure 5) Additionally, the fixed reference classifier (from step 3) be used measure the accuracy drop caused by remapping.
22

---

Table 8: Impact Transformations on the Performance for Legitimate Users. We show that the transformations applied per-account do not harm the performance of legitimate users on their downstream tasks. The victim encoders was trained on the ImageNet dataset using SimSiam and DINO frameworks.
TRANSFORMATION
ENCODER
CIFARIO
STLI0
SVHN
F-MNIST
NONE
Victim SimSiam
90.41
95.08
10.13 75.47
10.04 91.22 20.11
AFFINE
SIMSIAM
90.24 #0.1 95.05
=0.1
74.96
10.18 91.42 20.15
PAD+SHUFFLE
SIMSIAM
90.4 #0:05
95.34
#0.06 75.47
#0011 91.38 40.15
ARTNEE-PA-SSURFL
SIMSIAM
90.18
95.03
20.05 74.86
#0.1 91.35 20.1
BINARY
SIMSIAM
88.78 #0.2
94.72
#0.02
68.42
10.16
88.91 20.34
NONE
Victim DINO
94.51 #0.08
97.98
10.04 70.66
10.15 89.98
4.003
AFFINE
DINO
94.25 10.11
98.05
69.77
10.11 89.68 10.01
PAD+SHUFFLE
DINO
94.72 =0.2
98.07
00.3
70.44
1011 89.91 10.08
AFTINE-PAD-SHUFLE
DINO
94.26 20.06
98.02
10
69.49
102
89. 70 10.1
BINARY
DINO
92.96 #0.1
98.03
+0.03
59.53
+0.27
88.26 +0.04
23


Reviews:
[{"review_id": "Lk4cTHUOuO", "reviewer": "Reviewer_FtSs", "summary": "This paper proposes B4B as a defense against model stealing attacks for pretrained encoders. The main assumption of the defense is that attack queries generally have a much broader coverage of the embedding space than queries from normal users. Based on this assumption, B4B utilizes Local Sensitive Hash and separates the embedding space into a number of 2$^12$ buckets. B4B then calculates the fraction of buckets covered by the user queries, then adds Gaussian noise to the subsequent queries when the coverage is large, where this cost function exponentially increases. To defend against sybil attacks, different transformations are applied to the returned embedding vectors of different users, so that legitimate users can obtain high-quality models with the transformed embedding vectors, while attackers who aim to combine training data from different accounts cannot get performance boost.", "questions": "1. In the case when the users  query the model using ImageNet samples from a small subset of the label set, would the input coverage still be high and thus the user is identified as an attacker?\n\n2. Another question is whether the attack queries should be images included in the pretrained data, or does the defense also work for queries drawn from a similar distribution? For example, if you use STL-10 unlabeled data for pretraining and STL-10 labeled data to query, or if you use CIFAR-100 data for pretraining and CIFAR-10 to attack, does the defense work?\n\n3. The full defense includes a couple of hyperparameters that seem critical, i.e., the number of buckets, the cost function, and the transformations. Is the same set of hyperparameters used for all tasks and models in the experiments? How much work is required to tune the implementation for new tasks and model architectures?\n\n4. I do not fully understand the process of per-user representation transformation. How is it done for a new user? Does the defense randomly select/construct a transformation from possible choices?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Designing an active defense for model stealing attacks is a good topic. The proposed approach is interesting, and seems to be generally applicable to different models and tasks.\n\n2. The results show that B4B does not degrade the response quality for legitimate users, while significantly decreases the model performance trained by attackers. ", "weaknesses": "1. In general, I wonder what is the valid condition to consider a user as an attacker. In the current evaluation setup, all legitimate users are querying with images drawn from different data distributions. However, even if the users directly query the model using ImageNet samples, if the user queries are only limited to a small subset of labels instead of the full label set, then I also would not consider it an attack. Have you evaluated this setting? Would the input coverage in this case still be high?\n\n2. Another question is whether the attack queries should be images included in the pretrained data, or does the defense also work for queries drawn from a similar distribution? For example, if you use STL-10 unlabeled data for pretraining and STL-10 labeled data to query, or if you use CIFAR-100 data for pretraining and CIFAR-10 to attack, does the defense work?\n\n3. The full defense includes a couple of hyperparameters that seem critical, i.e., the number of buckets, the cost function, and the transformations. Is the same set of hyperparameters used for all tasks and models in the experiments? How much work is required to tune the implementation for new tasks and model architectures?\n\n4. I do not fully understand the process of per-user representation transformation. How is it done for a new user? Does the defense randomly select/construct a transformation from all possible choices?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the positive, encouraging, and constructive feedback. \n\n>**Answer to Weakness 1 (W1) & Question 1 (Q1): Querying with a small subset of ImageNet**\n\nB4B does not classify a user as adversarial or benign but only adaptively calibrates the defense strength based on the coverage of the representation space. We ran new experiments with a limited number of labels for ImageNet (using only the classes of dog breeds) and observe that *the smaller number of classes included in the query set, the lower the coverage of the representation space*. Indeed, a potential legitimate user might be interested in a few classes instead of all 1000 classes from ImageNet. B4B does not penalize such users. \n\nWe show the results of coverage (%) in the table below and in Figure 1 in the attached 1 pager PDF from the general response. We query the ImageNet pre-trained encoder (the same as in Table 1 in the submission) and consistently observe lower *coverage* of the representation space for ImageNet Dogs (with only the 118 classes for dog breeds) than for the ImageNet Full (with all 1000 classes included).\n\n|Dataset↓/Number of Queries→|5000|10000|20000|30000|40000|50000|\n|-|--------|--------|--------|--------|--------|--------|\n|ImageNet-Full (all 1000 classes)|31.66 |38.61|48.32|54.02|58.03|61.15|\n|ImageNet-Dogs (118 classes of dog breeds)|22.47|26.93|33.34|37.20|40.01|42.19|\n\nWe emphasize that there is no accuracy drop for legitimate users when querying with the ImageNet Dogs data. The accuracy on the ImageNet Dogs with and without B4B is 71.47 +- 0.02 and 71.53 +- 0.03, respectively. The table below shows that when querying with ImageNet Dogs our defense is not activated and all the accuracy values are comparable with or without B4B defense. On the other hand, when querying with the Full ImageNet dataset, the performance drop on downstream tasks is substantial. \nAll the results (accuracy on downstream tasks in %) in the Table below are for the ImageNet stolen encoder. In all cases we use 50k queries to steal the encoder. \n|Defense|Dataset|CIFAR10|STL10|SVHN|F-MNIST|\n|-|-|-|-|-|-|\n|None|ImageNet-Dogs|42.83±0.63|48.346±0.20|44.42±0.07|59.91±0.01|\n|B4B|ImageNet-Dogs|43.21±0.55|46.91±0.3|45.2±0.08|58.76±0.01|\n|None|ImageNet-Full|65.2±0.03|64.9±0.01|62.1±0.01|88.5±0.01|\n|B4B|ImageNet-Full|35.72±0.04|31.54±0.02|19.74±0.02|70.01±0.01|\n\n>**Answer to W2 & Q2: Different distribution for stealing than the victim's training set.**\n\nPlease, check the results for the LAION dataset used for stealing from ImageNet encoder in the general response.\n\nWe also carried out the suggested experiments, where in our case the victim encoder is trained on CIFAR10 and then stolen with CIFAR100. We show that: \n1. The stealing process is effective in this case and the attack queries do not need to be images included in the pretrained data. They can be from similar (stealing from CIFAR10 pre-trained encoder using CIFAR100 data) or even a different distribution (as shown for stealing ImageNet encoder with LAION).\n2. B4B defends against such attacks (compare two last rows), where the average performance drop is above 40%. \n\n|User|Defense|Number of Queries|Dataset|Type|CIFAR10|STL10|SVHN|F-MNIST|\n|-|-|-|-|-|-|-|-|-|\n|Legit|None|-|Downstream task|Query|86.91|74.64|80.57|86.55|\n|Attack|None|50k|CIFAR10|Steal|75.97|65.91|49.37|86.46|\n|Attack|None|50k|CIFAR100|Steal|70.78|60.59|61.97|84.75|\n|Attack|B4B|50k|CIFAR100|Steal|21.32|12.76|19.43|49.44|\n\nPlease, see also the attached PDF to the main paper, where in Figure 2, we present the corresponding occupation of the representation space for the CIFAR10 pre-trained encoder when queried with different datasets, namely: STL10, CIFAR10, CIFAR100, SVHN, and FashionMNIST. The results indicate that we can steal such encoder with equal effectiveness using STL10, CIFAR10, or CIFAR100 datasets. Overall, this demonstrates that B4B can be applied to encoders pre-trained on different datasets and with diverse architectures.\n\n>**Answer to W3 & Q3 B4B Hyperparameters**\n\nWe maintain consistency in our experiments and use the same set of hyperparameters. We added ablation study to demonstrate how the choice of the parameters influences the quality of the defense. The ablation study helps to tune (hyper)parameters of a new instance of the defense. For example, the number of buckets (Appendix F.4) and the cost function have to be tuned and we provide guidelines on how to find the optimal values. Regarding the number of buckets, we provide  in Section 4.1 an explanation that in the extreme: If we have a too large number of buckets, the number of buckets filled will correspond to the number of queries posed by a user which fails to capture that similar representations cover similar sub-spaces of the embedding space, and hence does not serve to approximate the total fraction of the embedding space covered. However, if we have too few buckets, even the representations for simple downstream tasks will fill large fractions of buckets, making it impossible to calibrate the cost function such that it only penalizes adversaries.\n\n>**Answer to W4 & Q4: Selection of per-user representation transformations**\n\nThis is correct, for a new user, the defense randomly selects the transformations from all possible choices. Note that the randomness is also added on a per-transformation basis, instead of only on the level of selecting the transformations. For example, a shuffling/permutation of the elements in the output representations should be different for each user. Additionally, the transformations can be combined/composed, for instance, we can combine shuffling with padding and an affine transformation. This further prevents an attacker from the possibility of simply guessing what transformation was used for a given user and reversing the applied transformations to obtain the original representations.\n\n---\n**We hope that the provided answers adequately address reviewers’ concerns and the score can be increased.**\n "}, {"review_id": "dnYwMUkpxJ", "reviewer": "Reviewer_N8vt", "summary": "The paper propose a defense over model stealing on-the-fly by assessing dataset distributions, and directly train on it. The paper is well-written and easy to read, while proposing a novel type of active defense over model stealing, without harming the performance of benign users.\n", "questions": "As stated in \"Weakness\" section. I'm particularly interested if the authors can clarify the threat model and potential applications of the proposed method.\n", "limitations": "The authors do not discuss potential negative social applications. However, I do not find any potentially negative impact in this paper.\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The authors propose an active defense over model stealing, which is based on the assumption that malicious queries will cover larger fraction of representation space, while benign queries are limited to a smaller fraction of space. The authors assign adversaries with noisy representations and benign users with clean representations. They also provide an additional transformation to avoid adversaries aggregate the stolen representation via creating multiple accounts.\n", "weaknesses": "I have particular concerns on the assumptions and experimental setup of this paper.\n\n* The paper is based on the assumption that individual queries in a much narrower distribution, while adversary queries in a broader one. However, I wonder if this is the case in practice. For large tech firms, they might have requests from diverse users and query a specific API for many times, which also results in diverse usage. For a model trained for specific downstream task (eg, to identify cancer), there can be only one domain and the queries and adversaries can be similar (both include images containing/not containing cancer). Can B4B defend over this? Otherwise, authors should quality the potential usage of their work. Indeed, this assumption is difficult to prove to be true or to be false, so at least discussions should be included.\n\n* I question the experiment setting of Table. 1, which follows the assumption that adversary query is inherently different from benign ones. However, this also means adding a simple classifier will also have high defend accuracy, since the distribution can be drastically different for ImageNet and CIFAR 10. While training on image classifiers is nowhere near to real world applications, I believe authors should test on a wider range or threats possible.\n\n* In Figure 3, it turns out ImageNet naturally occurs for more precentage of buckets (covers larger proportion of representation space), while other dataset covers smaller fraction. Thus, I am again skeptical of the result given in Table. 1. It turns out that, ImageNet naturally occurs for larger fraction of representation space, thus naturally occupies more buckets. In this way, what if the legit query is ImageNet, while attack query is CIFAR-10? Will the proposed defense crash, since the legit query occupies larger fraction of representation space?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments. We address individual points below one by one:\n\n>**“For large tech firms, they might have requests from diverse users and query a specific API many times, which also results in diverse usage.”**\n\nIf we correctly understand the setup, there are users who send queries to company 1 (a large tech firm), and company 1 sends the queries on behalf of the users to company 2 who exposes the API. This is why the queries are so diverse. We argue that in this case, company 1 (the large tech firm) should simply open a new account per each user. \n\n>**\"Specific downstream task.\"**\n\nIf we consider only a model trained for a specific downstream task (e.g., to identify cancer), then the API provider should use the active defense for *supervised models* proposed in [15]: the PoW (Proof-of-Work defense: Increasing the cost of model extraction with calibrated proof of work). In general, the *encoders* that we consider in this work are not specialized to specific downstream tasks but trained to cover a large space of possible inputs.\n \n>**\"ImageNet naturally occurs for larger fraction of representation space, thus naturally occupies more buckets. In this way, what if the legit query is ImageNet, while the attack query is CIFAR-10?\"**\n\nIf we select images from ImageNet but only a few classes then they do not occupy a large representation space. In contrast, if we select images from ImageNet at random, each from a different class, then they do occupy a high percentage of the representation space (see table below and Figure 1 in the attached PDF). When stealing images using CIFAR10, the adversary steals a small subset of the representation space and not the whole encoder. \n\nSpecifically, the encoder stolen with CIFAR10 does not perform well on other downstream tasks.\nOur defense does not want to prevent someone from stealing the CIFAR10 part of the encoder but the encoder's **general** functionality.\n\nStealing with ImageNet is more effective to extract general functionality than stealing with CIFAR10 and with ImageNet, we obtain much higher accuracy on downstream tasks, such as Flowers 102 or even CIFAR10. The very low performance of the stolen encoder on Flowers102 shows that the representation space covered by CIFAR10 does not include the required representation space for Flower102, where the accuracy is close to random. \n\nThe accuracy values (%) are for stolen encoder from the ImageNet pre-trained victim using 50k queries. No defense is used in these experiments.\n\n|Stealing-Dataset|CIFAR10|Flowers102|\n|----------|------------|------------|\n|ImageNet|65.2±0.03|16.3±0.01|\n|CIFAR10|60.61±0.05|1.4±0.02|\n||||\n|(Difference)|(-7.1%)|(-91.4%)|\n\nNote that the Flowers dataset contains 102 classes and full resolution (>225x225x3) pictures. This shows that stealing with CIFAR10 creates a CIFAR10 level encoder at most and does not generalize to other downstream tasks, especially the more complex ones.\n\nWe also show that querying with a subset of ImageNet (using only the dog images; 118 classes) is also not effective to steal the ImageNet encoder and our defense does not activate for such benign queries as compared to stealing with the full ImageNet dataset (when all 1000 classes are used).\n\nAll the results (accuracy on downstream tasks in %) in the Table below are for the ImageNet stolen encoder. In all cases we use 50k queries to steal the encoder. \n\n|Defense|Dataset|CIFAR10|STL10|SVHN|F-MNIST|\n|---------|---------------|--------------|---------------|---------------|--------------|\n|None|ImageNet-Dogs|42.83±0.63|48.346±0.20|44.42±0.07|59.91±00.01|\n|B4B|ImageNet-Dogs|43.21±0.55|46.91±0.3|45.2±0.08|58.76±0.01|\n|None|ImageNet-Full|65.2±0.03|64.9±0.01|62.1±0.01|88.5±0.01|\n|B4B|ImageNet-Full|35.72±0.04|31.54±0.02|19.74±0.02|70.01±0.01|\n\n>**The threat model.**\n\nOur setup and the resulting threat model are inspired by public APIs, such as Cohere, OpenAI, or Clarify [1–3] that expose encoders to users through a pre-defined interface (as described in Section 3.1). In the design of our defense, we consider adversaries who can query the encoder with arbitrary inputs to obtain high-dimensional representation vectors from the encoder. Our defense is independent of the protected encoder’s architecture and does not rely on any assumption about the adversary’s data and query strategy.\n\n---\n**We hope that we have adequately addressed the reviewer’s concerns and that the reviewer will consider raising the rating for the submission.**\n"}, {"review_id": "Hbnc7k45QE", "reviewer": "Reviewer_weVe", "summary": "The authors proposed B4B, which aims at actively preventing model stealing while preserving high-utility representations for\n133 legitimate users. The paper is overall well-organized and easy to follow. The proposed three blocks are clear and effective, namely the embedding space estimation, cost increasing, and per-user representation transformations. This topic also makes sense in practical to protect the commercial APIs. Experiments are well designed.", "questions": "refer to weakness", "limitations": "refer to weakness", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is easy to follow.\n- The motivation is clear enough, and the study is of value for protecting commercial APIs especially for those foundation models.\n- The proposed three blocks are simple yet effective.", "weaknesses": "- Is there any defense baselines in Table 1? \n- Have you tried an ablation study on \"SYBIL\" in Table1? For example, maybe the user has 5+ sybil accounts with 20k queries per account. I think it could be interesting to explore if there is a convergency.\n- I would like to suggest more experiments on existing public APIs. As emphasized in paper, it should be a valuable contribution to a safer sharing and democratization of high-utility encoders over public APIs. It could be better to conduct experiments to directly support this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**Baseline in Table 1**\n\nWe extend Table 1 to include the static addition of noise to all the returned representations (as proposed in [13 (Section A.4),26,32]) as another baseline. \n\nWe enumerate the following observations for such a defense:\n1. If the amount of noise is small ($\\sigma$=0.1) then the performance drop is negligible but for both a legitimate user (row 2) and an adversary (row 6). In this case, the defense does not affect the adversary at all (compare rows 5 & 6).\n2. If the amount of noise is large ($\\sigma$=10) then the performance drop is large for both a legitimate user (row 3) and an adversary (row 7). In this case, the encoder is worthless for legitimate users since the performance is too low. \n\nIn summary, these defenses can either effectively defend stealing (but harm legitimate users), or keep utility for legitimate users high (but not defend well against stealing). In contrast, our B4B defense solves the above problem by providing high performance for legitimate users (row 4) while effectively defending the encoder against stealing attacks (row 8). \n\nFor the Table below, we use the same setup as in Table 1 in our submission (with an ImageNet pre-trained encoder).\n\n|RowID|User|Defense|Number of Queries|Access Type|CIFAR10|STL10|SVHN|F-MNIST|\n|--------|-----------------|-------------------------|----------------|------------|-------------|-------------|-------------|-|\n|*1*|*Legit*|*None*|*Full downstream dataset*|*Query*|*90.41±0.02*|*95.08±0.13*|*75.47±0.04*|*91.22±0.11*|\n|2|Legit|Noise $\\sigma$=0.1|Full downstream dataset|Query|90.20±0.03|95.15±0.13|75.29±0.09|91.24±0.02|\n|3|Legit|Noise $\\sigma$=10|Full downstream dataset|Query|65.11±0.45|76.37±0.14|33.23±0.09|65.83±0.13|\n|4|Legit|**B4B**|Full downstream dataset|Query|90.24±0.11|95.05±0.1|74.96±0.13|91.7±0.01|\n|||||||||||\n|*5*|*Attack*|*None*|*50k ImageNet*|*Steal*|*65.2±0.03*|*64.9±0.01*|*63.1±0.01*|*88.5±0.01*|\n|6|Attack|Noise $\\sigma$=0.1|50k ImageNet|Steal|64.92±0.04|64.61±0.02|62.35±0.01|88.41±0.01|\n|7|Attack|Noise $\\sigma$=10|50k ImageNet|Steal|36.32±0.2|32.588±0.06|20.59±0.01|74.94±0.02|\n|8|Attack|**B4B**|50k ImageNet|Steal|35.72±0.04|31.54±0.02|19.74+-0.02|70.01+-0.01|\n\nThe original baseline in Table 1 (in our submission) was represented by the gray rows which show the performance of the attack without any defense. Thus, without any defense, the performance of the attack is high, while with B4B it decreases significantly. \n\nWe also add results from [13] to show how the static addition of noise can decrease the performance of a stolen model for an attacker but simultaneously lower by the same amount the performance for legitimate users. \nThe table below is for noise based perturbations of the output representations. The values represent the downstream accuracy (%) on CIFAR10 for a legitimate user and an adversary’s stolen encoder. All results are based on 9000 queries from the CIFAR10 test set. The noise added is Gaussian noise with a mean of 0 and standard deviation of $\\sigma$. The results highlight again that both legitimate users and adversaries are equally influenced by the defense.\n|$\\sigma$|Legitimate user|Adversary|\n|----------|-----------------|-----------|\n|0|66.7|66.3|\n|1|63.9|64.8|\n|2|67.2|66.5|\n|3|65|65.5|\n|4|63.1|63.4|\n\n>**5 Sybils**\n\nWe noted in Section 4.3 in the submission: “*Using more accounts for the adversary causes a larger query overhead and potentially more performance loss from remapping.*“ In Table 5 in our submission, we already evaluated more Sybils (4 with 25k per each fake account). We run the additional experiment with 5 Sybils and 20K queries per account. The same as in Table 1, we use 10K queries to obtain good quality remappings between representations from different accounts. With more Sybils, the performance of the stolen encoder drops since the adversary has to sacrifice more queries on remapping between different accounts than on training the stolen encoder. We observe a general trend that the Sybil-based attacks can achieve higher performance of the stolen encoder than a single user attack. However, with more Sybil devices, the cost of remapping grows substantially and saturates rapidly since with 5 Sybil accounts, the attack becomes much less effective than with fewer Sybils. \n\n|Attack|Defense|Number of Queries|Dataset|Type|CIFAR10|STL10|SVHN|F-MNIST|\n|-------------|---------|-------------------|----------|-------|-------------|--------------|-------------|-------------|\n|Single User|None|100k|ImageNet|Query|68.1+-0.03|63.1+-0.01|61.5+-0.01|89.0+-0.07|\n|Single User|B4B|100k|ImageNet|Query|12.01±0.07|13.94±0.05|19.96±0.03|69.63±0.07|\n|2 Sybils|B4B|2*50k|ImageNet|Query|39.56±0.06|38.50±0.04|23.41±0.02|77.01±0.08|\n|5 Sybils|B4B|5*20k|ImageNet|Query|32.65+-0.05|32.45+-0.05|29.63+-0.01|70.12+-0.08|\n\n>**Public APIs**\n\nWe model our experiments after public API setups as closely as possible so that our B4B defense method can be used to protect public APIs against encoder stealing attacks. Our work is motivated by cases from industry collaborators who identified the problems with protecting the intellectual property of their SSL encoders. We do hope that our work can eventually go into production and be used to prevent encoder extraction attacks. As we pointed out in the introduction of our submission, from a practical industry’s viewpoint, such active defenses as ours are required since many popular API providers, such as Cohere, OpenAI, or Clarify [1–3] already expose their high-value SSL encoders via APIs to a broad range of users.\n\n---\n**We hope that we have adequately addressed the reviewer’s concerns and that the reviewer will consider raising the rating for the submission.**\n"}, {"review_id": "kDj9zPiFHQ", "reviewer": "Reviewer_pRsJ", "summary": "This paper studies active defenses against model stealing attacks, with a specific focus on SSL encoders. The authors propose B4B, a framework including three modules: 1) a coverage estimation to distinguish legitimate users and adversaries; 2) a cost function that maps the coverage to a penalty to prevent stealing; 3) a per-user transformation function to prevent sybil attacks. The results show that B4B can prevent stealing and also preserve the utility of representations for legitimate users.", "questions": "see limitations", "limitations": "This paper focuses on a trendy topic: defenses against model stealing attacks. To tackle this problem, the authors propose an active defense framework, namely B4B. They evaluate its performance on two SSL vision encoders, i.e., SimSiam and DINO, and consider four downstream datasets, i.e., CIFAR10, STL10, SVHN, and FashionMNIST. \n\nI appreciate the authors' efforts in conducting extensive ablation studies to understand the impact of hyperparameters. These experiments provide valuable insights and guidelines for readers interested in implementing B4B in real-world applications.\n\nHowever, there are still some concerns:\n\n- Absence of comparison with previous work. The authors claim B4B is the first active defense that prevents stealing without degrading representation quality for legitimate API users. However, it is important to note that [13] has demonstrated active defenses, such as adding random noise, can already achieve this goal. Additionally, other research works have proposed multiple active defenses against encoder stealing attacks, such as [26] (top-k, feature rounding, feature poisoning) and [32] (noise, top-k, rounding). I would suggest the authors reassess their claims and compare B4B with existing defense methods to elaborate on the advantages and unique contributions of B4B.\n- Impractical assumption. The authors declare that \"B4B is independent of the protected encoder’s architecture and does not rely on any assumption about the adversary’s data and query strategy\" (line 148-149). However, the design of B4B's cost function suggests that it primarily works for users who query data from a small distribution, while users with queries from a larger distribution are considered adversaries. This assumption is strong and impractical for API providers. I recommend that the authors discuss this limitation of B4B. Furthermore, it would be valuable to analyze the impact of queried distribution and the corresponding utility to demonstrate the trade-offs of B4B in real-world scenarios.\n- Design intuition lacks persuasiveness. The design of B4B is based on an observation: the representations returned to adversaries cover a significantly larger fraction of the embedding space than representations of legitimate users. However, a previous study [32] has shown that a small surrogate dataset is sufficient to steal the encoder. This raises concerns about the effectiveness of B4B, as attackers are able to employ small datasets to bypass it. Furthermore, in Figures 2 and Figure 9 in the Appendix, the authors visualize downstream datasets in the encoder’s embedding space. It is confusing to observe that STL10 and CIFAR10, two datasets that shared 90% classes, appear to be separated in both figures. Ideally, these datasets should be located in a similar space. This further weakens the design intuition’s persuasiveness. I recommend that the authors review their statements and provide explanations for the two points.\n- Lack of experiments. As a defense method, it is important to show its effectiveness against different encoder stealing attacks [13, 26, 32]. However, the authors only consider [14] in this paper, which limits the utility of their work. I recommend the authors expand their experimental evaluation to include different types of stealing attacks and report the performance of B4B. This will provide a more comprehensive assessment of B4B. Furthermore, the authors propose five transformations, namely Affine, Pad, Add, Shuffle, and Binary. However, there is a lack of experimental analysis regarding the impact of these transformations on B4B’s effectiveness. Additionally, in Table 1, the authors should explicitly mention the specific transformations applied to the legitimate users, attackers, and sybil attacks. This will enhance the clarity and reproducibility of their experiments.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Trendy topic\n- Extensive ablation studies", "weaknesses": "- Absence of comparison with previous work.\n- Impractical assumption\n- Design intuition lacks persuasiveness.\n- Lack of experiments", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments.\n\n>**Comparison with previous work**\n\nAll the previous active defenses add the same amount of noise to each returned representation. All of the them can be integrated into our framework in Block 2, where we can replace the addition of random noise, for example, with top-k or rounding. All the previous defenses show substantial degradation in performance for legitimate users when these defenses are applied. We propose how to solve this problem by adjusting the strength of the perturbation to how much of the representation space is recovered by the querying party. Please see experiments in the general response.\n\nIt was stated in Section A.4 in [13] that “*we need a better method for the defense to make a wider gap in terms of accuracy between legitimate and malicious users*”. Tables 10 and 11 show that adding noise to prevent the stealing substantially lowers the accuracy for legitimate users. [26] stated in Section 9 that their defenses “*are insufficient to mitigate encoder stealing attacks*.” For instance, in Section 7.1 in [26]: “*Our results show that top-𝑘 features are insufficient.*” In particular, if accuracy for the attacker is reduced, then the accuracy for legitimate users is “*also reduced by similar or larger magnitudes.*” Similarly, from Section 7.3 in [26]: “*feature poisoning degrades the functionality of the stolen encoder by degrading the functionality of the target encoder*”. We addressed exactly this issue. \n\nPlease note that only the 2nd version of [32], which introduced active defenses and improved attack, appeared on arXiv less than two months before the NeurIPS submission deadline, thus this work is labeled as contemporaneous. \n\nDespite the above, we carefully analyzed [32] v2. In Section 4.5, they divide all defenses into perturbation-based defense [37] and watermark-based defense [2]. The three perturbation-based defenses proposed in [32] correspond exactly to the same ones from [13] and [26]. The conclusions are also the same as in previous work: “We can observe that while adding noise and top-k can reduce the model stealing attacks’ performance, it may also degrade the target model performance to a large extent.” And in general: “perturbation-based defense cannot defend against the encoder’s model stealing attack effectively since they cannot reach a good trade-off between attack performance and model utility.” We show that it is possible!\n\n[32] v2 lacks watermarks: SSL Guard [11] and [13].\n\n[32] v2 incorrectly assesses the dataset inference. The results in Table 8 do not show p-values and the effect sizes, which are two necessary metrics to be computed for dataset inference. \n\n[32] only reports the mutual information and cosine similarity, which are metrics to compare the quality of stolen encoder vs the victim encoder and should not be used for resolving the ownership resolution, as stated in the original paper on dataset inference for SSL.\n\n>**Impractical assumption**\n\nThe end users cannot afford to train SSL models on the scale done by large companies (e.g., Google or Meta). We model a real world use case where the query set of end users is much less diverse and smaller than the training set of SSL models. The trade-offs between query distribution and corresponding utility with B4B is shown in Table 1: legitimate users’ queries occupy relatively smaller representation space and their utility is preserved as opposed to attackers whose queries are spread over the whole representation space and B4B degrades their performance.\n\n>**Design intuition**\n\nThe occupation of the embedding space does not depend on the size of the dataset but the selection of queries. In the limit, a large dataset with the same image copied many times occupies a small fraction of the representation space. Similarly, a small dataset with diverse images occupies a large representation space. This is shown in Figure 3a: for the same number of queries, the images from FashionMNIST (representing only 10 classes) occupy much smaller representation space than ImageNet images (drawn from 1000 classes). Note that [32] states in A2 that “Cont-Steal can have better performance with a larger surrogate dataset size and more training epochs”.\n\nRegarding different clustering of CIFAR10 and STL10, the embedding space is *not* created based on classes but input images. STL10 is derived from ImageNet with high resolution images and many diverse classes, which is much different from TinyImages, with low resolution images that CIFAR10 is derived from. This is why STL10 and CIFAR10 are separated in Figures 2 and 9.\n\n>**Experiments**\n\nThe attacks proposed in [13] are the same as the ones used in [14]. The [32] v2 (contemporaneous work published less than two months before the NeurIPS submission) shows that attacks from [13] are better than from [26] and [32] claims only small improvement over [13]. The attacks from [13] are based on contrastive learning loss and with data augmentations, which then are replicated in [32]. Note that [32] incorrectly claims in their Section A.7 that attacks from [13] do not use data augmentations. Algorithm 1 and Figure 1 clearly show that attacks from [13] do use data augmentations. Thus, we do evaluate B4B against state-of-the-art attacks.\n\n[32] did not release their source code so it remains infeasible for us to run this evaluation in a limited time of the rebuttal.  \n\nThe transformations are used to defend encoders against Sybil attacks. The impact of the transformations and their comparison is discussed in Section 4.3, specifically the trade-offs between the effectiveness of the defense and the negative impact on legitimate users. We state at the beginning of Section 4.4 that we use the affine transformations for experiments in Table 1. It shows that B4B is effective against Sybil attacks as a result of applying the transformations. \n\n---\n**We hope we addressed the reviewer’s concerns and the score can be increased.**"}]

Author rebuttal:
We would like to thank all the reviewers for their positive feedback and insightful comments. The paper improved as a result of your reviews. 

We tackle a current urgent problem of designing an active defense for encoder stealing attacks (reviewers FtSs, N8vt, pRsJ). The proposed approach is interesting, and is generally applicable to different models and tasks (reviewer FtSs), making it practical to protect the commercial APIs (reviewer weVe). The proposed three blocks of the B4B defense are clear and effective, namely the embedding space coverage estimation, cost function that disincentivizes attackers, and per-user representation transformations to prevent against Sybil attacks (reviewers weVe, pRsJ). The results show that B4B does not degrade the response quality for legitimate users, while significantly decreasing the performance of stolen encoders (reviewers FtSs, N8vt). Overall, the paper is well-written, easy to read (reviewers N8vt, weVe), and experiments are well-designed (weVe) and further extended with extensive ablation studies (reviewer pRsJ).

Below in the general response, we provide answers to questions that occur in more than a single review and the other questions are answered one-by-one separately for each reviewer. 

We also attach the 1-page PDF that shows two Figures (corresponding to Figure 3a in the main submission) with the coverage of the representation space for: (1) ImageNet pretrained encoder when queried with diverse datasets, namely LAION, ImageNet with all classes, ImageNet with only classes of dog breeds, and CIFAR10, as well as for (2) the CIFAR10 pre-trained encoder queried with STL10, CIFAR10, CIFAR100, SVHN, and FashionMNIST.

>**1. Baseline & comparison with previous work** (reviewers weVe and pRsJ)

We extend Table 1 to include the static addition of noise to all the returned representations (as proposed in [13 (Section A.4),26,32]) as another baseline. 

We enumerate the following observations for such a defense:
1. If the amount of noise is small ($\sigma$=0.1) then the performance drop is negligible but for both a legitimate user (row 2) and an adversary (row 6). In this case, the defense does not affect the adversary at all (compare rows 5 & 6).
2. If the amount of noise is too large ($\sigma$=10) then the performance drop is large for both a legitimate user (row 3) and an adversary (row 7). In this case, the encoder is worthless for legitimate users since the performance is too low. 

In contrast, our B4B defense solves the above problem by providing high performance for legitimate users (row 4) while effectively defending the encoder against stealing attacks (row 8). 

For the Table below, we use the same setup as in Table 1 in our submission (with an ImageNet pre-trained encoder).

|RowID|User|Defense|Number of Queries|Access Type|CIFAR10|STL10|SVHN|F-MNIST|
|-|-|-|-|-|-|-|-|-|
|*1*|*Legit*|*None*|*Full downstream dataset*|*Query*|*90.41±0.02*|*95.08±0.13*|*75.47±0.04*|*91.22±0.11*|
|2|Legit|Noise $\sigma$=0.1|Full downstream dataset|Query|90.20±0.03|95.15±0.13|75.29±0.09|91.24±0.02|
|3|Legit|Noise $\sigma$=10|Full downstream dataset|Query|65.11±0.45|76.37±0.14|33.23±0.09|65.83±0.13|
|4|Legit|**B4B**|Full downstream dataset|Query|90.24±0.11|95.05±0.1|74.96±0.13|91.7±0.01|
|||||||||||
|*5*|*Attack*|*None*|*50k ImageNet*|*Steal*|*65.2±0.03*|*64.9±0.01*|*63.1±0.01*|*88.5±0.01*|
|6|Attack|Noise $\sigma$=0.1|50k ImageNet|Steal|64.92±0.04|64.61±0.02|62.35±0.01|88.41±0.01|
|7|Attack|Noise $\sigma$=10|50k ImageNet|Steal|36.32±0.2|32.588±0.06|20.59±0.01|74.94±0.02|
|8|Attack|**B4B**|50k ImageNet|Steal|35.72±0.04|31.54±0.02|19.74+-0.02|70.01+-0.01|

>**2. The LAION dataset used for stealing from the ImageNet pre-trained encoder** (reviewer FtSs)

We show that: (1) we can steal the ImageNet pre-trained encoder using the LAION dataset, which is from a different distribution than the victim's training set, and (2) B4B effectively defends against such attacks, where the accuracy on downstream tasks drops significantly (avg. > 20%) when B4B is applied compared to stealing without any defense. All the results (accuracy on downstream tasks in %) in the Table below are for the ImageNet victim encoder. In all cases we use 100k queries to steal the encoder (the same setup as in Table 1 in our submission).

|Defense|Dataset|CIFAR10|STL10|SVHN|F-MNIST|
|-|-|-|-|-|-|
|None|ImageNet|68.1±0.03|63.1±0.01|61.5±0.01|89.0±0.07|
|B4B|ImageNet|12.01±0.07|13.94±0.05|19.96±0.03|69.63±0.07|
|None|LAION|64.92+-0.03|62.51±0.03|59.02±0.02|84.54±0.01|
|B4B|LAION|40.96+-0.03|40.69±0.05|34.43±0.01|72.92±0.01|

We also present the corresponding coverage of the representation space (% of occupied buckets, please also see Figure 1 in the attached PDF). When querying the encoder with ImageNet-Full (includes all 1000 classes) and LAION datasets, they both occupy a large fraction of the representation space of the victim encoder. In contrast, CIFAR10 covers the smallest portion of the representation space as the simplest dataset tested. ImageNet-Dogs (with only 118 classes for dog breeds) falls in the middle, occupying more space than CIFAR10 but less than ImageNet-Full and LAION. Its intermediate coverage aligns with its mid-level difficulty compared to the other datasets. As indicated by representation space coverage, stealing the encoder is similarly effective with ImageNet-Full and LAION datasets.

|Query Dataset↓/Number of Queries→|5000|10000|20000|30000|40000|50000|
|-|-|-|-|-|-|-|
|ImageNet-Full|31.66|38.61|48.32|54.02|58.03|61.15|
|LAION|30.75|37.45|47.16|52.69|56.53|59.56|
|ImageNet-Dogs|22.47|26.93|33.34|37.2|40.01|42.19|
|CIFAR10|11.59|13.98|17.55|19.79|21.44|22.81|

---
**We hope that the provided answers address reviewers’ concerns and that the scores can be increased.**

Meta-review: This paper addresses robustness issues in MLaaS APIs to offer pre-trained encoders for generating vector representations of input data. These encoders are expensive to train and are vulnerable to model theft, where attackers use query access to replicate the encoder at a fraction of the cost. The proposed solution, called "Bucks for Buckets" (B4B), is an active defence mechanism that thwarts theft attempts. B4B identifies that representations returned to attackers cover a larger portion of the embedding space compared to legitimate users, and it adjusts representation quality based on user coverage. Additionally, B4B individually transforms each user's representations to prevent attackers from aggregating them across multiple accounts. 

The reviewers initially raised concerns regarding the paper's motivations, the need for additional experiments, and the effectiveness of its comparison with existing methods. During the rebuttal phase, the authors provided additional results and addressed the feedback, which led to a consensus among most reviewers in favour of accepting the paper. However, one reviewer did not engage in the rebuttal phase despite direct communication.

This particular reviewer's primary concerns centred around the paper's underlying assumptions, the clarity of its intuition, and the robustness of its comparisons and experiments. Upon thoroughly examining the authors' rebuttal, I believe all these concerns have been satisfactorily addressed. Consequently, I confidently recommend the acceptance of the paper.

To enhance the paper's final version, I advise the authors to incorporate the rebuttal responses and the new results into the camera-ready manuscript. Additionally, it would be beneficial to provide further clarification in the final version regarding the specific concerns raised by the reviewers during the evaluation process.
Decision: Accept (poster)