SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Utility Preserving Transformations. As a concrete instantiation for our B4B, we propose a set of transformations that fulfill the above-mentioned two requirements: (1) Affine where we apply affine transformations to representations, (2) Pad where we pad representations with constant values, (3) Add where we add constant values at random positions within representations, (4) Shuffle where we shuffle the elements in the representation vectors, and (5) Binary where the original representations are mapped to binary vectors relying on a random partitioning of the representation space. To preserve the full amount of information contained in the original representations, in our binary transformations, we tune the length of binary representations. We visualize the operation of each of these transformations in Appendix C. All these transformations can additionally be combined with each other, which further increases the possible set of transformations applied per user. This renders it impossible for an adversary to correctly guess and reverse the applied representation. Instead, the adversary has to remap the representations over all accounts into a single embedding space in order to unify them and leverage them for training of their stolen encoder copy. We present an exhaustive list of strategies that adversaries can apply for the remapping in Appendix D. All the attack methods reduce to the minimum of remapping between representations of a pair of users, i.e., they are at least as complex as mapping between two separate accounts. In the next section, we show that our defense already impedes stealing for an adversary with two accounts.
We first empirically evaluate our instantiation of B4B's three building blocks and show how to calibrate each of them for our defense. Finally, we provide an end-to-end evaluation that highlights B4B's effectiveness in preserving downstream utility for legitimate users while successfully preventing the stealing by adversaries.
Experimental Setup. We conduct experiments on various kinds of downstream tasks and two popular SSL encoders. To test our defense, we use FashionMNIST, SVHN, STL10, and CIFAR10 as our downstream datasets, each with standard train and test splits. For stealing, we utilize training data from ImageNet and LAION-5B. We rely on encoder models from the SimSiam [12] and the DINO [9] SSL frameworks. As our victim encoders, we use the publicly available ResNet50 model from SimSiam trained for 100 epochs on ImageNet and the ViT Small DINO encoder trained for 800 epochs on ImageNet, each using batch size 256. The ViT architecture takes as input a grid of non-overlapping contiguous image patches of resolution  $N \times N$ . In this paper, we typically use  $N = 16$ . The Simsiam encoder has an output representation dimension of 2048, while DINO returns 1536 dimensional representations. We examine the utility of downstream classifiers using SimSiam's or DINO's representations obtained for the respective downstream datasets. To implement LSH, we rely on random projections [34] that we implement from scratch. For a detailed description of our stealing and downstream training parameters, we refer to Appendix F.
(a) Fraction of Occupied Buckets.
(b) Number of Queries per Bucket.
Figure 3: Estimating Embedding Space Coverage through LSH on SimSiam Encoder. We present the fraction of buckets occupied by representations of different datasets as a function of the number of queries posed to the encoder (left). We observe that representations for the downstream datasets (FashionMNIST, SVHN, CIFAR10) occupy a smaller fraction of buckets than representations from the complex ImageNet dataset. Our evaluation of the number of queries whose representations are mapped to the same bucket (right) indicates that our total number of buckets  $(2^{12})$  is well calibrated for the estimation of covered representation space: over all datasets, we experience hash collisions, i.e., queries whose representations are mapped to the same buckets. This indicates that our LSH is capable of representing similarities in the representations.
4.1 Local Sensitive Hashing for Coverage Estimation
We first observe that the choice of the total number of hash buckets in the LSH influences the effectiveness of our method. In the extreme, if we have a too large number of buckets, the number of buckets filled will correspond to the number of queries posed by a user which fails to capture that similar representations cover similar sub-spaces of the embedding space, and hence does not serve to approximate the total fraction of the embedding space covered. However, if we have too few buckets, even the representations for simple downstream tasks will fill large fractions of buckets, making it impossible to calibrate the cost function such that it only penalizes adversaries. We experimentally find that for our evaluated encoders,  $2^{12}$  buckets represent a good trade-off. In Appendix F.6, we present an ablation study on the effect of the number of total buckets.
Our evaluation of the LSH to track coverage of the embedding space is presented in Figure 3. We observe that representations returned for standard downstream tasks (FashionMNIST, SVHN, CIFAR10) occupy a significantly smaller fraction of the total number of buckets than complex data from multiple distributions (ImageNet, LAION-5B). We present additional experimental results on measuring the coverage of the representation space in Appendix F.5. Specifically, we show that our method of measuring the embedding space coverage has broad applicability across various encoders and datasets used for pretraining. We further observe that the fraction of buckets occupied by the representations saturates over time. These findings highlight that LSH is successful in capturing the differences between legitimate users and adversaries—even in a low-query regime. Finally, we note that our total number of buckets  $(2^{12})$  is well calibrated since, over all datasets, it successfully maps multiple representations to the same hash bucket while still filling various fractions of the total number of buckets.
4.2 Calibrating the Cost Function
We experiment with different sets of hyperparameters to instantiate the cost function from Equation (1) in the previous section (3.3). As described there, we can calibrate the function (as shown in Figure 4) such that a desired penalty (in the form of a specific  $\sigma$ ) will be assigned at a certain fraction of buckets occupied. For B4B, we aim at penalizing high embedding space coverage severely. Therefore, we need to identify and optimize for two components: 1) which value of  $\sigma$  leads to significant performance drops, and 2) for what
Figure 4: Cost Function Calibration.
fraction of coverage do we want to impose this significant drop. We base both components on empirical observations. Our first observation is that for our four downstream tasks (FashionMNIST, SVHN, STL10, and CIFAR10), performance drops to  $10\%$  (i.e., random guessing) at roughly  $\sigma = 0.5$ . In Figure 3, we further see that with 50k queries, the downstream tasks occupy  $< 30\%$  of the buckets. Ultimately, setting  $\alpha$  and  $\beta$  are design choices that an API provider needs to make in order to specify what type of query behavior they want to penalize. As very loose bounds (weak defense), based on our observation, we consider  $\sigma = 1$  as a high penalty, which leads to  $\alpha = 1$ , and select  $\beta = 0.8$ . This  $\beta$  corresponds to considering  $80\%$  of buckets filled as a too-large coverage of the embedding space. We empirically observe that coverage of  $80\%$  of buckets occurs, for example, after around 100k of ImageNet queries. By choosing our target  $\beta$  so loose, i.e., significantly larger than the observed  $30\%$  for downstream tasks, we offer flexibility for the API to also provide good representations for more complex downstream tasks. Finally, to obtain a flat cost curve close to the origin—which serves to map small fractions of covered buckets to small costs—we find that we can set  $\lambda = 10^{-6}$ . In the Appendix, we evaluate our defense end-to-end with differently parameterized cost functions.
4.3 Assessing the Effect of Transformations
Transformations Do Not Harm Utility for Legitimate Users. We evaluate the downstream accuracy for transformed representations based on training a linear classifier on top of them. To separate the effect of the noise added by our defense from the effect of the transformations, we perform the experiments in this subsection without adding noise to the returned representations. For example, on the CIFAR10 dataset and a SimSiam encoder pre-trained on ImageNet, without any transformations applied, we obtain a downstream accuracy of  $90.41\%$  ( $\pm 0.02$ ), while, with transformations, we obtain  $90.24\%$  ( $\pm 0.11$ ) for Affine,  $90.40\%$  ( $\pm 0.05$ ) for Pad+Shuffle,  $90.18\%$  ( $\pm 0.06$ ) for Affine+Pad+Shuffle, and  $88.78\%$  ( $\pm 0.2$ ) for Binary. This highlights that the transformations preserve utility for legitimate users. This holds over all datasets we evaluate as we show in Appendix F.
Adversaries Cannot Perfectly Remap Representations over Multiple Sybil Accounts. To understand the impact of our per-user account transformations on sybil-attack based encoder stealing, we evaluate the difficulty of remapping representations between different sybil accounts. For simplicity, and since we established in Section 3.4 that multi-account attacks reduce to a two-account setup, we assume an adversary who queries from two sybil accounts and aims at learning to map the transformed representations from account #2 to the representation space of account #1. Using more accounts for the adversary causes a larger query overhead and potentially more performance loss from remapping. Our evaluation here, hence, represents a lower bound on the overhead caused to the adversary through our transformations.
We learn the mapping between different accounts' representations by training a linear model on overlapping representations between the accounts. We assess the fidelity of remapped representations as a function of the number of overlapping queries between the accounts. As a fidelity metric for our remapping, we compare the cosine distance between representations ( $a$  and  $b$  defined as:  $1 - \frac{a^T b}{||a||_2 \cdot ||b||_2}$ ). Once the remapping is trained, we evaluate by querying 10k data points from the test dataset through account #1 and then again through account #2. Then, we apply the learned remapping to the latter one and compute the pairwise cosine distances between the representations from account #1 and their remapped counterparts from account #2.
Our results are depicted in Figure 5. We show that the largest cosine distance is achieved with the binary transformations, making them the most protective against the adversary since they best prevent perfect remapping, even with an overlap of as many as 10k queries between both accounts. However, these binary transformations also incur the highest drop in accuracy for legitimate users. The defender has the possibility of selecting their preferred types of transformations between representations taking into account the trade-offs between the effectiveness of the defense and the negative impact on legitimate users.
Figure 5: Quality of Remappings.
Table 1: Stealing and Using Encoders With and Without our Defense. The USER column represents the type of the APIs' user, where LEGIT denotes a legitimate user, ATTACKER stands for a standard single-account adversary, and SYBIL represents an adversary using two sybil accounts. We use InfoNCE loss for encoder extraction. # Queries stands for the number of queries used for stealing with ALL denoting that the entire downstream dataset was used. The TYPE column expresses how the dataset is used. We follow the stealing setup from [17]. In the first row, we present the undefended victim encoder's performance as the accuracy for downstream tasks trained on the encoder's returned representations. In the following row, we show downstream utility for legitimate users when the victim encoder is defended by our B4B. Finally, (in the remaining rows) we assess the performance of stolen encoders on the downstream tasks. Our results highlight that while the performance of the encoder for legitimate users stays high, our B4B renders stealing inefficient with the stolen encoders obtaining significantly worse performance on downstream tasks.
USER	DEFENSE	# QUERIES	DATASET	TYPE	CIFAR10	STL10	SVHN	F-MNIST
LEGIT	NONE	ALL	TASK	QUERY	90.41 ±0.02	95.08±0.13	75.47±0.04	91.22±0.11
LEGIT	B4B	ALL	TASK	QUERY	90.24±0.11	95.05±0.1	74.96±0.13	91.7±0.01
ATTACK	NONE	50K	IMGNET	STEAL	65.2±0.03	64.9±0.01	63.1±0.01	88.5 ±0.01
ATTACK	B4B	50K	IMGNET	STEAL	35.72±0.04	31.54±0.02	19.74±0.02	70.01±0.01
ATTACK	NONE	100K	IMGNET	STEAL	68.1 ±0.03	63.1 ±0.01	61.5 ±0.01	89.0 ±0.07
ATTACK	B4B	100K	IMGNET	STEAL	12.01±0.07	13.94±0.05	19.96±0.03	69.63±0.07
ATTACK	NONE	100K	LAION	STEAL	64.92±0.03	62.51±0.03	59.02±0.02	84.54±0.01
ATTACK	B4B	100K	LAION	STEAL	40.96±0.03	40.69±0.05	34.43±0.01	72.92±0.01
SYBIL	B4B	2×50K	IMGNET	STEAL	39.56±0.06	38.50±0.04	23.41±0.02	77.01±0.08
SYBIL	B4B	3×33.3K	IMGNET	STEAL	33.87±0.05	38.57±0.06	21.16±0.01	72.95±0.05
SYBIL	B4B	4×25K	IMGNET	STEAL	33.98±0.04	34.52±0.08	21.21±0.02	70.71±0.05
SYBIL	B4B	5×20K	IMGNET	STEAL	32.65±0.05	32.45±0.05	29.63±0.01	70.12±0.08
SYBIL	B4B	6×16.7K	IMGNET	STEAL	26.62±0.04	26.85±0.05	24.32±0.02	70.51±0.04
4.4 End-to-End Stealing of an Encoder under our Defense
We perform an end-to-end study to showcase how our B4B defense affects legitimate users vs adversaries. The hyperparameters for B4B are chosen according to the empirical evaluation of the previous sections with  $2^{12}$  as the number of buckets,  $\alpha = 1$ ,  $\beta = 0.8$ ,  $\lambda = 10^{-6}$  as the hyperparameter of the cost function, and different random affine transformations per-user account. Our main results are presented in Table 1. We observe that instantiating our framework with B4B has a negligible impact on legitimate users while substantially lowering the performance of stolen encoders in the case of single-user and sybil attackers.
**Legitimate Users.** We compare the accuracy of downstream classifiers trained on top of unprotected vs defended encoders. The victim encoder achieves high accuracy on the downstream tasks when no defense is employed. With B4B in place, we observe that across all the downstream tasks, the drop in performance is below  $1\%$ . For example, there is only a slight decrease in the accuracy of CIFAR10 from  $90.41 \pm 0.02\%$  to  $90.24 \pm 0.11\%$ . B4B's small effect on legitimate users stems from the fact that their downstream representations cover a relatively small part of the representations space. This results in a very low amount of noise added to their representations which preserves performance.
Adversaries. For adversaries who create a stolen copy of the victim encoder, we make two main observations. The most crucial one is that when our B4B is in place, the performance of the stolen copies over all downstream tasks significantly drops in comparison to when the victim encoder is unprotected (grey rows in Table 1). This highlights that our B4B effectively prevents stealing. Our next key observation concerns the number of stealing queries used by the adversary: When no defense is applied, the more queries are issued against the API (e.g., 100k instead of 50k), the higher performance of the stolen encoder on downstream tasks (e.g., CIFAR10 or FashionMNIST). In contrast, with B4B implemented as a defense, the performance decreases when using more stealing queries from a single account. This is because with more queries issued, the coverage of embedding space grows which renders the returned representations increasingly noisy and harms stealing performance.
Moreover, we show that B4B can also prevent model stealing attacks with data from a different distribution than the victim encoder's training set. We highlight this in Table 1 where we also use the LAION-5B dataset to steal an ImageNet pre-trained encoder. Our results highlight first that without any defense in place, the LAION dataset is highly effective to extract the ImageNet pre-trained encoder. Second, B4B effectively defends against such attacks, and yields a significant drop in downstream accuracy (on average above  $20\%$ ) of the stolen encoder.
We also show that this performance drop cannot be prevented by sybil attacks. Therefore, we first consider an adversary who queries from two sybil accounts with 50k queries issued per account and the first 10k queries of both accounts used to learn the remapping of representations between them. When the adversary trains their stolen encoder copy on all the remapped representations, they increase downstream performance over querying from a single account. Yet, their performance is still significantly smaller than the performance of the victim encoder for legitimate users, or the encoder stolen from an undefended victim. Moreover, using more than two sybil accounts further reduces the attack performance as remapping complications accumulate. With ten sybils, remapping leaves no more usable data for training the stolen encoder. This demonstrates our method's advantage: increasing the number of sybil accounts makes encoder extraction impractical due to the growing remapping overhead. Overall, the results highlight that our B4B also successfully prevents sybil attacks.
4.5 Baseline Comparison
Finally, we compare our B4B against the current state-of-the-art baseline defense, namely a static addition of noise to all the returned representations (as proposed in [16] (Section A.4),[29, 36]). For direct comparability, we evaluate the defense using the our end-to-end experiment setup from the previous section. We present our results in Table 6 in Appendix F.4. Confirming the findings from [16] our results also show that defenses that rely on static noise have the great disadvantage to harm legitimate users and attackers equally. When adding noise with a small standard deviation of  $\sigma = 0.1$ , we observe a negligible  $(< 1\%)$  performance drop for both attackers and legitimate users. Adding noise with a large standard deviation of, for example,  $\sigma = 10$ , we observe that both legitimate users' and attackers' performance drops between  $15\%$  and  $>40\%$ . In summary, these defenses can either effectively defend stealing (but harm legitimate users), or keep utility for legitimate users high (but not defend well against stealing). In contrast, our B4B is able to provide high performance for legitimate users while effectively defending the encoder against stealing attacks.
We design B4B a new and modular active defense framework against stealing SSL encoders. All the previous approaches were either reactive, acting after the attack happened to detect stolen encoders, or lowered the quality of outputs substantially also for legitimate users which rendered such mechanisms impractical. We show that B4B successfully distinguishes between legitimate users and adversaries by tracking the embedding space coverage of users' obtained representations. B4B then leverages this tracking to apply a cost function that penalizes users based on the current space coverage, for instance, by lowering the quality of their outputs. Finally, B4B prevents sybil attacks by implementing per-user transformations for the returned representations. Through our experimental evaluation, we show that our defense indeed renders encoder stealing inefficient while preserving downstream utility for legitimate users. Our B4B is therefore a valuable contribution to a safer sharing and democratization of high-utility encoders over public APIs.
This research was supported by Warsaw University of Technology within the Excellence Initiative Research University (IDUB) programme, National Science Centre, Poland grant no 2020/39/O/ST6/01478, grant no 2020/39/B/ST6/01511, grant no 2022/45/B/ST6/02817, and in part by PL-Grid Infrastructure grant nr PLG/2022/016058. The authors applied for a CC BY license to any Author Accepted Manuscript (AAM) version arising from this submission, in accordance with the grants' open access conditions.
[35] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
[36] Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Can't steal? cont-steal! contrastive stealing attacks against image encoders. 2022. URL https://arxiv.org/abs/2201.07513.
[37] Yun Shen, Xinlei He, Yufei Han, and Yang Zhang. Model stealing attacks against inductive graph neural networks. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1175-1192. IEEE, 2022.
[38] Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry perspectives. In 2020 IEEE Security and Privacy Workshops (SPW), pages 69-75, 2020. doi: 10.1109/SPW50608.2020.00028.
[39] Malcolm Slaney and Michael Casey. Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal processing magazine, 25(2):128-131, 2008.
[40] Florian Tramér, F. Zhang, A. Juels, M. Reiter, and T. Ristenpart. Stealing machine learning models via prediction apis. USENIX Security Symposium, 2016.
[41] Jean-Baptiste Truong, Pratyush Maini, Robert J. Walls, and Nicolas Papernot. Data-free model extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021.
[42] Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pages 269-277, 2017.
[43] Yutong Wu, Han Qiu, Tianwei Zhang, Lin Jiwei, and Meikang Qiu. Watermarking pre-trained encoders in contrastive learning. ArXiv, abs/2201.08217, 2022.
[44] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03230, 2021.
A Broader Impacts
The goal of our work is to actively defend self-supervised encoders against model stealing attacks. Since we are directly defending encoders, any negative societal impacts of our work are minimal. One potentially negative impact could be the degradation of performance for legitimate users. However, as shown in our experimental results, we are able to preserve high utility for standard users.
B Limitations
We show how our defense method is tuned for SimSiam and DINO. There are more types of SSL encoders that can be tested with our method. The B4B defense method requires tuning the parameters, such as the number of occupied buckets that is allowed without any penalty for the cost function, or the selection of the transformations. These steps are rather difficult to automate but can be replaced with more data-driven approaches. For example, instead of designing a cost function from scratch, one could create an ML model to obtain a cost for a given occupation of the representation space. We explain more details in the Appendix C.2.
C Alternative Building Blocks to Instantiate B4B
While we present a reference implementation of B4B in our work that instantiates the three building blocks with (1) Local Sensitive Hashing, (2) Utility of the Representations, and (3) a set of concrete transformations, there exists a multitude of alternatives to concretely implement our B4B framework. In the following, we present these alternatives, grouped by building block.
C.1 Alternative Estimation of the Coverage of Embedding Space