Measuring Dejavu Memorization Efficiently
Authors: Narine Kokhlikyan
Bargav Jayaraman
Florian Bordes
Chuan Guo
Kamalika Chaudhuri
Venue: NeurIPS 2024 poster (2024)
Keywords: memorization
privacy
Primary area: privacy
TL;DR: The deja vu memorization test measures training data memorization, but is inefficient because it involves training another similar model.  This work provides a simpler, more efficient way to carry out the test.

Abstract: Recent research has shown that representation learning models may accidentally memorize their training data. For example, the déjà vu method shows that for certain representation learning models and training images, it is sometimes possible to correctly predict the foreground label given only the representation of he background – better than through dataset-level correlations. However, their measurement method requires training two models – one to estimate dataset-level correlations and the other to estimate memorization. This multiple model setup becomes infeasible for large open-source models. In this work, we propose alter native simple methods to estimate dataset-level correlations, and show that these can be used to approximate an off-the-shelf model’s memorization ability without any retraining. This enables, for the first time, the measurement of memorization in pre-trained open-source image representation and vision-language models. Our results show that different ways of measuring memorization yield very similar aggregate results. We also find that open-source models typically have lower aggregate memorization than similar models trained on a subset of the data. The code is available both for vision (https://github.com/facebookresearch/DejaVuOSS) and vision language (https://github.com/facebookresearch/VLMDejaVu) models.

Measuring Déjà vu Memorization Efficiently
Narine Kokhlikyan FAIR Meta
Bargav Jayaraman FAIR Meta
Florian Bordes FAIR at Meta
Chuan Guo FAIR at Meta
Kamalika Chaudhuri
FAIR at Meta
Abstract
Recent research has shown that representation learning models may accidentally memorize their training data. For example, the déjă method shows that for certain representation learning models and training images, it is sometimes pos- sible correctly predict the foreground label given only the representation of the background better than through dataset-level correlations. However, their measurement method requires training two models one to estimate dataset-level correlations and the other to estimate memorization. This multiple model setup becomes infeasible for large open-source models. In this work, we propose alter- native simple methods to estimate dataset-level correlations, and show that these can be used to approximate an off fff-the-shell model's s memorization ability without any retraining. This enables, for the first time, the measurement of memorization in pre-trained open-source image representation and vision-language models. Our results show that different ways of measuring memorization yield very similar aggregate results. We also find that open-source models typically have lower aggre- gate memorization than similar models trained on subset of the data. The code is available both for vision and vision language models.
Introduction
Representation learning has emerged as of the major tasks in computer vision. The goal in representation learning is to learn a model that produces semantically meaningful representations, where images image-text pairs that close in meaning occur close together in representation space. These learned representations can then be used in numerous downstream applications such as semantic segmentation [Kirillov al. 2023), image generation [Rombach et al., [022] and multi-modal LLMs [Liu et al.] 2024]. A natural question that arises is whether these representation learning models memorize their training data and what extent. Excessive memorization may call the generalization abilities of the models into question. Thus, there is a need to develop a way to measure if and what extent memorization is taking place.
Since learned representations are usually abstract and hard to interpret, memorization measurement for representation learning models requires careful design. Currently, standard way of doing this is the déjä vu method, which designs a causal task of predicting parts of the training sample given another disjoint part, and uses performance
this task to determine if the model memorizes. For example, Mcehan et al. 2023) designed the task of predicting the foreground object given the background crop of a training image, as shown in Figure (orange block). Achieving a high performance on this task indicates two possibilities: (i) if the model has memorized the association of the background crop with the foreground object for specific training sample, (ii) if the model has leamed the dataset-level correlation between the background crop and given foreground object. To rule out the second possibility, |Meehan al. [2023) opted for two-model approach, training two separate models on disjoint parts of the training set and using the gap in performance between
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Finding memorized examples
Maddrindd Find
Finding correlated examples
Correlated!
S
KNN Classite
black
SSLA
KNN Classifer
Argiope
aurantia
Train image
Train image 4,
Two-model
Two-model Test
SSL.g
KNN Classifer
water snale
SSLB
KNN
Argiope
aurantia
Correlated
Correltted
Reshet Clasiffer
wafer
BesNet
Argiope
aurantia
water, ripple;
Naive Bayes
Ofject
Spider
Nuive Bayes
Arglope
Clasifier
Amnotator
Clasitter
aurantla
One-model Test (Ours)
One-model Test (Ours)
Figure I: Illustration of our one-model déjā vu test for image representation learning. The task is predict the foreground object given background crop. The original déjă test [Meehan et al. 2023] trains two models SSLA and SSL on disjoint splits of the training set, and uses SSL to quantify the degree of dataset-level correlation between the foreground and background crop. Our one-model test replaces SSLg with classifier that directly predicts the foreground given background crop, and we show that both ResNet50 network and Naive Bayes classifier work well for this purpose.
the two models as indication of memorization. This approach has been extended by Jayaraman et al. 2024 to measure memorization of vision-language models such CLIP (Radford et al. [02T].
Despite the success of the déjà vu method in defining and measuring memorization, scaling this approach to state-of-the-art representation learning models is challenging. First of all, the two-model approach requires the model trainer to split the training set into disjoint halves, which severely constrains the valuable training data. Moreover, even if data is abundant, training the second model on internet-scale datasets is computationally expensive. Due to these limitations, the déjà vu test cannot be used to measure memorization of pre-trained models out-of-the-box.
In this work, we provide simple alternative ways of quantifying dataset-level correlations and show that they suffice for the purpose of measuring déjà memorization. Specifically, for image representation lcarning, we propose two alternative ways to derive reference models to predict the foreground label from background crop: training image classification network directly, and using a Naive Bayes classifier on top of a pre- trained object detection model. We then leverage these reference models to define one-model déjà vu test-a memorization test for representation learning models that only requires training a simpler reference model once per dataset. FFigure T] (green block) gives illustration of our proposed one-model déjà vu test. We also propose variant of the method for vasion-language models by leveraging pre-trained text embedding model.
We validate our proposed methods by comparing them to the two-model test on ImageNet-trained image representation learning models, as well as CLIP models trained on a privately licensed image-caption pair dataset. We find that the one-model test successfully identify memorized examples and obtain similar population-level memorization scores as the two-model test. We then apply the one-model déjà test pre-trained open-source models and provide for the first time principled memorization measurement on these models. Our results reveal that open-source models have significantly lower memorization rates than similar models trained smaller subset of data. We conclude that our one-model test can be a practical tool for evaluating memorization rates in representation learning models.
Contributions.
To summarize, main contributions follows:
1. We develop simple and efficient methods to quantify dataset-level correlations for both image-only and vision-language representation learning models. Our methods enable déjà vu memorization tests without training two models on disjoints splits of the training set.
We validate our proposed methods by comparing them to the two-model test, and analyze the strengths and weaknesses of both tests.
3. We evaluate the one-model déjà test open-source image-only and vision-language represen- tation models. Our test reveals that open-source models memorize specific training samples, but overall to a lesser degree than the same model trained smaller subsets of data.

---

2
Related Work
A body of literature has been built around how to detect and measure memorization of large foundation models.
The first line of work is on extraction attacks [Carlini et al.
where the goal is to extract snippets of training data from model. These attacks typically tend work well when models are trained on data that is duplicated many times [Kandpal ct al. 2022]. and are successful on a very small fraction of training data. Consequently, is challenging to use them to develop a consistent metric that can be used to compare different models in terms of their memorization capacity.
2019 2021
The second line is membership inference (Shokri et al. 2017]. 1. which involve statistic, such as,
a loss function or score, where low values suggest membership in the training set. State-of-the-ar membership inference attacks Carlini al. [2022]. Watson al. [202T] also involve training multiple "shadow models" that are used to calibrate the values of the statistics. Membership inference tests have close connections to overfitting Yeom et al. [0018] in that the statistic is chosen to be one that is overfitted during training. The challenges of membership inference tests is that low values of the statistic only suggest membership, and do not necessarily provide concrete sample-Ievel evidence. Additionally, they sometimes do not work well on large models (Duan et al., [2024]. Finally, we note that for representation learning methods such as DINO [Caron et al., 2021 that use self-distillation, loss minimization is not usually the training objective which might lead to failure of membership inference attacks based on loss statistics (Liu et al. [02T] He and Zhang [202T. In contrast, our measurement method is more concrete, agnostic to the method of training, and does not require training multiple similar models.
(ee
A third line of work is on astribute inference [Fredrikson et al. 2014), where we are given a model, and some attributes of training data point, and the goal is to use the model infer the rest rest Jayyaamann and Evans [2022] recently show that most attribute inference tests apply equally well to training and test data, and hence may not be very relevant in measuring privacy. In contrast, déjà vu memorization specifically looks at sample-level attribute inference in training data points beyond what could be achieved through dataset level correlations, which justifies its relevance.
Our work also has connections to prior work on measuring memorization in classification models and influence functions (Koh and Liang 2017]. Feldman (2020) proposes stability- -ased definition of memorization, where a classifier memorizes the label of (z. if fs(x) = where Is is trained on training set S., and (2x((2))) (xx) * y where (sx(2.g) is trained on S { {xx, y)). Unfortunately this
can be highly computationally demanding, as measuring memorization for a single example requires training a full model.
Stability- based memorization is also very related influence functions [Koh and Liang 2017]. which approximate the impact of single training example on test prediction. Specifically, if training point has high influence on its own prediction, then is likely memorized. However, calculating influences, while easier than re-training model, is also compute-heay. for large models, and involve many approximations. In contrast, our approach has lower computational cost.
3 Measuring Dataset-level Correlations
As explained before, the main challenge with prior work is that we need second model trained on similar data to determine the task could done by dataset-level correlations. Our main contribu- tion is to introduce alternative approaches for inferring datasct-level correlations, and empirically demonstrate that these approaches suffice for the purpose of measuring memorization.
3.1 Formal Definition
Formally, we define memorization as follows. We have a training dataset = {21,..., Zn. drawn i.dd from an underlying data distribution D; this is used to train a representation learning model f. Suppose that a data point z drawn from can be written as: z = (v, 1) where v and / represent disjoint but possibly correlated information. For example, v could be the background of an image, and the label of the foreground object it. Similarly, suppose that we can write the data distribution D as the product of the marginal p(v) over and the conditional distribution μ(tllu).
3

---

Loosely speaking, déjà va memorization happens when we can use f to infer t from e for points 2; in the training set D better than what we could do from knowing ((tle)). Formally, for a discrete label t. we can rigorously define memorization as follows.
Definition 1 (Déjà vu Memorization). Let z = (v.t) be a training data point. 2 is said to be memorized there exists a predictor h such that h(f.e) = while argmax, (v) # t.
Observe that the second part of the definition precludes inference through dataset-level correlations. (v, t) of a patch of water (background) with a black swan in the foreground. Suppose also that based on the representation [()), we can prediet there is a black swan in the image. Is this image memorized by f? It is possible, but might also be possible that all patches of water in the training dataset are adjacent to black swans, i.e. argmax, (u [e) t. Therefore determine if the image is being memorized, need to rule out this possibility.
Figure I shows concrete example. Suppose have an image
Related to, but different from us. Feldman 2020) provides stability- -baed definition of memorization. We adapt their original definition to our setting as follows.
Definition 2 (Stability-based Memorization). Let z; (v;, be a training data point, and let f'D denote model trained on the dataset D. zi is said to be stably memorized if there exists a predictor h such that h h(SD; V1) = 14 and ((SD (4) + 4.-
In other words, if we exclude 2; from the training set, then we cannot use the model f to predict correctly. Observe that this definition is very closely related the notion of stability in learning theory [Bousquet and Elisseeff 2002).
These two definitions related, but subtly different there can be examples that déjà va memorized, but not stably memorized and vere-versa. can be casily shown that the rate of déja vu memorization is upper-bounded by the generalization error of h; on the other hand, the rate of stability-based memorization is by definition the leave-one-out error [Bousquet and Elisseetf (002) of h. Classical learning theory (Bousquet and Elisseeff [002] predicts that the leave-one-ous error of a classifier is close to its generalization error. Therefore, the rates of these two notions are close for well-generalized classifiers that adapt themselves to the data such as neural networks.
Feldman 2020] provides a method to measure stability- memorization for a sub-sample of the
training data that involves training large number of auxiliary models; in contrast, our notion has the advantage that it can be measured in a much more computationally efficient manner.
3.2 Image Representation Learning Models
For image representation learning, déjà vu memorization measures the accuracy of inferring the foreground object given background crop. Let crop be a function that, when given any image æ, produces a background crop cr)p(z). Then for a sample 2; = (#i, y:) E D where x; is an image and y, is the label of the foreground object, we have v, = (2; and I, = y in the notation of Definition] T Observe that since we are looking at unsupervised representation learning, the label y; was not used to train the model f. Define
(v, t) = ((ho N)() = t) E 1),
(1)
where h is a predictor that takes the representation of fw) and outputs a foreground object label. Observe that accy(u, I) is 0/1 value which is 1 when the foreground prediction is correct. Since i; does not contain the foreground object, for training sample z; that is not memorized, one expects acc 4) = O. except by sheer chance. However, dataset-level correlations may in fact allow accurate prediction of the foreground object from a background crop, e.g. if the foreground object is a basketball and the background is basketball court. To isolate this effect, Mechan et al. [2023) proposed to split the training set D into disjoint sets and B, and train two models fA and JB on the two datasets. Then, for 2; A, if acCy (v,, f,)= 1 but acc (v, f,) = O. one can then infer that t; cannot be predicted from U/ from correlation alone, and thus fA has likely memorized 24.
In this analysis, acc determines if the foreground object can be predicted from crop(z;). To enable déjä vu memorization measurement with a single model, we propose to replace acCfs with the prediction of a reference model that directly classifies the foreground object given the background crop. We propose two ways to do this: training an image classification network end-to-end, and using naive Bayes classifier top of an object detector.

---

ViCReg KNN
rugby ball
Barlow T. KNN
valleybell
DINO KNN
Classifers
ResNet
anemone
Test
w/Top-1
"wo Models
One Model
baskettball
w/Top-2
Classifiers
KNN
barbell
Reshict
NB w/Top-5 CA
Naive Bayes
Accuracy
Examples1%%
(a) Aggregate accuracy(%) of foreground predic- tion from background for different models. We that ResNet50 and NB Top-2 similar to both VICReg and Barlow Twins, and the aggregate ac- curacy is low.
(b) Accuracy of the top-5 predicted classes based on dataset-level conelltions using three classifiers: KNN, Resnet and Naive Bayes. Naive Bayes classifier uses top-20 crop annota- tions as features.
Figure 2: Left: Population-level correlation accuracy scores across different models. The accuracies for two model tests are based on KNNs computed on top of VICReg, Barlow Twins and DINO representations. ResNet50 and Naive Bayes classifier are used for one model tests. The results show that ResNet50 and NB Top-2 are similar to both VICReg and Barlow Twins. Right: Corresponding Top-5 predicted dataset-level correlation classes and the percentage of per class correlated examples.
Image classification network. Our first approach straightforward: we train an image classifier to predict 4 directly given e cr) For ImageNet, train ResNet50 model over split D' of the training set D) and evaluate déjà vu memorization on D D', This ensures the reference model itself is not memorizing, but rather predicting the correlation between the background crop and the foreground object.
Naive Bayes classifier. If the training set D' for the image classifier is large, the above approach can be just as expensive as training the model fB- Our second approach alleviates this by fitting a simpler model, naive Bayes classifier, top of objects detected in crop(æ:). In detail, let objects be an object detection model with vocabulary set V: that is, for given image æ, objects(x:) E f0, [V] is a binary vector such that objects(r)) in the vocabulary set V. We then derive the empirical probability estimates a split D' of the training set:
if and only object оы exists in image æ for each
P(ox) = D
Σ Z,ED
objects(v()e: P(ov = H E D' =
1
Σ
objects(w))):
For sample 2; D D', the naive Bayes classifier predicts the probability P(t. t v;) for each foreground object t given the background i as:
P( 144 =t P(ok)
P(4 = t v.) P(4 = detected objects in vy) P(t)
II
k:objects(w))e000
where the last equality uses the independence assumption for naive Bayes. In practice, because the object detection result can be noisy, we truncate the list of detected objects to the top- K according to detection score.
Results. We now investigate how effective the two approaches are at measuring dataset-level correlations in comparison with a second model [Meehan et al. [023]. Specifically, we compare ResNet classifier and two versions of the Naive Bayes Classifier that uses the top-5 and top-20 crop annotations, well as three SSL models models-VICReg Bardes et al. 2002]. DINO Caron et al. [202T) and Barlow Twins Zhontar al. 2021 and look how much these classifiers agree on the predicted correlations.
5

---

1.0
ViGReg
1
0.61
0.46
0.42
0.4
0.43
SSL B
KNN
SCB
Barlow Twins
0.61
0.38
ResNet
Clowafish
DINO
0.46
0.43
0.34
0.33
0.35
ResNet
0.42
0.38
0.34
1
0.45
0.48
0.6
Naive Bayes
Clowafish
Top-5
0.4
0.38
0.33
0.45
0.99
Top 20
0.43
0.35
0.48
-04
SSL
KNN
cur, spike, capitulum
vicleg
Twita
DINO
NB Top5
N8 1op-20
ResNet
Naive Bayes
leathopper
(a) Pairwise sample-level correlation agreement fraction among six reference models. VICReg. Barlow Twins and DINO are used for two model tests whereas ResNet50, NB Top-5 and NB Top- 20 for one model tests.
(b) Examples demonstrating when one model tests (ResNet and Naive Bayes classifiers) suc- ceed and two model tests (KNN) fail and vice versa.
Figure 3: Left: Pairwise sample-level agreement in measuring dataset-level correlations and Right: Examples demonstrating when model tests (Resnet and Naive Bayes classifiers) succeed and two model tests (KNN) fail and vice versa. One model tests learn the correlations between foreground and background better since it is enforced by the classifier training, however, they are less accurate when the relationships between foreground and background are ambiguous. One model tests, in contrast, better disambiguating the foreground and background relationships. They, however, sometimes tend to predict what's s on the background and not what foreground it is associated with.
Figure 2a shows that the overall accuracy across these classifiers are largely comparable. We then zoom into top-5 most correlated classes in Figuue 26) where we show the number of correctly predicted correlations for the top-5 most correlated classes. Across the three methods that we compared, namely KNN, ResNet and Naive Bayes, the top-5 most correlated classes are identical. However, at a sample level, there in fact a large divergence in prediction across different methods, [Figure 3a shows the fraction of samples where the correlation prediction agreed for the different reference models. Here, we see that the agreement is quite low, only about 40%. This suggests that the methods have different inductive biases from the SSLeb classifiers when measuring dataset-level correlations and thus can overestimate memorization when used in the one-model test. Appendix [B.0 looks deeper into the intersection of common memorized examples across multiple reference models. It shows that ResNet classifier agrees with the intersection of three two model tests for approximately 86% and Naive Bayes for 78% of top-1 correlated examples. Figure 33b showcases different scenarios when the reference models agree and disagree. It unveils the strengths and the weaknesses of the one and two model tests and suggests that these methods can be used conjointly.
3.3 Vision Language Models
For vision- language models (VLMs), the training dataset D consists of image-text pairs & (Zing. 7. The model f learns to simultancously embed 2mg and zixt into low-dimensional repre- sentations, with the training objective of aligning the representations f (Zing) and ff (zwat). Following the setup of Jayaaaman et al. 2024), we consider v = Ztext and t objects( (Zimg) E {0, 111 where objects( Zimg) the set of detected objects in vocabulary V. Déjà memorization occurs when one can leverage f to infer objects in 2img using 4ux significantly beyond dataser-level correlation. Specifically, consider predictor h that operates f(V) and outputs binary vector of predicted objects. We can define the precision and recall metrics for the predictor:
(rec, t) = ((h (v),
[0. 1]. (u, = ((h f)(). ||t||
E 10.
(2)
[1(h f)
One might expect prec, () (v. = when ʃ does not memorize. However, dataset- level correlations may in fact enable the prediction of objects in Zimg from Etext, e.g. if 2txt table full of fruits and vegetables and Zing contains objects such as apples, oranges, carrots, etc. To design one-model déjà vu memorization tests, we would like to capture this type

---

of dataset-level correlation with a reference model. This is especially hard for VLMs since these models are typically trained on internet-scale datasets consisting billions of diverse samples under a long-tailed distribution. Training this reference model from scratch on subset of D requires similar effort as training the VLM itself, which defeats the purpose of one-model test.
Using pre-trained text embedding models as reference models. To tackle this challenge, we leverage pre-trained text embedding model g that transforms text into vector representations, with the requirement that (g(z 9 (() is high when Aixt and elex are semantically similar (and vice versa). We can then utilize g to define reference model similar to the two-model setup of Jayaramann et al. [2024). Given a public set pub of image-text pairs and training sample 2, the reference model first performs inner product search in the embedding space of g to find the K most similar captions in puu.
(z )ext- Then, we predict O₁: E Zimg if and only if (z)) Jing for some E f1...., K}; see Figure 13 in Appendix C for an example.
Result. We investigate how well the LLM (g) captures the dataset-level correlations for predicting ground-truth objects in images when compared to the reference VLM (fB) of Jayaraman et al.] [2024), that has not seen the target images in its training. We plot heatmaps for pairwise sample- level agreement similar to the vision model case above. However, since this setting has multiple objects per image, we calculate the Jaccard sim- ilarity between the correct object pre- dictions per sample for the two mod-
NN
Top-1
0.8
0.25
VLM
NNS NNS
-0.6
Top. 10-10
0.6
0.29
0.59
0.62
0.21
0.64
0.47
100 NNSI
0.5
-0.4
-0.4
0.25
0.61
0.84
0.13
0.45
0.84
-0.3
-0.2
els and fa. and report the averaged value across all the training samples.
INN
NNs100 I.M
Top-1
Top-10
All
1.
(a) Predicting all objects with varying NNs
(b) Predicting top-k objects with 100 NNs
shows the pairwise sample- level agreement between the two mod- els for predicting various top-k ob- ject labels and for different number of NNs. As shown, even when predict- ing all objects, the two models agree only on 84% objects on average. This agreement decreases as we limit the number of top-k object predictions or alternatively limit the number of NNs. We see a similar trend that reference models do not always agree the predictions. We show some examples of what the two models, VLM fa and LLM g, predict for a given caption in Figure 14 in the appendix.
Figure 4
Figure 4: Pairwise sample-level agreement (using Jaccard similarity for predicting correct objects) between the ref- erence VLM fB in previous two-model test and the GTE language model g. The heatmap shows that the agreement fraction for one model and two model tests are comparable.
Measuring Déjà vu Memorization using One Model Test
In this section, we investigate how effective new methods for measuring dataset-level correlations are when we use them for measuring memorization. Specifically, we look at two main questions: How close are the results of the single-model deja-vu test to the two-model test? 2. What is the fraction of memorization in open-source (OSS) pre-trained representation learning models? These questions are addressed in the context of both image representation learning models and vision language models.
4.1 Image Representation Learning
Dataset. We conduct all our image representation learning experiments on ImageNet Deng al. (2009] dataset.
We use 300k (300 per class) examples to train the reference models learn dataset-level correlations. We measure memorization accuracy on an additional disjoint set of 300k images. For the two model tests, these images are included in the training set of the target models, but not the reference models. Finally, we use another additional distinct 500k images to predict the nearest foreground object given the representation of background crop through KNN.
Models. Two model tests are conducted analogous Meehan et al. 2023 One model tests rely on classifier that is trained once to predict datasct-level correlations for SSL models. The dataset used
7

---

to train this classifier overlaps with the training dataset of source models but is disjoint from the subset of the examples for which we measure memorization.
We compare two kinds of classifiers to detect dataset-level correlations. trained on the background crops predict the foreground object. We used LARS optimizer and 0.1 weight decay for L2 regularization to avoid overfitting. The second classifier is Naive Bayes classifier. It uses background crop annotations as features. We automatically annotate background crops using Grounded-SAM (Liu et al. 2023 Ren et al. [2024]. Annotations represent textual tags associated with probability scores. We use these probability scores to pick top 1, and features to compute final Naive Bayes probability scores. The reference models are trained on single machine with Nvidia v100 GPUs, 32GB per GPU using 128 batch size. All other experiments are performed on the same machine.
The first is a ResNet50
Metrics. Following Meehan et al. 2023) we report the déjà vu score and the déjà score at p%6 The déjà vu score for a model / is the difference between two accuracy values: the first is the accuracy of predicting the foreground label y from the representation f(v) of the background crop based on KNN. The second is the accuracy of predicting from reference model. The déjă vu score 5% is the difference between the same two accuracies, but now calculated only on the top p% of the most confident examples.
4.1.1 How close is the déjà vu memorization of one-model and the two-model tests?
discusses how close one and two model tests are in terms of dataset-level correlation accuracy. In this section we compare déjà vu memorization scores for one and two model tests. Figure 5|shows that KNN classifier (two model test) and ResNet classifier (one model test) identify similar amount of déjà vu memorization for VICReg (Bardes et al. [222] and Barlow Twins models.
Section 3.2
Déjà vu memorization is substantially lower in DINO [Caron et al.) [02T]] Similar findings are reported in Meehan et al. (2023) as well. addition, we observe that déja vu score decreases as we increase the number of features (crop annotations) in Naive Bayes. This is due to the increasing
accuracy of dataset-level correlation as we increase the number of crop annotations.
30
35
Classifier
30
DejaVu Score
KNN
ResNet
DejaVu Score
20
NB Top-1 CA
Top-2
15
NB Top
WMAA
All
10
Top 20%
5
VICReg
Barlow Twins
DINO
0
VICReg Barlow Twins
DINO
Figure 5: Comparison of overall and Top 20% most con- fident Déjà vu (DV) scores using one model (ResNet Classifier, Naive Bayes w/ Top-k Crop Annotations (CA)) and two model (KNN Classifier) tests for VI- CReg, Barlow Twins and DINO trained on a 300k sub- set of ImageNet.
Figure 6: Comparison of overall and Top 20% most confident Déjà vu (DV) scores using one model (ResNet Classi- fier, Naive Bayes w/ Top-k Crop Annota- tions (CA)) tests for pre-trained VICReg, Barlow Twins and DINO.
4.1.2 Do pre-trained representation learning models in the wild exhibit déjà vu memorization?
In this section we present déjà memorization for pre-trained OSS representation learning models on population-Iewe using one model tests. Two model tests aren't 't applicable in this scenario since pre-trained models are trained the entire ImageNet dataset and the validation dataset is relatively small to be considered for training a second representation learning model.
Hence, Figure 6 compares only one model tests. A comparison of one model tests between Figure 5 Figure 6 shows that pre-trained models memorize less compared the same models trained on smaller subset of the training data. We hypothesis that this is due to the lower generalization error of
8

---

2 Model Top Object
1 Model Top Object
Top Object
2- Moddel Top-10 ( Objects
1-Mode) Top-1 D Objects
Top-10 Objects
Matric Store
2-Model All Objects
1-Model Objects
All Objects
Metric
0.2
0.020.03 0.062.07
0.06
0.07
0.020.03 03
PPG
PRG
PPG
PRG
(a) One-model vs two-model tests for Shutterstock models.
(b) OSS model pre-trained on YFCC13M.
Figure 8: Data set level memorization of various VLMs.. We use top- public set NNs to predict the top-k objects and report PPG and PRG as done in Jayaraman et al. [0244]
the pre-trained models result of having larger training We provide additional examples of common dataset-level correlations and memorized images in appendix subsection B.
4.1.3 Sample-level memorization
Figure visualizes the distribution of memorization confidence scores for pre-trained VICReg OSS model with ResNet correlation detector. The memorization confidence for the i-th example is computed based on the following formula:
MemConf (x:z)
Classifier)
Entropusst (K is computed according to Meehan et al. 2023 's Section 4 description and (Coreelatooo Classifier) correlation classifier s entropy over the softmax values.
7 shows that the memorized examples with high memo- rization confidence scores are rarer and more likely to be memorized. The ex- amples in the middle of the distribution are
High Entropy VICReg
Similer entropy VICReg and ReeNet
Entropy VICReg
Low Entropy ResNet
High Entropy ReeNet
VicRag: Aircraft Carrier VicRegi Leaf Beetle (2.35)
VicReg: Polaroid (0.056)
ResNet: Dock (0.55)
Dragonfly (2.36)
ResNeti Parking meter (6.33)
ViaRege Passenger oar (3.93) VicReg: Black and gold garden spider (0.83): VicReg: Cross Word (0.0)
ReaNet: Snowmobile
ResNet: European gorden spider (0.83)
RooNet: Black-footod forrot (6.48)
easy to be confused with another class. E.g. Black and gold garden spider with European garden spider. On the other hand the examples with negative mem- orization confidence have higher memo- rization and slightly lower correlation entropy.
3500
3005
2500
Counts
2000
3500
1000
Memorization Confidence
Figure 7: A histogram of sample-based memorization confidence for VICReg OOB model. Given a background patch, VICReg predicts the correct class (green). ResNet (correlation classifier) predicts the incorrect (red) class.
4.2 Vision Language Models
Experiment setup. We train CLIP models using the OpenCLIP Ilharco al. [021 framework on the Shutterstock dataset (a private licensed dataset consisting of 239M image-caption pairs). See subsection C. C details on dataset preparation and training. We quantify dataset-level memorization using the population precision gap (PPG) and population recall gap (PRG) metrics of Jayaraman et al. (2024] These metrics capture the population-level gap between the fraction memorized objects and fraction of objects inferred through correlation; subsection C.I for details.

---

4.2.1 How close is the déjä vu memorization of one-model and the two-model tests?
As explained in subsection 3.3 in our one-model test we use a GTE language model as a reference model to quantify data set level memorization of a target VLM f. We compare this test to the previous work Jayaraman et al. [024], which trains reference VLM from scratch on separate hold-out set. Figure 8aj campaaess the two tests terms of the PPG and PRG metrics for predicting top-k object labels in training images with 10 nearest neighbors from the Shutterstock public set. While the previous two-model test achieves 0.06 PPG and PRG values for predicting top-10 objects, approach obtains 0.07 PPG and 0.06 PRG values for the same setting. Our test thus slightly overestimates the memorization as in the vision case above. We also compare the dataset-level metrics for the two tests for different settings where we vary both the number of nearest neighbors used in the test and also the number of top-k objects predicted in Table 2 and Table 3 respectively in Appendix C
4.2.2 Do pre-trained vision-language models in the wild exhibit déjä vu memorization?
We perform our one-model test against an out-of-the-box ResNet-50 CLIP model pre-trained on the YFCC15M data set from OpenCLIP, Figure 8b) shows the PPG and PRG values for predicting different top-k objects. These results are comparable to our one-model test results in Figure 8a where we evaluate our CLIP model trained on 40M Shutterstock data. More specifically, for predicting top-10 objects with 10 nearest neighbors from public set, our Shutterstock model achieves 0.07 PPG and 0.06 PRG, whereas the OSS YFCC15M pre-trained model achieves 0.07 PPG and PRG values. Additional results can be found in Table 4) and Table in the appendix. We include the most memorized examples for our Shutterstock models Figure 15 in AApenddix C.
4.2.3 Sample-level memorization
Figure 9 shows samples with higher degree of memorization. The samples are sorted from high to low memorization such that the top-L samples have higher precision and recall gaps for recovering objects using target and reference models. We find the gap between the objects recovered from target and reference models for each training record, and estimate the precision and recall gaps. positive gap indicates that the target model memorizes the training sample and the magnitude of the gap indicates the degree of memorization.
1.00
1.00
1.00
Presision Gap
0.75
Recall Gap
0.75
F-Sorre 8
0.75
0.50
0.50
0.50
0.25
0.25
0.25
0.00
0.00
0.00
10a
10a
10a
10a
10"
10
10a
10"
10a
10
10a
Top-L Records
Top-L Records
Top-L Records
Top-1 NNs
Top-5 NNs
Top-10 10 NNs
Figure 9: Sample-level memorization in VLM trained on 40M Shutterstock images, quantified in terms of precision and recall gap between target VLM and off-the-shelf GTE LMM
5 Discussions and Conclusion
This paper proposes a principled method for measuring memorization in vision and vision- language encoder models that does not rely on training similar shadow models. This enables, for the first time, direct measurement of memorization in open-source representation learning and vision-language models. One consequence of these new measurements is that now we can find out how much different OSS models memorize. In particular, we find that VicReg and Barlow Twins memorize more than DINO. Additionally, all standard OSS models memorize less than their versions trained subsets of the data.
Finally, our method of measurement involves approximations to theoretical quantities, and as such, has some limitations when these approximations do not hold. One such limitation is that our alternative dataset-level correlation estimation might be poor approximation to the Bayes optimal, or might itself memorize its own training set, thus skewing the results. However, given that these are much simpler classifiers, their own rate of memorization is expected to be lower. Another limitation is that the additional annotations that we use for our measurements may be lower quality, which might also lead to biased results. A closer analysis of the impact of these factors is avenue for future work.
10

---

Acknowledgements
We thank Maxime Oquab for pointing us the challenges of the two-model test.
References
Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Varraane-invarranee-cvvraaaan regularization for self-suuervised learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URLLattss:/ net neet/forumPid=xm66D66200
Olivier Bousquet and André Elisseeff. Stability and generalization. The Journal of Machine Learning Research, 2:499-526, 2002.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267-284, 2019.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633-2650, 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897-1914. IEEE, 2022.
Mathilde Caron, Hugo Touvron, Ishan Misra, Herve Jegou, and Julien Mairal Piotr Bojanowski Ar- mand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database, In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255 lece, 2009.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work large language models? arXiv preprint arXiv:2402.07841 2024.
Vitaly Feldman. Does learning require memorization? a short tale about long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954-959, 2020.
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy pharmacogenntics: (End-to-End] case study of personalized warfarin dosing. In 23rd USENIX security symposium (USENIX Security 14), pages 17-32, 2014.
Xinlei He and Yang Zhang. Quantifying and mitigating privacy risks of contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 845-863, 2021.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. OpenCLIP, 2021. URL http///i..rrr/1..211//111// 5143773
Bargav Jayaraman and David Evans. Are attribute inference attacks just imputation? arXiv preprint arXiv:2209.01292 2022.
Bargav Jayaraman, Chuan Guo, and Kamalika Chaudhuri. Déjà vu memorization in vision-language models, 2024.
Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pages 10697-10707. PMIR, 2022.
11

---

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:3304002643, 2023.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885-1894, PMLR, 2017.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong. Encodermi: Membership inference against pre-trained encoders in contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 2081-2095, 2021.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303 0,0999, 2023.
Casey Mechan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, and Chuan Guo. Do ssl models have déjà vu? case of unintended memorization in self-supervised learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 42775-42799. Curran Associates, Inc., 2023. URL https: httpps://proceedinggs itw. neurips cc/ papprp .pff
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh. Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models from Natural Language Supervision. International conference on machine learning. PMLR, 2021.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang. Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2 2021.
Kaiyu Yang, Jacqueline Yau, Fei-Fei, Jia Deng, and Olga Russakovsky. A study of face obfuscation in imagenet. In International Conference on Machine Learning (ICML). 2022.
Samuel Yeom, Trene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. In 2018 IEEE 3/ Computer Security Foundations Symposium (CSF). IEEE, 2018
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint arxiv:2103.03230. 2021.
Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting thhntadt Classes using Image-level Supervision. In ECCV, 2022.
12

---

A License of the assets
A.1 License for the code
We use the code from Mechan et al. [003] which is under the Attibution-NonnCommercall 4.0 International license according to https://githinn comf/facabolrreesarcch//.jjijj
Lab=License-1-o0 1e----ov--leetrradmee
We also the code from Ren et al. [2024] which
is under the Apache 2.0 licence according to https://github. com/ IDEA ReeA-Reeeachh
Grounded-Segment- hatpree-eeeeee---tttnnaaaanaaaaa hwy.hhinggaa-//pcha--------
A.2 License for the datasets
We use ImageNet Yang et al. 2022] which license can be found at /https://www. image-maeee. net org/ download. php We also use private licensed dataset consisting of 239M image-caption pairs.
B Additional results for Image Representation Learning
B.0.1 How close are the results of the single-model deja-vu test and the two-model test?
We observe that the intersection of correctly predicted examples between the reference models is relatively small. This intersection approximately 40% for model tests. This tells us that there is significant noise in predicting dataset level correlations even if the models are trained on the same dataset. order better understand this phenomenon,
intersect common subsets of two model tests with one model test. Table [ shows that Resnet50 and Naive Bayes with Top 20 annotation tags are able predict the same correlations for almost 60% of the examples that were also predicted as correlated by by two model tests. This percentage increases if we look into top-20, top-5 and top-1 predictions. For top-1 predictions Resent50 reaches over 86%. In addition to that we also observe that example-level correlation accuracy increases for Naive Bayes by increasing the number of features. This tells us that Naive Bayes becomes more accurate if we increase the number of features describing the crop.
Intersection between VICReg, Barlow Twins, DINO
Accuracy Top20
Accuracy Top-5
Accuracy Top-1
AND
Accuracy
NB w/Top-I Crop Annotation
32.04%
31.12%
29.78%
13.51%
NB w/Top-2 Crop Annotations
45.31%
50.19%
45.74%
35.13%
NB w/Top-5 Crop Annotations
54.52%
65.82%
70.21%
75.67%
NB w/Top-20 Crop Annotations
59.32%
69.66%
75.53%
78.37%
ResNet
58.02%
72.58%
76.01%
86.48%
Table 1: Example-level correlation accuracy between the intersection of two model tests and each one model test.
B.1 Common memorized vs. correlated examples
In this section we showcase examples of common dataset-level correlations and memorization by the OSS pre-trained representation models such as VICReg, Barlow Twins and Dino.
Figure 10 showcases examples of two common dataset-level correlations between 'kitchen, store' and 'microwave', 'gondola' and 'pole, water', Resnet and Naive Bayes classifiers learn these correlations effectively and help us distinguish memorization from dataset-level correlations. In addition, we observe that memorization tends to happen in examples where there is no clear dataset- level correlations between the background crop and the foreground object. |Figure II demonstrates an example a memorized image by VICReg pre-trained model. Here reference models incorrectly predict the foreground object whereas KNN correctly classifies the VICReg representation of the crop. In this case, our approach identifies the image with the shopping cart as memorized, Figure 12 demonstrates top-5 images memorized by the VICReg OSS model. We observe that there no clear
13

---

Test Image
label: "microwave"
Similar crops with the label "microwave" the classifters' training set
background crop
Similar crops with the label "gondola" the classifiers' training set
Test Image
label: gondola"
background crop
Figure 10: Two common dataset-level correlations: 1) "stove, kitchen' and 'microwave', 2) 'sky, pole, water' and 'gondola'. ResNet and Naive Bayes classifiers learn to associate the background crops with the image label.
Reference Models
Image E
Crup
prediction: thatched roof
ResNet Classifier
backyard, bush, fence
Crop Annolator
Naive Bayes Classifier
prediction: picket fence
label: shopping cart
Memorization Test
VICReg
KNN Classificr
prediction: shopping cart
Trained on A E A
Figure 1I: KNN predicts correct class 'shopping cart' given VICReg's representation of the back- ground crop. Here the original image of the crop is part of the VICReg's 's training. Resnet and NB classifiers, however, fail to predict the correct class which concludes that the image is memorized.
correlation between the background and foreground and some images could be unique than the rest of the ImageNet.
C Additional Results for VLM One-Model Déjà Vu Memorization
C.1 Experiment setup
For the VLM experiments, we trained CLIP model from scratch on the Shutterstock dataset-a private licensed data set consisting of 239M image-caption pairs. Since this dataset has many duplicate captions, first perform caption-level de-duplication by considering only per caption. This resulted 103M samples. We created three random splits this data set of sizes 40M, 40M and 20M, call these sets D, D and Dpub respectively. We use the first two splits for training the CLIP models and use the Dpub set for our nearest neighbor test to find the most relevant images the target image from the training set D.
Dataset.
image
For object annotation on Shutterstock images, we use an open-source annotation tool, called De- tie [Zhou et al. 2022], that can annotate all the 21K ImageNet objects. We use threshold of 0.3 to identify object bounding boxes (i.e., any bounding box that more than 0.3 confidence considered for annotation), the default 0.5 threshold results in nearly 17% images with no
14

---

Original Imaga
Craps
grail jPlt
loaer pants
Figure 12:
Top-5 images memorized by VICReg OSS model. Both Naive Bayes and ResNet classifiers fail to predict the correct class based on the background crops.
Comparison
Using Top- I NNs
Using Top- NNs PPG
Using Top- 100 NNs
PPG
PRG
PRG
PPG
PRG
Two Model
0.030
0.030
0.060
0.064
0.054
0.063
fr2i vs 9
0.014
0.034
0.094
0.082
0.190
0.107
feze vs g
0.033
0.038
0.098
0.065
0.218
0.061
Table 2: Comparing the population-level memorization for predicting all objects for various settings where the 40M D set is used as the target set. For we use the GTE model where we match the target caption with the public set captions. For VLMs, is the cross-moda setting where target caption matched with public set images for kNN search, f2t is the unimodal setting where only the text modality of the model used for kNN search, i.., target caption is matched with public set captions similar the case. do not consider the image-to-image search as the target image is not known to the adversary.
annotations. Detic.
Figure
13 shows the sample images with multiple object annotations obtained using
Model training. The architecture we use is the ViT-B-32 model from OpenCLIP (Ilharco et al. 202T]. We train our models for 200 epochs with a learning rate 0.0005 and a warmup of 2000 steps for cosine learning rate scheduler. Our training runs use 512GB RAM and use 32 Nvidia A100 GPUs with global batch size of 768. A single training run on 40M data size takes around 10 days. Our CLIP models trained on 40M data sets achieve around 41 11.26% zero-shot classification accuracy on ImageNet data set, which is line with CLIP models trained on similar size data sets.
Metrics. To quantify déjà vu memorization for the target sample 2, we consider the precision and recall metrics defined in |Equatio 2 These metries are scaled in range [0, and quantify memorization at an individual sample level. We can also quantify memorization at the dataset level using the population precision gap (PPG), population recall gap (PRG) and AUC gap (AUCG) metrics defined by Jayaraman et al. [2024] AUCG measures the gap between the cumulative object recall distributions and nB. The equations for PPG and PRG metries are given below:
1 PPG
(1( A preclz, prec(z, fB)) - K= e A: (e) preclz, fo)H).
A
PRG
(1 A A:c(z, fA) rec(z, - 1[= e A: red(z fa) red(z.f))H)
A
(4)
C.2 Additional experiments
15

---

Target Image Frest
Imuge Cuption Xuu Healthy Vitamin D C
Public Image-Caption Set Dra
VITAMIN D C
K-NN Similarity Search
Text Embedding fixe)
Image Embeddings RM tie E Drs
Fis VLM trained with
AWELLCOME
Detected: 39/44
Precision: 0.53
Recall: 0.89
K-NN Similarity Search
Text Embedding g(am)
Text Emmheddings glo Aou E Dru
pretrnined GTE
Detected: 12/44
Precision: 0.18
Recall: 0.27
Figure 13: An example where an OpenCLIP model trained on a 40M subset of a Shutterstock data set exhibits déjà memorization of objects present in training image. Public set is separate collection of 20M images from Shutterstock that has no overlap with the training set. The objects annotated in orange true positives, i.e., the ones present in the target image, and the objects annotated in blue are false positives. For the OpenCIIP model f trained on the target image, our test recovers significantly more memorized objects compared to the pretrained GTE language model g that finds the closest captions from the public set using the k-NN search in the text embedding space.
Comparison
Predicting Top-1 Object
Predicting Top- 10 Objects
Predicting All Objects
PPG
PRG
PPG
PRG
PPG
PRG
Two Model
0.022
0.022
0.055
0.056
0.060
0.064
fezi vs g
0.034
0.034
0.081
0.074
0.094
0.082
feze vs g
0.026
0.026
0.068
0.064
0.098
0,065
Table 3: Comparing the population-level memorization for various settings with top- 10 public NNs where the 40M D set is used the target set. For g, we the GTE model where we match the target caption with the public set captions. For the VLMs, /2i is the cross-modal setting where target caption is matched with public images for kNN search, #2t is the unimodal setting where only the text modality of the model is used for kNN search, i.c., target caption is matched with public set captions similar to the case. We do not consider the image-to-image search the target image is known the adversary.
Comparison
Using Top-1 NNs
Using
Top- NNs
Using
Top-100
100 NNs
PPG
PRG
PPG
PRG
PPG
PRG
fezi vs g fezi vs g
0.140
0.164
0.156
0.165
0.257
0.097
0.053
0.061
0.054
0.092
0.066
0.080
Table 4: Population-levé memorization for predicting all objects with the pre-trained YFCC15M OSS model. For g. we use the GTE model where we match the target caption with the public set captions. For the VLMs. (2i is the cross-modal setting where target caption is matched with public set images for kNN search, (21 is the unimodal setting where only the text modality of the model is used kNN search, i.e., target caption is matched with public set captions similar to the g case. We do not consider the image-to-image search as the target image not known to the adversary.
16

---

Comparison
Predicting Top-1 Object
Predicting Top-10 Objects
Predicting All Objects
PPG
PRG
PPG
PRG
PPG
PRG
fizi vs g frze g g
0.092
0.092
0.143
0.142
0.156
0.165
0.054
0.054
0.071
0.071
0.054
0.092
Table 5: Population-level memorization for predicting top-k objects with top-10 10 public NNs with the pre-trained YFCC15M OSS model. For g. we use the GTE model where we match the target caption with the public set captions. For the VLMs, f2i is the modal setting where target caption matched with public set images for kNN search, f2t is the unimodal setting where only the text modality the model is used for kNN search, i.c., target caption is matched with public set captions similar to the g case. We do not consider the image-to-image search as the target image is not known to the adversary.
17

---

Caption: butter, traditional ceramic pan bread and snuce on a rustic wooden table, Top view Top-5 of
Predicted Labels: diniin table, 'chnkuri', 'carving, fork", bread", 'iingr. how!', bbaked, potato', dinner napkin', platre, etc.
[Objects Recovered: 11/26, Precision: 0.21. Recall: 0.42]
Top-5 NN of LLM.s g:
Predicted Labels: diiiing lable' 'hukkrr,, 'carving lock', 'bread' Tinger buwl', dinner mapkin', "shullof, platic" stc. [Objects Recovered: 12/26, Precision: 0,24, Rocalli 0.46)
(a) Example where both fo and perform similarly.
Cuption: bunddha bowl wood background
Top-5 NNs of VLM
Predicted tomatillo', chickposl, 'dining table', 'Jerusllm 'yellow salsify. 'pigean, priceon_peal, [Objocts Recevered: 19:20, Precision: 0,31, Rocall: 0.95)
Top-5 L.Mg:
Predicted Labels: 'dining, table', bowl
[Objects Recovered: 2/20, Precision: 0.11, Recall: 0.10]
(b) Example where fa performs better than g.
Caption: Colorful spices in ceramii and metal containers beautiful kitchen image. Top-5 NNs of VLM f::
Prodicted I Labels: 'lingr bowl", 'cinnabar chanterelie' 'dining-room table', 'turmeric, 'howl', 'komondar, 'pesite", [Objects 8/38, 0.11, 0.21]
Top-5 NNs of o'LLMM
Predicted Labels: jgroundmof, 'shmend', 'carambolaa, ppoprikal, 'trmeeric', bell pupper', 'pestlc', pistachio', etc.
Recovered: Precisson Recall:
(c) Example where 9 performs better than fa-
Figure 14: Examples showing correlations captured by the reference VLM (fB) and the LLM (g).
18

---

Target Image
L-NNN of VLM
trained with It:
Desected:
Prectsion: 0.29
Recall: 0.36
I-NNN of langunge model
Caption: top view of crispy chicken with french
Desected: 5/04
Precision: 0.14
meat diakes
Recall: 0.11
Target
I-NN of VLM F
trained with Tt
Detected: 2931
Precision: 0 58
Recall: 0.94
I-NNs of language model
Caption: Flag of Saudi Arabia and
Desected: 1/31 Recall: 0.03
white background
Target Image
L-NNN vfVLM with
Desected: 30141
Precision: 0.38
Recall: 0.55
L-NNa
Caption: Madem Livinu, soom stile with urey soda yellow miblle table and hooksheff, Eurn and hamp.
Desected: 16:41
Precision: 0,42
Torget Imsage
VLM Inined
Desectod: 25/37
Precision: 0.16
L-NNs of language model
Caption: Flouting
huncl
Phuket
Detected:
Precision: 0
Recall: 0
Figure 15: Additional qualitative examples from Shutterstock data set showing examples memorized by the VLM.
19

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] is generally preferable to "[No]" it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Section 3 formally describes our methods identify dataset-level correlations. It showcases the effectiveness of our method for vision (VICReg, Barlow Twins and DINO) and vision language (CLIP) models. Section discusses the memorization both OSS models and models trained on subsets of the training data. The experimental results show that our method effectively identifies memorization in those models.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
20

---

Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We present the limitations of our methods at the end of section and 4.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We put the experimental details in the appendix.
21

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel urchitecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification:
The code is released on github. The links are provided in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
hhipss.c/public/uuddds http:///SoSoSSooooooooeoooo
www.obiiiiiiioooooooiiooo for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
22

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We believe that by using different models, we have already good estimate of how much noise there can be our process.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources are described in sections 4.1 and
Guidelines:
The answer NA means that the paper does not include experiments.
23

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The paper conforms with the NeurIPS Code of Ethics every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work has potential to lead to better tools for measuring privacy leakage stemming from memorization.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]

---

Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite the licenses in the appendix
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, haperswithcode com/ datasets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
25

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "bDmTKfjqaD", "reviewer": "Reviewer_ASwd", "summary": "The paper introduces a method to measure memorization in representation learning models without the need for training multiple models. Previous déjà vu memorization estimation methods require two models to estimate dataset-level correlations and memorization, which is computationally expensive. The authors propose a simplified approach using a single model, with alternative methods to estimate dataset-level correlations. This approach is validated on various image representation learning and vision-language models, showing that the simplified method yields similar results to the traditional two-model approach.", "questions": "1. Two-Model Dependency: Does this one-model method still rely on the use of two models, with one being a pretrained model used for estimating dataset-level correlations? How does the choice of such a pretrained model affect the results?\n\n2. Sample-level Metrics: Providing some discussion as to why aggregate metrics were chosen and how they compare to sample-level results could offer more detailed insights. Can the method provide sample-level memorization scores, and how would this affect the interpretation of the results?\n\n3. Are there specific cases where this approach may fail or provide? It was hard to decipher the limitations discussed at the end of sections 3 and 4.", "limitations": "1. Background Annotation Model Concerns: The potential overlap in data between background annotation models like Grounded-SAM and the dataset being tested should be addressed. If these models were not trained on disjoint sets, it could lead to misleading results regarding memorization.\n\n2. Sample-level Insights: The lack of sample-level memorization scores is a significant limitation. Providing some a discussion as to why aggregate metrics were chosen and how they compare to sample-level results could offer more detailed insights into which specific samples are being memorized and help in understanding the model's behavior better.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors present a simplified method for measuring memorization that avoids the need for training multiple models.\n2.  The method is validated across multiple datasets and models, including ImageNet-trained models and CLIP models, showing consistent results with the traditional approach.\n3. The proposed method can be applied to both image-only and vision-language representation learning models.\n4. The paper is well-written and organized, with clear explanations of the methods, experiments, and results.", "weaknesses": "1.  The method builds on existing ideas of dataset-level correlation estimation and déjà vu memorization, which have been previously explored in the literature. This reduces the novelty of the contributions.\n2.  The concern that background annotation models like Grounded-SAM used in the paper may not have disjoint sets is important. If the sets were not disjoint, it could affect the results by introducing unintended correlations between the training and testing sets. The paper should address whether this was considered and controlled for in the experiments.\n3.  The results are reported for the entire dataset, but memorization metrics are more meaningful at the sample level. The lack of sample-level memorization scores limits the granularity and insights of the analysis. Including sample-level scores would provide a clearer picture of which specific samples are being memorized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments!\n\n**Novelty of the work**\n\nOriginal Déjà Vu memorization [2] work requires training two different models with the same SSL architecture on two disjoint splits of the training data and is computationally expensive.\nThe novelty of our work lies in proposing creative ways of estimating déjà vu memorization for any vision and vision language (VLM) pre-trained model using a generic correlation detection classification model. Our approach is less expensive and can be used for multiple pre-trained SSL architectures and exhibits accuracy that is comparable with two model tests proposed in the original Déjà Vu paper [2].\n\n**Grounded-SAM's training set**\n\nThis is a good point, and this was one of our major considerations in annotation model selection. As our annotation model, we use Recognize Anything (RAM), a component of Grounded-SAM. While our representation learning models are trained on ImageNet, RAM does not use ImageNet during training. Their paper also says that ImageNet has an unusual tag presence, and is hence inappropriate for their training task. \n\n**Sample-level memorization**\n\nIn order to obtain dataset-level memorization, we aggregate sample-level memorization results. \n\n**Vision**\n\nAs described in [2]'s Figure 1, for each background crop we predict the foreground label both with a SSL model (e.g. pre-trained VICReg) and correlation classifier. If SSL predicts the correct class and the correlation classifier fails to predict the correct class then we mark that example as memorized. We then aggregate all memorized examples to obtain a dataset level view.\n\n**VLM**\n\nFor VLM learning, we do the evaluation of sample-level precision and recall gap between objects recovered from target and reference models and report the aggregate population level statistics in the main paper. \n\nWe include the sample-level statistics both for Vision and VLM in the1-pager PDF (see Figures 1 and 2) in the attached pdf. \nSee global rebuttal for more details.\n\nWe would be happy to include these visualizations in the revision.\n\n**Two-Model Dependency**\n\n**Vision**\n\n- one of the models is the target model for which we want to measure the memorization. This can be any pre-trained SSL model. \n- the other model is the reference model that is used to detect dataset-level correlations. It is trained once on (image crop, foreground label) pairs.\nWe still have two models but 1) we don’t have to train a target model  - we can use a pre-trained model and 2) we train a correlation classification model only once and use it as a reference across all pre-trained SSL Vision models.\n\n**VLM**\n\n- one of the models is the target model for which we want to measure the memorization. This could also be any pre-trained VLM.\n- the other model is a pre-trained language model (such as GTE) which we use to capture correlations. We only need to get the captions embeddings for kNN search once with this model. This can be reused for all the target models.\n\n**Pre-trained models**\n\nFor vision, pre-trained models are the target models which we measure the memorization for. We didn't use a pre-trained reference model for vision since we didn't see any such OSS model appropriate for our task.\n\nFor VLM we use a pre-trained language model (GTE) [1] as a reference model in the VLM one-model tests. This model is fairly complex and has strong generalization in terms of language model (LM) capabilities as it has been trained on 788M text pairs crawled from various sources from the internet. While this can be replaced with any off-the-shelf language representation model, we would expect similar memorization gaps.\nIn section 4.2.2, the pre-trained models correspond to the target VLMs that are pre-trained on existing datasets and are readily available as checkpoints for evaluation. Here we use the OpenCLIP's checkpoint pre-trained on YFCC15M dataset. The choice of the pre-trained target model will heavily impact the memorization results as the size and quality of the pre-training data will directly control the memorization capability of the model. For instance, the OpenAI CLIP model pre-trained on their 400M private data corpus will have different memorization than the YFCC15M pre-trained model. However, we would need access to the pre-training set to evaluate the memorization. Larger pre-training data typically leads to smaller population-level memorization.\n\n**Limitations of our work**\n\n**Vision**\n\n- The approach might be less successful if the training set for the correlation reference model is too small and the model is not able to learn dataset level correlations effectively. In our experiments we ensured a large subset (300k per class) for ImageNet. We recommend using large and representative sample size for training correlation classifier.\n- Although [2] shows that supervised models memorize significantly less compared to the SSL models, it is still possible that ResNet memorizes some of its training data. In contrast Naive Bayes doesn’t memorize its training data, however, it is a much simpler optimizer. Overall we have similar observations based on both types of correlation classifiers.\n\n**VLM**\n\n- The pre-training data of the reference LM either has an overlap with the target VLM’s training data, or has a superior generalization capability than the VLM. In either of the cases, the reference model might match or outperform the VLM in which case the memorization gap will be underestimated. This could happen if, for instance, we use a superior LM that has better reasoning capability to infer beyond dataset level correlations.\n- On the other hand if the reference LM is simplistic or has a poor language understanding, then the memorization gap will be overestimated.\n\n[1] Zehan Li, et.al. \"Towards general text embeddings with multi-stage contrastive learning\" arXiv, 2023\n\n[2] Casey Meehan et. at. “Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning”, NeurIPS, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eQmMfG7a2T", "reviewer": "Reviewer_wk9m", "summary": "This paper proposes methods to gauge the ability of a model to memorize", "questions": "1. In Section 4.1.2, I understand that the authors make the observation that pre-trained models memorize less than the same models trained on a subset of the data and they offer as an explanation that the pre-trained models have lower test error and larger training set. Can the authors clarify here what they mean? To me, if a model memorizes less and generalizes better I don't see any evidence that one is the cause of the other, these might both be the result of some other variable. Maybe the answer is as simple as larger training set leads to both phenomena, but I'd then like to see a test where the training routing (pre-training or not) is held constant and memorization is studied as it varies with trainset size. In any case, some more clarity is needed here.\n1. I feel a paper with this many results could be stronger if it ended with a bit more advice or conclusion. Do the authors have a clear ranking of models/techniques that a practitioner should use to avoid memorization or a set of available models that a user should use if they want a model with less memorization? These points would better motivate the paper as a whole. I think section 4.2.2 aimed to do this but it left me with these questions, so a clearer more direct bit of actionable advice to the reader would be appreciated.", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Originality: to my knowledge, this is an original work with novel results.\n1. Clarity: The writing is clear.\n1. Significance: The memorization of foundation models is critical to study, these methods and findings are significant to the community.", "weaknesses": "1. The results hinge on the smaller classifier not memorizing training data. The authors list this as a limitation, but some empirical result to convince the reader that this is in fact unlikely would be strengthen the argument.\n1. The comparison in Section 4.1.2 is slightly confusing (question below).\n1. A concluding ranking or thought about commonly used models is missing (question below)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed review of our paper and the questions!\n\n**Question 1- Clarification on Section 4.1.2**\n\nVision Representation\n\nIn our work we are not stating that there is a causal relationship between training size and memorization. We hypothesize that pre-trained models memorize less due to larger training sets. [1] shows in Figure 4 that increasing the size of training data doesn’t always increase the memorization score. This result, however, is reported on smaller subsets of the training data. Retraining the model on larger  subsets is computationally expensive and we haven’t run that experiment. Our goal was to show the effectiveness of our approach for pre-trained models and not retraining new models.\n\nVLM\n\nFor VLMs, previous work [2] (that requires training 2 models; target and reference models; for their memorization test) did show an experiment with different dataset sizes, where they concluded that larger training set size leads to lower population-level memorization. Our contribution is to show that their method can be applied with just one model, but our conclusion regarding dataset size should remain the same as theirs.\n\n\n**Questions 2 - Advice and stronger conclusion of the work**\n\nVision Representation\n\n+ Our key contribution is to purpose effective one-model tests that can be used for pre-trained vision models and are comparable to the two model tests in [1]. We do not rank the models based on the amount of memorized data. \n+ Nonetheless, we observe that DINO memorizes the least amount of data compared to VICReg and Barlow Twins. \n+ We plan to release a list of ImageNet examples with large dataset-level correlation scores; a list of examples for which the foreground can be inferred easily based on the background crop. This can help the vision community researcher to better estimate whether their models are memorizing certain examples or whether the models make correct predictions due to strong dataset-level correlations.\n\n\nVLM\n\n+ Our key contribution is a method to evaluate the déjà vu memorization with only one model, by utilizing an out-of-box pre-trained language model as a reference model. With this setup we show that the prior work of [2] can be extended to quantify the memorization of pre-trained models provided we have access to the training set. We show that one such pre-trained model (OpenCLIP model checkpoint pre-trained on YFCC15M) suffers from déjà vu memorization, similar to the models we train from scratch on a 40M subset of Shutterstock dataset. \n+ We hope our evaluation methodology helps future researchers and ML practitioners to evaluate the memorization in different models and compare which model is safe for deployment. While the mitigation strategies are not the focus of the paper, existing mitigation results from [2] can be extrapolated to our setting as our setup mimics theirs except that we do not need to train a second (reference) model from scratch.\n\n**Weakness 1 - smaller classifier not memorizing training data**\n\n[1] shows that classifiers memorize significantly less than self-supervised models. Apart from that we ensured that Grounded SAM, a tool that was used to annotate background images for Naive Bayes, did not rely on ImageNet dataset during training.\n\nWe'd be happy to make those clarifications in the paper.\n\n[1] Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, and Chuan Guo. “Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning.”, NeurIPS,  2023.\n\n[2] Jayaraman, Bargav, Chuan Guo, and Kamalika Chaudhuri. \"D\\'ej\\a Vu Memorization in Vision-Language Models.\" arXiv preprint arXiv:2402.02103 (2024).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aBzp1qO924", "reviewer": "Reviewer_Uwm9", "summary": "this paper proposes a method to measure memorization in representation learning models without the need for training two separate models. The proposed method aims to address the computational challenges and practical limitations of the existing déjà vu method by using simpler alternative approaches to quantify dataset-level correlations.", "questions": "see Weaknesses section", "limitations": "see Weaknesses section", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method achieves similar aggregate results in measuring memorization as the traditional two-model setup.\n- The use of a single model and simpler classifiers (like Naive Bayes and ResNet50) to approximate dataset-level correlations is interesting but also a limitation :) see the weaknesses section.", "weaknesses": "- The reliance on simpler models like Naive Bayes classifiers might limit the method's accuracy in certain complex scenarios.\n- There is a risk that the simpler models used for approximating dataset-level correlations might overfit their training data, which could skew the results.\n- The reference models used to approximate dataset-level correlations might introduce their own biases, affecting the accuracy of the memorization measurement.\n\n- The method provides an aggregate measure of memorization but lacks detailed insights into the types of memorized information. A more granular analysis could help in understanding the nature and implications of memorization in different models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper and raising those important questions!\n\n **Weakness 1  - Sample-level granular analysis** \n\nIn our paper we  obtain dataset-level statistics by aggregating sample level results. Appendix C.1 provides examples for Vision and C.2 for Vision language models. We ran additional experiments and shared the results in the 1 page PDF and in the global rebuttal.\nPlease, check global rebuttal section.\n\n**Weakness 2 - The reliance on simpler models like Naive Bayes classifiers might limit the method's accuracy in certain complex scenarios**\n\nThis is a good point. In our scenario for the ImageNet dataset Naive Bayes provided sufficiently good results. \nOverall, whether it is Naive Bayes or ResNet we observe similar memorization patterns. Our experiments show that many of those observations (for instance lower memorization in pre-trained models compared to the models trained on the subsets of the dataset) are also true for Vision Language Models (VLMs). \n\n\n**Weakness 3 - A risk that the simpler models used for approximating dataset-level correlations might overfit their training data, which could skew the results.**\n\nYes, this can happen. We used early stopping to avoid overfitting. Apart from that, we retrained the models multiple times with random seeds and showed a small variance across five training runs.\n\nSince Naive Bayes is a deterministic classifier, we used a bootstrapping approach to estimate the variance. We randomly sampled half of the examples per class and retrained Naive Bayes classifiers on those subset. We repeated the experiment 5 times to estimate the mean and the variance.\n\nThe results are presented in the table below:\n\n\n| Correlation Model  | Mean  | Std.  |\n|:----------------------|:--------|:------|\n| VICReg + KNN       |  7.53   | 0.26\n| Barlow T. + KNN   |  7.16   | 0.18\n| DINO + KNN         |  6.33   | 0.04\n| ResNet                  |  7.5     | 0.27\n| NB w/Top-1 CA     |  4.5     | 0.08\n| NB w/Top-2 CA     | 6.93    | 0.12\n| NB w/Top-5 CA     | 8.799  | 0.21\n\n\nThe table above represents the error bars for the 2a plot in our paper. Due to limited space in 1-pager rebuttal PDF we represent the results in a table. \n\n**Weakness 4 - Biases in the reference models**\n\nYes, this can be the case, however, we have similar observations for dataset, class and sample-level memorization across different types of correlation models (Naive Bayes and ResNet) and different types of representation learning models such as Vision Language Models and Vision Representation models.\nOur goal in this paper is to show that we can effectively replace two-model déjà vu memorization test with one model. We leave detailed analysis of the biases in the reference models for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**On the lack of sample-level memorization scores**
----

Reviewers Aswd and Uwm9 brought up an important question about the lack of sample-level memorization metrics and scores.

In our work we obtain dataset-level metrics by aggregating sample-level memorization results. Hence, it is straightforward to report the distribution of sample-level memorization scores. Figure 1 in the uploaded 1-page PDF visualizes the histogram of memorization confidence scores for the VICReg open source and Vision Language models.

**Vision Representation model**

Here we used ResNet as a correlation detection model. The same experiment can be repeated for Barlow Twins, DINO and other representation learning models. Instead of ResNet we can also use Naive Bayes correlation classifier. 

The memorization confidence for the `i`-th example is computed based on the following formula:

$$
MemConf (x_i)= Entropy(Correlation\ Classifier) - Entropy_{SSL}(KNN)  
$$

$Entropy_{SSL} (KNN)$ is computed according to the description in [1]’s Section 4, Quantifying Déjà Vu Memorization.

$Entropy (Correlation\ Classifier)$ is the entropy over the softmax values for the correlation classifier.

Figure 1 in 1-pager visualizes examples from both ends and the middle of the memorization confidence scores. We observe that the memorized examples with high memorization confidence scores are rarer and more likely to be memorized. The examples in the middle of the distribution have a label which is easy to be confused with another class. E.g. Black and gold garden spider with European garden spider. On the other hand the examples with negative memorization confidence have higher memorization and slightly lower correlation entropies. These seem to be examples for which the true label is not clear and visually obvious.

**Vision Language Models**

Figure 2 in the attached pdf shows the samples that have higher degree of memorization. The samples are sorted from high to low sample-level memorization such that the top-L samples have higher precision and recall gaps for recovering objects using target and reference models. For this test, we find the gap between the objects recovered from target and reference models for each training record, and estimate the precision and recall gaps. A positive gap indicates that the target model memorizes the training sample and the magnitude of the gap indicates the degree of memorization. Some of these worst case examples are shown in Figure 13 in the paper.

We will be happy to add these results to our paper.

[1] Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, and Chuan Guo. “Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning.”, NeurIPS,  2023.

Meta-review: The paper introduces a more efficient method to measure de ja vu memorization in representation learning models without the need for training multiple models. The method is validated across multiple datasets and models, including ImageNet-trained models and CLIP models, showing consistent results with the traditional approach.
The proposed method can be applied to both image-only and vision-language representation learning models

Please incorporate the changes asked by the reviewers and the clarifying discussions from the rebuttal period, such as lack of sample level memorization scores.
Decision: Accept (poster)