Captured by Captions: On Memorization and its Mitigation in CLIP Models
Authors: Wenhao Wang
Adam Dziedzic
Grace C. Kim
Michael Backes
Franziska Boenisch
Venue: ICLR 2025 Poster (2025)
Keywords: memorization
multi-modal
clip
vision language models
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose a metric to measure memorization in CLIP models and study the memorization behavior in the multi-modal setup.

Abstract: Multi-modal models, such as CLIP, have demonstrated strong performance in aligning visual and textual representations, excelling in tasks like image retrieval and zero-shot classification. Despite this success, the mechanisms by which these models utilize training data, particularly the role of memorization, remain unclear. In uni-modal models, both supervised and self-supervised, memorization has been shown to be essential for generalization. However, it is not well understood how these findings would apply to CLIP, which incorporates elements from both supervised learning via captions that provide a supervisory signal similar to labels, and from self-supervised learning via the contrastive objective.
To bridge this gap in understanding, we propose a formal definition of memorization in CLIP (CLIPMem) and use it to quantify memorization in CLIP models. Our results indicate that CLIP’s memorization behavior falls between the supervised and self-supervised paradigms, with "mis-captioned" samples exhibiting highest levels of memorization. 
Additionally, we find that the text encoder contributes more to memorization than the image encoder, suggesting that mitigation strategies should focus on the text domain. 
Building on these insights, we propose multiple strategies to reduce memorization while at the same time improving utility---something that had not been shown before for traditional learning paradigms where reducing memorization typically results in utility decrease.

Published as a conference paper at ICLR 2025
CAPTURED BY CAPTIONS: ON MEMORIZATION AND ITS MITIGATION IN CLIP MODELS
Wenhao Wang', Adam Dziedzic', Grace C. Kim², Michael Backes', Franziska Boenisch CISPA, 2Georgia Institute of Technology
ABSTRACT
Multi-modal models, such as CLIP, have demonstrated strong performance in aligning visual and textual representations, excelling in tasks like image retrieval and zero-shot classification. Despite this success, the mechanisms by which these models utilize training data, particularly the role of memorization, remain unclear. In uni-modal models, both supervised and self-supervised, memorization has been shown to be essential for generalization. However, it is not well understood how these findings would apply to CLIP, which incorporates elements from both supervised learning via captions that provide supervisory signal similar to labels, and from eef-suuer learning via the contrastive objective. To bridge this gap in understanding, we propose formal definition of memorization in CLIP (CLIPMem) and use to quantify memorization CLIP models. Our results indicate that CLIP's memorization behavior falls between the supervised and self- supervised paradigms, with mis-captioned" samples exhibiting highest levels memorization. Additionally, we find that the text encoder contributes more memorization than the image encoder, suggesting that mitigation strategies should focus on the text domain. Building on these insights, we propose multiple strategies to reduce memorization while at the same time improving Solity-something that had not been shown before for traditional learning paradigms where reducing memorization typically results in utility decrease.
INTRODUCTION
Multi-mode models, such CLIP (Radford et al. (021). have demonstrated strong performance in representation learning. By aligning visual and textual representations, these models achieve state-of- the-art results in tasks like image retrieval (Baldrati al., 2022a b). visual question answering (Pan
et al. 2023 Song et al.) 2022). and zero-shot classification (Radford et al. 2021 Ali & Khan 2023 Wang al. 2023 Zhang al., 2022). Despite these successes, the mechanisms by which modti-modal models leverage their training data to achieve good generalization remain underexplored.
In uni-modal setups, both supervised (Feldman 2020 Feldman & Zhang 2020) and self- supervised (Wang et al. (2024b), machine learning models have shown that their ability to memorize their training data essential for generalization. was indicated that, supervised learning, mem- orization typically occurs for mislabeled samples, outliers (Bartlett et al. 2020 Feldman 2020 Feldman & Zhang 2020), or data points that were seen towards the end training (Jagielski et al. 2022). while in self supervivdd learning, high memorization is experienced particularly for atypical data points (Wang et al. 2024b). However, it is unclear how these findings extend to models like CLIP which entail elements from both supervised learning (through captions supervisory signals) and self-supervised learning (through contrastive loss functions).
Existing definitions of memorization offer limited applicability to CLIP and therefore cannot fully address the gap in understanding. The standard definition from supervised learning (Feldman 2020) relies one-dimensional labels and the model's ability to produce confidence scores for these labels, whereas CLIP outputs high- nigh-dimensional representations. While the SSLMem metric (Wang et al. 2024b), developed for sule-super vision models, could, in prineiple, be applied to CLIP's vision encoder outputs, it neglects the text modality, which is a critical component of CLIP. Additionally,
"Correspondence boenisch@ cissp.dde

---

Published as a conference paper at ICLR 2025
Anthon of was cometiing
prcup of peoele canding
get inper pweaker outho thus besls over yeltow fhe hystoae
do and take hydlows
leo make andia hamale in red too
colo with some yelloo aghbiin of namoo bualoig
Crosco of eneer sipne that eeed Mersort Fery" and "Kamnal
sipnis dap syedion pole
A wormn opening trunk her
parling for with our cars
A sig with Oxicesal liinng and
irainees stedl
cauncing
pepor discory
pas Sity monroveee shatt
enoulder
Manii ide""
An open paint of new morm warlos things around
Lm unsble noo imsge sbows
soying "Don Periity on polo.
aardidd affhordss.oom slokkydd in fone
(a) Most Memorized: CLIPMem > 0.89
(b) Least Memorized: CLIPMem = 0.0
Figure| 1: Examples of data with different levels of memorization. Higher memorization scores indicate stronger memorization. We observe that atypical or distorted images, as well as those with incorrect or imprecise captions, experience higher memorization compared to standard samples and easy-to-label images with accurate captions. Results are obtained on OpenCLIP (Ilharco et al. 2021 with encoders based on the ViT-Base architecture trained on the COCO dataset.
measuring memorization in only one modality, or treating the modalities separately, risks diluting the signal and under-reporting memorization. Our experimental results, shown in Section 4.3 confirm this concern. Therefore, new definitions of memorization tailored to CLIP's multi-modal nature are necessary.
The only existing empirical work on quantifying memorization in CLIP models (Jayarman et al. 2024) focuses on Déjà Vu memorization (Meehan et al., 2023), a specific type of memorization. The success of their method relies on the accuracy of the integrated object detection method and on the availability of an additional public dataset from the same distribution as CLIP's training data, limiting practical applicability. To overcome this limitation, we propose CLIPMem that measures memorization directly on CLIP's output representations. Specifically, compares the alignment-i.e. the similarity between representationss-of given image-text pair in a CLIP model trained with the pair, to the alignment in a CLIP model trained on the same data but without the pair.
In our empirical study of memorization in CLIP using CLIPMem, we uncover several key findings. First, examples with incorrect or imprecise captions ("mis-captioned" examples) exhibit the highest levels of memorization, followed by atypical examples, illustrated in Figure [ Second, removing these samples from training yields significant improvements in CLIP's generalization abilities. These findings are particularly noteworthy, given that state-of-the-are CLIP models usually trained on large, uncurated datasets sourced from the internet with no guarantees regarding the correctness of the text-image pairs. Our results highlight that this practice not only exposes imprecise or incorrect data pairs to more memorization, often recognized as cause for increased privacy leakage (Carlini al. 2019 2021; 2022) Song et al.] 2017; Liu et al.] (021). but that it also negatively affects model performance. Furthermore, by disentangling CLIP's two modalities, we are able to dissect how memorization manifests within each. Surprisingly, we find that memorization does not affect both modalities alike, with memorization occurring more in the text modality than in the vision modality. Building on these insights, we propose several strategies to reduce memorization while simultaneously improving generalization-22 result that has not been observed in traditional supervised or self-supervised learning, where any reduction of memorization causes decreases in performance. Finally, at a deeper level, our analysis of the model internals, following Wang et al. (2024a), shows that CLIP's memorization behavior sits between that of supervised and self-supervised learning. Specifically, neurons in early layers responsible for groups of data points (e.g., classes), similar to models trained using supervised learning, while neurons later layers memorize individual data points, as seen in self-super learning.
In summary, we make the following contributions:
We propose CL metric measure memorization in multi-modal vision language models.
2

---

Published as a conference paper at ICLR 2025
Through extensive evaluation, identify that "mis-captinn captsonet and "atypical" data points expe- rience the highest memorization, and that the text encoder is more responsible for memorization than the image encoder.
Based on our insights, we propose and evaluate multiple strategies to mitigate memorization in CLIP. We show that in CLIP, contrary to traditional supervised and sulf-super learning, reduction of memorization does not need to imply a decrease in performance.
2 BACKGROUND AND RELATED WORK
CLIP. Contrastive Language-Iman Pretraining (CLIP) (Radford et al. 2021) trains mudai encoders to map text-image pairs into shared latent space with semantically equal representations. The core of CLIP is a two-encoder architecture with an image encoder fing and a text encoder fia that are trained maximize the similarity between the image and text features for correct text-image pairs, while minimizing the similarity for incorrect pairs. This is achieved using a contrastive loss function L defined as:
exp(sim([fng (z.). (()()))/TT)
C= N 1=1 Σn= exp(sim (2,,, fua(U)))/T)
log
where sim( is the cosine similarity, is the temperature parameter, and N the batch size. This training makes CLIP versatile across various downstream tasks, including image classification, retrieval, captioning, and object recognition. There different versions of CLIP. The popular Language augmented CLIP (LaCLIP) (Fan et al. (023) augments original CLIP by introducing text augmentations during training in addition to the image augmentations (crops) performed in the original CLIP training to reduce overfitting. We study the impact of this practice on memorization
and find be suitable mitigation method.
Memorization. Memorization refers to model's tendency to store specific details of individual training examples, rather than generalizing patterns across the dataset (Zhang et al. 2016 Arpit et al. 2017: Chatterjee 2018 Feldman 2020). This becomes problematic when models memorize sensitive data, as it has been shown to increase privacy risks (Carlini et al. 2019 2021 2022, Song cet al. 2017). To date, memorization has been studied within single modalities for supervised and self-supervised learning. In supervised learning, has been shown that models tend to memorize mislabeled (Feldman 2020), difficult, atypical examples (Arpit al. 2017 Sadrtdinov al. 2021
and that this memorization improves generalization, especially on long-tailed data (Feldman 2020
Feldman & /Zhang 2020). Similar findings have been observed in self-supervised learning (SSL. in
the vision domain Wang et al.) 2024b). where atypical samples experience high memorization, and a reduction of memorization in SSL encoders leads decreased performance in various downstream tasks, such as classification, depth depth-estimation, and segmentation. A connection between memorization and generalization has also been observed in the language domain (Antoniades al. 2024 Tirumala et al. (002)). In contrast to our work, these papers consider single-modality models. How those insights transfer to muulti-moal models remains unclear.
Memorization in self-supervised learning. Our CLIPMem builds concepts from the SSLMem metric introduced by Wang et al. (2024b). This metric measures the memorization an individual data point v: by an SSL. encoder, based on the alignment of representations from augmented views of Rd be an SSL encoder trained using an SSL algorithm A on an unlabeled dataset The data augmentations transformation function applied to the data point æ, mapping from RR > R" The encoder's output
x. Let R R" + S (-, Lm
represented as Aug(z) {a(x) la E Aug), where is
representation for given data point is denoted as f fz). For trained SSL encoder f. the alignment loss for a data point & is defined as
Calign( (f, r'.""~
E
(f((z))
where x', x are augmented views of I and d(...) is a distance metric, typically the & distance. SSLMem then defined
SSLMem(x) = E g~((()())
Calign x)
E (~J((S)
Catign (f,
(2)
with f being an SSL encoder trained with data point x, and g. an encoder trained without x but otherwise the same dataset. While this framework measures memorization using alignment loss
3

---

Published as a conference paper at ICLR 2025
for single-modaliyy encoders, this approach is unsuitable to leverage the signal over both modalities from multi-modas encoders like CLIP, as we also highlight empirically in Section 4.3 However, we can build the main concepts from SSLMem to define new metric that can evaluate memorization in CLIP, by considering both image and text representations, as we will detail in Section
Memorization in CLIP. Even though CLIP is widely used vision-language encoder, there has been limited work on measuring memorization in CLIP. The only existing work (Jayaraman et al. 2024) applies the empirical Déjà Vu memorization framework from (Meehan al.) 2023) to CLIP. measures memorization by computing the overlap between unique objects in potentially memorized images and their nearest neighbors-identiee iddettiiidd in the CLIP embedding space space-frome from a public dataset. However, the reliance on externl public data from the same distribution, along with the required accuracy of the object detection (which may not perform well for all samples, especially atypical ones (Kumar et al., 2023; Dhamija et al. (220),, limits the applicability of this approach. We further expand on this in Appendix |A.1] In contrast, our CLIPMem operates directly on CLIPP's output representations and returns joint score over both modalities.
DEFINING MEMORIZATION OVER MULTI-MODAL ENCODERS
3.1 PROBLEM SETUP
Consider a single image-text pair (T, T) from a dataset and two CLIP models: a model and a reference model g, trained on dataset S and S' = S {(T,T)) respectively. We aim to quantify the memorization of (1, T) in f. trained on this data point, by leveraging model not trained on the data point but otherwise on the same data, in a leave-oneoout style of defining memorization (Feldman 2020). We denote the image encoder in CLIP as fing- Image + Rd and the text encoder as fia Text + Rd For the image-text pair (I, T),, we denote with fimg the output representation of f's image encoder image 1 and with (T) the output representation of f's text encoder text T. To evaluate the alignment between the image and text representation,,, iee, to quantify how similar the two representations are, we use cosine similarity sim(simg! (4), fex (T)), as defined in the original CLIP paper (Radford et al. 2021).
3.2 ALIGNMENT WITH CONTRASTIVE ORJECTIVE
During training. the contrastive objective in CLIP maximizes the cosine similarity for correct image- text pairs while minimizing the cosine similarity for all the other N 1 incorrect pairs in any given training mini-batch with N training samples. This means that for given image / and text T. the training objective pulls (7m and (T) closer together in the latent space, while pushing fing (I) away from the representations of all other N - unrelated texts, and fia(T) away from all other images. Hence, the intuition is that the quality of alignment in f. unlike in uni-modal self-supervised learning (Wang et al., 2024b), depends not only on the model's ability to create well-aligned text and image representations for a given text-image pair, but also on its ability to create distant representations for the N 1 other representations.
To formalize this intuition into a metric that quantifies the alignment of f on the image-text pair (I, T), we define Tiast as set of N randomly chosen testing samples that were not used in training f or g. Furthermore, when applicable, we denote random augmentations of the training data-e.g, text augmentations in versions like LaCLIP (Fan et al., 2023 -as as T" ~ ATg for texts
and I' ~ Aug(T) for images. Then, we define the alignment score of f on (I,
Aatign Aatign (J,I,T) = E (sim[fimg (I'), Sox (T')) (1 (r:Tr)-Ag(1I,))
(3)
E (.)eTT
[sim( (f), fux(t))]
E (
[sim(fin (i), fex (T)),
where high scores indicate a better alignment of f on (I,T). In case no text augmentations are applied, as in standard CLIP training, the first term is calculated only over T.
4

---

Published as a conference paper at ICLR 2025
3.3 DEFINING MEMORIZATION CLIP
Given our definition of alignment scores, we can define our CLIPMem in a similar vein to the definition of memorization in supervised learning (Feldman) 2020), in the leave-one-out style. Given the image-text pair (1,T) from dataset S and two CLIP models, f and trained on dataset S and S' S {(I,T)) respectively, we define CLIPMem as
CLIPMem(1, (1 = Aalign
(f,1,T) Aa, Anlign (g. 1,
(4)
If model f has a significantly higher alignment score than model 9 on (I,T), T), this means that memorizes this data point. Note that taking the difference between f and is crucial to get solid estimate of memorization. This is because without "context", a high or low alignment score of does not express much information. The aligmment of f can be high without memorizing (I, for example, if (I, (T) ) is simple (but not memorized) training example. In this case, the reference model g will also have high score, such that the difference is again small. Thanks to this design of our CLIPMem, it will then correctly report low memorization.
EMPIRICAL EVALUATION
4.1 EXPERIMENTAL SETUP
Models and training. We build our experiments on OpenCLIP (Cherti et al. (023), an open- source Python version of Open-CLIP (Ilharco et al. 202I). The standard architecture used for the experiments builds on ViT-Base, but we also include experiments using ViT-Large. We train the model on the COCO dataset (Lin et al., 2014). Since COCO is much smaller than OpenCLIP's standard training datasets, we reduce the training batch size to 128 and increase the epoch number from 32 to 100 to achieve similar performance. All other settings strictly follow OpenCLIP. For training DINO, as example of SSL. vision encoder, follow the default setting Ca et al.) (202T). The supervised model is trained as a multi-label classifier, also based on ViT-Base (with an additional fully connection layer) based on the first-level annotation captions in the COCO dataset. A full specification of our experimental setup is detailed in Appendix A.2 Additional experiments for measuring memorization on the BLIP (Li et al.
2022) model are presented in Appendix AA.6
Datasets. We use COCO Lin et al. 2014), CC3M (Sharma et al. 2018). and the YFCC100M (Thomee et al. 2016a) datasets to pre-train the OpenCLIP models. For the CC3M dataset, we randomly sample 7500) examples from the total of 2.91 M data points. We evaluate the models by testing the linear probing accuracy on ImageNet (Deng al., 2009) with an added classification layer trained on top of the output representations. We use the YFCC100M dataset to simulate an infinite data regime, i.e., using a single training run where no data point is repeated whereas we train iteratively using CC3M and COCO.
Mcasuring memorization. We follow Wang et al. (024b) to approximate our CLIPMem. Since training separate pair of models for every data point whose memorization would be computationally intractable, we measure memorization of multiple data points at the same time. Therefore, we divide the original training in four subsets: (1) Ss, data points that both model f and g were trained on, (2) Sc, data points used only for training f. (3) Sr, data points used only for training g. and (4) Sg. external "test" data points that none of the models was trained
aim to measure
on. Note that |Sc| |5r], such that f and have the same number of training data points in total. For our experiments, following similar approach to Wang et al. (0224)), we want to strike a balance when choosing the size of Sc. If the size is too large, then f and might differ too much and not yield strong memorization signal, but if memorization signal for too few data points. Concretely, for COCO and CC3M, we set [Ssl = 65000 and |Scl = |Sr| | = 5000. Memorization is reported as an average over all data points in Se for model f. or per individual data point in Sc.
too small, we would only have
Generating captions and images. For generating additional captions for the training images, we rely on GPT-3.5
For each input image, we provide the representation produced by our trained OpenCLIP model and ask GPT to generate five new captions. Generated sample captions are presented in Figure [8] To generate additional images for the COCO dataset, we use Stable Diffusion v1.5 to generate five new images, one corresponding to each of the five per-image captions in the COCO dataset. Sample generated images are presented in Figure 17
5

---

Published as a conference paper at ICLR 2025
600
Frequency
500
Sc
400
S:
300
S:
200
Cleam Se
Poisoned Sig
100
Clean Model
0.38 0.440
NIA 0.586
-1.00
-0.75
-0.50
-025
0.00
0.25
0.50
0.75
1.00
Poisoned Model
(b) Average CLIPMem scores.
CLiPMem
(a) Memorization scores across data subsets.
Figure 2: Memorization with CLIPMem. We train a CLIP model on COCO using standard image cropping and no text augmentations. (a) We present memorization scores according to CLIPMem per data subset. The significantly higher scores for Sc compared to Sş indicate that f memorizes Sc. (b) We also study how inserting training samples with imprecise or incorrect captions ("mis- captioned")) affects memorization. We refer to the model trained with correct captions as Clean Model, and the model trained with Sc containing 4500 standard canaries (Clean) and 500 mis-captioned (Mis-captioned) as Poisoned Model. We report CLIPMem over the different subsets of candidates. We observe that the mis-captioned samples experience significantly higher memorization while the memorization of the clean data points is (almost) not affected.
4.2 STUDYING MEMORIZATION USING CLIPMEM
We first set out to analyze the general memorization in CLIP in order identify which data points are memorized. To do this, we quantify CLIPMem over the different training subsets. Our results are presented in Figure 2a] In particular, we observe that CLIPMem for Sc, the data points only used to train model J. is significantly higher than for Ss, the data points shared between the two models. Memorization for Sş comparable to that for S5, Le.,, the external data not seen during training, indicating that does not memorize these samples. The data in SJ causes negative CLIPMem scores, indicating that this data is memorized by g. not by f. This is the expected behavior according to the definition of our metric. In Appendix A.4, we additionally highlight that memorization increases with model size, i.e., CLIP based on Vir-Large has a higher overall memorization with an average of 0.457 while CLIP based on ViT-Base only reaches 0.438 on average.
Additionally, we analyze individual data points according to their reported CLIPMem. We give examples of highly memorized data points in CLIP in Figure S and more highly vs. little memorized samples in Figures 13] 14 15) T6 in Appendix A.9 Overall, the samples with high CLIPPMem, e.g., in Figure seem to be difficult examples and examples with imprecise or incorrect captions whereas the samples with low CLIPMem are simpler and (potentially consequently) more precisely captioned. In Appendix A.5 we show that these findings also hold when we operate in the infinite data regime, i.e., when we perform only single training run where no data point is repeated.
Motivated by this insight and by observations from supervised learning where it was shown that models can memorize random labels (Zhang et al., 2016) and where mislabeled data experiences highest memorization (Feldman 2020), we test if the same effect can also be observed in CLIPP Therefore, we "poison" our CLIP'S training data by randomly shuffling the captions among 500 of the 5000 candidate data points in Sc. Thereby, these 500 data points are "mis-captioned". We train a model based on this data and see that the mis-captioned examples experience significantly higher memorization (CLIPMem of 0.586) compared to the "clean" data points (CLIPMem of 0.440). Even though CLIP trains using contrastive training objective, the memorization of clean data points is not significantly affected by training the model with the mis-captioned examples, as we can see by their CLIPMem that is 0.438 the clean model and 0.440 on the poisoned model.
4.3 MEASURING MEMORIZATION IN ONE MODALITY DOES NOT YIELD STRONG SIGNAL
To understand how important it is to take both modalities into account in definition of CLIPMem, we set out to evaluate whether existing practical methods to measure memorization over uni-modal encoders (Wang et al. 2024b) yield sufficiently strong memorization signal in CLIP. Therefore,

---

Published as a conference paper at ICLR 2025
600
700
600
frequent)
(
500
5, Avg. 0.001
S-Aug 0.168
600
500
5- Awp.
Avg.
0200
0.013
500
400-
300
400
300
200
200
200
100
100
100
-1.00
-0.15
-0.50
-0.2%
00
0.25
0.50
0.15
1.00
-1.00
-0.75
50
-0.25
0.00
0.25
0.50
0.75
1.00
-0.15
-0.50
-0.25
0.
0.00
0.25
0.50
0.75
1.00
SSLMem
SSLMem
CLIPMem
(a) SSLMem (Img).
(b) SSLMem (Text).
(c) CLIPMem.
Figure 3: Measuring memorization on individual modalities is not able to extract strong signal. (a)-b) We measure SSLMem (Wang et al. 2024b) on the individual encoders of our CLIP model trained on COCO. (c) Our CLIPMem extracts stronger memorization signal by using both modalities in CLIP jointly.
apply their SSLMem to the individual encoder parts of CLIP. Since SSLMem relies on augmentations of the encoder input, we use image crops, like during CLIP training for the vision encoder, and the COCO captions as augmentations for the text, like (Fan et al., (023). Our results in Figure highlight that SSLMem and its naive adaptation to CLIP fail to yield a strong signal for memorization. In particular, there is high overlap in scores between the non-memorized samples from Ss, and candidate examples for memorization Sc. Additionally, the highest reported memorization scores for Sc go up to around 0.65 (for SSLMem on the vision encoder) and 0.73 (for SSLMem on the text encoder). In contrast, our new CLIPMem is able to get distinct signal for the candidates So with respect to Ss and reports a much higher memorization of 0.91. Thereby, our CLIPMem prevents under-reporting the actual memorization in CLIP.
4.4 MEMORIZATION BETWEEN MODALITIES
Our results in Figure indicate that memorization is higher in CLIP's text encoder than in the image encoder (the average SSLMem on Sc in the text encoder is 0.209 vs. 0.168 in the image encoder). To provide further insights into how memorization behaves between the modalities CLIP, first analyze the use of augmentations. We compare five cases: (1) no additional augmentations beyond the baseline (image cropping), (2) generating one image using a diffusion model for a given original caption, (3) generating five variations of each image using diffusion model and randomly selecting one for each training iteration while keeping the caption fixed, (4) using the original image but randomly selecting one of the five COCO captions for each training iteration, and (5) randomly pairing each the five generated images with one the five COCO captions.
As shown in Figure 14 there is quite variabil- ity in the COCO captions for the same sample. Hence, some images might not fit well with the chosen training caption. This imprecise caption- ing cause increase in memorization. We observe that the effect is mitigated when using the 5 images with the 5 captions (5th case, see Table T). This phenomenon results most likely from the increased number of possible image-text pairs (25), such that individual incorrect or imprecise pairs are not seen often during training. For the third case, i.e. row three in Table we generate five images with diffusion model based on all five captions per image from the COCO dataset. However, as we only use the first caption during training, this would introduce many mis-captioned images which significantly lowers performance and increases memorization. To avoid this problem, we removed 6000 mis-captioned samples.
Table Impact of augmentations.
Con
C1.PPMom
Lin. Prob Ap: (ImagsNir)
Image, Captinn
1
Imtage (generized), Caption
Images (genermesh. Capcion
Imugs, 5 Captions
61.58%
Images (poneuted), Captions
64.79%
Our results in Table I highlight that augmenting text during training reduces memorization and increases performance more than augmenting images. However, applying augmentations of both text and images strikes the right balance between the reduction in memorization and the increase in performance. In fact, applying both augmentations reduces memorization most significantly. Overall, these results indicate that memorization in CLIP's is tightly coupled to the captions assigned to the training images with imprecise captions having a destructive effect on CLIP performance and memorization.
7

---

Published as a conference paper at ICLR 2025
4.5 RELATION TO CLIP MEMORIZATION TO (SELF-)SUPERRISED MEMORIZATION
We further provide insights whether CLIP's memorization behavior is more alike to the of supervised learning or SSL.. This question is highly interesting since the captions in CLIP can be considered form of labels, like in supervised learning, whereas the contrastive training objective on the dataset resembles more SSL. We perform two experiments to gain a better understanding of the memorization behavior of CLIP with respect to supervised learning and SSL.
First, we compare an SSL vision encoder pair f and g with the same architecture as CLIP's vision encoder but trained from scratch on COCO using DINO, i.e., standard SSL training. We train f and g using the same candidates as the pair of CLIP models in our previous experiments. Then, we use the SSLMem metric from Wang et al.) (024bb quantify memorization in the CLIP vision encoder and the SSL encoder, respectively. The CLIP vision encoder has a significantly lower SSLMem than the SSL encoder (0.209 vs. 0.279). Hence, CLIP vision encoders experience lower SSL memorization than SSL trained encoders. To further investigate the difference, we also report the overlap between the top 10% memorized samples between the two models, measured according to SSLMem. With an overlap of only 47 out of 500 (9.4%) samples, we find that CLIP memorizes significantly different samples than SSL encoders. WWang et al. (2024b) had performed a similar experiment on SSL vs. supervised learning and found that the two paradigms also lead to different samples being memorized. While this is, on the one hand, an effect of the different objective function, the difference between the memorized samples in CLIP and SSL is likely also closely connected the additional captions that CLIP takes into account. While SSL-trained encoders can memorize atypical images, CLIP encoders can memorize typical images when they have an atypical, imprecise, or incorrect caption.
Additionally, we compare the memorization behavior of CLIP against supervised and SSL-trained models on the neuron-level. Therefore, we train two additional ViT- Base models on COCO using supervised training and SSL.
UnitMem
CLI Acc. +3225%
0.45
SOL (P Acc. 61.55
Suservizz Aep. procalon 70.2%%
0.40
BLP Aer. 66 18%
training with DINO. Then, we apply the UnitMem met- ric (Wang et al. 2024a) to measures how much individual neurons memorize individual samples from the training data. A high UnitMem suggests that neurons highly mem- orize individual data points instead of groups/classes of points. It had been shown that supervised learning causes neurons in lower layers to experience low UnitMem, i.e., being responsible for learning joint groups of data points, while neurons in later layers highly memorize individual data points. contrast, for SSL, UnitMem was shown to remain relatively constant over layers with neurons in lower layers also being able to memorize individual data points. This difference was autri to the different objective functions where supervised learning's s cross entropy loss pulls together data points from the same class, whereas SSL's contrastive loss leads to individual data points being pushed away from each other (Wang et al., 2024a). Our results in Figure highlight that CLIP, in terms of its memorization behavior, is between supervised learning and SSL. At the lower layers, it is much less selective than models trained with SSL, i.e., it focuses on groups of data points rather than memorizing individual data points, similar to supervised learning. Yet. in later layers, CLIP becomes more selective than SSL, i.e., memorizes individual data points more in individual neurons, but still less than supervised learning which there has a very high average per-layer UnitMem.
0.35
0.30
0.25
1
2
3
4
5
6
7
101112
Layer
Figure 4: UnitMem metric: CLIP is between supervised and SSL models.
4.6 MITIGATING MEMORIZATION WHILE MAINTAINING GENERALIZATION
The experiments from Table T suggest that using augmentations during training can improve general- ization while also reducing memorization. This an unexpected synergy since for both supervised learning (Feldman) 2020) and SSL (Wang al.) 2024b), generalization was shown to decline when memorization decreases. To further study the impact of mitigating memorization in CLIP on down- stream generalization, we explore two orthogonal strategies for "augmenting" the text modality during CLIP training. first in the input space and second directly in the embedding space. Additionally, we analyze the effect of removing memorized samples from training.
Multiple captions. We vary the number of captions used during training and report fine-grained insights into the resulting memorization and downstream performance in Figure 5a Our results
8

---

Published as a conference paper at ICLR 2025
0.4400
0.660
0.45
0.67
Memorization
Memorization
0.4375
0.655
0.66
0.4350
0.650
0.44
0.65
0.4325
0.645
0.64
0.43
0.4300
0.640
0.63
0.4275
0.635
0.42
0.62
0.4250
0.630
0.61
0.4225
0.625
0.41
0.60
0.4200
0.620
2
3
4
5
# of Caption Used
Noise Strength (Std.)
(a) Different numbers captions.
(b) Noising text embedding during training.
Figure 5: Mitigating memorization CLIP improves downstream generalization. We train CLIP models with different "augmentations" in the textual domain. (a) We multiple captions for the same image during training. (b) We directly noise the text embeddings during the training using Gaussian noise with mean of 0 and different standard deviations (adding the Gaussian noise (0, 0.15) gives us the sweet spot with the smallest memorization and highest performance). Both strategies successfully reduce memorization while improving performance.
Linear probing Acc.
Linear probing Acc.
0.67
Randem
0.68
Random
0.66
Clip Similarity
Cip similarity OipMern B Based
0.65
OipMem Based
0.67
0.64
0.63
0.66
0.62
0.65
0.61
0.60
0.64
50
100
200
400
800
1600
3200
50
100
200
400
800
1600
3200
# of Sample removed
# of Sample removed
(a) CLIP trained on COCO.
(b) CLIP trained on CC3M.
Figure 6: Removing memorized samples according to CLIPMem has stronger influence on the linear probing accuracy than removing random data points. Removing the mislabeled samples based on CLIPMem improves the performance significantly. followed by a sharper drop when removing atypical samples.
highlight the trend that the more captions are used during training, the lower memorization and the higher the linear probing accuracy. Our additional results in Table highlight also that choosing all captions equally often is beneficial for utility while keeping memorization roughly the same. experiment with generating these where we train CLIP with captions generated extremely similar
Since not in every dataset, multiple captions captions with a language model. Our results in There
available.
by GPT3.5 show that the results both in terms of utility and memorization to the original captions, making this improved training strategy widely applicable. Our findings that modifying the text during training can reduce memorization align with the insights presented by Jayaraman et al. (2024). For datasets where only single captions are available, they proposed text randomization, i.e., masking out fraction tokens during training as a mitigation for their Déjà Vu memorization. In contrast to our GPT3.5-generated captions, this masking, however, causes drop performance when mitigating memorization. We hypothesize that this due to the higher distribution shift introduced by the masked tokens.
Noising the text embedding during training. overcome such shortcomings altogether and avoid any inherent distribution shifts, propose to perform the "augmentations" directly in the embedding space. More precisely, we experiment with an approach where, during training, before calculating the cosine similarity between text and image embeddings for the contrastive loss, add small amounts of Gaussian noise to the text embeddings. Our results in Figure [5b and Table X highlight that this strategy is highly effective in reducing memorization while improving downstream generalization.
Removing memorized samples. Finally, we investigate the effect of removing memorized samples understand how it impacts downstream performance. We perform additional experiment where

---

Published as a conference paper at ICLR 2025
we first train CLIP model, then identify the highest memorized training data points, remove them, and retrain on the remaining data points only. We compare this method to two baselines where we either randomly remove samples filter out the samples with the lowest CLIP similarity between the training data points" two modalíties. We showcase the effect on the downstream linear probing accuracy ImageNet in Figure 6 with CLIP models trained COCO and on the CCM3 dataset. For the COCO dataset, when removing up to 100 most memorized data points, we first observe a sharp increase in downstream performance in comparison removing random samples. Then, the downstream performance starts dropping significantly more when removing memorized instead of random samples, until between 400 and 800 removed samples, the cutoff point is reached where model performance is worse when removing according to highest memorization instead of randomly. For the CC3M dataset, this cutoff occurs later, between 1600 and 3200 removed samples. While the CLIP similarity also manages to increase performance through removal, it is not as effective as CLIPMem, highlighting the value of considering memorization as a lens to identify noisy samples. This finding is significantly different than for supervised learning and SSL, where the removal of highly memorized samples constantly harms performance more than the removal of random samples (Feldman 2020 Wang al.] 2024b). We hypothesize that the effect observed in CLIP might result from the distinction between "mis-captioned" and atypical samples, where the former harm generalization while the latter help the model learn from smaller sub-populations (Feldman 2020). We empirically support this hypothesis in Appendix | The finding that CLIP generalization can be improved by identifying inaccurately captioned data points using our CLIPMem and removing them from training is of high practical impact, given that state-of- offte-of-the-ar CLIP models are usually trained on large, uncurated datasets sourced from the internet with no guarantees regarding the correctness of the text-image pairs. Overall, our results suggest that CLIPMem can help reduce memorization in CLIP while improving downstream generalization.
CONCLUSION
We presented CLIPMem, a formal measure to capture memorization in mulli-modal models, such as CLIP. By not only quantifying memorization but also identifying which data points are memorized and why, we provide deeper insights into the underlying mechanisms of CLIP. Our findings highlight that memorization behavior of CLIP models falls between that of supervised and self-supervised models. In particular, CLIP highly memorizes data points with incorrect and imprecise captions, much like supervised models memorize mislabeled samples, but it also memorizes atypical examples. Furthermore, we find that memorization in CLIP happens mainly within the text encoder, which motivates instantiating mitigation strategies there. By doing so, we not only reduce memorization in CLIP but also improve downstream generalization, result that challenges the typical trade-offs seen in both supervised and self-supervised learning.
ACKNOWLEDGEMENT
This research was funded by the Deutsche Forschungsgemeiisccaftt (DFG, German Research Foun- dation), Project number 550224287. We would like to also acknowledge our sponsors, who support our research with financial and in-kind contributions, especially the OpenAI Cybersecurity Grant.
REFERENCES
Muhammad Ali and Salman Khan. Clip-decoder: Zeroshot multilabel classification using multimodal clip aligned representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4675-4679, 2023.
Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs. memorization: Tracing language models' capabilities back to pretraining data. ICMI. 2024 Workshop on Foundation Models in the Wild, 2024.
Devansh Arpit, Stanistaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning. pp. 233-242. PMLR, 2017.
10

---

Published as a conference paper at ICLR 2025
Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Conditioned and composed image retrieval combining and partially fine-tuning clip-based features. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pp. 4959 4969, 2022a.
Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Effective conditioned and composed image retrieval combining clip-based features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21466-21477 2022b.
Peter I. Bartlett, Philip M Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 117(49):30033-30077 2020.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jemnj Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. 28th USENIX Security Symposium (USENIX Security 19), pp. 267-284, 2019.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650, 2021.
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. Advances Neural Information Processing Systems,
13276, 2022.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings the IEEE/CVF international conference on computer vision, pp. 1660-9660, 2021.
Satrajit Chatterjee. Learning and memorization. In International conference on machine learning, pp- 755-763. PMLR, 2018.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 2818 28188220 2829, June 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. leee, 2009.
Akshay Dhamija. Manuel Gunther, Jonathan Ventura, and Terrance Boult. The overlooked elephant of object detection: Open set. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1021-1030, 2020.
Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yonglong Tian. Improving clip training with language rewrites. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances in Neural Information Processing Systems, volume 36. pp. 35544-35575 Curran Asso- ciates, Inc., 2023. URL ht http::/://pppccceeeeeeeeeceeee papp 2223://11//4444555c2202222222333 httpp//pppppppppppeeeeeeee////e files/paper pdf
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 954-959, 2020.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881-222 12891, 2020.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openelip, July 2021. URL ht https: / /oi. orq/10.
zerodo.51440 5143773
11

---

Published as a conference paper at ICLR 2025
Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Guha Thakurta, Nicolas Papernot, et al. Measuring forgetting of memorized training examples. In The Eleventh International Conference Learning Representations, 2022.
Bargav Jayaraman, Chuan Guo, and Kamalika Chaudhuri. Déjà vu memorization in vision-language models, 2024. URL https:// arxiv. org/ abs/ 02103
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Nishant Kumar, Siniša Segvić, Abouzar Eslami, and Stefan Gumhold. Normalizing flow based feature synthesis for outlier-aware object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5156-5165, 2023.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-ina -mmge pre- training for unified vision-language understanding and generation. In International conference on machine learning. pp.
PMLR, 2022.
Tsung- Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pictro Perona, Deva Ramanan, Piotr Doll' and C. Lawrence Zitnick. Microsoft COCO: common objects in context. CoRR, abs/1405.0312, 2014. URL http: arxiv. axxiv.org/abs/ 1405. 0312
Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong, Encodermi: Membership inference against pre-trained encoders in contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2081-2095, 2021.
Casey Meehan, Florian Bordes, Pascal Vincent, Kamalika Chaudhuri, and Chuan Guo. Do ssl models have déjà vu? a case of unintended memorization in self-supervised learning, arXiv e-prints, pp. 2023.
Junting Pan. Ziyi Lin, Yuying Ge, Xiatian Zhu, Renrui Zhang, Yi Wang, Yu Qiao, and Hongsheng Li. Retrieving-to-answw Zero-shot video question answering with frozen large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 272-283,, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.
PMLR, 2021.
Ildus Sadrtdinov, Nadezhda Chirkova, and Ekaterina Lobacheva. On the memorization properties of contrastive learning. arXiv preprint 1rXiv:2107.101433 2021.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of ACL, 2018.
Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. Machine learning models that remember too much. In Proceedings of the 2017 ACM SIGSAC Conference on computer and communications security, pp. 587-601, 2017.
Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, and Furu Wei. Clip models are few-shot learners: Empirical studies vqa and visual entailment. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 6088-6100, 2022.
Bart Thomee, David A. Shamma. Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfec100m: the new data in multimedia research. Commun. ACM, 52(5))994473.... January 2016a. ISSN 0001-0782. doi: 10.1 145/281 2802. URL https
//doi. 0. 1..1.../2122800
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016b.
12

---

Published as a conference paper at ICLR 2025
Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274-38228 382200, 2022.
Wenhao Wang, Adam Dziedzic, Michael Backes, and Franziska Boenisch. Localizing memorization in ssl vision encoders. arXiv preprint arXiv:2409. 19069, 2024a.
Wenhao Wang, Muhammad Ahmad Kaleem, Adam Dziedzic, Michael Backes, Nicolas Papernot, and Franziska Boenisch. Memorization self seff-sueee learning improves downstream generalization. In The Twelfth International Conference on Learning Representations (ICLR), 2024b.
Zhengbo Wang. Jian Liang, Ran He, Nan Xu, Zilei Wang, and Tieniu Tan. Improving zero-shot generalization for clip with synthesized prompts. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp- 1132-3042, 2023.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations, 2016.
Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training- free adaption of clip for few-shot classification. In European conference on computer vision, pp. 493-510. Springer, 2022.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 EXTENDED BACKGROUND
Déjà Vu Memorization in CLIP. The Déjà Vu memorization framework (Jayaraman et al. 2024) is the only existing other work that attempts to quantify memorization in vision- language models. uses the text embedding of a training image caption to retrieve relevant images from a public dataset of images. It then measures the fraction of ground-truth objects from the original image that are present in the retrieved images. If the training pair is memorized, retrieved images have higher overlap in ground truth objects, beyond the simple correlation. While valuable, several aspects warrant further consideration for broader applicability of the framework. First, its focus on object-level memorization ignores non-object information like spatial relationships or visual patterns that can also influence memorization (Feldman) 2020) Wang et al.) 2024b). To perform object retrieval, the framework also relies on object detection and annotation tools, which may introduce variability based on the accuracy and robustness of these tools. Additionally, the assumption that public datasets with similar distributions to the training data are readily available may not always hold, necessitating alternative approaches. Moreover, the framework does not analyze why certain images are memorized limiting detailed analysis. Finally, while Déjà Vu must address the challenge of distinguishing between memorization and spurious correlations, CLIPMMm avoids this by directly assessing memorization on the output representations of the model. One notable difference between the results of our approach and Déjà Vu's is that their findings show that their mitigation strategies can reduce memorization, but at the cost of decreased model utility. CLIPMem, in contrast, does not observe trade-offss between memorization and performance.
A.2 EXTENDED EXPERIMENTAL SETUP
General Setup. All the experiments in the paper are done on server with 4 A100 (80 GB) GPUs and work station with one RTX 4090 GPU(24 GB). We detail the setup for our model training, both CLIP and SSL (relying DINO) in Table 2
Table 2: Experimental Setup. We provide details our setup for encoder training and evaluation.
Model Training
Linear Probing
CLIP
DINO
Supervised VIT
CLIP
DINO
Supervised VIT
Training Epoch
100
300
100
45
45
45
Warm-up lipach
30
5
Batch Size
128
1024
128
4096
4096
4096
Optimizer Learning rale
Adam
Adamm
Adam
LARS
LARS
LARS
1.2e-3
2e-3
le.3
1.6
Learning rate Schedule
Cos. Docay
Cos. Decay
Cos. Decay
Cos. Decay
Cos. Decay
Cos. Decay
To experimentally evaluate memorization using the SSLMem framework (Wang et al.) 2024b), the training dataset is split into four sets: shared set (S.) used for training both encoders and gi candidate set (Sc) used only for training encoder f; independent set (Sr) data used only for training encoder g: and an additional extra set (S,) from the test set not used for training either or g. For training encoders, encoder f is trained on Sş U Se,, while encoder g is trained on Sg Sy. The alignment losses Lalign ff x)) and Latign x) are computed for both encoders, and the memorization score m((r) for each data point is derived as the difference between these alignment losses, normalized to range between - 1 and 1. A score of 0 indicates no memorization, +1 indicates the strongest memorization by f. and 1 indicates the strongest memorization by g.
Experimental Setup for SSLMem.
Normalization on CLIPMem. For improved interpretability, we normalize our CLIPMem scores to a range of [-1, 1]. A memorization score of 0 indicates no memorization, +1 indicates the strongest memorization on CLIP model f, and -1 indicates the strongest memorization on CLIP model g. We find the normalized CLIPMem score for a dataset using the following process: For each image-text pair T), we first calculate the CLIPMem score as the difference in alignment scores between two CLIP models f and g. Once CLIPMem scores are computed for all data points, we normalize them by dividing each score by the range, which is the difference between the maximum and minimum scores in the dataset. Finally, we report the normalized CLIPMem score for a dataset as the average of these normalized values.
14

---

Published as a conference paper at ICLR 2025
Linear probing Acc.
0.70
ClipMem Based
0.69
0.68
0.67
0.66
0.65
0.64
0.63
50
800 1600 3200 # of Sample removed
100 200 400
Figure 7: Removing memorized samples. We show the effect on downstream performance in terms of ImageNet linear probing accuracy and CLIPMem for CLIP model trained COCO using 5 text captions instead of 1, like done in Figure We observe the same trend, with the difference that the peak is roughly 500 removed samples rather than 100. This is likely due the increase in captions (by factor 5) that causes increase in mis-captioned samples.
0.62
Linear probing Acc.
Random
Memorization Based
0.61
0.60
0.59
0.58
0
50
100 200 400 800 1600 3200 of Sample removed
Figure 8: Removing memorized samples in supervised learning. We train a ViT-tiny on CI- FARIO (Krizhevsky al. (009) using supervised learning. We use our evaluation setup with Sc, Ss. Sr. and SE to approximate the memorization metric from [Feldman (2020). We use 5000 samples in Sc, but before training, we flip the labels of 200 samples. We calculate memorization over all samples in Sc and test the linear probing accuracy with ImageNet resized to 32*32 on the representations output before the original classification layer.
A.3 ADDITIONAL EXPERIMENTS
A.3.1 MEMORIZATION VS. GENERALIZATION IN CLIP
Extending evaluation. In Figure 7 perform the same experiment as in Figure 6 but CLIP model trained with 5 captions instead of 1. We observe the same trend, with the difference that the peak is roughly 500 removed samples rather than 100. This likely due the increase in captions (by factor 5) that causes increase in mis-captioned samples.
Verifying the pothesss on memorizing mis-captioned samples through supervised learning. We repeat the same experiment in the supervised learning setup to understand where the increase and then decrease linear probing accuracy stems from. To our hypothesis that it stems from "mis-captioned" samples, we "poison" our supervised model by flipping the labels of 200 data points before training. Then, we approximate the memorization metric from Feldman (020)) in our setup and remove highly memorized vs. random data points. In the same vein in Appendix A.3. first observe an increase in linear probing accuracy when removing memorized samples (instead of random samples). The peak is at roughly 200 data points, i.e., the number of deliberately mislabeled samples. Until the cutoff point at roughly 3200 examples, linear probing accuracy still higher when removing most memorized rather than random samples, which might suggest that there other
15

---

Published as a conference paper at ICLR 2025
Table 3:
Using diffferent/multiple captions during training. We evaluate CLIPMem how mem- orization on different data subsets and linear probing accuracy on ImageNet differ when using caption (baseline). 5 COCO captions, one chosen at random at every round (random), and 5 COCO captions, but all chosen equally often, i.e., 20 out of 100 training epochs (balanced), We observe that increasing the number of captions reduces highest memorization. Yet, only when we balance the usage of caption, also model performance increases.
baseline
random
balanced
Avg. CLIPMem (Top 10 samples) Avg. CLIPMem (Top 20%) Linear Probing Acc.
0.792
0.788
0,790
0.552
63.11% 0.91 % 62.44% + 1.18%
64. 84.88% = 0.83%
Table 4: The CLIPMem and linear probing accuracy of model trained with original coco captions and captions generated by GPT3.5. For 'Single Caption', only one caption is used during training. For "Five Caption', all five caption are used equally during training (every caption trained for 20 epoch out of 100). The linear probing accuracy tested ImageNet
COCO
GPT3.5
Single Caption
Five Caption
Single Caption
Five Caption
CLIPMem I.P. Ace.
0.423 64.88% + 0.83%
0,430
63.1 + 0.91%
63.09% 1.12%
64.47 + 0.72%
outliers or inherently mislabeled samples whose removal improves model performance. After the cutoff, we observe the behavior as observed in prior work (Wang et al. 22224b Feldman 2020) that reducing memorization harms generalization more than reducing random data points from training.
A.3.2 THE EFFECT OF CAPTIONS
In Table we show that using more captions during training reduces memorization and that by using each caption at the same frequency over the training epochs, additionally improve model performance, Additionally, we show that captions generated by GPT3.5 have the same effect as the original COCO captions on memorization and linear probing accuracy in Table
A.4 THE EFFECT OF MODEL SIZE
In Table 5 we present how the model size affects the memorization level of CLIP models. Both models trained using the same dataset and settings. We observe that with more parameters (larger model size), encoders have higher memorization capacity. This aligns with findings from previous
research (Wang al. 2024b Feldman 2020 Meehan et al. 2023).
A.5 VERIFICATION INFINITE DATA REGIMES
To evaluate CLIPMem over infinite data regimes (i.e., using a single training run where no data point is repeated), we subset D (containing 7050000 samples) of YFCC100M dataset (Thomee) et al. 2016b) to train another pair of ViT-Base models for only 1 epoch. Following our definition of CLiPMem, we further divide D into Sy with 6950000 samples. Sc with 50000 samples, and Sy with 50000 samples. The reason we use 7M (6950000+50000) samples to train either model f or model g is make sure the newly trained model has the same number of training samples as the model trained with K-epoch runs (70000 samples/epoch 100 epoch). The results in Table show that the model trained with infinite data regimes has higher linear probing accuracy on ImageNet as a downstream task and lower memorization scores, as measured by CLIPMem. This aligns with the fact that duplicated data points increase the memorization level and make the model over-fit, hence reducing the generalization (Wang et al. 2024b; Feldman 2020). The results in Figure 9 show that the most memorized samples according to CLIPMem in the model trained with infinite data regimes also samples with imprecise or incorrect captions. This aligns with statements in Section 4.5
16

---

Published as a conference paper at ICLR 2025
Table CLIPMem and linear probing accuracy of models with difffernt sizes. The models are trained using identical settings and the same subset of the COCO dataset. Linear probing accuracy is tested on the ImageNet dataset as the downstream task.
Model
CLIPMemm
Lin. Proh. Acc. (ImageNet)
hTse (Baseline main paper) ViT-large
0.438
63.11% d. 0.91% 67.04% + 1.05%
0.457
Series taken during heritage surveys in 2007.
Photo By Terri Hodges
have not had to develop individual paptions for these, however happy provide some comment if required.
www.cratingthegalax..oom Useage is Creative Commons but credit is required
Cumbres and Toltec Scenic Railroad, historical narrow gouge railroad between Chama, New Mexico, and Antonito, Colorado. These photos are from the station and railyard in Chama.
Mellwood Art Center Louisville, KY (1-27-06)
Day Four here, Entire Japan collection here.
Lisa Brewster Sent from my Palm Pre
Bridge Rackshelter Wisconsin State Natural Area m105 Natural Bridge State Park Sauk County
At Hellfest with After Forever
Mountain Empire Trip- Bristol, TN, and environs Photo Amy C. Evans, SFA oral hiscorian June 2009
On the way to visit Western Kentucky University (9/20/08)
www.sputhornfooddeays.org
Figure 9: Top 10 memorized samples according to CLIPMem in the model trained under infinite data regimes on YFCC100M. The model trained for one epoch, i.e., seeing each training data point exactly once. Even in this setup, the most memorized samples are still the ones with imprecise or incorrect captions.
A.6 EVALUATION ON BLIP
To verify the effectiveness CLIPMem over other similar multi-modal models, we train a BLIP model on COCO dataset following the same settings the baseline model in the main paper. We present the results for CLIPMem on BLIP over all four data subsets in Figure [0] which is in agreement with the results of the BLIP model Figure 2al We also present the results for UnitMem in Figure which is also very similar to the results of CLIP models
A.7 MEMORIZATION DISTRIBUTION DURING TRAINING
We present the distributions of neurons with highest UnitMem during training in Figure II These results highly consistently indicate that in the early stages of training, neuronal memory occurs mainly in the lower layer the clip model, while in the middle and later stages of training, neuronal memory more concentrated in the later layer of the model.
A.8 HUMAN MACHINE GENERATED CAPTIONS
For each image in the COCO dataset, we use GPT 3.5 (specifically, gpt-3.5-turbo) to generate captions (from scratch). We use the following instruction in the OpenAl API:
def satrraat__eerripttiin
for_iiagreeeeieeeeeeeeeeeerreeee (image_caption, clip_ : prompt f"Here is an image with the caption: prompt += f"Based on this caption and the visual features represented by this embedding please generate new detailed description."
17

---

Published as a conference paper at ICLR 2025
Table Evaluation of CLIPMem under infinite data regimes, i.e,, seeing every data point only once during training vs training with 100 epochs. We observe that both setups reach comparable downstream accuracy and memorization.
Model
CLIPMem
Lin. Prob, Acc. (ImageNet)
VIT Base (YFCC 7M, 1 epoch)
0.425
64.83% 1: 1.04% 63. 11% + 0.91%
VTT Rose (CO00 70K, 100 epochs)
0.438
600
Frequency
500
S
400
300
5s
200
100
0
-1.00
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
CLiPMem for BLIP
Figure 10: Memorization scores across data subsets on BLIP models We train BLIP model COCO standard image cropping and no text augmentation. We present the results for CLIPMem over all 4 data subsets, which in agreement with the results of the CLIP model in Figure 2a
response openai. Cha.Coomleettoonccreeetttt model 1eapp-3.5-ture. messages=
{ ("role": "system", "content": "You are a helpful assistant that generates captions for images. "), ("role": "user", "content"; prompt.)
return response [' choices' [[]] message") content')
We present the obtained captions in Figure 18 In Figure 12b analyze the pairwise cosine similarity in the original COCO and the GPT3.5 generated captions. We find that the GPT3.5 generated captions are slightly more uniform than the original COCO captions, reflecting in higher pairwise cosine similarity.
A.9 EXAMPLES FOR MEMORIZED SAMPLES
18

---

Published as a conference paper at ICLR 2025
12
11
0.25
10
Percentage of total
9
0.20
Layer
0.15
0.10
0.05
0.00
40
100
Epoch
(a) Top 1% neurons
12
10
Percentage of total
0.20
Layer
0.15
0.10
0.05
0.00
20
40
100
Epoch
(b) Top 3% neurons
12
10:
0.20
Percentage of total
0.15
Layer
0.10
0.05
0.00
40
100
Epoch
(c) Top 5% neurons
Figure Distribution of top % 3, and 5% neurons with highest UnitMem during training. We train a CLIP model on COCO standard image cropping and no text augmentation following the settings of baseline model main paper. We record the neurons with top 1%, 3%, and 5% of highest UnitMem during training (every epoch).
19

---

Published as a conference paper at ICLR 2025
Table 7: The machine generated captions provide similar performance to the original human- generated captions. We report the CLIPMem and linear probing accuracy of model trained with original COCO captions and captions generated by GPT 3.5. For the 'Single Caption', only single caption is used during training. For 'Five Captions', all five captions are used equally during training (every caption trained for 20 epochs out of 100). The linear probing accuracy is tested on the ImageNet dataset as the downstream task.
COCO
GPT 3,5
Single Caption
Five Captions
Single Caption
Five Captions
CLIPMem
0.438
0.423
0.430
0.411
Linear Probing Accuracy (ImageNet)
+
64.88% 0.83%
63.09% 1.12%
64.47 0.72%
10000
10000
8000
8000
Frequency
Frequency
4000
4000
2000
2000
8,65
0.70
0.75
0.60
0.65
0.90
0.95
1.00
8,55
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Cosine Similarity
Cosine Similarity
(a) COCO (Average: 0.798)
(b) GP73.5 (Average: 0.851)
Figure 12: Pairwise cosine similarity of captions from COCO and generated by GPT3.5.
Noise
CLIPMein
Lin. Proh Avc. (ImageNet)
Nome
0.438
(0.00
0.435
63.36%
0.88%
(0.005)
0.428
64.02%
1.12%
(00.10)
0.421
64.95%
0.96%
(0.1.5)
0.417
65.34% + 0.84%
(0
0.422
64.83% 0.92%
N00.25)
0.436
63.28% = 0.79%
(C(0300)
(00500)
0.491
57.04% 1.11%
(U.775)
0.501
52.28% 0.98%
(11000)
0.504
51.92% 1.03%
Table 8: Noising text embedding during training. We present the impact of adding noise to the text embedding during training for the ViT-base trained on COCO.
20

---

Published as a conference paper at ICLR 2025
Captin biis fhing
Caosin 2 X prpe tock of biic tyrng high in the
Capuion A lapicpaating latle
Cassicn 3 Many bind by over houses white the sun setting
Cuepioo Aaplopaating cria lable with micss booidee of Cappiard lightp camp saling med to compasee mosse upon the
Captinn beis ounade cuneet
ainds
Caption spoop
Copsion Young on mhowe area sipeno
Castion 2 grapp paople with
lbaech
C à A group of people by large sher a beach,
Copsion Two people in goor woorwihh ih mountain in badig cound.
Castic betch full poople with lots parachuee overhoad
Coprion & Two peaple who weoring snowirss on snow
C people the walling flynng
Caption boys sking
mounte
Caotion:: 1- cog siepping in dog bed on florr
Capoion DDG DRBSS TO SLEPP UN
BED
Caption Di swithhogg in fritey 1.tth
Coption 31 edice of dinkk and cate Captioon oteac in pitztin searn inriisiipp
Caption Armmll dog
that has trkkett matchee the
doge dictiing in dhrosed in. Capeion S dop getting ready to down
bad
Careion:: vames thon mands color adtingg de vade
Captin pair of veyy umousttnng on floor
Cuption A street seare with loous
Caption on A:A sion swa Hows you paid and hes on
4 it Sepaioto vosts, ond to emoty with the other has precn Towey Capticn Blue Block seod jars with one ding plant.
Copttoo imver skgn that asloing you hooe cead.
sips
Capsion sbject floodin
cotggding something top of lolle
2 2 An eepitar changs our a body of water
Caption
Casticn 3 Aud opnant wolling doep water the other side thruugh ceep ca waners objootis micdle
Coption on BIA bear th loo ling a1 some water.
Caption dlachone
Coption 4:A targe boow beor seering into the woter sogo of cream. Coption oceris looking into
Figure 13: The 10 samples with lowest CLIPMem in the CLIP model trained with all 5 captions. We can see that these samples contain clear concepts and precise captions.
The beis
burshhd stulfidd wrimal
Castion 2 The image ahors the dlseo of biee zemzing
Caprion d: Teskly beas and for sale in lou market loodss
Caskion lime sppe photo moloroydd sunt.
Caprion 3 thsre and merp stufled animada that are henn ng of the top
Cassion ding dictoreyd owel tamp ander sig:
Captiond: Eovaral lodily sears of difirunt colersand baing laydd.d dipplry
mosoroydees
Caption
Caption wall of bathroom
Coption slaneboorder looking at his feet while staring
Castion A a house with the walls bovnn cerolished
Caption sianetooodde mov switty down
Caption partullyy by demollahedion wall in run down house.
siombbard
Caption in room with imoccio out.
Daplion Thee
riding akutedacerc
Caption S: Fax, peisenn viee of panan vicknng sibea locand.
Daption osllctiona of secrned ferm amimal photos
Caption A driwing down
pouning at
down
Daption & comproine phitoo of meckeiing chidoees tomes, and
Caption & Nigh tine soons with now cornnng down at inafhicitteeeeee dion.
Caction Form orinals, horses sheep, and chidiens, pose for the mera.
Caption Nu merous iages of bimm on mote indluding sheap poots and person
Ceptera A An ineevation on unovy right with poserad adreat lampa
Captisn 1 A Axetup for obt untu the openatinn nonteer, with men caelc Ceptien22 besementt office heaues tach suppon Ulturna Caption 3 working on laptop wooden
Depiion it lecnooonng
care chivee down therdrres atreed
Dapiaon 3 Night lame city same with mome comnang dowen at buffis in andinasiios
Deciion inaassciion anooy rightt ouceral lemrze
buildrr
Caption city soene of strentights
Caption 5. a man ar doskintoo to some computters
Caction a man
vececen bench find
Caption scaiptuwe made up secraal trstiic lichts.
Caption man siting on lop of part banch.
Caction man laking on bench aunii day.
Capdon trtfic monserosity
ignts
Caction bench with
blttinn
Capson 4 nottic light tice sou poout in the middo of motcocton Capsio 5: pole with many different traffic ligts on ciyy stree
snovy
Figure 14: The 10 samples with highest CLIPMem in the CLIP model trained with all 5 captions. We can see that these samples contain atypical, difficult samples with imprecise or incorrect captions.
21

---

Published as a conference paper at ICLR 2025
girl in pink sweater putting blue umbrella over a yellow fire hydrant.
A yellow and blue fire hydrant surrounded by leaves
AIRPOR
Closeup of two street signs that read "irpprrt Pkwy" and "Karmill Ave."
A sign is displayed on pole that says bump.
ChILLLN ME
A sign with Oriental writing and the words saying Hyatt on the Bund.
A man maroon shirt standing next to stainless steel refrigerator.
pink Hello Kitty microwave on store shelf.
A blue street sign that reads "Thelonius Monk Circle."
A sign saying "Don't Hank, $350 Penalty" on pole.
A group artistic surfboards are displayed in a tent.
Figure 15: The 10 samples with lowest CLIPMem in the CLIP model trained with 1 caption. We can see that these samples contain clear concepts and precise captions.
An item of a vase with something inside of it.
A group of people standing on top of field together.
males and female red top holding some flowers
pole some yellow lights front of a narrow building
parking lot with a bunch of cars
A woman opening the trunk of her car.
A person is taken in this very picture.
a sleeping toddler laying on womans shoulder
An open point view room with various things all around
am unable to see an image above
Figure 16: The 10 samples with highest CLIPMem in the CLIP model trained with 1 caption. We that these samples contain atypical, difficult samples with imprecise or incorrect captions.
22

---

Published as a conference paper at ICLR 2025
COCO Image
COCO Caption
Stable Diffusion Image
group people that are souching enimal. group of people walking through and penting a shoop.
woman petting sheep that has been sheered.
woman pering a shoep insid pon a fair.
sheep stall being by
A man standing ned to paim trees rext to a bunch of surfboarcs,
SURFING BCARD STAND WITH A PERSON STANDING NEAR BY.
man standing in front of surlborrds the beach,
tree line beach contains
beard dard rack
people milling about.
group people standinn the sands the bozch
A tree filld with lots of lemons and green leaves.
large yellow fresh fruit wet from Vellow apples hanging ning
leafy
clese up several oranges Several round, yallow fruits growing on trae.
Figure 17: Samples of images generated by Stable Diffusion. We present the generated images based on the COCO captions.
COCO Captions
GPT3.5 Captions
sitting wooden brandh next to each tro.
three parrots sitting on a branch three blub and yellow parrots sitinng aranch theee blue and yellww parrots parrtt siting
bightly parroes interacting
Theee very colorfal parrots parch ed on different sticks.
branch
Three brightly solored biros on top a cluser of branches
branch
Thrce parres attingg on branches and groaming their festhers.
tho of parrote stinng
dim bethroom with light the
a bachroom with a sink and mimrrr small bathroom with sink and mirror bothroom
bathroom dimlly
single
dimiy light bothrocm with dark
bathrom and
bathroom sink, mirror and head light.
A bathroom sink with a light hanging above.
a and
th putting his bag
walks the colored holding luggage.
shou
man standing next to a car talliing on cell talking
standing
man coryying large through parking
standing
talliig
man with hend bag jalk on cell phone
a standing e car telking a man standing nest oer talking on the phooe
A talking on cell phone, with bag on his shoulder.
The two gl offees stand facing away from each other nost to the brown wall.
two gitalles star
other waoden fence
Twre gitafles walking away from each ocher against fencee
stand
two grarles wa king opposite of each other
fance bahind giraffies
girafles stand berind inc. Two giraties wallingg apposite directiors
two girafies stand ng a wanden fonoo two gitalles star ni front of fence
wall.
this is two rops riding on back of harses
solice officers horees
Two uniformed persorne are riding horses e trowd
police officers riding horsos
of people
on the back of hrrses horses.
sitting online officees horsahaces couple of police officers on horees
and offinem ricing through crowd
Figure 18: Sample captions generated by GPT3.5. We present the generated captions and the
original image and captions from COCO.
23

---

Published as a conference paper at ICLR 2025
600
600
Sc Avg. 0.168
Sc Avg. 0.213
500
Sş Avg. -0.001
500
S; Avg. -0.008
Frequency
Frequency
200
200
100
100
-1.00 -0.75
0.75
1.00
-1.00 -0.50 -0.25 0.25
SSLMem
SSLMem
(a) SSLMem Image Encoder (1 caption)
(b) SSLMem Image Encoder (5 captions)
700
700
Avg.
Sc Avg.
600
600
Ss Avg. 0.013
Sş Avg. 0.002
Prequency
500
Frequency
500
400
400
300
300
200
200
100
-1.00
0.75
0.50
0.25 0.00
0.25
0.50
0.75
1.00
-1.00
-.75
-0.00
D.25 0.00
0.25
0.50
0.75 1.00
SSLMem
SSLMem
(c) SSLMem Text Encoder caption)
(d) SSLMem Text Encoder captions)
600
Frequency
500
Sc Avg. - 0.438
5y Avg. = 0.001
Frequency
500
400
CApitte Aug. 0.488
55LMemToxt Avg. - 0.209
Avg. 0.298
300
300
200
200
100
0
-1.00-0.75
-0.75
-0.50
-025
0.00
0.25
0.50
0.75
1.00
-1.00
0.75 0.50
-.25 0.00 0.25
0.50 0.75 1.00
CLiPMem
SSLMem CLIPMem
(c) CLIPMem
(f) SSLMem, Naive Sum of SSLMem and CLIPMem
Figure 19: Evaluation of SSLMem and CLIPMem on a CLIP model trained on COCO. Extended version of Figure 3 where also include SSLMem calculated encoders trained with 5 captions instead 1. The trends both cases are the same. SSLMen for the CLIP Models trained with the captions is slightly higher since SSLMem uses the captions augmentations for the calculation of the memorization. Overall, our CLIPMem reports the strongest memorization signal for CLIP.
24


Reviews:
[{"review_id": "aHxl4i07sM", "reviewer": "Reviewer_khe4", "summary": "**Summary:** Understanding the memorization / generalization tradeoff is important to properly quantify modern ML models. This work focuses on CLIP which applies InfoNCE between image and text pairs and attempts to quantify the extent to which CLIP models memorize. The authors introduce CLIPMem to quantify memorization and find that the text encoder contributes more to memorization than the vision encoder. They also propose strategies to mitigate / remove memorized samples to improve performance.", "questions": "**Questions:**\n\n- Is CLIPMem bounded? \n- What dimension is kept constant for Table 1? Are the total training samples [count] seen the same? \n- It would be interesting to evaluate infinite data regimes (i.e. no repeated data) rather than classical K-epoch  runs. Would the results for - memorization hold here? Just like in language this setting is becoming more and more common.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**Stong points:**\n- Well highlighted literature on memorization.\n- Defines CLIPMem based on hold one out strategy (similar to Feldman et. al in supervised learning).\n- Interesting results on mis-captioned text labels, multi-caption and removal of memorized examples.\n- Reasonable pretraining datasets like CC3M\n- Clean separation of training and test splits for measuring memorization.", "weaknesses": "**Weak points:**\n- Missing ability for CLIPMem to be applicable to general off-the-shelf CLIP models. Currently if I understand correctly it requires retraining on specific splits. \n- Clarity of specifics of CLIPMem is used for vision only and joint vision + text can be improved.\n- The noising results (Table 5-b) are not very convincing. Almost all the results are within the same +/- std range.\n- The linear probe accuracy seems quite low (Table 1, 5-a, 5-b,  6-a/b).  \n\n**Nit:**\n- Text and images in Figure one are very hard to read. Suggest larger and fewer images and move rest to appendix. \n- Figure 3 can be make larger / more readable by doing share-y and increasing font sizes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QjuG7ivHVt", "reviewer": "Reviewer_CAJS", "summary": "The paper introduces CLIPMem, a novel metric to quantify memorization in CLIP, which combines elements of supervised and self-supervised learning. The paper shows that memorization within CLIP often arises from mis-captioned or atypical samples, particularly within the text modality rather than the image modality. They propose mitigation strategies that reduce memorization without sacrificing model utility, unlike traditional methods that often degrade performance when reducing memorization. It reports several interesting findings, including 1) CLIP's memorization lies between supervised and self-supervised paradigms, with high memorization for data with inaccurate or misaligned captions; 2) Text domain adjustments, such as using varied or augmented captions, reduce memorization and improve generalization, defying the usual trade-offs seen in other paradigms.", "questions": "Would you please comment on how the metric adapt to other multi-modal models besides CLIP?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ It introduces a new metric -- CLIPMem to provide a new way for measuring memorization in multi-modal settings, a gap in previous research.\n+ It performs empirical analysis to show differences in memorization between the text and image modalities, providing actionable insights.\n+ It proposes techniques to successfully reduce memorization while preserving or even enhancing model utility, challenging established norms.\n+ By highlighting the risks of training with uncurated, potentially mis-captioned data, the paper suggests guidelines that can benefit real-world multi-modal model training practices.", "weaknesses": "- While tailored to CLIP, the metric and findings may need adaptation to apply effectively to other multi-modal models with different architectures.\n- The experiments focus on datasets like COCO and CC3M, so it’s unclear how well these findings generalize to other large-scale or domain-specific datasets.\n- The mitigation strategies, such as augmenting captions or generating variations, may incur additional computational costs in training, which could limit practicality for some users.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w12lKu0Yqv", "reviewer": "Reviewer_F6JJ", "summary": "They study the memorization problem in the CLIP model unlike existing studies focusing on the unimodal memorization problem. \nThey propose a new metric to analyze it and find that memorization seems to be more significant in text encoder than in image encoder. Their analysis indicates that augmenting captions can be a key to mitigating memorization in the CLIP model. Their experiments confirm that augmenting captions can improve the quality of the image encoder's representations while reducing memorization.", "questions": "My rating is mainly based on 1 and 2. Please respond to those points.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. They propose a new metric to measure memorization in the CLIP model. The design of the metric is reasonable. \n2. Their insight that memorization is more significant in the text encoder is new and might interest readers. \n3. They conduct analysis on the augmentation of the text and images, which might be also interesting to some readers. \n4. This paper is well-organized and easy to follow.", "weaknesses": "1. They do not discuss how model size can affect memorization. Although I am not very familiar with this topic, I guess the model size can affect their arguments. For example, if they utilize a larger image encoder, the memorization might be more significant on the image side. Therefore, I think their conclusion about which encoders suffer more from memorization can change by the size of the encoders, but they do not discuss much. \n\n2. Most of their findings sound a bit too reasonable and are not surprising. Their finding that augmenting text improves the CLIP model has already been observed in many previous papers though they probably did not discuss memorization. Also, it is not hard to imagine that augmenting datasets mitigates memorization. In these points, I think their findings are not impressive. \n\n3. They mention that their metric is effective in removing noisy samples. But, they compare their approach only with random replacement. I think they need to add more baseline, such as naive CLIP's similarity as done in many works. \n\n4. In Table 1, the authors augment images by using a diffusion model. But, as they imply, such augmentation can cause a distribution shift in the image side and does not give much intuition about image augmentation. \n\nOverall, I think this paper is well-organized and delivers a clear statement to readers, which I like. However, I think their findings are not very surprising and lack impact due to the reasons described in 1, 2. My rating is based on it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)