Paper: Memorization in Self-Supervised Learning Improves Downstream Generalization (ICLR-2024)

Reviewer comment (Evaluation):
The proposed framework lacks enough practical implications. It is not so clear how we should treat samples with higher/lower memorization scores differently.

Author response:
As pointed out by the reviewer, our work is “useful for understanding model generalization”. We study the memorization and generalization properties of self-supervised learning (SSL) to deepen our understanding of this new learning paradigm and to reveal similarities and differences within different SSL frameworks and with respect to the well-studied supervised learning.

**Coreset Selection:** Following the reviewer’s suggestion, we ran additional experiments to investigate the suitability of our method for coreset selection (see our last answer). Our results highlight that by training only on the subset of most memorized data points, we can even outperform the encoder trained on the full dataset, or match its performance with a significantly smaller training dataset (up to 25% smaller). Thereby, our method can lead to new learning strategies that could dramatically reduce (1) training and computation time and (2) data and memory requirements for the SSL encoders (which are both extremely high under current methods).

**Data Privacy:** Our method supports studying which data points experience highest memorization by the encoder. These data points are particularly prone to privacy leakage. Based on the insights from our memorization score, depending on the type of use case of such encoders, appropriate actions (such as differential privacy, potentially with stronger guarantees for the memorized data points [A, B]) can be taken during or after the training to limit the leakage.

We added a new Appendix Section D.5 to the updated version of our paper to highlight these practical implications of our framework. 

[A] Alaggan, Mohammad, Sébastien Gambs, and Anne-Marie Kermarrec. "Heterogeneous Differential Privacy." Journal of Privacy and Confidentiality 7, no. 2 (2016).
[B] Yaodong Yu, Maziar Sanjabi, Yi Ma, Kamalika Chaudhuri, Chuan Guo. “ViP: A Differentially Private Foundation Model for Computer Vision.” https://arxiv.org/abs/2306.08842 (June 2023).

Reviewer comment (Evaluation):
The definition of (per-example) memorization score is a bit strange. From my perspective, a higher memorization score indicates that sample $x$ has larger impact to model performance. As such, it seems natural that pre-trained models with higher memorization perform well on downstream tasks: if not, it means all training samples do not contribute much to pre-training, and it will be strange that pre-training on such data set produces good models. Some more explanations may be needed on why we need to introduce memorization score here, instead of some other performance metrics.

Author response:
Our work’s main focus is on measuring the impact of individual data points on the performance of an encoder. Therefore, it is necessary to compute a per-example memorization score. In particular, we are interested in understanding which data points that contribute most to the training algorithm (and experience the highest memorization). The relevance of this research area, and the interest to study a per-example impact on models and training is based on a long line of work in that area. For example, Arpit et al., [2017] study memorization in supervised learning through the lens of hardness to learn individual examples throughout training. Our definition is inspired by Feldman [2020] and Feldman&Zhang [2020] who study the per-example memorization in supervised learning with a leave-one-out definition and use it to identify the data points that experience highest memorization.

Regarding the use of other performance metrics, we would like to point out that our work does not consider the limited setup studied in prior work where the training data is the same as the data used for evaluating performance. Instead, we pretrain an encoder on a dataset (e.g., CIFAR10 in Figure 3), and evaluate the performance of the resulting encoder on downstream tasks from entirely different data distributions (CIFAR100, STL10). In this setup, we cannot tie back easily the performance on the test data to the training data because performance of the test data only results from representations that are based on the training data (but cannot be traced back to individual training data points).

Reviewer comment (Experiments):
While the authors claimed that their experiments cover different architectures and downstream data sets, the experiments seem a bit restricted from my perspective. For architectures, it would be better if the authors can report i) same architecture (e.g., ResNet) with different depths, widths, etc. and ii) similar number of parameters with different architectures (ViT and ResNet here). That will make the experimental results more comprehensive and we may also gain some insights on how memorization differs across different architectural settings.

Author response:
We also report results for the same architecture (e.g., ResNet) with different depths, and widths (Wide ResNet). It shows how the memorization score differs for various number of parameters and their arrangement and how it can influence the memorization score. We observe that with more parameters, models have higher memorization capacity. We added these new results to Appendix B.5 of the updated paper.

|  | **Avg. Mem.** | **Acc.** | **# of Parameters** |
|---|-----|-----|------|
| SimCLR with Wide-ResNet50-2 | 0.350 ± 0.008 | 81.23 ± 1.01%  | 69M  |
| SimCLR with ResNet50  | 0.339 ± 0.011 | 77.12% ± 1.42% | 25M   |
| SimCLR with ResNet18  | 0.315± 0.013  | 71.07 ± 1.08%  | 11M   |

Reviewer comment (Experiments):
Downstream data sets used in the current paper (CIFAR-10/100, SVHN, STL-10, ImageNet) are all coarse-grained general classification data sets. The authors may consider also adding some experiments on fine-grained data sets (e.g., Food-101 or Flower102, which are popularly used in literature) and see if their conclusions are changed.

Author response:
We also added more experiments on fine-grained downstream data sets, namely Food-101 and Flower102 to obtain results similar to Figure 3 and Table 3.
We pre-trained a ViT-base encoder with MAE on ImageNet and then removed the 10k and 20k most memorized and random data points before training again. Then, we trained downstream classifiers on ImageNet, Food-101, and Flower102 on top of the resulting 5 encoders, and report below the downstream accuracies.

| **Downstream Accuracy** | **Without Removing** | **Remove 10k** | **Remove 10k** | **Remove 20k** | **Remove 20k** |
|-------|-----|----|----|---|-----|
|   |   | **Memorized**  | **Random**     | **Memorized**  | **Random**     |
| ImageNet  | 68.21% ± 0.98%       | 64.33%± 0.84%  | 66.82%± 0.77%  | 58.90± 1.05%   | 62.88%± 0.77%  |
| Food-101 | 58.96% ± 1.33%       | 55.15± 0.96%   | 56.83± 1.13%   | 50.07± 0.76%   | 53.14± 1.21%   |
| Flower102  | 60.11% ± 1.12%       | 56.27%± 1.14%  | 58.08%± 0.89%  | 51.48± 1.01%   | 55.29± 0.93%   |

Our results show the same effect as the experiments from our original submission: also on these more fine-grained datasets, removing memorized data points harms downstream performance (on the same and different distributions) significantly more than removing random training points. We also added these new results to Appendix B.5 of the updated paper.

Reviewer comment (Experiments):
Regarding experiments on removing samples for downstream tasks, the performance gap seems not so large. What will happen if we directly try to remove some samples to obtain largest performance drop? Will these samples found have larger memorization scores? Some additional experiments are welcome.

Author response:
Our metric is able to identify exactly the data points whose removal causes the performance drop. Could the reviewer suggest a method to identify the samples that are not marked by our metric as highly memorized but can cause a higher performance drop than the highly memorized training data points? A naive approach would be to check each point individually by training with or without this data point, however, it is computationally too expensive to run. Our method provides a good proxy to find data points that have a high impact on the performance. 
Regarding the identification of samples that cause the largest performance drop, we would like to point out again that we consider pretraining with one dataset (e.g., CIFAR10 in Figure 3), and evaluating performance of the resulting encoder on entirely different data. The pretraining data hence only influences representations which makes it hard to quantify the impact that a pretraining data point has on the downstream performance. 

The new experiments on the more fine-grained datasets (see previous answer) highlight a much larger performance gap (7-8%), especially for the removal of 20k samples. This is a significant difference and enforces the findings from the original submission.

Reviewer comment (Experiments):
I also wonder how it is connected to research on coresets [2], which aims to find a (small) subset of training data that can help obtain models with performances close to full training data. The authors may report something opposite to Figure 3 and Table 3: solely train on samples with high memorization scores, and see how the model works.

Author response:
We thank the reviewer for this excellent question and performed the suggested experiments. In the same setup as Figure 3 in Section 4.4, we trained a ResNet50 on SimCLR with CIFAR10 (25k training data points) and calculated the memorization score over all data points. We then trained the model again from scratch with the [25k (all), 24k, 22k, 20k, 16k, 12k] most memorized data points. We report the downstream accuracy on CIFAR10, CIFAR100, and STL10  in the table below. 

We observe that indeed by training only on the most memorized data points, we can achieve higher downstream accuracy on the same and different downstream data distributions. We bolded the higher downstream accuracy obtained on the encoders trained with fewer training data points than their full training set. We further systematically decrease the number of samples for the encoder training and until the performance degrades below the one for the training on the full data. With <16k training samples vs. the full 25k, the performance is significantly worse. This result shows that our method can be used to obtain coresets, i.e., smaller subsets of data that yield performance that matches and even outperforms the performance of the encoder trained on the entire dataset.

| **Retained Points ** | **CIFAR10**       | **CIFAR100**   | **STL10**      |
|----------------------|-------------------|----------------|----------------|
| 25k (full encoder)   | 63.3% ± 0.92%     | 61.1%±1.14%    | 61.6%±0.83%    |
| 24k (most memorized) | **64.4% ± 1.03%** | **61.3±0.98%**     | **61.7±1.18%**     |
| 22k (most memorized) | **63.8% ± 0.76%**     | **61.8±1.24%** | **62.4±1.05%** |
| 20k (most memorized) | 63.2% ± 1.07%     | 60.8%±0.68%    | 61.1±1.05%     |
| 16k (most memorized) | 61.8 ± 1.11%      | 58.4%±0.91%    | 59.9±0.89%     |
| 12k (most memorized) | 59.7% ± 0.74 %    | 55.6%±1.32%    | 55.2±1.24%     |

We added the above results in the new Appendix Section D.5 titled “Practical Implications of Memorization” of the updated version of our paper.

Reviewer comment (Novelty):
There are several works discussing the issue of memorization in self-supervised learning, which haven't received thorough attention in the literature review. While I understand that not all of these works are directly related, a broader discussion of related literature would enhance the paper's readability (e.g., [1, 2]).

Author response:
We would like to thank the reviewer for pointing out these papers from the broader domain of the scope of our work. [1] follows Arpit et al., [2017] in terms of setup and does not formalize a notion of memorization. Instead, they rely on training dynamics to distinguish between examples that are *hard* and *easy to learn* and experimentally evaluate this approach with SimCLR (the SSL framework that offers a form of “classes” through the positive and negative pairs in the mini-batch which is required for the approach from [1]) and the ResNet18 architecture on CIFAR10. In contrast, our work proposes a definition of memorization and empirically evaluates the metric on a broad range of different SSL frameworks, architectures, and datasets. 
[2] studies the impact of pre-training with SSL on the downstream generalization in the form of generalization gaps between training and test data. The factor they identify as most relevant for the generalization gap is “memorization”. Yet, they consider memorization over the entire resulting model (encoder + added downstream classifier), and find that it is bound by the complexity of the downstream classifier. In contrast, we are concerned with memorization of the SSL encoder only and measuring it on the representations, independent of any downstream task. We added discussion on [1,2] as an extension to the related work section of our updated paper to follow the reviewer’s advice.

Reviewer comment (Theory):
In the machine learning research community, it's generally accepted that there exists a tension between memorization and generalization. However, I would appreciate more clarity and intuition from the authors regarding why memorization leads to improved generalization.

Author response:
We thank the reviewer for this comment and are happy to provide more intuition. As stated by Feldman [2020], most data distributions exhibit a “long tail”, i.e., contain different small sub-populations.
In Appendix D.4, we show that the memorized samples (from these small sub-populations) of the encoder’s training data lead to a lower alignment loss in larger regions of the representation space. Figure 11 shows that without the inclusion of memorized samples (Figure 11c), the alignment loss is only low in the regions where the main body of the data distribution clusters. However, with the memorized (long-tail, atypical) samples (Figure 11b), larger regions of the representation space exhibit a lower alignment loss. A low alignment loss means that similar inputs obtain similar representations, which helps a downstream classifier to learn better decision boundaries and classify samples in these regions better which boosts generalization.

Reviewer comment (Experiments):
Regarding the statement, "Our definition compares the difference in alignment of representations for data points and their augmented views returned by both encoders that were trained on these data points and encoders that were not," I'd like to seek clarification. It seems that the training sets of g and f overlap. Could you please provide a justification for this design choice? I had expected a distinct encoder with a different training set or random initialization.

Author response:
We thank the reviewer for this question and are happy to clarify this design choice: Please keep in mind that we are interested in memorization of individual data points, i.e., in the question, how a given individual data point “impacts” the training. Therefore, the cleanest form to experimentally assess our memorization from Equation 2 would be to have f and g identical in architecture and training dataset with the simple difference being the *one data point $x$* whose memorization we would like to measure. This data point would be included in the dataset of f but not of g.  By then comparing f and g, we would bet the most accurate estimate of the memorization on $x$.

Unfortunately, since SSL models are very large, training many f and many g that each differ in one $x$ only is computationally prohibitive. Therefore, we approximate the memorization by making f and g as close as possible (same architecture, same training), but differing in a few points at the same time. By then comparing f and g’s behavior on all those points, we can make approximate statements on the memorization of these points. 
We discussed all the details of this trade-off and the alternative design choices in Appendix A.1 of our original submission.

Reviewer comment (Theory):
“we consider a data point as having a high level of memorization by an encoder $f$ if its alignment is significantly higher on $f$ than on encoder g that was not trained with the considered data point” – Why is this the case? Are there any exceptions where this doesn't hold true?

Author response:
There are no exceptions to the above since the memorization score will only be positive if the alignment loss of model f is lower than the alignment loss on model g. Additionally, the memorization score will be higher the greater the difference of alignment losses between f and g is.
This can be explained as follows: If we have a data point $x$  which is very common (i.e., similar to other training data points), then the models f and g will both have a good alignment (low alignment loss) on the data point. The alignment loss of f will be low because it was trained on $x$, the alignment loss of g will be low  since g learned on the many similar data points to $x$. In that case, f and g both have a low alignment loss, and we measure a low memorization (difference in alignment losses). However, if the point $x$ is very different from all other points, only when the point is in the training set,  the model can achieve a low alignment loss. Hence, f will have low alignment loss, and g will not, which results in a high difference between the two, and hence a high memorization. Figure 2a of our original submission visualizes this effect.
We see that only data points that have significantly lower alignment loss in f than in g have a high memorization score.

Reviewer comment (Experiments):
I understand that the memorization score is relative. However, can one freely choose $g$ as our interest primarily lies in $f$? Have you conducted any ablation studies on the choice of model architectures for $g$ while keeping $f$ constant? Is it necessary for both $g$ and $f$ to share the same architecture?

Author response:
In a similar vein to our previous answer, we would like to note that one cannot freely choose g when interested in measuring memorization of f. As can be seen in our definition of the memorization score (Definition 2), both models f and g are trained with the same learning algorithm A (which encompasses the choice of model architecture). The only intended difference between models f and g is in their training dataset with model g not including datapoint x, which allows to quantify exactly the effect of $x$ on the training algorithm and, thereby, lets one assess the memorization.

Reviewer comment (Experiments):
What if $g$ is pre-trained on a large but distinct dataset instead of being randomly initialized? Would that result in a reduced memorization score?

Author response:
As mentioned in the previous response, everything for models f and g is kept the same except for the training dataset. Therefore, if model g is pre-trained, then model f must also be pre-trained. Over the dataset of interest for quantifying memorization, this is not likely to have an effect on the types of scores obtained.

Reviewer comment (Experiments):
The training data is divided into 80%, 10%, and 10%. The last two sets do not overlap between $g$ and $f$. How does memorization change when we vary the overlapping ratios from 80% to 70%, 50%, and 30%?

Author response:
We selected the 80% overlapping ratio to approximate well the precise measure. Ultimately, if we had unlimited compute resources (as we point out in the previous answers), we would train encoders f and g that only differ in a single data point, and repeat that experiment for a significant number of different data points. However, due to the high computational complexity of SSL training, we have to approximate our memorization score (see Appendix A.1). We carried out additional experiments to showcase that the memorization score does not change with higher overlapping ratios (85%) but decreases for smaller ratios (70%). Thus, the ratios below 80% do not provide us with a sufficiently precise measure of memorization. We repeated experiments from Table 1 with ResNet50 trained with SimCLR on CIFAR10 with different splits for the overlap (70% overlap, and 85% overlap). For the best comparability, we made sure to have the same number of training data points over all setups (45k). 

| Setup: Overlap (%) for S_S, counts for S_C, and C_I | Avg. Mem. score  | Acc (%)        |
|-----------------------------------------------------|------------------|----------------|
| S_S=35k, S_C=10k, S_I=10k                           | 0.325 ± 0.008    | 77.95% ± 1.23% |
| S_S=40k, S_C=5k, S_I=5k (See Table 1)               | 0.339 ± 0.011    | 77.12% ± 1.42% |
| S_S=42.5k, S_C=2.5k, S_I=2.5k                       | 0.337 ± 0.010    | 76.84% ± 0.85% |

Reviewer comment (Evaluation):
“We formally verify that data points from Sc (Si ) have statistically significantly higher (lower) memorization scores m than those from Ss and Se.” “They support the claim that Sc (Si ) is substantially more (less) memorized than Ss and Se”. Why is it the case that Sc is substantially more memorized than Ss? Isn't this because memorization is a relative score, and Ss was used to train both $g$ and $f$?

Author response:
Indeed, the reviewer is perfectly correct with this conclusion. $S_C$ is used only as training data for f, hence f will have a low alignment loss (while g will not necessarily) which results in higher memorization. $S_S$ is training data of both encoders, hence both will have a relatively low alignment loss for both models, and hence, the difference between the alignment losses, i.e., the memorization, will be low.

Reviewer comment (Evaluation):
The paper lacks sufficient intuition to answer 'why' questions such as: why does memorization lead to generalization, why is the proposed metric suitable for measuring memorization, and why were certain design choices made, like training percentages and architecture selections for $g$ and $f$. The authors need to provide more comprehensive reasoning for their results. Simply presenting empirical findings is insufficient and can lead to confusion for readers.

Author response:
We thank the reviewer for this comment and summarize our answers below. For more details, please see our answers to the reviewer’s comments and questions above:
- The intuition behind why memorization leads to better generalization can be found in Appendix D.3 and D.4. Memorized (atypical) samples lower the alignment loss over larger regions of the representation space. This leads to more consistent representations in these regions which facilitates learning of the downstream classifiers.
- Our metric is suitable to measure memorization because it captures the impact that a single data point makes on the trained encoder. Our work is based on the prior research on memorization in supervised learning by Feldman [2020] and Feldman and Zhang [2020]. While in supervised learning, the impact can be measured w.r.t. to the model’s prediction on the data point’s ground truth label, such labels do not exist in SSL. Instead of optimizing for labels, all SSL frameworks implicitly or explicitly optimize for alignment, which makes alignment difference the suitable metric to assess the impact on the trained encoder. 
- The design choices made in the paper are discussed in detail in Appendix A.1. Taking the trade-offs between an *accurate* and *computationally possible* approximation into account, we chose g as close as possible to f (same architecture) with as much data overlap as possible. Our additional experiments presented in this answer highlight that our choice of percentage yields very close results to larger overlap in percentage (with desired but computationally infeasible overlap of near 100%, difference in just one data point). Hence, our approximation is sufficiently accurate.

Reviewer comment (Evaluation):
The results seem to show that memorization benefits downstream performance. However, *why* this is happening is not quite clear to me. For instance, when removing points with high memorization scores from the training data, does the performance of the model primarily drop on points "similar" to the removed ones? If this was the case, memorization here might actually be more of a long-tail generalization phenomenon.

Author response:
We would like to highlight that, in contrast to most prior work, we do not focus our study on the limited setup where the training data of the SSL encoder is the same as the training data of the downstream classifier (see for example Figure 3, where we remove the most memorized data points of an encoder trained on CIFAR10, and observe a drop in downstream accuracy for CIFAR100 and STL10 classifiers). The expressiveness of “similarity” for assessing performance is not straightforward in this setup with data from different distributions and with different classes and their number. 

However, what we show in Appendix D.3 and formalize in Appendix D.4 is that the atypical memorized samples of the encoder’s training data lead to a lower alignment loss in larger regions of the representation space. Figure 10 shows that without the inclusion of memorized samples (Figure 10c), the alignment loss is only low in the regions where the main data distribution clusters. However, with the memorized samples (Figure 10b), larger regions of the representation space exhibit a lower alignment loss. A low alignment loss means that similar inputs obtain similar representations, which helps a downstream classifier to learn better decision boundaries and classify samples in these regions better which boosts generalization.

Reviewer comment (Reproducibility):
In Section 4, first paragraph, what is the normalization procedure applied to constrain memorization scores between -1 and 1?

Author response:
We added an explanation of the normalization procedure in Appendix A of the updated paper. We normalize (using the $\ell_2$ norm) the representations output by encoders $f$ and $g$. Then, we calculate the differences in alignment loss per data sample over both encoders. We then normalize these differences by dividing them by the range (largest minus smallest difference), and report the memorization score as the average of the resulting scores over all data points in $S_C$.

Reviewer comment (Presentation):
Table 1, what is "Frac. Mem."? Is it the same as "Avg. Mem." defined in the caption?

Author response:
We would like to thank the reviewer for pointing out this inconsistency. Indeed, this is a typo and should be Avg. Mem. Following the reviewer’s suggestion, we gave a name to our metric (SSLMem), and now refer to our results as that in the updated version of the paper.

Reviewer comment (Experiments):
In 4.4, do you remove the 500, 1K, etc. datapoints with highest memorization from the set of 25K points, or from the full CIFAR10 training set?

Author response:
We removed them from the 25k data points. This is because only these 25k were used to train the SSL encoder and we measured the memorization of the encoder on these training data points.

Reviewer comment (Evaluation):
In 4.4, why do you use cosine similarity here vs l2 distance earlier?

Author response:
We used these two metrics interchangeably since the representations are normalized and L2-Distance(u,v) = 2 * (1- Cosine-Similarity(u,v)). To make it consistent, we will change the Cosine-Similairty to L2-Distance in the experiment.

Reviewer comment (Evaluation):
Does the metric agree with previous metrics for quantifying memorization in supervised learning? I.e. given a supervised learning model (where you can apply prior supervised learning work), would the metric highlight the same points as memorized as previous metrics for supervised learning?

Author response:
To provide an answer to the reviewer’s insightful question, we implemented an approximation of the leave-one-out memorization score by Feldman [2020]. Therefore, we trained a model $f$ and a model $g$ (both ResNet50 on CIFAR10) in a supervised manner. For best comparability with our results, we chose 40k data points overlap between the two models and 5k difference. On the 5k data points used to train f but not g, we calculated the difference in softmax outputs between f and g. The data points with the highest difference are the ones with the highest memorization according to Feldman [2020]. To calculate our metric on the same models, we removed the classification layer and then calculated our metric on the output representations. In the updated paper, we show in Figure 8 of our updated paper, the first 10 most memorized images from the supervised and the self-supervised models. We observe a substantial overlap. 
These preliminary results indicate a similarity between the two metrics. We hypothesize, based on our experiments, that both metrics (from Feldman [2020] and our) report similar atypical samples as memorized.

Reviewer comment (Experiments):
What are atypical datapoints? Is the judgement of typicality just based on visual inspection or are there other indicators as well?

Author response:
We refer to atypical data points as data points that are uncommon in the data distribution and different in terms of their features. Visual inspection is indeed the most straightforward way to judge how atypical a data point is. We present Figure 1 to allow for such a visual inspection.  However, other factors, such as distance in the image space or the similarity between extracted features of the data point may also be used. Those could be assessed using, for example, clustering approaches. We highlighted in the updated version of the paper that we refer to our visual inspection in that context.

Reviewer comment (Theory):
The idea behind the metric is that quantifying memorization via alignment differences between different augmentations of the same input point is the common denominator between different SSL approaches (contrastive, non-contrastive, reconstruction-based). Would it be possible to define a "stronger" notion of memorization if one were to only consider one family of SSL approaches, e.g. contrastive ones?

Author response:
Indeed, it might be possible to define a stronger notion of memorization if the types of SSL approaches are restricted. For example, in the case of contrastive approaches, one may incorporate the uniformity metric as both alignment and uniformity are optimized in this SSL type. However, the main issue is that memorization should be defined per data point while uniformity depends on all the training data points at the same time.

Reviewer comment (Writing):
Giving the memorization metric a name would make it easier to refer to it.

Author response:
We thank the reviewer for this valuable suggestion. We modified the submission accordingly and referred to our metric as “SSLMem”.

Reviewer comment (Experiments):
Given that the proposed metric operates at the representation level, it might be interesting to quantify memorization at intermediate layers in the model, to potentially localize where it is happening.

Author response:
This is a very valuable observation and we will make sure to follow up on this work to pinpoint where the memorization happens in the SSL models.

Reviewer comment (Experiments):
Regarding the experiment considering differential privacy, only one algorithm was evaluated, I was not able to see other evidence (evaluation using different setups and algorithms) that supports the case about memorization in this context.

Author response:
We would like to point out that the differential private stochastic gradient descent (DP-SGD) is the de facto standard algorithm for performing machine learning models with privacy protection. Therefore, we evaluated with this algorithm. Differential privacy (implemented through DP-SGD), by definition, limits the degree of memorization since it limits the impact that any individual data point is allowed to have on the model updates during training. We evaluated different privacy setups ($\varepsilon=\infty$, i.e., no privacy protection, $\varepsilon=20$, i.e. weak privacy protection, and $\varepsilon=8$, i.e., stronger privacy protection).

Reviewer comment (Experiments):
Does a particular augmentation or a set of augmentations plays a role in this empirical evaluation?

Author response:
With respect to the role of augmentations, we performed additional experiments during the rebuttal period on assessing memorization with different sets of augmentations than the ones used during training. The results can be found in the updated Appendix B.1, especially Table 8. The results indicate that using the same original augmentations that were used during training also for measuring memorization yields the highest memorization score, i.e., gives the strongest signal to measure memorization. Yet, the other augmentations’ scores are not significantly different and hence can be used equally to approximate the degree of memorization.

Reviewer comment (Evaluation):
How does similar data samples play a role with respect to the memorization vs generalization claims?

Author response:
We would like to highlight that, in contrast to most prior work, we do not focus our study on the limited setup where the training data of the SSL encoder is the same as the training data of the downstream classifier (see, for example, Figure 3, where we remove the most memorized data points of an encoder trained on CIFAR10, and observe a drop in downstream accuracy for CIFAR100 and STL10 classifiers). The expressiveness of “similarity” for assessing performance is not straightforward in this setup with data from different distributions, with different classes and their number. 

However, what we show in Appendix D.3 and formalize in Appendix D.4 is that the atypical memorized samples of the encoder’s training data lead to a lower alignment loss in larger regions of the representation space. Figure 10 shows that without the inclusion of memorized samples (Figure 10c), the alignment loss is only low in the regions where the main data distribution clusters. However, with the memorized samples (Figure 10b), larger regions of the representation space exhibit a lower alignment loss. A low alignment loss means that similar inputs obtain similar representations, which helps a downstream classifier to learn better decision boundaries and classify samples in these regions better, which boosts generalization.