Title: Measuring Dejavu Memorization Efficiently
Venue: NeurIPS 2024
Accepted: yes
Keywords: memorization

privacy
Abstract: Recent research has shown that representation learning models may accidentally memorize their training data. For example, the déjà vu method shows that for certain representation learning models and training images, it is sometimes possible to correctly predict the foreground label given only the representation of he background – better than through dataset-level correlations. However, their measurement method requires training two models – one to estimate dataset-level correlations and the other to estimate memorization. This multiple model setup becomes infeasible for large open-source models. In this work, we propose alter native simple methods to estimate dataset-level correlations, and show that these can be used to approximate an off-the-shelf model’s memorization ability without any retraining. This enables, for the first time, the measurement of memorization in pre-trained open-source image representation and vision-language models. Our results show that different ways of measuring memorization yield very similar aggregate results. We also find that open-source models typically have lower aggregate memorization than similar models trained on a subset of the data. The code is available both for vision (https://github.com/facebookresearch/DejaVuOSS) and vision language (https://github.com/facebookresearch/VLMDejaVu) models.


Peer reviews:
comments: SUMMARY:
this paper proposes a method to measure memorization in representation learning models without the need for training two separate models. The proposed method aims to address the computational challenges and practical limitations of the existing déjà vu method by using simpler alternative approaches to quantify dataset-level correlations.

STRENGTHS:
- The proposed method achieves similar aggregate results in measuring memorization as the traditional two-model setup.
- The use of a single model and simpler classifiers (like Naive Bayes and ResNet50) to approximate dataset-level correlations is interesting but also a limitation :) see the weaknesses section.

WEAKNESSES:
- The reliance on simpler models like Naive Bayes classifiers might limit the method's accuracy in certain complex scenarios.
- There is a risk that the simpler models used for approximating dataset-level correlations might overfit their training data, which could skew the results.
- The reference models used to approximate dataset-level correlations might introduce their own biases, affecting the accuracy of the memorization measurement.

- The method provides an aggregate measure of memorization but lacks detailed insights into the types of memorized information. A more granular analysis could help in understanding the nature and implications of memorization in different models.

QUESTIONS:
see Weaknesses section

LIMITATIONS:
see Weaknesses section
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes methods to gauge the ability of a model to memorize\n\nSTRENGTHS:\n1. Originality: to my knowledge, this is an original work with novel results.\n1. Clarity: The writing is clear.\n1. Significance: The memorization of foundation models is critical to study, these methods and findings are significant to the community.\n\nWEAKNESSES:\n1. The results hinge on the smaller classifier not memorizing training data. The authors list this as a limitation, but some empirical result to convince the reader that this is in fact unlikely would be strengthen the argument.\n1. The comparison in Section 4.1.2 is slightly confusing (question below).\n1. A concluding ranking or thought about commonly used models is missing (question below)\n\nQUESTIONS:\n1. In Section 4.1.2, I understand that the authors make the observation that pre-trained models memorize less than the same models trained on a subset of the data and they offer as an explanation that the pre-trained models have lower test error and larger training set. Can the authors clarify here what they mean? To me, if a model memorizes less and generalizes better I don't see any evidence that one is the cause of the other, these might both be the result of some other variable. Maybe the answer is as simple as larger training set leads to both phenomena, but I'd then like to see a test where the training routing (pre-training or not) is held constant and memorization is studied as it varies with trainset size. In any case, some more clarity is needed here.\n1. I feel a paper with this many results could be stronger if it ended with a bit more advice or conclusion. Do the authors have a clear ranking of models/techniques that a practitioner should use to avoid memorization or a set of available models that a user should use if they want a model with less memorization? These points would better motivate the paper as a whole. I think section 4.2.2 aimed to do this but it left me with these questions, so a clearer more direct bit of actionable advice to the reader would be appreciated.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method to measure memorization in representation learning models without the need for training multiple models. Previous déjà vu memorization estimation methods require two models to estimate dataset-level correlations and memorization, which is computationally expensive. The authors propose a simplified approach using a single model, with alternative methods to estimate dataset-level correlations. This approach is validated on various image representation learning and vision-language models, showing that the simplified method yields similar results to the traditional two-model approach.\n\nSTRENGTHS:\n1. The authors present a simplified method for measuring memorization that avoids the need for training multiple models.\n2.  The method is validated across multiple datasets and models, including ImageNet-trained models and CLIP models, showing consistent results with the traditional approach.\n3. The proposed method can be applied to both image-only and vision-language representation learning models.\n4. The paper is well-written and organized, with clear explanations of the methods, experiments, and results.\n\nWEAKNESSES:\n1.  The method builds on existing ideas of dataset-level correlation estimation and déjà vu memorization, which have been previously explored in the literature. This reduces the novelty of the contributions.\n2.  The concern that background annotation models like Grounded-SAM used in the paper may not have disjoint sets is important. If the sets were not disjoint, it could affect the results by introducing unintended correlations between the training and testing sets. The paper should address whether this was considered and controlled for in the experiments.\n3.  The results are reported for the entire dataset, but memorization metrics are more meaningful at the sample level. The lack of sample-level memorization scores limits the granularity and insights of the analysis. Including sample-level scores would provide a clearer picture of which specific samples are being memorized.\n\nQUESTIONS:\n1. Two-Model Dependency: Does this one-model method still rely on the use of two models, with one being a pretrained model used for estimating dataset-level correlations? How does the choice of such a pretrained model affect the results?\n\n2. Sample-level Metrics: Providing some discussion as to why aggregate metrics were chosen and how they compare to sample-level results could offer more detailed insights. Can the method provide sample-level memorization scores, and how would this affect the interpretation of the results?\n\n3. Are there specific cases where this approach may fail or provide? It was hard to decipher the limitations discussed at the end of sections 3 and 4.\n\nLIMITATIONS:\n1. Background Annotation Model Concerns: The potential overlap in data between background annotation models like Grounded-SAM and the dataset being tested should be addressed. If these models were not trained on disjoint sets, it could lead to misleading results regarding memorization.\n\n2. Sample-level Insights: The lack of sample-level memorization scores is a significant limitation. Providing some a discussion as to why aggregate metrics were chosen and how they compare to sample-level results could offer more detailed insights into which specific samples are being memorized and help in understanding the model's behavior better.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}