Title: Quantifying Cross-Modality Memorization in Vision-Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Memorization

VLM Memorization
Abstract: Understanding what and how neural networks memorize during training is crucial, both from the perspective of unintentional memorization of potentially sensitive information and from the standpoint of effective knowledge acquisition for real-world, knowledge-intensive tasks. While previous studies primarily investigate memorization within a single modality, such as text memorization in large language models or image memorization in diffusion models, unified multimodal models are becoming increasingly prevalent in practical applications. In this work, we focus on the unique characteristics of cross-modality memorization and conduct a systematic study centered on vision-language models. To facilitate controlled experiments, we first introduce a synthetic persona dataset comprising diverse synthetic person images and textual descriptions. We quantify factual knowledge memorization and cross-modal transferability by training models on a single modality and evaluating their performance in the other. Our results reveal that facts learned in one modality transfer to the other, but a significant gap exists between recalling information in the source and target modalities. Furthermore, we observe that this gap exists across various scenarios, including more capable models, machine unlearning, and the multi-hop case. At the end, we propose a baseline method to mitigate this challenge. We hope our study can inspire future research on developing more robust multimodal learning techniques to enhance cross-modal transferability.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies cross-modality memorization in vision-language models for sensitive attribute prediction task on a synthetic dataset introduced by the authors. The paper quantifies memorization in terms of predicting correct attribute in both source and target modalities. Here the source is the modality that is used for training the model and target is the modality that is used only for inference.\nThe paper suggests that memorization transfers between modalities but it is significantly lower in the target modality. Lastly, the paper proposes mitigations using data augmentation and studies cross-modality unlearning.\n\nSTRENGTHS AND WEAKNESSES:\n**Pros**\n\n+ The paper is clearly and well written it conducts thorough experiments for different hyper parameters such as model size, increasing data diversity to avoid overfitting.\n+ The paper proposes a mitigation for the cross modality transferability gap and shows that proposed mitigation helps to close the gap. \n+ It’s also interesting to see multi-hop unlearning results for cross-modality dataset.\n\n**Cons**\nCons\n+ The experimental evidence is provided only for the benchmark proposed by the paper. It is unclear whether the observations made by the authors are generalizable cross other datasets and modalities.\nIt could be that the observations made in the paper reflect some of the properties in the dataset. It might be good to try the experiments on other attribute prediction dataset and task.\n+  It seems that the authors use the COCO dataset as OOD dataset in one of their experiments. It might have been good to run all experiments on COCO dataset instead of the synthetic dataset and use synthetic dataset as OOD. It would be interesting to see whether the claims made by the authors will hold for this setup too.\n+ In terms of VLM memorization there is this other paper that is not cited\nhttps://arxiv.org/abs/2402.02103\n\nQUESTIONS:\n+  The paper first mentions that there are ~64B profiles but then later that there are only 100 unique profiles. What are the differences between profile and persona ? How large is the final dataset ?\n+ How did you ensure that the dataset is diverse and representative enough for the study ?\n+ How much data was added during training with variants ? How did you know how many variants you need to add ?\n\nLIMITATIONS:\nThe paper discussed future work but does not discuss the limitations of the work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper carries out studies of cross-modal knowledge extraction in a multimodal LLM. To do this, they introduce a new dataset of personas, consisting of synthetically generated images of (fake) people, together with their textual descriptions. The MLLMs are then fine-tuned with this information in one modality (image or text), and extraction is attempted from the other. The main finding of the paper is that image to text extraction is significantly easier than text to image extraction.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ Overall I thought this is a very solid paper -- the dataset is well-made and interesting, the experiments are quite well-done, the conclusions are presented crisply. Execution-wise this is a really strong paper.\n\nWeaknesses:\n\n- To me, the main weakness of this paper is in the way is it framed. The current framing of the paper suggests that the results have serious implications for privacy, showing that models memorize private information, and that this memorization could be \"cross-modal\". \n\nHowever usually memorization in the the privacy/memorization literature refers to incidental memorization where there is a very small amount of data about a single person in a very large crowd, such as a single data point in a large crowd. My understanding is that the setting of the current paper is a little bit different in a subtle way -- since we are directly fine-tuning with each person's data for many epochs; in my view, this paper's setting is closer to the Physics of LLMs: Knowledge Extraction (https://arxiv.org/abs/2309.14316) paper but for multimodal models. This is not to say that the results are uninteresting -- in fact they are extremely well-done and interesting -- but the implications for privacy are perhaps not quite there. \n\n- Minor: It's also not quite correct that there has no work at all on multimodal memorization -- [1, 2] look at multimodal memorization in a different type of multimodal (CLIP) models. \n\n[1] https://arxiv.org/abs/2402.02103\n[2] https://arxiv.org/abs/2504.05651\n\nQUESTIONS:\nn/a\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the memorization behavior in a multi-modal (image-text) foundation model using a novel synthetic dataset that is easy to control. Carefully designed experiments test the model’s ability to recognize facts in both modalities, which were only presented in one modality during training. The findings reveal that knowledge does not fully transfer from the source modality to the target modality, and larger model sizes, nor diverse training variations can bridge the gap. Training with more in-distribution image-caption pairs, on the other hand, does help. Finally, the paper reports asymmetries in the results depending on which modality was used as the source modality (e.g., for cross-modality transfer, effects of machine unlearning and the multi-hop curse).\n\nSTRENGTHS AND WEAKNESSES:\nI will list perceived strengths and weaknesses regarding Quality [Q], Clarity [C], Significance [S}, and Originality [O] below.\n\n# Strengths\n1. [C] The paper is beautifully written and very well structured. The introduction makes the aims and contributions clear from the start, and lays out nicely what the reader can expect (and what not to). The take-home boxes are very helpful too. The type of language used (not overly technical) makes it accessible to non-experts. \n2. [Q] The experiments are carefully designed to address specific questions and they delivered clear take-home messages. This was made possible through a novel synthetic dataset that offers good experimental control.\n3. [O] The custom synthetic dataset makes for an original addition to the work and describes a way for others to build their own.\n4. [S] I can see this work being an important first steps for many follow-up papers to build on, speaking to many members in the NeurIPS community. Especially given the far-reaching impact of multi-modal foundation models, understanding their behavior and mitigating privacy and other memorization-related concerns is a valuable endeavor. \n\n# Weaknesses\n1. [Q] The paper is very engaging, which made me want to know more about potential causes for the observed patterns, something that wasn't touched upon very much.\n2. [Q] In a similar vein, I would have loved to see more results on other mitigation strategies, although it was clearly acknowledged in the introduction that this was for future work. Could it not have been one big paper?\n3. [S] The results are based on one (specialized) dataset and one model family (the focus on solely image-text modalities was already acknowledged in the paper).\n\nQUESTIONS:\nMy first impression was that it was already a strong paper. To consider increasing an already good score, I would want to see how the conclusions generalize to other datasets/models/modalities and get more insights on causes or mitigation strategies.\n\nSome minor suggestions:\n- it could be nice to have the take-home in the figure caption. I found myself referring back to the text to check my interpretation of the figure.\n- this might be personal taste, but I was comparing graph-lines between figures and it would have helped me if the xlims and ylims were consistent across figures. And to have it pointed out which lines were identical to one from a previous figure (same condition).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work takes the first step toward systematically quantifying cross-modal memorization in vision–language models. Rather than evaluating on arbitrary real-world data, it introduces a synthetic “persona” benchmark of paired images and text captions labeled with unique names. By fine-tuning VLMs to predict names from one modality and testing on the other, it uncovers a persistent, asymmetric performance gap and a multi-hop inference failure in cross-modal settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper tackles an important and understudied problem by systematically quantifying how factual knowledge learned in one modality transfers to another in vision-language models.\n2. The synthetic persona dataset is carefully designed with controlled image–text pairs, enabling clear attribution of memorization and transfer effects.\n3. Experiments are thorough which covering single-input vs. variant training, model scaling, data augmentation, unlearning, and multi-hop reasoning, providing a comprehensive empirical picture.\n4. Clear asymmetries (image-to-text and text-to-image transfer) and failure modes (overfitting, multi-hop curse, unlearning gaps) are identified, offering concrete takeaways for future methods.\n\nWeaknesses:\n1. Reliance on fully synthetic personas may limit external validity, as real-world data distributions are not evaluated.\n2. The study focuses solely on name-recall tasks, leaving open whether richer or higher-level knowledge (e.g., events, attributes beyond names) exhibits similar cross-modal behavior.\n3. All experiments use a single VLM family (Gemma-3-IT), so conclusions may not generalize to other architectures or pretraining regimes.\n4. The impact of synthetic data quality (artifacts from Imagen 3 or Gemini 2.0) on memorization and transfer is not examined, risking biased results.\n5. The limitations of this study is not discussed in the paper.\n\nQUESTIONS:\nMy main concerns are in the method section, please see the weakness above.\nSince this work is interesting to me, I would be willing to raise my score if the mentioned weaknesses are addressed in the rebuttal.\n\nLIMITATIONS:\nThe limitations of this study is not discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}