Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges (2025)

Abstract: Privacy risks in text-only Large Language Models (LLMs) are well studied, particularly their tendency to memorize and leak sensitive information. However, Multi-modal Large Language Models (MLLMs), which process both text and images, introduce unique privacy challenges that remain underexplored. Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images, posing new privacy risks. We reveal that some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory. Specifically, in this paper, we (1) introduce MM-Privacy, a comprehensive dataset designed to assess privacy risks across various multi-modal tasks and scenarios, where we define Disclosure Risks and Retention Risks. (2) systematically evaluate different MLLMs using MM-Privacy and demonstrate how models leak sensitive data across various tasks, and (3) provide additional insights into the role of task inconsistency in privacy risks, emphasizing the urgent need for mitigation strategies. Our findings highlight privacy concerns in MLLMs, underscoring the necessity of safeguards to prevent data exposure. Part of our dataset and code can be found here.

## introduction privacy risks in text-only large language models (llms) are well-documented. studies have shown that these models can memorize and leak private information, such as personal identifiers or confidential text, during inference @xcite @xcite @xcite . however, the privacy risks of multi-modal large language models (mllms), which process both images and text @xcite ; zhu et al., 2023), remain insufficiently studied. compared to their text-only counterparts, mllms can expose sensitive information embedded in images or output sensitive information in memory induced by images, which poses new security risks. most previous works on evaluating mllms' privacy issues mainly focus on privacy awareness @xcite , i.e. whether the mllms can recognize private information in the image. therefore, the actual risks of sensitive data leakage remain largely unexplored. given this gap, a key research question arises: q: how severe is the privacy issues of mllms? to answer the research question and address the new privacy challenges, in this paper, we first define two privacy risks: 1) disclosure risks and 2) retention risks, which are designed specifically for mllms. disclosure risks assess the model's immediate behavior when processing sensitive input, which is similar to the privacy awareness test while retention risks evaluate how the model handles information it has learned during training. to evaluate our defined risks, we propose a novel dataset, mm-privacy. mm-privacy includes a shared image set that can be used for both disclosure risks and retention risks and two distinct text prompts based on assessing different privacy risks. for disclosure risks, mm-privacy only contains an evaluation set and for retention risks mm-privacy contains a memory set and an evaluation set. for both risks, evaluation sets are designed to assess whether mllms reveal private information while the memory set contains images with synthetic private information, which serves as the memory of mllms. this basic dataset comprises 1,000 memory samples and 2,500 evaluation samples. to ensure a more comprehensive evaluation, we extend our basic dataset to cover multiple tasks, including image caption or sentence rephrasing as different tasks may introduce varying levels of privacy risk. this expansion results in a final mm-privacy dataset containing over 13,000 samples. ## experiments in this part, we evaluate mm-privacy on both closed-source llms and open-source llms to analyze the privacy issues in mllms systematically. we will first introduce results for disclosure risks and then retention risks. we also explore how the defense prompt will influence the results later. ## conclusion in this study, we assessed the privacy vulnerabilities in mllms by introducing the mm-privacy. the experimental results indicate that mllms may be susceptible to privacy leaks through direct and memory-based tests, particularly in tasks such as image captioning and rephrasing. besides, our results indicate that closed-source models generally offer better privacy safeguards while open-source models are notably vulnerable. these results underscore the urgent need for enhanced privacy protection mechanisms in mllms especially in the open-source domain to ensure their safe use. limitation though our dataset contains the memory set, we cannot evaluate the correctness of email output by closed-sourced mllms. besides, we do not provide a comprehensive comparison between realworld data and data generated by our code due to the low volume of real-world data. exploring the different privacy risks provided by real-world data could be our future work.