Tracing Training Footprints: A Calibration Approach for Membership Inference Attacks Against Multimodal Large Language Models (2025)

Abstract: With the increasing scale of training data for Multimodal Large Language Models (MLLMs) and the lack of data details, there is growing concern about privacy breaches and data security issues. Under black-box access, exploring effective Membership Inference Attacks (MIA) has garnered increasing attention. In real-world applications, where most samples are non-members, the issue of non-members being over-represented in the data manifold, leading to misclassification as member samples, becomes more prominent. This has motivated recent work to focus on developing effective difficulty calibration strategies, producing promising results. However, these methods only consider text-only input during calibration, and their effectiveness is diminished when migrated to MLLMs due to the presence of visual embeddings. To address the above problem, we propose PC-MMIA, focusing on visual instruction fine-tuning data. PC-MMIA is based on the idea that tokens located in poorly generalized local manifolds can better reflect traces of member samples that have been trained. By employing bidirectional perturbation of image embeddings to capture tokens critical to MIA and assigning them different weights, we achieve difficulty calibration. Experimental results demonstrate that our proposed method surpasses existing methods.

## introduction in recent years, the rapid development of large language models (llms) has brought new opportunities for multimodal large language models (mllms) research, leading to the emergence of vision-language models such as gpt-4o and qwen2.5-vl @xcite . by aligning visual embeddings with the textual space, mllms can process and reason on multimodal data, enabling them to perform complex tasks such as image captioning and visual question answering @xcite . two wheelbarrows are creatively repurposed as planters, overflowing wheelbarrows token difficulty text-only calibration pc-mmia this rapid development is heavily reliant on the availability of large-scale internet datasets, which have tremendously improved model performance @xcite . however, this also raises concerns about privacy leakage and unauthorized data usage. for instance, image data used in commercial model training may include private photos or copyrighted content @xcite . to explore these issues, membership inference attacks (mia) serve as a common technique, where attackers attempt to infer whether specific data samples are part of the model's training set @xcite . mia has now evolved from a mere attack method into an effective tool for auditing black-box models @xcite , providing a new perspective on data privacy protection. mia can generally be divided into three attack scenarios: black-box access, gray-box access, and white-box access @xcite . in next-token prediction, black-box access provides only the probability of the predicted token, while gray-box access allows retrieval of probabilities for all candidate tokens in the vocabulary @xcite . under white-box access, attackers can access the model's architecture and parameters and even train the model. research on mia is crucial for protecting data security and maintaining user privacy, especially for mia targeting visual instruction fine-tuning data, which is more likely to contain unauthorized or private data @xcite @xcite . despite numerous mia methods proposed in the llms domain, their direct naive migration to mllms is unsatisfactory due to the multimodal inputs of mllms @xcite . thus, there is an urgent need for a mia framework specifically designed for mllms. the major challenge faced by mia against mllms is the need to consider the impact of image embeddings on the difficulty of token prediction. previous conventional methods rely on the notion that the word probability distribution based on text can reveal whether it is included in the training set @xcite . for instance, when a token's predicted probability is high, these methods assume that the token is likely present in member samples. however, they overlook the fact that the token may simply be simple and easy to predict, meaning non-member samples might themselves be over-represented in the data manifold @xcite . to address this, @xcite introduce the concept of difficulty calibration, highlighting the need to mitigate biases arising from the intrinsic complexity of samples, and calibrate using the predictive difficulty of the tokens themselves. recent works attempt to utilize various methods including keyword marking @xcite and divergencefrom-randomness @xcite for calibration, yielding promising results. nonetheless, as illustrated in figure 1 , in multimodal data, these calibration methods are only effective for text tokens and fail to account for the impact of image embeddings on the probability distribution, leading to suboptimal performance. facing the challenges above, our research aims to address the question: in the presence of image embedding input, if a specific text token in the response has a high probability, is this due to verbatim memorization (overfitting of member data), or is it simply because the token itself is easily predictable? inspired by @xcite , we propose a perturbation-based calibration method, named pc-mmia, to calibrate token probabilities for membership inference attacks against mllms. our core idea is that tokens located in poorly generalized local manifolds can better reflect traces of training on member samples. after perturbing images, we observe probability changes for each token in the samples to determine whether the token's probability contribution stems more from generalized prediction or verbatim memorization of member data. subsequently, we normalize the changes in log-likelihood for each token before and after perturbation to serve as correction weights. we summarize our main contributions as follows: i) we propose pc-mmia, a novel membership inference attack method that focuses on the issue of non-member samples potentially being over-represented in the data manifold. ii) to the best of our knowledge, we are the first to consider the visual embeddings in the difficulty calibration of mllms, opening up new directions for mia research. iii) compared to other black-box mia methods, our approach achieves optimal performance in commonly used detection metrics tpr at low fpr and auc. additionally, we have opensourced the code to facilitate further research foot_0 . ## pc-mmia in order to achieve image understanding, mllms like llava @xcite and cogvlm @xcite project the image embeddings from visual encoders into the feature space of llms. consequently, when considering the prediction difficulty of text tokens, the semantic impact of images must also be taken into account, hindering the effective migration of most existing mia methods designed for llms to mllms. to address this, we propose pc-mmia, a novel approach tailored for mllms. ## experiments in this section, we conduct comprehensive experiments to validate the effectiveness of pc-mmia. ## conclusion conventional mia overlook the intrinsic complexity of samples, prompting an increasing number of studies to introduce various difficulty calibration methods. however, these approaches focus solely on calibrating text tokens, neglecting the rich semantics brought by image embeddings. to address this, we introduced pc-mmia, a black-box membership inference attack method specifically tailored for multimodal large language models. it captures tokens in the text that better reflect training traces through perturbation of image embeddings, assigning different weights to achieve difficulty calibration. experiments demonstrate that pc-mmia outperforms all black-box access baselines. we believe our work lays the groundwork for advancing mia technology, thereby enhancing privacy preservation in multimodal large language models.