Membership Inference Attacks against Large Vision-Language Models

Authors: zhan  li
yongtao  wu
yihang  chen
francesco  tonin
elias abad rocamora
volkan  cevher
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 00:53:14

Abstract: Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records, in their training datasets. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue, mainly due to the lack of standardized datasets and suitable methodologies. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new metric called MaxRényi-K%, which is based on the confidence of the model output and applies to both text and image data. We believe that our work can deepen the understanding and methodology of MIAs in the context of VLLMs. Our code and datasets are available at https://github.com/LIONS-EPFL/VL-MIA.

Main paper: 
 introduction: 
 The rise of large language models (LLMs) [9, 60, 45, 11] has inspired the exploration of large models across multi-modal domains, exemplified by advancements like GPT-4 [1] and Gemini [59] . These large vision-language models (VLLMs) have shown promising ability in various multi-modal tasks, such as image captioning [33] , image question answering [13, 35] , and image knowledge extraction [26] . However, the rapid advancement of VLLMs also causes user concerns about privacy and knowledge leakage. For instance, the image data used during commercial model training may contain private photographs or medical diagnostic records. This is concerning since early work has demonstrated that machine learning models can memorize and leak training data [3, 56, 63] . To mitigate such concerns, it is essential to consider the membership inference attack (MIA) [23, 53] , where attackers seek to detect whether a particular data record is part of the training dataset [23, 53] . The study of MIAs plays an important role in preventing test data contamination and protecting data security, which is of great interest to both industry and academia [24, 19, 44] .When exploring MIAs in VLLMs, one main issue is the absence of a standardized dataset designed to develop and evaluate different MIA methods, which comes from the large size [16] and multimodality of the training data, and the diverse VLLMs training pipelines [66, 35, 18] . Therefore, one of the main goals of this work is to build an MIA benchmark tailored for VLLMs.Beyond the need for a valid benchmark, we lack efficient techniques to detect a single modality in VLLMs. The closest work to ours is [30] , which performs MIAs on multi-modal CLIP [46] by detecting whether an image-text pair is in the training set. However, in practice, it is more common to detect a single modality, as we care whether an individual image or text is in the training set. Therefore, we aim to develop a pipeline to detect the single modality from a multi-modal model. Moreover, existing literature on language model MIAs, such as Min-K% [52] and Perplexity [62] , mostly are target-based MIAs, which use the next token as the target to compute the prediction probability. However, we can only access the image embedding instead of the image token in VLLMs, and thus only target-free MIAs [48] can be directly applied.Therefore, we first propose a cross-modal pipeline for individual image or description MIAs on VLLMs, which is distinguished from traditional MIAs that only use one modality [61, 62] . We feed the VLLMs with a customized image-instruction pair from the target image or description. We show that we can perform the MIA not only by the image slice but also by the instruction and description slices of the VLLM's output logits, see Figure 1 . Such a cross-modal pipeline enables the usage of text MIA methods on image MIAs. We also introduce a target-free metric that adapts to both image and text MIAs and can be further modified to a target-based way.Overall, the contributions and insights can be summarized as follows.• We release the first benchmark tailored for the detection of training data in VLLMs, calledVision Language MIA (VL-MIA) (Section 4). By leveraging Flickr and GPT-4, we construct VL-MIA that contains two images MIA tasks and one text MIA task for various VLLMs, including MiniGPT-4 [66] , LLaVA 1.5 [35] and LLaMA-Adapter V2 [18] . • We perform the first individual image or description MIAs on VLLMs in a cross-modal manner. Specifically, we demonstrate that we can perform image MIAs by computing statistics from the image or text slices of the VLLM's output logits (Figure 1 and Section 5.1). • We propose a target-free MIA metric, MaxRényi-K%, and its modified target-based ModRényi (Section 5.2). We demonstrate their effectiveness on open-source VLLMs and closed-source GPT-4 (Section 6). We achieve an AUC of 0.815 on GPT-4 in image MIAs.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract and introduction accurately state our contribution and findings.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 Membership Inference Attack (MIA) aims to classify whether a data sample has been used in training a machine learning model [53] . Keeping training data confidential is a desired property for machine learning models, since training data may contain private information about an individual [62, 24] . Popular MIA methods can be divided into metric-based and shadow model-based MIAs [24] . Metric-based MIAs [62, 48, 57, 52] determine whether a data sample has been used for training by comparing metrics computed from the output of the target model with a threshold. Shadow model-based MIAs need shadow training to mimic the behavior of the target model [53, 48] , which is computationally infeasible for LLMs. Therefore, we focus on the metric-based methods in this work.MIAs have been extensively researched in various machine learning models, including classification models [38, 58, 10] , generative models [20, 22, 7] , and embedding models [54, 40] . With the emergence of LLMs, there are also a lot of work exploring MIAs in LLMs [42, 17, 52, 41] . Nevertheless, MIAs for multi-modal models have not been fully explored. [30, 25] perform MIAs using the similarity between the image and the ground truth text, which detects the image-text pair instead of a single image or text sequence. However, detecting an individual image or text is more practical in real-world scenarios and poses additional challenges. To the best of our knowledge, we are the first to perform the individual image or text MIA on VLLMs.In this paper, we also consider a more difficult task, to detect the pre-training data from the fine-tuned models, that is, to detect the base LLM pertaining data from VLLMs. First, compared with detecting fine-tuning data [55, 51] , pretraining data come from a much larger dataset and are used only once, reducing the potential probability for a successful MIA [27, 32] . In addition, compared to the detection of pretraining data from the pre-trained models [53, 52] , catastrophic forgetting [29, 28, 16] in the fine-tuning stage also makes it harder to detect the pre-training data from downstream models.To the best of our knowledge, we are the first to perform pre-training data MIAs on fine-tuned models.Large Vision-Language Models (VLLMs) incorporate visual preprocessors into LLMs [9, 60, 45] to manage tasks that require handling inputs from text and image modalities. A foundational approach in this area is represented by CLIP [46] , which established techniques for aligning modalities between text and images. Further developments have integrated image encoders with LLMs to create enhanced VLLMs. These models are typically pre-trained on vast datasets of image-text pairs for feature alignment [33, 64, 39, 2] , and are subsequently instruction-tuned for specific downstream tasks to refine the end ability. MiniGPT [66, 8] , LLaVA [36, 35] , and LLaMA Adapter [65, 18] series have demonstrated significant capabilities in understanding and inference in this area.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations in Appendix G.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem setting: 
 In this section, we introduce the main notation and problem settings for MIAs.Notation. The token set is denoted by V. A sequence with L tokens is denoted by X := (x 1 , x 2 , . . . , x L ), wherex i ∈ V for i ∈ [L]. Let X 1 ⊕ X 2 be the concatenation of sequence X 1 and X 2 .An image token sequence is denoted by Z. In this work, we focus on the VLLM, parameterized by θ, where the input is the image Z followed by the instruction text X ins , and the output is the description text X des . We use D des and D image to represent the description training set and image training set, respectively. Detailed notations are summarized in Table 5 of the appendix.Attacker's goal. In this work, the purpose of the attacker is to detect whether a given data point (image Z or description X des ) belongs to the training set. We formulate this attack as a binary classification problem. Let A image (Z; θ) :→ {0, 1} and A des (X; θ) :→ {0, 1} be two binary classification algorithms for image and description respectively, which are implemented by comparing the metric Score(Z ⊕ X ins ⊕ X des ; θ) with some threshold λ.When detecting image Z, we feed the model with the target image with a fixed instruction prompt such as "Describe this image in detail", denoted as X ins . The model then generates the description text X ′ des . The algorithm A image (Z; θ) is defined byEQUATIONWhen detecting a description sequence X des , we feed the model with an all-black image, denoted as Z ept , as the visual input, followed by an empty instruction X ept . The algorithm A des (X des ; θ) is defined byEQUATIONAttacker's knowledge. We assume a grey-box setting on the target model, where the attacker can query the model by a custom prompt (including an image and text) and have access to the tokenizer, the output logits, and the generated text. The attacker is unaware of the training algorithm and parameters of the target model.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA]Answer: [Yes] Justification:We have provided open access to the data and code in https://github.com/LIONS-EPFL/VL-MIA. Guidelines:• The answer NA means that paper does not include experiments requiring code.  
dataset construction: 
 We construct a general dataset: Vision Language MIA (VL-MIA), based on the training data used for popular VLLMs, which, to our knowledge, is the first MIA dataset designed specifically for VLLMs. We present a takeaway overview of VL-MIA in Table 1 . We also provide some examples in VL-MIA, see Table 16 in the appendix. The prompts we use for generation can be found in Table 6 . Target models. We perform MIAs open-source VLLMs, including MiniGPT-4 [66] , LLaVA-1.5 [35] , and LLaMA-Adapter V2.1 [18] . The checkpoints and training datasets of these models are provided and public. The training pipeline of a VLLM encompasses several stages. Initially, an LLM undergoes pre-training using extensive text data such as LLaMA [18] . Meanwhile, a vision preprocessor, e.g., CLIP [46] , is pre-trained on a large number of image-text pairs. Subsequently, a VLLM is constructed based on the LLM and the vision preprocessor and is pre-trained using image-text pairs. The final stage involves instruction-tuning the VLLM, which can be performed using either image-text pairs or image-based question-answer data. In the instruction tuning stage of a VLLM, every data entry contains an image, a question, and the corresponding answer to the image. We use the answer text as member data and GPT-4 generated answers under the same question and same image as non-member data. Specifically, for LLaVA 1.5 and LLaMA-Adapter v2, we use the answers in LLaVA 1.5's instruction tuning as member data.VL-MIA/DALL-E. MiniGPT-4, LLaVA 1.5, and LLaMA-Adapter V2 use images from LAION [49] , Conceptual Captions 3M [6] , Conceptual 12M [6] and SBU captions [43] datasets (collectively referred to as LAION-CCS) as pre-training data. BLIP [33] provides a synthetic dataset with imagecaption pairs for LAION-CCS used in MiniGPT-4 and LLaVA 1.5. We first detect the intersection of the training images used in these three VLLMs. From this intersection, we randomly select a subset to serve as the member data for our benchmark. For the non-member data, we use the corresponding BLIP captions of the member images as prompts to generate images with DALL-E 2 3 . This process yields one-to-one corresponding pairs of the generated images (non-member) and the original member images. Consequently, our dataset comprises an equal number of member images from the LAION-CCS dataset and non-member images generated by DALL•E, allowing us to evaluate MIA performance comprehensively. We have 592 images in VL-MIA/DALL-E in total.VL-MIA/Flickr. MS COCO [34] co-occurs as a widely used dataset in the training data of the target models, so we use the images in this dataset as member data. Given the fact that such member data are collected from Flickr 4 , we filter Flickr photos by the year of upload and obtain new photos from January 1, 2024, as non-member data, which are later than the release of the target models. We additionally prepare a set of corrupted versions, where the member images are deliberately corrupted to simulate real-world settings. More results of the corrupted versions are discussed in Section 6.5. This dataset contains 600 images.VL-MIA/Text. We prepare text MIA datasets for the VLLMs instruction-tuning stage. LLaVA 1.5 and LLaMA Adapter v2.1 both use the LLaVA-Instruct-150K [35] in instruction-tuning, which consists of multi-round QA conversations. We first select entries with descriptive answers of 64 words. Next, we feed the corresponding questions and images into GPT-4 3 , and ask GPT-4 to generate responses of the same length, treating these generated responses as non-member text data. In addition, since MiniGPT-4 employs long descriptions of images for instruction-tuning, typically beginning with "the image", we prompt GPT to generate descriptions based on the MS COCO dataset, starting with "this image" to ensure similar data distributions. We also prepare different versions of the datasets by truncating the text into different word lengths such as 16, 32, and 64. Each text dataset contains 600 samples.We synthesize two new MIA datasets: VL-MIA/Geometry and VL-MIA/Password. The image in the VL-MIA/Geometry consists of a random 4x4 arrangement of geometrical shapes, and the image in the VL-MIA/Password consists of a random 6x6 arrangement of characters and digits from MNIST [15] and EMINST [12] . The associated text for each image represents its content (e.g., specific characters, colors, or shapes), ordered from left to right and top to bottom. Half of the dataset can be selected as the member set for VLLM fine-tuning, with the remainder as the non-member set. This partition ensures that members and non-members are independently and identically distributed, avoiding the latent distribution shift between the members and non-members in current MIA datasets [14] . Our synthetic datasets are ready to use, and can be applied to evaluate any VLLM MIA methods through straightforward fine-tuning. We provide some examples of this dataset in Figure 5 in the Appendix C. 
a cross-modal pipeline to detect image: 
 VLLMs such as LLaVA and MiniGPT project the vision encoder's embedding of the image into the feature space of LLM. However, a major challenge for image MIA is that we do not have the ground-truth image tokens. Only the embeddings of images are available, which prevents directly transferring many target-based MIA from languages to images. To this end, we propose a token-level image MIA which calculates metrics based on the output logit of each token position.This pipeline consists of two stages, as demonstrated in Figure 1 . In generation stage, we provide the model with an image followed by an instruction to generate a textual sequence. Subsequently, in inference stage, we feed the model with the concatenation of the same image, instruction, and generated description text. During the attack, we correspondingly slice the output logits into image, instruction, and description segments, which we use to compute various metrics for MIAs. Our pipeline considers the information from the image, the instructions and the descriptions following the image. In practice, even if there is no access to the logits of the image feature and instruction slice, we can still detect the member image solely from the model generation. We visually describe a prompt example with different slice notations presented in Appendix A.2.Our pipeline operates on the principle that VLLMs' responses always follow the instruction prompt [60] , where the images usually precede the instructions and then always precede the descriptions. For causal language models used in VLLMs that predict the probability of the next token based on the past history [45] , the logits at text tokens in the sequence inherently incorporate information from the preceding image.We propose our MaxRényi-K%, utilizing the Rényi entropy of the next-token probability distribution on each image or text token. The intuition behind this method is that if the model has seen this data before, the model will be more confident in the next token and thus have smaller Rényi entropy.Given a probability distribution p, the Rényi entropy [47] of order α, is defined asH α (p) = • H 1 (p) = -j p j log p j , • H ∞ (p) = -log max p j .To be more specific, given a token sequenceX := (x 1 , x 2 , . . . , x L ), let p (i) (•) = P(•|x 1 , • • • , x i )be the probability of next-token distribution at the i-th token. Let Max-K%(X) be the top K% from the sequence X with the largest Rényi entropies, the MaxRényi-K% score of X equalsMaxRényi-K%(X) = 1 |Max-K%(X)| i∈Max-K%(X) H α (p (i) ).When K = 0, we define the MaxRényi-K% score to be max i∈[L-1] H α (p (i) ). When K = 100, the MaxRényi-K% score is the averaged Rényi entropy of the sequence X.In our experiments, we vary α = 1 2 , 1, 2, and +∞; K = 0, 10, 100. As α increases, the top percentile of distribution p will have more influence on H α (p). When α = 1, H 1 (p) equals the Shannon entropy [50] , and our method at K = 100 is equivalent to the Entropy [48] . When α = ∞, we consider the most likely next token probability [31] . In contrast, Min-K% [52] only deals with the target next token probability. When the sequence is generated by the target model deterministically, i.e., when the model always generates the most likely next token, our MaxRényi-K% at α = ∞ is equivalent to the Min-K%.We also extend our MaxRényi-K% to the target-based scenarios, denoted by ModRényi. We first consider linearized Rényi entropy,H α (p) = 1 1-α j (p j ) α -1 , 0 < α < ∞, α ̸ = 1. H α (p) is also further defined at α = 1, as H 1 (p) = lim α→1 H α (p) = H 1 (p).Assuming the next token ID is y, recall that a small entropy value or a large p y value indicates membership, we want our modified entropy to be monotonically decreasing on p y and monotonically increasing on p j , j ̸ = y. Therefore, we propose the modified Rényi entropy on a given next token ID y, denoted by H α (p, y):H α (p, y) = - 1 |α -1|   (1 -p y )p |α-1| y -(1 -p y ) + j̸ =y p j (1 -p j ) |α-1| -p j   . Let α → 1, we have H 1 (p, y) = lim α→1 H α (p, y) = -j̸ =y p j log(1 -p j ) -(1 -p y ) log p y ,which is equivalent to the Modified Entropy [58] . In addition, our more general method does not encounter numerical instability in Modified Entropy as p j → 0, 1 at α ̸ = 1. For simplicity, we let the ModRényi score be the averaged modified Rényi entropy of the sequence. 
experiments: 
 In this section, we conduct MIAs across three target models using various baselines, MaxRényi-K%, and ModRényi. Experiment setup is provided in Section 6.1. The results on text MIAs and image MIAs are present in Section 6.2 and Section 6.3, respectively. In Section 6.4, we show that the proposed MIA pipeline can also be used in GPT-4. Ablation studies are present in Section 6.5. The versions and base models of VLLMs we use are listed in Table 7 of the appendices.Evaluation metric. We evaluate different MIA methods by their AUC scores. AUC score is the area under the receiver operating characteristic (ROC) curve, which measures the overall performance of a classification model in all classification thresholds λ. The higher the AUC score, the more effective the attack is. In addition to the average-case metric AUC, we also include the worst-case metric, the True Positive Rate at 5% False Positive Rate (TPR@5%FPR) in Appendix D suggested by [5] .Baselines. We take existing metric-based MIA methods as baselines and conduct experiments on our benchmark. We use the MIA method from [37] , which compares the feature vectors produced by the original image with the augmented image. We use KL-divergence to compare the logit distributions and term it Aug-KL in this paper. We also use Loss attack [62] , which is perplexity in the case of language models. Furthermore, we consider ppl/zlib and ppl/lowercase [4] , which compare the target perplexity to zlib compression entropy and the perplexity of lowercase texts respectively. [52] proposes Min-K% method, which calculates the smallest K% probabilities corresponding to the ground truth token. Min-K% is currently a state-of-the-art method to detect pre-training data of LLMs. For both Min-K% and MaxRényi-K%, we vary K = 0, 10, 100. In addition, we consider K = 20 for Min-K% as suggested in [52] . We further include the max_prob_gap metric that can represent the extreme confidence in certain tokens by the model. That is, we subtract the second largest probability from the maximum probability in each token position and calculate the mean as metric.We first conduct MIAs on images using VL-MIA/Flickr and VL-MIA/DALL-E in three VLLMs. For the image slice, it is not possible to perform target-based MIAs, because of the absence of ground-truth token IDs for the image. However, our MIA pipeline presented in Figure 1 can still handle target-based metrics by accessing the instruction slice and description slice.As demonstrated in Table 2 , MaxRényi-K% surpasses other baselines in most scenarios. An α value of 0.5 yields the best performance in both VL-MIA/Flickr and VL-MIA/DALL-E. As α increases, performance becomes erratic and generally deteriorates, though it remains superior to all target-based metrics. Overall, target-free metrics outperform target-based metrics for image MIAs. Another interesting observation is that instruction slices result in unstable AUC values, sometimes falling below 0.5 in target-based MIAs. This can be partially explained by the fact the model is more familiar with the member data. As a result, after encountering the first word "Describe", the model is more inclined to generate the description directly than generating the following instruction of X ins , i.e., "this image in detail". This is an interesting phenomenon that we leave to future research.The performance of the image MIA model is influenced by its training pipelines. Recall that MiniGPT-4 only updates the parameters of the image projection layer in image training, and LLaMA Adapter v2 applies parameter-efficient fine-tuning approaches. In contrast, LLaVA 1.5 training updates both the parameters of the projection layer and the LLM. The inferior performance of MIAs on MiniGPT-4 and LLaMA Adapter compared to LLaVA 1.5 is therefore consistent with [52] that more parameters' updates make it easier to memorize training data.We find that VL-MIA/DALL-E is a more challenging dataset than VL-MIA/Flickr, reflected in the AUC being closer to 0.5. In VL-MIA/DALL-E, each non-member image is generated based on the description of a member image. Therefore, member data have a one-to-one correspondence with non-member data and depict a similar topic, which makes it harder to discern. MIA on VLLM instruction-tuning texts We detect whether individual description texts appear in the VLLMs instruction-tuning. We use the description text dataset VL-MIA/Text of lengths (32, 64) , constructed in Section 4. We present our results in the first column of Table 3 . We observe that target-based MIA methods are significantly better than target-free ones, confirming our hypothesis.MIA on LLM pre-training texts. We use the WikiMIA benchmark [52] , which leverages the Wikipedia timestamp to separate the early Wiki data as the member data, and recent Wiki data as the non-member data. The early Wiki data are used in various LLMs pre-training [60] . We use WikiMIA of different lengths (32, 64, 128, 256) , and expect the membership of longer sequences will be more easily identified. We present our results in the second column of Table 3 . We observe that on LLaVA, our target-free MIA methods on large α consistently outperform target-based MIA methods, which again confirms our hypothesis since the base LLM model of LLaVA has full parameter fine-tuning from LLaMA-2 and thus changed a lot.6.4 Image MIA on GPT-4 4 . We omit the result of perplexity and Min-K% since they are equivalent to MaxRényi-K% with α = ∞ in the greedygenerated setting, as discussed in Section 5.2. Surprisingly, we observe that in VL-MIA/DALL-E, the best-performed method MaxRényi-K% (α = 0.5) can achieve an AUC of 0.815. This indicates a high level of effectiveness for MIAs on GPT-4, demonstrating the potential risks of privacy leakage even with closed-source models.Does the length of description affect the image MIA performance? We conduct ablation experiments on LLaVA 1.5 targeting the length of generated description texts with MaxRényi-10%. In the generation stage, we restrict the max_new_tokens parameter of the generate function to (32, 64, 128, 256) to obtain description slices of different lengths. As presented in Figure 2a , when the length of the description increases, the AUC of the MIA becomes higher and enters a plateau when max_new_tokens reaches 128. This may be because a shorter text contains insufficient information about the image, and in an excessively long text, words generated later tend to be more generic and not closely related to the image, thereby contributing less to the discriminative information that helps discern the membership.Can we still detect corrupted member images? The motivation is to detect whether sensitive images are inappropriately used in VLLM's training even when the images at hand may get corrupted. We leverage ImageNet-C [21] to generate corrupted versions of member data in VL-MIA/Flickr: Snow, Brightness, JPEG, and Motion_Blur, with the parameters in Table 8 . The corrupted examples and corresponding model output generations are demonstrated in Appendix C Table 17 and Table 18 . We take MaxRényi-K% (α = 0.5) as the attacker and the results of LLaVA are presented in Figure 2b .Corrupted member images make MIAs more difficult, but can still be detected successfully. We also observe that reducing model quality (JPEG) or adding blur (Motion_Blur) degrade MIA performance more than changing the base parameter (Brightness) or overlaying texture (Snow).We conduct image MIAs on VL-MIA/Flickr with LLaVA through three different instruction texts: "Describe this image concisely.", "Please introduce this painting.", and "Tell me about this image.". We present our results in Table 14 of the appendix. Our pipeline successfully detects member images on every instruction, which indicates robustness across different instruction texts.  
conclusion: 
 In this work, we take an initial step towards detecting training data in VLLMs. Specifically, we construct a comprehensive dataset to perform MIAs on both image and text modalities. Additionally, we uncover a new pipeline for conducting MIA on VLLMs cross-modally and propose a novel method based on Rényi entropy. We believe that our work paves the way for advancing MIA techniques and, consequently, enhancing privacy protection in large foundation models. We can see that in this process, image embeddings are directly obtained from the vision encoder and there are no image tokens. There are no causal relations between consecutive image embeddings as well. Threfore, as we stated in Section 1, target-based MIA that requires token ids cannot be directly applied. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss these in Appendix F.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes] Justification: We discuss the safeguards in Appendix F.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite every reference we use and CC-BY 4.0 is applicable for our work.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: We present the instruction prompts we use to construct the dataset in Section 4 in Table 6 . detect instruction-tuning texts in VLLMs, the model has just repeatedly seen the member data, and the target-based ones are significantly superior to target-free ones. Overall, the phenomena can be explained by our hypothesis.Table 9 : MIA on VLLM instruction-tuning texts. We present the MIA on VLLM instruction-tuning texts with description length (32, 64) . Complete results of Table 3 .Given a sequence of entropies calculated on each position, it would be natural to ask whether the higher percentile or lower percentile should be used to determine the score of the whole sequence for MIA. In the main paper, we use max, which coincides with Min-K% [52] under some special cases. We provide more evidence of the advantages of max or min. We define MinRényi-K% as similar to MaxRényi-K%, but consider the smallest K% entropies in the sequence X.We conduct the pre-training text MIA on LLAVA. We also use the WiKiMIA [52] benchmark of different lengths (32, 64, 128, 256) . We set K = 0, 10, 20, since when K = 100, MaxRényi-K% coincides with MinRényi-K%. We observe that usually MinRényi-K% is slightly inferior to MaxRényi-K%. Therefore, we adopt MaxRényi-K% in our main paper. In this part, we extend VL-MIA/Flickr and VL-MIA/Text size to 2000, which includes 1000 members and 1000 non-members. From Table 12 and Table 13 , we see a similar trend with small-size (600 samples) VL-MIA. Considering this computing source, our main experiments are conducted on small-size datasets.We conduct image MIA on VL-MIA/Flickr with LLaVA through three different instruction texts: "Describe this image concisely.", "Please introduce this painting.", and "Tell me about this image.". We present our results in Table 14 of the appendix. Our pipeline successfully detects member images on every instruction, which indicates robustness across different instruction texts.In Table 16 , we give some examples of our proposed VL-MIA datasets. In Table 17 and Table 18 , we show the corrupted images used in the ablation stuidies (Section 6.5) and the model's descriptions of these corrupted images. In Figure 5 , we present some examples of our VL-MIA/Synthetic dataset.In this section, we provide the True Positive Rate (TPR) at 5% False Positive Rate (FPR) results as supplementary evaluation, as presented in Table 15 . We can see a similar trend in Table 2 . In this part, we investigate the pipelines of vLLMs, to explain why we have access to the text tokens, but only have access to the image embeddings instead of image tokens. The VLLMs consist of a vision encoder, a text tokenizer, and a language model. The vision encoder and the text tokenizer have the same embedding dimension d. When feeding the VLLMs with an image and the instruction, a vision encoder transforms this image to L 1 hidden embeddings of dimension d, denoted by e image ∈ R d×L1 . The text tokenizer first tokenizes the instruction into L 2 tokens, and then looks up the embedding matrix to get its d-dimensional embedding e ins ∈ R d×L2 . The image embedding and the instruction embedding are concatenated as e img-ins = (e image , e ins ) ∈ R d×(L1+L2) , which are then fed into a language model to generate a description that has L 3 tokens. The cross-entropy loss (CE loss) is calculated based on the predicted token id and the ground truth token id on the instruction and description tokens.We can see that in this process, image embeddings are directly obtained from the vision encoder and there are no image tokens. Threfore, as we stated in Section 1, target-based MIAs that require token ids cannot be directly applied.Furthermore, given the ouput logits of the shape (L 1 + L 2 + L 3 ) × |V|, where V is the vocabulary set, we can access the logits of the image by the slice [0 : L 1 ], the logits of instruction by the slice[L 1 : L 1 + L 2 ], and the logits of description by the slice[L 1 + L 2 : L 1 + L 2 + L 3 ].In this paper, we present the first multi-modalities MIA benchmark for VLLMs, and propose a novel metric MaxRényi-K% for MIA. We recognize that our research has significant implications for the safety and ethics of VLLMs and may lead to targeted MIA defense by developers. Nevertheless, our findings provide valuable insights into data contamination, which could contribute to the training data split. Additionally, our method empowers individuals to detect their private data within the training dataset, which is essential for ensuring data security. We believe our work can raise awareness about the importance of privacy protection in multi-modal language models.The first limitation of this work is that the best methods on the pre-training dataset can only achieve an AUC of 0.688. This is because the detection of pre-training data will become more challenging as the VLLMs are further fine-tuned. We believe increasing the performance for detecting pre-training data would be promising in the future. Secondly, the proposed MaxRényi-K% method requires access to the full probability distribution over the predicted tokens. Although we have demonstrated the The image shows a bedroom with a wooden headboard and nightstands on either side of the bed. The bed is made with a white comforter and pillows, and there are two lampsThe image shows a bathroom with cream-colored walls. On the left, there is a vanity with a granite countertop and wooden cabinets below. A soap dispenser is placed on the countertop, andThis image shows a blue pickup truck, which appears to be a Volkswagen Beetle, parked in a driveway in front of a house. The hood of the truck is open, exposing theThe image depicts a well-used kitchen with various cooking utensils and food items scattered throughout. On the left, there is a gas stove with a white oven beneath it. Above the stove VL-MIA/Text for LLaVA 1.5 and LLaMA Adapter v2To enjoy the last two pieces of cake equally and fairly, I suggest using a knife which, according to the image, is already present on the table.To enjoy the remaining pieces of this delectable cake fairly, I would recommend dividing the slices equally among those present, ensuring that each person gets an identical portion, or alternatively, one couldThe young boy is demonstrating the important habit of maintaining good oral hygiene by brushing his teeth. In the image, he is standing in front of a mirror holding a toothbrush, whichIn the image, a young boy is engaging in the imperative habit of brushing his teeth, which is fundamental for maintaining oral hygiene, preventing dental issues like cavities and gum disease, and Motion_Blur The image features a large, orange four-wheel-drive truck driving down a dirt road. The truck is the main focus of the scene, occupying a significant portion of the image. The truck appears to be in motion, possibly on a trail or a dirt road.The image features a large, red pickup truck parked on a dirt road. The truck is equipped with massive tires, giving it a distinctive appearance. The truck is positioned in the middle of the scene, with the front end facing the viewer.The image features a large, bright orange truck with a black front end, parked on a dirt road. The truck is equipped with massive tires, giving it a monster-like appearance. The truck is positioned in the center of the scene, with the front end facing the vie NeurIPS Paper ChecklistQuestion: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We state computer resources-related information in Appendix A.4.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).Justification: We follow the NeurIPS Code of Ethics in every respect.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).https://platform.openai.com/docs/modelshttps://www.flickr.com/1-α log j (p j ) α , 0 < α < ∞, α ̸ = 1. H α (p) is further defined at α = 1, ∞, as H α (p) = lim γ→α H γ (p) by,BLIP2/eva_vit_g uses MS COCO as one of the pre-training datasets.A concatenation of LAION400M, COYO, MMC4, SBU, Conceptual Captions, and COCO.Including LLaVA Instruct 150k and MS COCO.https://github.com/hendrycks/robustness/blob/master/ImageNet-C/create_c/make_ cifar_c.py 


Decision: The paper introduces the first benchmark specifically for MLA on LVLMs and proposes novel metrics and methods for detecting training data usage. Reviewers found the research problem significant and the approach innovative, appreciating the extensive experiments and novel benchmarks. However, concerns were raised about dataset size, the clarity of some methodological choices, and the practical applicability of the results. The authors' rebuttal provided additional datasets, expanded evaluations, and further clarifications, addressing most concerns satisfactorily. The recommendation is to accept.

Reviews: review: paper_summary: paper_summary: The rise of large vision-language models (VLLMs) has significantly advanced multi-modal tasks but also brought forth concerns about data security and privacy. This paper introduces a novel membership inference attack (MIA) benchmark specifically designed for VLLMs to detect training data, addressing the lack of standardized datasets and methodologies in this domain. The authors propose a new MIA pipeline for token-level image detection and introduce the MaxRényi-K% metric for improved detection. The key contributions include the development of the first VLLM-specific MIA benchmark, a cross-modal MIA pipeline for individual image or description detection, and the new MaxRényi-K% and ModRényi metrics, which show effectiveness in experiments.
strength_weakness: strength_weakness: - **Small Evaluation Dataset**: The MIA evaluation dataset consists of around 600 images for each evaluation, which is relatively small. This limited dataset size can lead to less statistically robust results and may not fully capture the variability and challenges present in real-world scenarios. It would strengthen the paper to include evaluations on larger and more diverse datasets.

- **Lack of Standard Metrics**: The paper does not consider the _TPR at low FPR_ (True Positive Rate at low False Positive Rate) metric, which is a standard for evaluating worst-case membership privacy. Including this metric would provide a more comprehensive assessment of the model's privacy risks, especially in high-stakes applications where false positives must be minimized.

- **Generalizability**: While the proposed metrics and methods show effectiveness, it's important to discuss their generalizability to other types of vision-language models and datasets. Providing insights or experiments on different architectures or domains could enhance the applicability of the findings.
questions: questions: - How to choose $K$ for the MaxRényi-K% metric?
limitations: limitations: The size and the diversity of the benchmark is limited.
paper_summary: paper_summary: The paper introduced a benchmark for membership inference attack on VLMs, proposed a pipeline for token-level image detection, and proposed a target-free metric for image MIA detections. The pipeline relies on the fixed sequence of the VLM output to obtain the output image, instruction, and description segments of logits and use them for evaluation metrics.
strength_weakness: strength_weakness: 1. The explanation of some concepts are not clear. For example, when proposing target-free MIA metrics, the author mentioned that it's because we only have access to the image embeddings but not image tokens. How are these two terms defined here? Why do we not have access to the image tokens? Additionally, image tokens are defined again in line 95. If this is not available, why does the paper need to define the concept again? I think the authors should clarify these assumptions properly in the main text.
questions: questions: 1. For the cross-modal pipeline for detecting images, how do you determine which logits are for image, instructions, or description text, during the attack phase?
2. The proposed metrics can have different K and $\alpha$ values. Are there any ablation studies on when should what K and $\alpha$ values being used?
limitations: limitations: The authors clarified the limitations in the paper.
paper_summary: paper_summary: This paper presents an interesting idea of detecting training data in large vision-language models (VLLMs) through membership inference attacks (MIAs). The authors introduce the following interesting points :

- An MIA benchmark, specifically designed for VLLMs, called Vision Language MIA (VL-MIA). This benchmark includes tasks for both image and text detection across various VLLM architectures.
- A pipeline for MIAs on VLLMs. This approach does not require both side information as in the prior work. This includes a new detection metric called MaxRényi-K%.
- Extensive evaluation results. The authors demonstrate the effectiveness of their approach to diverse VLLMs.
strength_weakness: strength_weakness: This paper requires further clarification regarding the proposed methods and evaluation results. Please see the following questions.
questions: questions: Regarding dataset generation:
If the member data is drawn from LAION_CCS, and non-member data is drawn from the generated dataset, is there any underlying bias caused by the nature of differences between natural and synthetic data (in VL-MIA/DALL-E)? Moreover, why did we choose instruction text for members and generated text for non-members in VT-MIA/Text?

Regarding the evaluation:
Based on Table 2, it seems that different datasets show different behaviors. For example, for VL-MIA/Flickr, Max_100% shows the best performance in most cases, and the performance gap is quite large. On the other hand, we cannot see a similar observation from VL-MIA/DALL-E. I wonder whether the authors have some insight into why it shows a clear difference between datasets. 
Moreover,  it seems that we need to vary a lot of parameters to find better performance (e.g., different alpha, and different K). How did the authors choose the hyperparameters from other baselines? Since the authors present a new benchmark, it might not be a good choice to use optimal hyperparameters from other sources.
limitations: limitations: The authors discuss the limitations and broader impacts in the appendix.
paper_summary: paper_summary: This paper focuses on the membership inference attack (MIA) of large vision language models. It creates multiple MIA datasets and proposes a new method to identify whether an image or a text belongs to the training dataset.
strength_weakness: strength_weakness: The setting for detecting a description sentence is a little confusing for me. Why do you feed the model with a black image rather than others, like those generated by DALL-E? Also, why do you skip the instructions here?  It is less possible to produce the target description for the target VLM just based on a black image and an empty instruction. Why don't skip the input image?
questions: questions: 1. As illustrated in the weakness section, could you please explain the setting of MIA in the description sentence?
2. How to set the threshold value \lambda in the MIA process?
limitations: limitations: The paper has analyzed its limitations in the Appendix, which I believe is reasonable.

Comments: title: Response to authors
response to reviewer e87W
response to authors
Thank you
Clarification on the assumption
Thank you for the response
comment: Thanks for the detailed response. Although some presentations can still be revised, it is an interesting paper on its threat model design. Therefore, I will maintain my current score.
We appreciate the insightful feedback from reviewer e87W. Note that our results across all datasets exhibit a consistent trend. Specifically, in Table 4 of the rebuttal PDF, our proposed Rényi metric demonstrates a considerable improvement in AUC for the newly constructed synthetic dataset: from 0.65 to 0.69 with the description slice, and from 0.55 to 0.65 with the instruction+description slice, compared to the previous baselines. From Table 2 of our paper, we also notice that "Rényi ($\alpha = 0.5$)" generally outperforms previous baselines in image MIA for both Flickr and Dalle-E datasets.

Regarding the choice of datasets, we believe **both** the synthetic dataset and the Flickr dataset **should be considered**. Previous literature in machine learning [1,2,3] suggest that evaluating across multiple datasets, such as MNIST, CIFAR-10, ImageNet, and synthetic (Gaussian) data, can demonstrate its effectiveness in various scenarios. In our benchmark, the synthetic and real-world dataset have their own benefit. The advantage of our synthetic geometry and password dataset is that it adheres to the i.i.d. assumption of member and non-member data, and the membership can be fully determined. Meanwhile, the Flickr dataset aligns more closely with the real-world data distributions. We verify our method's performance in both synthetic and real-world datasets. 

---
Reference:

[1] "Membership inference attacks against machine learning models." 2017.

[2] "Descending through a crowded valley-benchmarking deep learning optimizers." 2021.

[3] "ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data." 2023.
Thank the authors for your efforts in responding to questions and conducting additional experiments. I still have some questions as follows:

In the Text MIA experiments along with DALL-E, the authors compare generated data and real-world data, which might already contain underlying biases that could affect performance. I understand that the authors also used Flickr by splitting the data between the target model's release date, and it has even shown better performance with the authors' proposed method (e.g., Flickr has consistently outperformed results from Renyi) than others. Also, as shown in the additional results from the PDF, the performance difference on synthetic data seems marginal.

In this case, which dataset should we rely on more if the two datasets show different trends?

For example, if we can find one real-world text pair (before and after the target models' release date), and it also shows better performance with the proposed method, as presented in image experiments, why is there a difference? Which one should we trust more if the authors want to present a benchmark?
Thank you for the reply. It appears that the threshold-based MIAs rely on somewhat strong assumptions, particularly the need for a validation dataset, which may limit their broader applicability. However, I still believe this paper offers valuable new insights into the privacy issue of vision-language models.

I will maintain my current score.
We thank reviewer M6Nm for the meaningful feedback. Regarding the concern that our method assumes that the attacker has collected some member and non-member data in advance, we explain that this assumption is practical in real-world scenarios. For open-source models trained on open-source data, we can confidently obtain both member and non-member data, as demonstrated in this paper's Section 4. For closed-source models, such as GPT4, we speculate that commonly used datasets, such as MS COCO, and a wide collection of copyrighted materials [1], are used as training data by these closed-source models. 

Note that all of the baseline methods (e.g., PPL, min-k) also require a validation set to estimate a threshold ($\lambda$ in Equation 1) to determine membership. The above choice of member and non-member dataset can be universally used for all MIA methods.  

We would like to add these clarifications after line 305 with the revised version to enhance understanding on member and non-member data selection.

Refs: 

[1] "Speak, memory: An archaeology of books known to chatgpt/gpt-4." arXiv 2023.

---
Remark: We have further **extended our benchmark size to 10k** and observed similar results, which will be released in the final version. In addition, our heuristic findings in the paper suggest that for text MIA, one can select a smaller k, e.g.,0 or 10; while for image MIA, one can select k=100.
Thanks for the response and the updated results.

Using a validation dataset to tune hyperparameters is a reasonable approach. However, this method assumes that the attacker has collected some member and non-member instances in advance. I’m curious about how practical this assumption is in real-world scenarios. I would appreciate it if the authors could provide further clarification on this point.