Title: Membership Inference Attacks against Large Vision-Language Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Membership inference attack; Large Vision-Language Models
Abstract: Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records, in their training datasets. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue, mainly due to the lack of standardized datasets and suitable methodologies. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new metric called MaxRényi-K%, which is based on the confidence of the model output and applies to both text and image data. We believe that our work can deepen the understanding and methodology of MIAs in the context of VLLMs. Our code and datasets are available at https://github.com/LIONS-EPFL/VL-MIA.

Membership Inference Attacks against Large Vision-Language Models

Abstract

Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. However, their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records, in their training datasets. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue, mainly due to the lack of standardized datasets and suitable methodologies. In this study, we introduce the first membership inference attack (MIA) benchmark tailored for various VLLMs to facilitate training data detection. Then, we propose a novel MIA pipeline specifically designed for token-level image detection. Lastly, we present a new metric called MaxRényi-K% , which is based on the confidence of the model output and applies to both text and image data. We believe that our work can deepen the understanding and methodology of MIAs in the context of VLLMs. Our code and datasets are available at https://github.com/LIONS-EPFL/VL-MIA.

1 Introduction

The rise of large language models (LLMs) [9, 60, 45, 11] has inspired the exploration of large models across multi-modal domains, exemplified by advancements like GPT-4 [1] and Gemini [59]. These large vision-language models (VLLMs) have shown promising ability in various multi-modal tasks, such as image captioning [33], image question answering [13, 35], and image knowledge extraction [26]. However, the rapid advancement of VLLMs also causes user concerns about privacy and knowledge leakage. For instance, the image data used during commercial model training may contain private photographs or medical diagnostic records. This is concerning since early work has demonstrated that machine learning models can memorize and leak training data [3, 56, 63]. To mitigate such concerns, it is essential to consider the membership inference attack (MIA) [23, 53], where attackers seek to detect whether a particular data record is part of the training dataset [23, 53]. The study of MIAs plays an important role in preventing test data contamination and protecting data security, which is of great interest to both industry and academia [24, 19, 44].

When exploring MIAs in VLLMs, one main issue is the absence of a standardized dataset designed to develop and evaluate different MIA methods, which comes from the large size [16] and multimodality of the training data, and the diverse VLLMs training pipelines [66, 35, 18]. Therefore, one of the main goals of this work is to build an MIA benchmark tailored for VLLMs.

Beyond the need for a valid benchmark, we lack efficient techniques to detect a single modality in VLLMs. The closest work to ours is [30], which performs MIAs on multi-modal CLIP [46] by detecting whether an image-text pair is in the training set. However, in practice, it is more common

An adventurous

skier skillfully

moved down the

steep, snowy

slope at a high

rate of speed.

Target image

Z

Target VLLM

A skier descending a steep,

snow-covered slope.

Generated text (description)

X'

des

+

Describe this image.

+

Describe this image.

+

Instruction

X

ins

description

…

…

…

…

…

…

…

…

…

…

…

…

MaxRényi-K%

Generation stage

Inference stage

Member

Non-member

✅

❌

A skier descending a steep,

snow-covered slope.

…

…

Target image

Z

Target text

X

des

Position: Rényi entropy

Position: Rényi entropy

sort

1) Rényi entropy of each position     2) select largest K% Rényi entropy

Rényi entropy

Member

Non-member

✅

❌

logits

instruction

image

OO

1

13

2

5

3

4

20

16

to detect a single modality, as we care whether an individual image or text is in the training set. Therefore, we aim to develop a pipeline to detect the single modality from a multi-modal model. Moreover, existing literature on language model MIAs, such as Min-K% [52] and Perplexity [62], mostly are target-based MIAs, which use the next token as the target to compute the prediction probability. However, we can only access the image embedding instead of the image token in VLLMs, and thus only target-free MIAs [48] can be directly applied.

Therefore, we first propose a cross-modal pipeline for individual image or description MIAs on VLLMs, which is distinguished from traditional MIAs that only use one modality [61, 62]. We feed the VLLMs with a customized image-instruction pair from the target image or description. We show that we can perform the MIA not only by the image slice but also by the instruction and description slices of the VLLM's output logits, see Figure 1. Such a cross-modal pipeline enables the usage of text MIA methods on image MIAs. We also introduce a target-free metric that adapts to both image and text MIAs and can be further modified to a target-based way.

Overall, the contributions and insights can be summarized as follows.

We release the first benchmark tailored for the detection of training data in VLLMs, called V ision L anguage MIA ( VL-MIA ) (Section 4). By leveraging Flickr and GPT-4, we construct VL-MIA that contains two images MIA tasks and one text MIA task for various VLLMs, including MiniGPT-4 [66], LLaVA 1.5 [35] and LLaMA-Adapter V2 [18].

We perform the first individual image or description MIAs on VLLMs in a cross-modal manner. Specifically, we demonstrate that we can perform image MIAs by computing statistics from the image or text slices of the VLLM's output logits (Figure 1 and Section 5.1).

We propose a target-free MIA metric, MaxRényi-K% , and its modified target-based ModRényi (Section 5.2). We demonstrate their effectiveness on open-source VLLMs and closed-source GPT-4 (Section 6). We achieve an AUC of 0.815 on GPT-4 in image MIAs.

2 Related work

Membership Inference Attack (MIA) aims to classify whether a data sample has been used in training a machine learning model [53]. Keeping training data confidential is a desired property for

machine learning models, since training data may contain private information about an individual [62, 24]. Popular MIA methods can be divided into metric-based and shadow model-based MIAs [24]. Metric-based MIAs [62, 48, 57, 52] determine whether a data sample has been used for training by comparing metrics computed from the output of the target model with a threshold. Shadow model-based MIAs need shadow training to mimic the behavior of the target model [53, 48], which is computationally infeasible for LLMs. Therefore, we focus on the metric-based methods in this work.

MIAs have been extensively researched in various machine learning models, including classification models [38, 58, 10], generative models [20, 22, 7], and embedding models [54, 40]. With the emergence of LLMs, there are also a lot of work exploring MIAs in LLMs [42, 17, 52, 41]. Nevertheless, MIAs for multi-modal models have not been fully explored. [30, 25] perform MIAs using the similarity between the image and the ground truth text, which detects the image-text pair instead of a single image or text sequence. However, detecting an individual image or text is more practical in real-world scenarios and poses additional challenges. To the best of our knowledge, we are the first to perform the individual image or text MIA on VLLMs.

In this paper, we also consider a more difficult task, to detect the pre-training data from the fine-tuned models, that is, to detect the base LLM pertaining data from VLLMs. First, compared with detecting fine-tuning data [55, 51], pretraining data come from a much larger dataset and are used only once, reducing the potential probability for a successful MIA [27, 32]. In addition, compared to the detection of pretraining data from the pre-trained models [53, 52], catastrophic forgetting [29, 28, 16] in the fine-tuning stage also makes it harder to detect the pre-training data from downstream models. To the best of our knowledge, we are the first to perform pre-training data MIAs on fine-tuned models.

Large Vision-Language Models (VLLMs) incorporate visual preprocessors into LLMs [9, 60, 45] to manage tasks that require handling inputs from text and image modalities. A foundational approach in this area is represented by CLIP [46], which established techniques for aligning modalities between text and images. Further developments have integrated image encoders with LLMs to create enhanced VLLMs. These models are typically pre-trained on vast datasets of image-text pairs for feature alignment [33, 64, 39, 2], and are subsequently instruction-tuned for specific downstream tasks to refine the end ability. MiniGPT [66, 8], LLaVA [36, 35], and LLaMA Adapter [65, 18] series have demonstrated significant capabilities in understanding and inference in this area.

3 Problem setting

In this section, we introduce the main notation and problem settings for MIAs.

Notation. The token set is denoted by V . A sequence with L tokens is denoted by X := ( x 1 , x 2 , . . . , x L ) , where x i ∈ V for i ∈ [ L ] . Let X 1 ⊕ X 2 be the concatenation of sequence X 1 and X 2 . An image token sequence is denoted by Z . In this work, we focus on the VLLM, parameterized by θ , where the input is the image Z followed by the instruction text X ins, and the output is the description text X des. We use D des and D image to represent the description training set and image training set, respectively. Detailed notations are summarized in Table 5 of the appendix.

Attacker's goal. In this work, the purpose of the attacker is to detect whether a given data point (image Z or description X des) belongs to the training set. We formulate this attack as a binary classification problem. Let A image ( Z ; θ ) : →{ 0 , 1 } and A des ( X ; θ ) : →{ 0 , 1 } be two binary classification algorithms for image and description respectively, which are implemented by comparing the metric Score ( Z ⊕ X ins ⊕ X des ; θ ) with some threshold λ .

When detecting image Z , we feed the model with the target image with a fixed instruction prompt such as 'Describe this image in detail', denoted as X ins. The model then generates the description text X ′ des . The algorithm A image ( Z ; θ ) is defined by

When detecting a description sequence X des, we feed the model with an all-black image, denoted as Z ept, as the visual input, followed by an empty instruction X ept. The algorithm A des ( X des ; θ ) is defined by

Attacker's knowledge. We assume a grey-box setting on the target model, where the attacker can query the model by a custom prompt (including an image and text) and have access to the tokenizer, the output logits, and the generated text. The attacker is unaware of the training algorithm and parameters of the target model.

4 Dataset construction

We construct a general dataset: V ision L anguage MIA ( VL-MIA ), based on the training data used for popular VLLMs, which, to our knowledge, is the first MIA dataset designed specifically for VLLMs. We present a takeaway overview of VL-MIA in Table 1. We also provide some examples in VL-MIA, see Table 16 in the appendix. The prompts we use for generation can be found in Table 6.

Target models. We perform MIAs open-source VLLMs, including MiniGPT-4 [66], LLaVA-1.5 [35], and LLaMA-Adapter V2.1 [18]. The checkpoints and training datasets of these models are provided and public. The training pipeline of a VLLM encompasses several stages. Initially, an LLM undergoes pre-training using extensive text data such as LLaMA [18]. Meanwhile, a vision preprocessor, e.g., CLIP [46], is pre-trained on a large number of image-text pairs. Subsequently, a VLLM is constructed based on the LLM and the vision preprocessor and is pre-trained using image-text pairs. The final stage involves instruction-tuning the VLLM, which can be performed using either image-text pairs or image-based question-answer data. In the instruction tuning stage of a VLLM, every data entry contains an image, a question, and the corresponding answer to the image. We use the answer text as member data and GPT-4 generated answers under the same question and same image as non-member data. Specifically, for LLaVA 1.5 and LLaMA-Adapter v2, we use the answers in LLaVA 1.5's instruction tuning as member data.

VL-MIA/DALL-E. MiniGPT-4, LLaVA 1.5, and LLaMA-Adapter V2 use images from LAION [49], Conceptual Captions 3M [6], Conceptual 12M [6] and SBU captions [43] datasets (collectively referred to as LAION-CCS) as pre-training data. BLIP [33] provides a synthetic dataset with imagecaption pairs for LAION-CCS used in MiniGPT-4 and LLaVA 1.5. We first detect the intersection of the training images used in these three VLLMs. From this intersection, we randomly select a subset to serve as the member data for our benchmark. For the non-member data, we use the corresponding BLIP captions of the member images as prompts to generate images with DALL-E 2 3 . This process yields one-to-one corresponding pairs of the generated images (non-member) and the original member images. Consequently, our dataset comprises an equal number of member images from the LAION-CCS dataset and non-member images generated by DALL·E, allowing us to evaluate MIA performance comprehensively. We have 592 images in VL-MIA/DALL-E in total.

VL-MIA/Flickr. MS COCO [34] co-occurs as a widely used dataset in the training data of the target models, so we use the images in this dataset as member data. Given the fact that such member data are collected from Flickr 4 , we filter Flickr photos by the year of upload and obtain new photos from January 1, 2024, as non-member data, which are later than the release of the target models. We additionally prepare a set of corrupted versions, where the member images are deliberately corrupted

to simulate real-world settings. More results of the corrupted versions are discussed in Section 6.5. This dataset contains 600 images.

VL-MIA/Text. Weprepare text MIA datasets for the VLLMs instruction-tuning stage. LLaVA 1.5 and LLaMA Adapter v2.1 both use the LLaVA-Instruct-150K [35] in instruction-tuning, which consists of multi-round QA conversations. We first select entries with descriptive answers of 64 words. Next, we feed the corresponding questions and images into GPT-4 3 , and ask GPT-4 to generate responses of the same length, treating these generated responses as non-member text data. In addition, since MiniGPT4 employs long descriptions of images for instruction-tuning, typically beginning with 'the image', we prompt GPT to generate descriptions based on the MS COCO dataset, starting with 'this image' to ensure similar data distributions. We also prepare different versions of the datasets by truncating the text into different word lengths such as 16, 32, and 64. Each text dataset contains 600 samples.

VL-MIA/Synthetic We synthesize two new MIA datasets: VL-MIA/Geometry and VLMIA/Password . The image in the VL-MIA/Geometry consists of a random 4x4 arrangement of geometrical shapes, and the image in the VL-MIA/Password consists of a random 6x6 arrangement of characters and digits from MNIST [15] and EMINST [12]. The associated text for each image represents its content (e.g., specific characters, colors, or shapes), ordered from left to right and top to bottom. Half of the dataset can be selected as the member set for VLLM fine-tuning, with the remainder as the non-member set. This partition ensures that members and non-members are independently and identically distributed, avoiding the latent distribution shift between the members and non-members in current MIA datasets [14]. Our synthetic datasets are ready to use, and can be applied to evaluate any VLLM MIA methods through straightforward fine-tuning. We provide some examples of this dataset in Figure 5 in the Appendix C.

5 Method

5.1 A cross-modal pipeline to detect image

VLLMs such as LLaVA and MiniGPT project the vision encoder's embedding of the image into the feature space of LLM. However, a major challenge for image MIA is that we do not have the ground-truth image tokens. Only the embeddings of images are available, which prevents directly transferring many target-based MIA from languages to images. To this end, we propose a token-level image MIA which calculates metrics based on the output logit of each token position.

This pipeline consists of two stages, as demonstrated in Figure 1. In generation stage , we provide the model with an image followed by an instruction to generate a textual sequence. Subsequently, in inference stage , we feed the model with the concatenation of the same image, instruction, and generated description text. During the attack, we correspondingly slice the output logits into image, instruction, and description segments, which we use to compute various metrics for MIAs. Our pipeline considers the information from the image, the instructions and the descriptions following the image. In practice, even if there is no access to the logits of the image feature and instruction slice, we can still detect the member image solely from the model generation. We visually describe a prompt example with different slice notations presented in Appendix A.2.

Our pipeline operates on the principle that VLLMs' responses always follow the instruction prompt [60], where the images usually precede the instructions and then always precede the descriptions. For causal language models used in VLLMs that predict the probability of the next token based on the past history [45], the logits at text tokens in the sequence inherently incorporate information from the preceding image.

5.2 MaxRényi MIA

We propose our MaxRényi-K% , utilizing the Rényi entropy of the next-token probability distribution on each image or text token. The intuition behind this method is that if the model has seen this data before, the model will be more confident in the next token and thus have smaller Rényi entropy.

̸

Given a probability distribution p , the Rényi entropy [47] of order α , is defined as H α ( p ) = 1 1 -α log ( ∑ j ( p j ) α ) , 0 < α < ∞ , α = 1 . H α ( p ) is further defined at α = 1 , ∞ , as H α ( p ) = lim γ → α H γ ( p ) by,

To be more specific, given a token sequence X := ( x 1 , x 2 , . . . , x L ) , let p ( i ) ( · ) = P ( ·| x 1 , · · · , x i ) be the probability of next-token distribution at the i -th token. Let Max-K% ( X ) be the top K% from the sequence X with the largest Rényi entropies, the MaxRényi-K% score of X equals

When K = 0 , we define the MaxRényi-K% score to be max i ∈ [ L -1] H α ( p ( i ) ) . When K = 100 , the MaxRényi-K% score is the averaged Rényi entropy of the sequence X .

In our experiments, we vary α = 1 2 , 1 , 2 , and + ∞ ; K = 0 , 10 , 100 . As α increases, the top percentile of distribution p will have more influence on H α ( p ) . When α = 1 , H 1 ( p ) equals the Shannon entropy [50], and our method at K = 100 is equivalent to the Entropy [48]. When α = ∞ , we consider the most likely next token probability [31]. In contrast, Min-K% [52] only deals with the target next token probability. When the sequence is generated by the target model deterministically, i.e., when the model always generates the most likely next token, our MaxRényi-K% at α = ∞ is equivalent to the Min-K% .

̸

We also extend our MaxRényi-K% to the target-based scenarios, denoted by ModRényi . We first consider linearized Rényi entropy, H α ( p ) = 1 1 -α ( ∑ j ( p j ) α -1 ) , 0 < α < ∞ , α = 1 . H α ( p ) is also further defined at α = 1 , as H 1 ( p ) = lim α → 1 H α ( p ) = H 1 ( p ) . Assuming the next token ID is y , recall that a small entropy value or a large p y value indicates membership, we want our modified entropy to be monotonically decreasing on p y and monotonically increasing on p j , j = y . Therefore, we propose the modified Rényi entropy on a given next token ID y , denoted by H α ( p, y ) :

̸

̸

̸

Let α → 1 , we have H 1 ( p, y ) = lim α → 1 H α ( p, y ) = -∑ j = y p j log(1 -p j ) -(1 -p y ) log p y , which is equivalent to the Modified Entropy [58]. In addition, our more general method does not encounter numerical instability in Modified Entropy as p j → 0 , 1 at α = 1 . For simplicity, we let the ModRényi score be the averaged modified Rényi entropy of the sequence.

6 Experiments

In this section, we conduct MIAs across three target models using various baselines, MaxRényi-K% , and ModRényi . Experiment setup is provided in Section 6.1. The results on text MIAs and image MIAs are present in Section 6.2 and Section 6.3, respectively. In Section 6.4, we show that the proposed MIA pipeline can also be used in GPT-4. Ablation studies are present in Section 6.5. The versions and base models of VLLMs we use are listed in Table 7 of the appendices.

6.1 Experimental setup

Evaluation metric. We evaluate different MIA methods by their AUC scores. AUC score is the area under the receiver operating characteristic (ROC) curve, which measures the overall performance of a classification model in all classification thresholds λ . The higher the AUC score, the more effective the attack is. In addition to the average-case metric AUC, we also include the worst-case metric, the True Positive Rate at 5% False Positive Rate (TPR@5%FPR) in Appendix D suggested by [5].

Baselines. We take existing metric-based MIA methods as baselines and conduct experiments on our benchmark. We use the MIA method from [37], which compares the feature vectors produced by the original image with the augmented image. We use KL-divergence to compare the logit distributions and term it Aug-KL in this paper. We also use Loss attack [62], which is perplexity in the case of language models. Furthermore, we consider ppl/zlib and ppl/lowercase [4], which compare the target perplexity to zlib compression entropy and the perplexity of lowercase texts respectively. [52] proposes Min-K% method, which calculates the smallest K % probabilities corresponding to the

̸

ground truth token. Min-K% is currently a state-of-the-art method to detect pre-training data of LLMs. For both Min-K% and MaxRényi-K% , we vary K = 0 , 10 , 100 . In addition, we consider K = 20 for Min-K% as suggested in [52]. We further include the max_prob_gap metric that can represent the extreme confidence in certain tokens by the model. That is, we subtract the second largest probability from the maximum probability in each token position and calculate the mean as metric.

6.2 Image MIA

We first conduct MIAs on images using VL-MIA/Flickr and VL-MIA/DALL-E in three VLLMs. For the image slice, it is not possible to perform target-based MIAs, because of the absence of ground-truth token IDs for the image. However, our MIA pipeline presented in Figure 1 can still handle target-based metrics by accessing the instruction slice and description slice.

As demonstrated in Table 2, MaxRényi-K% surpasses other baselines in most scenarios. An α value of 0.5 yields the best performance in both VL-MIA/Flickr and VL-MIA/DALL-E. As α increases, performance becomes erratic and generally deteriorates, though it remains superior to all target-based metrics. Overall, target-free metrics outperform target-based metrics for image MIAs. Another interesting observation is that instruction slices result in unstable AUC values, sometimes falling below 0.5 in target-based MIAs. This can be partially explained by the fact the model is more familiar with the member data. As a result, after encountering the first word 'Describe', the model is more inclined to generate the description directly than generating the following instruction of X ins , i.e., 'this image in detail'. This is an interesting phenomenon that we leave to future research.

The performance of the image MIA model is influenced by its training pipelines. Recall that MiniGPT4 only updates the parameters of the image projection layer in image training, and LLaMA Adapter v2 applies parameter-efficient fine-tuning approaches. In contrast, LLaV A 1.5 training updates both the parameters of the projection layer and the LLM. The inferior performance of MIAs on MiniGPT-4 and LLaMA Adapter compared to LLaVA 1.5 is therefore consistent with [52] that more parameters' updates make it easier to memorize training data.

We find that VL-MIA/DALL-E is a more challenging dataset than VL-MIA/Flickr, reflected in the AUC being closer to 0.5. In VL-MIA/DALL-E, each non-member image is generated based on the description of a member image. Therefore, member data have a one-to-one correspondence with non-member data and depict a similar topic, which makes it harder to discern.

6.3 Text MIA

Text member data might be used in different stages of VLLM training, including the base LLM model pre-training and the later VLLM instruction-tuning. We hypothesize that after the last usage of the member data in its training, the more the model changes, the better the targetfree MIA methods compared to targetbased ones, and vice-versa. The heuristic is that if the model's parameters have changed a lot, target-free MIA methods, which use the whole distribution to compute statistics, are more robust than target-based methods, which rely on the probability at the next token ID. On the other hand, if the member data are seen in recent fine-tuning, the next token will convey more causal relations in the sequence remembered by the model, and thus target-based ones are better.

VL-MIA/DALL-E

MIA on VLLM instruction-tuning texts We detect whether individual description texts appear in the VLLMs instruction-tuning. We use the description text dataset VL-MIA/Text of lengths (32, 64), constructed in Section 4. We present our results in the first column of Table 3. We observe that target-based MIA methods are significantly better than target-free ones, confirming our hypothesis.

MIA on LLM pre-training texts. We use the WikiMIA benchmark [52], which leverages the Wikipedia timestamp to separate the early Wiki data as the member data, and recent Wiki data as the non-member data. The early Wiki data are used in various LLMs pre-training [60]. We use WikiMIA of different lengths (32, 64, 128, 256), and expect the membership of longer sequences will be more easily identified. We present our results in the second column of Table 3. We observe that on LLaV A, our target-free MIA methods on large α consistently outperform target-based MIA methods, which

again confirms our hypothesis since the base LLM model of LLaVA has full parameter fine-tuning from LLaMA-2 and thus changed a lot.

6.4 Image MIA on GPT-4

In this section, we demonstrate the feasibility of image MIAs on the closed-source model GPT-4. Our experiments use two image datasets: VL-MIA/Flickr and VLMIA/DALL-E, detailed in Section 4. We choose GPT-4vision-preview API, which was trained in 2023 and likely does not see the member data in either dataset. We randomly select 200 images per dataset and prompt GPT-4 to describe them in 64 words. We then apply MIAs based on the generated descriptions. Since GPT-4 can only provide the top-five probabilities at each token position, we can not directly use the proposed MaxRényi-K% that requires the whole probability distribution. To address this issue, we assume the size of the entire token set is 32000 and the probability of the remaining 31995 tokens are uniformly distributed. The AUC results are present in Table 4. We omit the result of perplexity and Min-K% since they are equivalent to MaxRényi-K% with α = ∞ in the greedygenerated setting, as discussed in Section 5.2. Surprisingly, we observe that in VL-MIA/DALL-E, the best-performed

method MaxRényi-K% ( α = 0 . 5 ) can achieve an AUC of 0 . 815 . This indicates a high level of effectiveness for MIAs on GPT-4, demonstrating the potential risks of privacy leakage even with closed-source models.

6.5 Ablation study

Does the length of description affect the image MIA performance? We conduct ablation experiments on LLaVA 1.5 targeting the length of generated description texts with MaxRényi-10% . In the generation stage, we restrict the max_new_tokens parameter of the generate function to (32, 64, 128, 256) to obtain description slices of different lengths. As presented in Figure 2a, when the length of the description increases, the AUC of the MIA becomes higher and enters a plateau when max_new_tokens reaches 128. This may be because a shorter text contains insufficient information about the image, and in an excessively long text, words generated later tend to be more generic and not closely related to the image, thereby contributing less to the discriminative information that helps discern the membership.

Can we still detect corrupted member images? The motivation is to detect whether sensitive images are inappropriately used in VLLM's training even when the images at hand may get corrupted. We leverage ImageNet-C [21] to generate corrupted versions of member data in VL-MIA/Flickr: Snow , Brightness , JPEG , and Motion_Blur , with the parameters in Table 8. The corrupted examples and corresponding model output generations are demonstrated in Appendix C Table 17 and Table 18. We take MaxRényi-K% ( α = 0.5) as the attacker and the results of LLaV A are presented in Figure 2b. Corrupted member images make MIAs more difficult, but can still be detected successfully. We also observe that reducing model quality ( JPEG ) or adding blur ( Motion_Blur ) degrade MIA performance more than changing the base parameter ( Brightness ) or overlaying texture ( Snow ).

Can we use different instructions? We conduct image MIAs on VL-MIA/Flickr with LLaVA through three different instruction texts: 'Describe this image concisely.', 'Please introduce this painting.', and 'Tell me about this image.'. We present our results in Table 14 of the appendix. Our pipeline successfully detects member images on every instruction, which indicates robustness across different instruction texts.

16

32

64

128

256

max_new_tokens

0.60

0.62

0.64

0.66

0.68

0.70

0.72

AUC

Perplexity

Rényi (

= 0.5)

Rényi (

= 1)

Rényi (

= 2)

Rényi (

= inf)

(a) Ablation study on max_new_tokens .

Snow

Brightness

JPEG

Motion_Blur

Corruption

0.50

0.55

0.60

0.65

0.70

0.75

0.80

AUC

Marginal

Moderate

Severe

(b) MaxRényi-K% on corrupted images.

7 Conclusion

In this work, we take an initial step towards detecting training data in VLLMs. Specifically, we construct a comprehensive dataset to perform MIAs on both image and text modalities. Additionally, we uncover a new pipeline for conducting MIA on VLLMs cross-modally and propose a novel method based on Rényi entropy. We believe that our work paves the way for advancing MIA techniques and, consequently, enhancing privacy protection in large foundation models.

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023.

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966 , 2023.

Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19) , pages 267-284, 2019.

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2633-2650, 2021.

Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP) , pages 1897-1914. IEEE, 2022.

Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3558-3568, 2021.

Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. Gan-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security , pages 343-362, 2020.

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechu Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478 , 2023.

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) , 2(3):6, 2023.

Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Labelonly membership inference attacks. In International conference on machine learning , pages 1964-1974. PMLR, 2021.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research , 24(240): 1-113, 2023.

Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 international joint conference on neural networks (IJCNN) , pages 2921-2926. IEEE, 2017.

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems , 36, 2024.

Debeshee Das, Jie Zhang, and Florian Tramèr. Blind baselines beat membership inference attacks for foundation models. arXiv preprint arXiv:2406.16201 , 2024.

Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine , 29(6):141-142, 2012.

Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841 , 2024.

Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. Practical membership inference attacks against fine-tuned large language models via self-prompt calibration. arXiv preprint arXiv:2311.06062 , 2023.

Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010 , 2023.

Shahriar Golchin and Mihai Surdeanu. Time travel in llms: Tracing data contamination in large language models. In The Twelfth International Conference on Learning Representations , 2023.

Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. Logan: Membership inference attacks against generative models. arXiv preprint arXiv:1705.07663 , 2017.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations , 2019.

Benjamin Hilprecht, Martin Härterich, and Daniel Bernau. Monte carlo and reconstruction membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies , 2019.

Nils Homer, Szabolcs Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLoS genetics , 4(8):e1000167, 2008.

Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Membership inference attacks on machine learning: A survey. ACMComputing Surveys (CSUR) , 54(11s):1-37, 2022.

Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, and Minhui Xue. M 4 i: Multi-modal models membership inference. Advances in Neural Information Processing Systems , 35:1867-1882, 2022.

Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. Bliva: A simple multimodal llm for better handling of text-rich visual questions. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 2256-2264, 2024.

Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning , pages 1069710707. PMLR, 2022.

Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.

Myeongseob Ko, Ming Jin, Chenguang Wang, and Ruoxi Jia. Practical membership inference attacks against large-scale multi-modal models: A pilot study. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4871-4881, 2023.

Robert Konig, Renato Renner, and Christian Schaffner. The operational meaning of min-and max-entropy. IEEE Transactions on Information theory , 55(9):4337-4347, 2009.

Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated { White-Box } membership inference. In 29th USENIX security symposium (USENIX Security 20) , pages 1605-1622, 2020.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pages 740-755. Springer, 2014.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems , 36, 2023.

Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https: //llava-vl.github.io/blog/2024-01-30-llava-next/ .

Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong. Encodermi: Membership inference against pre-trained encoders in contrastive learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , pages 2081-2095, 2021.

Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen. Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889 , 2018.

Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, and Rongrong Ji. Cheap and quick: Efficient vision-language instruction tuning for large language models. Advances in Neural Information Processing Systems , 36, 2024.

Saeed Mahloujifar, Huseyin A Inan, Melissa Chase, Esha Ghosh, and Marcello Hasegawa. Membership inference on word embedding and beyond. arXiv preprint arXiv:2106.11384 , 2021.

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. Membership inference attacks against language models

via neighbourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 11330-11343, 2023.

Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David K Evans, and Taylor BergKirkpatrick. An empirical analysis of memorization in fine-tuned autoregressive language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 1816-1826, 2022.

Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using 1 million captioned photographs. Advances in neural information processing systems , 24, 2011.

Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Proving test set contamination in black-box language models. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum? id=KS8mIvetg2 .

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. OpenAI , 2018.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Alfréd Rényi. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume 4, pages 547-562. University of California Press, 1961.

Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246 , 2018.

Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114 , 2021.

Claude Elwood Shannon. A mathematical theory of communication. ACM SIGMOBILE mobile computing and communications review , 5(1):3-55, 2001.

Virat Shejwalkar, Huseyin A Inan, Amir Houmansadr, and Robert Sim. Membership inference attacks against nlp classification models. In NeurIPS 2021 Workshop Privacy in Machine Learning , 2021.

Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations , 2024. URL https: //openreview.net/forum?id=zWqr3MQuNs .

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) , pages 3-18. IEEE, 2017.

Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security , pages 377-390, 2020.

Congzheng Song and Vitaly Shmatikov. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &Data Mining , pages 196-206, 2019.

Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. Machine learning models that remember too much. In Proceedings of the 2017 ACM SIGSAC Conference on computer and communications security , pages 587-601, 2017.

Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2615-2632, 2021.

Liwei Song, Reza Shokri, and Prateek Mittal. Membership inference attacks against adversarially robust deep learning models. In 2019 IEEE Security and Privacy Workshops (SPW) , pages 50-56. IEEE, 2019.

Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. Enhanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security , pages 3093-3106, 2022.

Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF) , pages 268-282. IEEE, 2018.

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM , 64(3): 107-115, 2021.

Pan Zhang, Xiaoyi Dong Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Hang Yan, et al. Internlm-xcomposer: A vision-language large model for advanced text-image comprehension and composition. arXiv preprint arXiv:2309.15112 , 2023.

Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199 , 2023.

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representations , 2023.

Contents of the appendix

The Appendix is organized as follows:

In Appendix A, we provide some supplementary of the main body.

Our additional experiment results are presented in Appendix B.

A preview of some examples of VL-MIA can be found in Appendix C.

We give broader impacts of this work in Appendix F.

We state the limitation of our work in Appendix G.

A Supplementaries to the main text

A.1 Detailed notation

We summarize the notations of this work in Table 5.

A.2 Explanation and visualization of slices

We give an example of the different slices in the MiniGPT-4 prompt in Figure 3.

Give the following image: <Img>ImageContent</Img>. You will be able to see the

image once I provide it to you. Please answer my questions.

 <Img> img

feature </Img> Describe this image in detail. 
 This is an image of a woman

Image slice

Instruction slice

Generated text slice

There is no

CEloss

CEloss

CEloss

cross entropy

(CE)loss

here

Distribution of

next text token

Language model

Embedding

vector

Vision encoder

Text token id

There is no

13

322

1352

(return by

image token

tokenizer)

qqq

This

image

shows

Input

As shown in Figure 4, the VLLMs consist of a vision encoder, a text tokenizer, and a language model. The output of the vision encoder (image embedding) have the same embedding dimension d as the text token embedding.

When feeding the VLLMs with an image and the instruction, a vision encoder transforms this image to L 1 hidden embeddings of dimension d , denoted by E image ∈ R d × L 1 . The text tokenizer first tokenizes the instruction into L 2 tokens, and then looks up the embedding matrix to get its d -dimensional embedding E ins ∈ R d × L 2 . The image embedding and the instruction embedding are concatenated as E img -ins = ( E image , E ins ) ∈ R d × ( L 1 + L 2 ) , which are then fed into a language model to perform next token prediction. The cross-entropy loss (CE loss) is calculated based on the predicted token id and the ground truth token id on the text tokens.

We can see that in this process, image embeddings are directly obtained from the vision encoder and there are no image tokens. There are no causal relations between consecutive image embeddings as well. Threfore, as we stated in Section 1, target-based MIA that requires token ids cannot be directly applied.

A.3 Datasets construction prompts

We present the instruction prompts we use to construct the dataset in Section 4 in Table 6.

A.4 Additional experimental information

The versions, base models and training datasets of target VLLMs are listed in Table 7. We run experiments on a single NVIDIA A100 80GB GPU, where the image MIA costs less than 2 hours for one model.

In our experiments on description text MIA, we do not detect the member text during the VLLM image-text pre-training stage (the 3rd row) because the captions used in pre-training are usually fewer than 10 words and do not contain sufficient information. Therefore, we only detect member text during the VLLM instruction tuning stage.

A.5 Parameters for image corruption

We utilize the code 8 from ImageNet-C [21] to corrupt member images. The related analysis can be found in Section 6.5. Here we provide the corruption parameters in the code in Table 8.

B Additional experiments

B.1 Text MIA: Complete results

In addition to Section 6.3, we present our complete Text MIA results on LLaVA, MiniGPT-4, and LLaMA-Adapter in Table 9 and Table 10. We find that no current method can effectively detect LLM pre-training text corpus from MiniGPT-4, even the AUC on 256-length Wiki text is only 0.6. Apart from that, we note that on the LLaMA-Adapter, target-based methods perform similarly or better than target-free ones. In the following, we explain this phenomenon by examining the training pipelines of two different models.

The wiki data is used in the pre-training stage of LLaMA 7b, which is further fine-tuned to LLaMA 7b Chat. The base MML of LLAVA is fine-tuned from Vicuna 7b v1.5, which is further fine-tuned from LLaMA 7b Chat. Therefore, the LLM model of LLAVA has changed a lot since wiki data's last usage, and target-free methods are preferable here. In contrast, LLaMA-Adapter is parameter-efficiently fine-tuned from LLaMA 7b, and the model's parameters have not changed much since the wiki's data last usage, and the target-based MIA methods can still retain their performance. In addition, to

detect instruction-tuning texts in VLLMs, the model has just repeatedly seen the member data, and the target-based ones are significantly superior to target-free ones. Overall, the phenomena can be explained by our hypothesis.

B.2 Abalation on MaxRényi and MinRényi

Given a sequence of entropies calculated on each position, it would be natural to ask whether the higher percentile or lower percentile should be used to determine the score of the whole sequence for MIA. In the main paper, we use max , which coincides with Min-K% [52] under some special cases. We provide more evidence of the advantages of max or min . We define MinRényi-K% as similar to MaxRényi-K% , but consider the smallest K% entropies in the sequence X .

We conduct the pre-training text MIA on LLAVA. We also use the WiKiMIA [52] benchmark of different lengths (32, 64, 128, 256). We set K = 0 , 10 , 20 , since when K = 100 , MaxRényi-K% coincides with MinRényi-K% . We observe that usually MinRényi-K% is slightly inferior to MaxRényi-K% . Therefore, we adopt MaxRényi-K% in our main paper.

B.3 Ablation on extended dataset

In this part, we extend VL-MIA/Flickr and VL-MIA/Text size to 2000, which includes 1000 members and 1000 non-members. From Table 12 and Table 13, we see a similar trend with small-size (600 samples) VL-MIA. Considering this computing source, our main experiments are conducted on small-size datasets.

B.4 Different instruction texts

We conduct image MIA on VL-MIA/Flickr with LLaVA through three different instruction texts: 'Describe this image concisely.', 'Please introduce this painting.', and 'Tell me about this image.'. We present our results in Table 14 of the appendix. Our pipeline successfully detects member images on every instruction, which indicates robustness across different instruction texts.

C Dataset examples

In Table 16, we give some examples of our proposed VL-MIA datasets. In Table 17 and Table 18, we show the corrupted images used in the ablation stuidies (Section 6.5) and the model's descriptions of these corrupted images. In Figure 5, we present some examples of our VL-MIA/Synthetic dataset.

D TPR at 5% FPR

In this section, we provide the True Positive Rate (TPR) at 5% False Positive Rate (FPR) results as supplementary evaluation, as presented in Table 15. We can see a similar trend in Table 2.

E VLLM pipelines

In this part, we investigate the pipelines of vLLMs, to explain why we have access to the text tokens, but only have access to the image embeddings instead of image tokens.

The VLLMs consist of a vision encoder, a text tokenizer, and a language model. The vision encoder and the text tokenizer have the same embedding dimension d . When feeding the VLLMs with an image and the instruction, a vision encoder transforms this image to L 1 hidden embeddings of dimension d , denoted by e image ∈ R d × L 1 . The text tokenizer first tokenizes the instruction into L 2 tokens, and then looks up the embedding matrix to get its d -dimensional embedding e ins ∈ R d × L 2 . The image embedding and the instruction embedding are concatenated as e img -ins = ( e image , e ins ) ∈ R d × ( L 1 + L 2 ) , which are then fed into a language model to generate a description that has L 3 tokens. The cross-entropy loss (CE loss) is calculated based on the predicted token id and the ground truth token id on the instruction and description tokens.

We can see that in this process, image embeddings are directly obtained from the vision encoder and there are no image tokens. Threfore, as we stated in Section 1, target-based MIAs that require token ids cannot be directly applied.

Furthermore, given the ouput logits of the shape ( L 1 + L 2 + L 3 ) ×|V| , where V is the vocabulary set, we can access the logits of the image by the slice [0 : L 1 ] , the logits of instruction by the slice [ L 1 : L 1 + L 2 ] , and the logits of description by the slice [ L 1 + L 2 : L 1 + L 2 + L 3 ] .

F Broader impacts

In this paper, we present the first multi-modalities MIA benchmark for VLLMs, and propose a novel metric MaxRényi-K% for MIA. We recognize that our research has significant implications for the safety and ethics of VLLMs and may lead to targeted MIA defense by developers. Nevertheless, our findings provide valuable insights into data contamination, which could contribute to the training data split. Additionally, our method empowers individuals to detect their private data within the training dataset, which is essential for ensuring data security. We believe our work can raise awareness about the importance of privacy protection in multi-modal language models.

G Limitation

The first limitation of this work is that the best methods on the pre-training dataset can only achieve an AUC of 0 . 688 . This is because the detection of pre-training data will become more challenging as the VLLMs are further fine-tuned. We believe increasing the performance for detecting pre-training data would be promising in the future. Secondly, the proposed MaxRényi-K% method requires access to the full probability distribution over the predicted tokens. Although we have demonstrated the

VL-MIA/DALL-E

inst+desp img

0.057

0.081

0.041

0.030

0.051

0.064

0.041

0.057

0.054

0.085

0.095

0.081

0.044

0.064

0.081

0.054

0.068

0.098

0.068

0.091

0.098

N/A

N/A

N/A

N/A

0.014

0.051

N/A

N/A

N/A

0.061

0.068

0.088

0.054

0.074

0.061

0.051

0.071

0.088

0.068

0.074

0.095

MiniGPT-4

inst desp

0.014

0.054

0.054

0.064

0.064

0.037

0.024

0.020

0.037

0.061

0.061

0.064

0.037

0.037

0.078

0.030

0.030

0.054

0.027

0.027

0.037

0.034

0.030

0.020

0.020

0.030

0.037

0.030

0.027

0.030

0.020

0.017

0.017

0.027

0.014

0.027

0.030

0.020

0.027

0.044

0.020

0.034

inst+desp

0.034

0.051

0.057

0.054

0.037

0.030

0.027

0.061

0.030

0.061

0.061

0.020

0.044

0.037

0.017

0.027

0.017

0.014

0.014

0.017

0.014

LLaMA Adapter inst

desp inst+desp

0.030

0.037

0.020

0.020

0.030

0.047

0.037

0.041

0.034

0.112

0.162

0.030

0.088

0.071

0.027

0.051

0.034

0.030

0.047

0.034

0.024

0.081

0.051

0.030

0.034

0.034

0.074

0.081

0.074

0.068

0.051

0.061

0.047

0.054

0.041

0.051

0.054

0.037

0.051

0.051

0.054

0.064

0.051

0.037

0.020

0.017

0.027

0.071

0.064

0.041

0.061

0.112

0.044

0.034

0.081

0.051

0.034

0.047

0.057

0.061

0.047

0.034

0.041

α

α

= 0

.

= 1

α

= 2

Max_0%

Max_10%

Max_0%

Max_10%

Max_0%

Max_10%

Max_0%

Max_10%

img

N/A

N/A

N/A

N/A

0.020

0.037

N/A

N/A

N/A

0.081

0.152

0.003

0.091

0.220

0.003

0.118

0.172

0.017

0.128

0.142

0.024

LLaVA

inst desp

0.027

0.081

0.081

0.064

0.081

0.108

0.020

0.034

0.024

0.088

0.088

0.098

0.101

0.101

0.101

0.091

0.091

0.101

0.112

0.112

0.122

0.081

0.051

0.068

0.071

0.037

0.085

0.088

0.095

0.088

0.047

0.064

0.095

0.064

0.061

0.081

0.051

0.051

0.101

0.051

0.068

0.081

feasibility of this approach on GPT-4, where the top-5 probabilities are available, it can be challenging if only the probability of the target token is provided.

Metric

∗

Perplexity

Min_0% Prob

∗

Min_0% Prob

∗

Min_20% Prob

∗

Aug_KL

Max_Prob_Gap

∗

ModRényi

Rényi (

α

= 0

Rényi (

Rényi (

Rényi (

α

α

α

.

5

= 1

)

= 2

)

=

∞

)

)

5

Moae MmsLkhioe

BELGIUM HOUSE

nDa

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

0

口

口

■

Q

6

2

h

4

9

3

P163

m

6S

y

n

7

8

P

0

s

N

e

G

n

6

G

msh

2

G

F

2

n

B

F

G

8

h

1

b

1

n

6

Z

X3

V

8

3

4

9

4

78

ywy

6

5

3

+

a

K

2

×

r

Y

5

3

V

3

9

S

2

h

。

S

5

0

S

x

3

oont

&D

7

5ot

Severity

Original image

Brightness

Motion_Blur

Snow

JPEG

Marginal

Moderate

Severe

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction accurately state our contribution and findings. Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations in Appendix G.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: There is no assumption or theoretical result in this work.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in the appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We disclose all the information needed to reproduce the main experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided open access to the data and code in https://github.com/LIONS-EPFL/VL-MIA.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We introduce the setup of our experiment in Section 6.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in the appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: There is no randomness in our experiments because we set the temperature of LLM to 0.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We state computer resources-related information in Appendix A.4.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We follow the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss these in Appendix F.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We discuss the safeguards in Appendix F.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite every reference we use and CC-BY 4.0 is applicable for our work.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We will provide the datasets on huggingface upon acceptance with comprehensive documentation.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper focuses on the membership inference attack (MIA) of large vision language models. It creates multiple MIA datasets and proposes a new method to identify whether an image or a text belongs to the training dataset.

STRENGTHS:
1. The authors have proposed an intriguing research problem, which focuses on the membership attack of large vision language models.
2. The paper proposes a creative method to verify if the input is a member of the training set. It also creates a new benchmark for evaluation.
3. The overall writing is clear and easy to follow.

WEAKNESSES:
The setting for detecting a description sentence is a little confusing for me. Why do you feed the model with a black image rather than others, like those generated by DALL-E? Also, why do you skip the instructions here?  It is less possible to produce the target description for the target VLM just based on a black image and an empty instruction. Why don't skip the input image?

QUESTIONS:
1. As illustrated in the weakness section, could you please explain the setting of MIA in the description sentence?
2. How to set the threshold value \lambda in the MIA process?

LIMITATIONS:
The paper has analyzed its limitations in the Appendix, which I believe is reasonable.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an interesting idea of detecting training data in large vision-language models (VLLMs) through membership inference attacks (MIAs). The authors introduce the following interesting points :\n\n- An MIA benchmark, specifically designed for VLLMs, called Vision Language MIA (VL-MIA). This benchmark includes tasks for both image and text detection across various VLLM architectures.\n- A pipeline for MIAs on VLLMs. This approach does not require both side information as in the prior work. This includes a new detection metric called MaxRényi-K%.\n- Extensive evaluation results. The authors demonstrate the effectiveness of their approach to diverse VLLMs.\n\nSTRENGTHS:\nThis paper is well-written and proposes an interesting approach that can be applicable to one side of information for MIA for VLLMs, which might not be well-studied in prior works.\nThe authors also provide a benchmark for evaluating MIA on VLLMs and present extensive results based on the proposed benchmark.\n\nWEAKNESSES:\nThis paper requires further clarification regarding the proposed methods and evaluation results. Please see the following questions.\n\nQUESTIONS:\nRegarding dataset generation:\nIf the member data is drawn from LAION_CCS, and non-member data is drawn from the generated dataset, is there any underlying bias caused by the nature of differences between natural and synthetic data (in VL-MIA/DALL-E)? Moreover, why did we choose instruction text for members and generated text for non-members in VT-MIA/Text?\n\nRegarding the evaluation:\nBased on Table 2, it seems that different datasets show different behaviors. For example, for VL-MIA/Flickr, Max_100% shows the best performance in most cases, and the performance gap is quite large. On the other hand, we cannot see a similar observation from VL-MIA/DALL-E. I wonder whether the authors have some insight into why it shows a clear difference between datasets. \nMoreover,  it seems that we need to vary a lot of parameters to find better performance (e.g., different alpha, and different K). How did the authors choose the hyperparameters from other baselines? Since the authors present a new benchmark, it might not be a good choice to use optimal hyperparameters from other sources.\n\nLIMITATIONS:\nThe authors discuss the limitations and broader impacts in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper introduced a benchmark for membership inference attack on VLMs, proposed a pipeline for token-level image detection, and proposed a target-free metric for image MIA detections. The pipeline relies on the fixed sequence of the VLM output to obtain the output image, instruction, and description segments of logits and use them for evaluation metrics.

STRENGTHS:
1. Novelty: There is no existing MIA benchmark datasets for VLM. The authors also proposed a new metrics for detecting MIA in single modality, especially in images. 
2. The paper conducts extensive experiments with their proposed methods.

WEAKNESSES:
1. The explanation of some concepts are not clear. For example, when proposing target-free MIA metrics, the author mentioned that it's because we only have access to the image embeddings but not image tokens. How are these two terms defined here? Why do we not have access to the image tokens? Additionally, image tokens are defined again in line 95. If this is not available, why does the paper need to define the concept again? I think the authors should clarify these assumptions properly in the main text.

QUESTIONS:
1. For the cross-modal pipeline for detecting images, how do you determine which logits are for image, instructions, or description text, during the attack phase?
2. The proposed metrics can have different K and $\alpha$ values. Are there any ablation studies on when should what K and $\alpha$ values being used?

LIMITATIONS:
The authors clarified the limitations in the paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe rise of large vision-language models (VLLMs) has significantly advanced multi-modal tasks but also brought forth concerns about data security and privacy. This paper introduces a novel membership inference attack (MIA) benchmark specifically designed for VLLMs to detect training data, addressing the lack of standardized datasets and methodologies in this domain. The authors propose a new MIA pipeline for token-level image detection and introduce the MaxRényi-K% metric for improved detection. The key contributions include the development of the first VLLM-specific MIA benchmark, a cross-modal MIA pipeline for individual image or description detection, and the new MaxRényi-K% and ModRényi metrics, which show effectiveness in experiments.\n\nSTRENGTHS:\n- The paper introduces the first benchmark specifically tailored for VLLMs in the context of MIAs.\n- The MaxRényi-K% and ModRényi metrics demonstrate significant effectiveness in detecting training data.\n\nWEAKNESSES:\n- **Small Evaluation Dataset**: The MIA evaluation dataset consists of around 600 images for each evaluation, which is relatively small. This limited dataset size can lead to less statistically robust results and may not fully capture the variability and challenges present in real-world scenarios. It would strengthen the paper to include evaluations on larger and more diverse datasets.\n\n- **Lack of Standard Metrics**: The paper does not consider the _TPR at low FPR_ (True Positive Rate at low False Positive Rate) metric, which is a standard for evaluating worst-case membership privacy. Including this metric would provide a more comprehensive assessment of the model's privacy risks, especially in high-stakes applications where false positives must be minimized.\n\n- **Generalizability**: While the proposed metrics and methods show effectiveness, it's important to discuss their generalizability to other types of vision-language models and datasets. Providing insights or experiments on different architectures or domains could enhance the applicability of the findings.\n\nQUESTIONS:\n- How to choose $K$ for the MaxRényi-K% metric?\n\nLIMITATIONS:\nThe size and the diversity of the benchmark is limited.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}