Title: Private Attribute Inference from Images with Vision-Language Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: privacy

vision-language models

multimodal models

large language models
Abstract: As large language models (LLMs) become ubiquitous in our daily tasks and digital interactions, associated privacy risks are increasingly in focus. While LLM privacy research has primarily focused on the leakage of model training data, it has recently been shown that LLMs can make accurate privacy-infringing inferences from previously unseen texts. With the rise of vision-language models (VLMs), capable of understanding both images and text, a key question is whether this concern transfers to the previously unexplored domain of benign images posted online. To answer this question, we compile an image dataset with human-annotated labels of the image owner's personal attributes. In order to understand the privacy risks posed by VLMs beyond traditional human attribute recognition, our dataset consists of images where the inferable private attributes do not stem from direct depictions of humans. On this dataset, we evaluate 7 state-of-the-art VLMs, finding that they can infer various personal attributes at up to 77.6% accuracy. Concerningly, we observe that accuracy scales with the general capabilities of the models, implying that future models can be misused as stronger inferential adversaries, establishing an imperative for the development of adequate defenses.

Private Attribute Inference from Images with Vision-Language Models

Abstract

As large language models (LLMs) become ubiquitous in our daily tasks and digital interactions, associated privacy risks are increasingly in focus. While LLM privacy research has primarily focused on the leakage of model training data, it has recently been shown that LLMs can make accurate privacy-infringing inferences from previously unseen texts. With the rise of vision-language models (VLMs), capable of understanding both images and text, a key question is whether this concern transfers to the previously unexplored domain of benign images posted online. To answer this question, we compile an image dataset with human-annotated labels of the image owner's personal attributes. In order to understand the privacy risks posed by VLMs beyond traditional human attribute recognition, our dataset consists of images where the inferable private attributes do not stem from direct depictions of humans. On this dataset, we evaluate 7 state-of-the-art VLMs, finding that they can infer various personal attributes at up to 77.6% accuracy. Concerningly, we observe that accuracy scales with the general capabilities of the models, implying that future models can be misused as stronger inferential adversaries, establishing an imperative for the development of adequate defenses.

1 Introduction

Since the release of ChatGPT [1], large language model-based (LLM-based) applications and chatbots have enjoyed a rapid adoption, surpassing hundreds of millions of daily active users [2]. Towards making these models universally applicable, there has been a recent push for vision-language models (VLMs) capable of understanding not only text but also reasoning over text and images jointly [3, 4]. The rapid adoption of LLM-based applications and the concurrent advances in the underlying models' capabilities raises several safety and privacy concerns among the general public, researchers, and regulators alike [3, 5-7]. In response, model providers are under increasing pressure from existing data protection regulations, such as the EU's GDPR [8] and the California Consumer Privacy Act (CCPA) [9], as well as from substantial ongoing regulatory efforts directly concerning AI [10, 11]. For instance, in 2023, Italy temporarily banned ChatGPT, citing data protection and privacy concerns [12]. As such, exploring the potential privacy concerns of VLMs is a crucial first step towards a wider deployment of VLM applications that are privacy-preserving and regulation-compliant.

Privacy Implications of LLMs Weidinger et al. [6] lay out the privacy implications of LLMs from two separate perspectives: (i) memorization and (ii) inference. Although several works have examined private information memorization and leakage in LLMs [13-15], until recently, inference has remained unexplored. Enabled by the strong inferential capabilities [16] and vast world-knowledge [17] of current frontier models, Staab et al. [18] were first to examine

the potential of LLMs being misused for accurate inference of personal attributes from previously unseen online texts. Their results indicate that current frontier models such as GPT-4 [19] already achieve close to human-level accuracy across various personal attributes (e.g., age, gender, location) while incurring only a fraction of the cost and time investment of a human.

The recent rise of VLMs lifts this discussion from a text-only domain to include also images, raising the question of how the findings of Staab et al. [18] translate to the multi-modal setting. This is particularly relevant as even though image and video are ubiquitous in most influential social media platforms (e.g., Instagram, TikTok), privacy risks associated with automated VLM inferences have not been explored yet.

Example To motivate our setting, consider Fig. 1 depicting a dormitory room. This picture could have been posted on a pseudonymized social media platform, such as Reddit (e.g., asking for arrangement advice) under the general assumption that one's privacy remains uncompromized. Despite no person being visible in the image, a human investigator may infer some personal attributes by reasoning over probable cues, such as recognizing the football team's logo or reverse image searching it. However, the involvement of a human detective prohibits the scalability of this approach, making its application on large-scale real-world data infeasible (already in 2014, there were 1.8 billion daily image uploads [20]). Yet, when feeding the image to a VLM (in this case to GPT4-V), the model

Where does the person living in this dormitory room come from?

I recognize the logo of the Green Bay Peckers, the local football team of Green Bay, Wisconsin. Location: Wisconsin, USA.

can do the investigator's work, detecting the relevant cues and correctly inferring that the person living in the dormitory is a Green Bay Packers fan, and as such, has a high probability of living or having lived in Wisconsin. As model inferences are inherently scalable, VLMs enable such privacy violations at an unprecedented scale, requiring us to re-evaluate our understanding of online privacy.

This Work For the first time, we systematically analyze the capability of VLMs to infer private information from inconspicuous images posted online. Our findings indicate that similarly to the text-only domain, VLMs are able to infer a variety of personal attributes from real-world images both accurately and at an unprecedented scale. Notably, as we show in our evaluation, current safeguards against such privacy-infringing queries are ineffective in the face of simple evasion techniques, allowing for a low entry barrier for potential malicious actors. As such, we believe that with the advent of VLMs, threats to our online privacy are currently underestimated.

Circumventing Safeguards & Resolution Limitations Current VLMs are commonly equipped with safeguards (both via model alignment and additional pre-processing) intended to prevent the model from answering privacy-violating queries [3]. Several recent works [21-23] have shown that such safeguards can be broken both by manual intervention or by automated algorithms. To explore the privacy risks posed by an adversary misusing a VLM for privacy-infringing inferences, we develop a simple inference attack consisting only of the target image and a prompt to circumvent the safeguard. Notably, we found that once the safeguard of the model has been (easily) evaded, the model cooperates on the inference task by providing helpful guidance. In particular, we observe that it can recognize relevant parts of the image (e.g., a small note posted on a fridge) that could help the inference but which due to technical resolution limitations are too small to be analyzed. Building on this observation, we further develop an automated pipeline in which the model can decide to zoom into parts of the image that it believes to be relevant, effectively improving its inference capabilities.

A Visual Inference-Privacy Dataset Due to their extensive reasoning capabilities and worldknowledge VLMs could draw conclusions about private attributes not just from direct depictions of

people, but also from other contextual cues, e.g., in a picture of a kitchen, products of local brands could be visible, or images of rooms with recognizable items such as college logos could reveal the posting person's educational background. Therefore, in order to evaluate the privacy-inference risks of VLMs beyond traditional human attribute recognition (HAR), we require a dataset of seemingly inconspicuous images alongside personal attribute labels. Current image datasets focusing on private attributes are insufficient for this task as: (1) They focus on a small specific set of attributes, such as gender, age, or the presence of certain clothing items [24] and (2) they almost exclusively consist of depictions of natural persons [25, 26]. To reflect the inference-based threat arising from the extended capabilities of VLMs appropriately, we create a dataset by collecting and manually annotating images posted on the pseudonymized social media platform Reddit, particularly focusing on images where private information is not sourced from direct depictions of humans.

We evaluate the performance of two widely adopted proprietary models, GPT4-V[3] and Gemini-Pro [4], together with five open-source models available on Hugging Face [27]. We find that although the safeguards of some of the models reject up to 54.5% of our queries when using a naive prompt, they can be easily circumvented via prompt engineering, making the models infer up to 77.6% of the private attributes correctly. Allowing the models to act autonomously and zoom in on details further improves the accuracy on certain features, e.g., precise location accuracy rises from 59.2% to 65.8%. Concerningly, this demonstrates that even safety-aligned VLMs can be misused as adversarial agents autonomously acting against their original safety objectives. Additionally, as with LLMs on text [18], we observe that the personal attribute inference accuracy is strongly correlated with the general capabilities of the models, implying that future iterations will pose an even larger privacy threat. Finally, VLM inferences are 480 × faster and ∼ 117 × cheaper than human annotation, indicating a paradigm shift in privacy considerations of images posted online, where the previous large time and monetary cost of inferences does not protect users anymore. Therefore, we advocate for further research into developing defenses against inference-based privacy attacks in the image domain, where the current safeguards are insufficient.

Main Contributions Our main contributions are:

The first identification and formalization of the privacy risks posed by vision-language models at inference time.

Extensive experimental evaluation of 7 frontier VLMs at inferring personal attributes from real-world images.

An open-source implementation 1 of our dataset labeling tool and our inference pipeline to advance privacy research.

Responsible Disclosure Before making any copy or derivative of this work public, we contacted OpenAI and Google about our findings, providing them access to our data, prompts, and results.

2 Background and Related Work

Vision-Language Models For the context of this work, we collectively refer to multimodal instruction-tuned foundational (large) language models with image understanding capabilities as vision-language models (VLMs). While combining different modalities for machine learning exhibits a long line of research [28], the first influential VLMs building upon foundational models have only appeared recently [29-33]. These methods achieve image understanding either by combining LLMs with pre-trained image encoders, or through joint training across modalities. Fundamentally, both methods rely on both the image and the textual input being translated to token embeddings and fed to a, usually decoder only, transformer model for processing. This approach is widely applied across both proprietary, i.e., GPT4-V[3] and Gemini [4], and open-source [34] VLMs. Additionally, these models are often equipped with learned safeguards (i.e., they are aligned ) to refuse queries that would lead to the generation of harmful responses [3, 4].

Personal Identifiable Information and Personal data Both personal identifiable information (PII) as well as personal data refer to information that can be attributed to a specific (natural) person. In

the EU, the term personal data is defined via Article 4 in the EU's General Data Protection Regulation (GDPR) [8] as "any information relating to an identified or identifiable natural person." While PII definitions in the USA are commonly less comprehensive than the GDPR, they similarly include all information from which "the identity of an individual [...] can be reasonably inferred by either direct or indirect means." Notably, this includes attributes like gender, geographic indicators, or economic status. We note that as in [18], most attributes considered in this work (e.g., age, location, income, sex) fall under both personal data and PII definitions.

Large Language Models and Privacy As the pre-training datasets of LLMs consist of vast amounts of data across diverse sources, they often contain sensitive personal (identifiable) information. Therefore, studying the phenomenon of training data memorization , i.e., the verbatim repetition of training data sequences at inference time, has become an important area of research in the context of LLMs [35-38, 13-15]. However, the restricted setting of exact memorization does in many cases fall short of covering other often highly contextual privacy notions [36]. In particular, as it is limited to the models' training data, it cannot account for privacy-infringing inferences on previously unseen texts [16]. Staab et al. [18] were the first to investigate the privacy risks of inferring personal information from text using LLMs, showing that current models can recover personal information even from seemingly anonymized text. However, their analysis was restricted to only the single modality of text, while current widely used frontier models are equipped with visual reasoning capabilities as well. In our work, we aim to bridge this gap by exploring the inference-based privacy threats of VLMs.

HumanAttribute Recognition Human attribute recognition (HAR) focuses on recognizing features of natural persons from their visual depictions. These feature recognitions are formulated as binary or multi-label classification tasks on a single person, commonly focusing on a specific feature such as the person's sex, age, or dressing style [25]. Before VLMs, state-of-the-art HAR models were trained by standard supervised learning, requiring access to highly task-specific and labeled (image-only) training data. Trained models then focused on singular tasks, e.g., recognizing specific attributes of pedestrians (PAR) [39]. Recently, VLMs have also been successfully explored on various PAR datasets [40-42], showing promising results over prior, non-VLM-based methods. Although VLMs prove to be performant methods on PAR, their capabilities extend beyond the commonly restricted HAR settings. Notably, as existing HAR datasets are centered around direct depictions of humans, they do not cover the privacy risk arising from the application of frontier VLMs with advanced reasoning capabilities and broad lexical knowledge. In particular, as we show in Section 5, VLMs enable the automated inference of personal attributes from images that do not necessarily contain the subjected person in the image but, e.g., only an inconspicuous depiction of their living room.

3 Privacy Infringing Inferences with VLMs

In this section, we first introduce the considered threat model. Then, we proceed by presenting our prompting strategy that allowed us to circumvent the safeguards of even the most recent VLMs of OpenAI [3] and Google [4]. Finally, we present our automated zooming scheme, enabling models to autonomously enlarge parts of the image it deems relevant for further inspection.

Threat Model To capture a general threat scenario, we assume an adversary with only black-box query access to a (frontier) VLM. The goal of the adversary is to get as much and as detailed personal information as possible from online images. At the same time, the attack shall remain simple and practical, keeping the entry requirements for any potential adversary low. Such an attack is particularly concerning, as its potential for automation enables execution at a scale unattainable by pre-VLM methods or human investigators. Crucially, this potential for scaling challenges our current understanding of online privacy, which in many cases and for many users relies heavily on the prohibitively high cost of obtaining private information from seemingly benign images and posts.

Circumventing Safeguards & Prompt Engineering Often, the training of VLMs such as GPT4-V [3] and Gemini [4] includes a separate safety alignment stage with the goal of creating a model capable of refusing queries that lead to potentially harmful generations. However, as highlighted in Section 1, it has been shown that such training-based safeguards can easily be circumvented both by hand-crafted prompts or even fully automated attacks [21-23]. As such, to cover the full extent of privacy risks associated with inferences made by VLMs, it is imperative to construct an evaluation

method that escapes such safeguards. Additionally, the prompt has to make use of the full capabilities of the model, avoiding a potential false sense of privacy through insufficient evaluation. To construct such a prompt we follow popular reasoning prompting practices, such as chain-of-thought prompting [43] to improve performance, and gamify the inference task in a similar vein to [18] to escape any safeguards. Additionally, we provide task-independent reasoning examples in the prompt, with the goal of increasing the model's attention to detail. We examine the impact of our prompting choices in Section 5, clearly demonstrating that "naive" prompts ( "Where was this picture taken?" ) severely underestimate the inference-based privacy risks posed by current frontier vision-language models.

Automated Zooming Small details in an image often contribute to privacy-infringing inferences, e.g., a letter hanging on the wall in the background revealing the state one resides in, or recognizing a small university emblem on a larger item in the image signifying the person's educational background. However as most current VLM are limited in input resolution, they struggle to properly extract these small yet important details. As exemplified in Fig. 2, our experiments indicate that even though in some cases VLMs are not able to process small details (e.g., writing on a tax form), they are still able to recognize their potential importance for inference (a tax form contains personal information). In fact, the model can be prompted to return a bounding box for such a recognized clue, which in turn can be automatically processed to feed the model a cropped image enlarging the corresponding section. Based on this, we automate the zooming procedure by prompting the model for 3 regions to zoom into via outputting bounding boxes. Then, we adjust the bounding box to cover 16% of the image and be within image limits. Finally, we return the zoomed-in images in a second request to the model. In Section 5, we show the impact of zooming, e.g., it improves GPT4-V's precise location inference accuracy by up to 6.6%.

4 A Visual Inference-Privacy Dataset

In this section, we first argue that current image datasets for (private) attribute inference do not cover the novel privacy-infringing inference threat that VLMs pose. Bridging this gap, we then present our visual inference-privacy (VIP) dataset used for our empirical evaluation in Section 5.

Not Only Images of Humans Leak Information Although there exist several datasets in the literature for (personal) human attribute recognition, they primarily focus on extracting and inferring features of persons included in the images, commonly in nonprivacy related settings, e.g., pedestrian identification [25, 26]. This focus is also present in current HAR privacy benchmarks, with the explicit goal of a per-

Where does the person with this

kitchen live? In case you discover

clues that are too small to recognize,

provide a bounding box for zooming.

The appliances likely indicate USA.

From a close-up of the document I could

maybe infer more information. I can

provide you the following bounding box:

Left: 0%               Upper: 0%

Right: 70%           Lower: 55%

This is a tax filing to Colorado state.

Location: CO, USA.

ceptual protection of humans included in the images [26]. However, with the rise of VLMs, which are capable of visual reasoning and are equipped with vast lexical knowledge, considering only images that include humans does not fully cover the potential privacy threat posed by these models. This is highlighted by our examples in Fig. 1 and Fig. 2, where private attributes are inferred from other objects in the depicted environment. Therefore, in this paper, we focus on evaluating the risk of

r/bedrooms

r/travel

…

1.

Extract images & comments from Reddit

2.

Label based only on the image

Place of Living:

Madison, USA

Hobbies/Other:

Dog owner

…

Keep only labels confirmed by comments

Image labels

Madison, USA

Middle Income

Dog Owner

Comment labels

Madison, USA

High Income

Dog Owner

+

×

private attribute inferences from images that primarily do not contain depictions of humans, a setting not considered under current benchmarks. To enable the evaluation of this arising privacy risk, we formulate three key criteria that a dataset for inference-based privacy evaluation has to fulfill.

Key Criteria As VLMs are no longer limited to the recognition of attributes of human visuals, we require a dataset that reflects this change in domain. In particular, the images should: (i) try to avoid containing full depictions of natural persons, (ii) be representative of what real people may post on (pseudonymized) online platforms, and (iii) come with a diverse set of labels covering a large set of private attributes as introduced in privacy regulations such as the GDPR [8].

Building a Visual Inference-Privacy

Dataset To the best of our knowledge, there currently does not exist any dataset that fulfills all three criteria. Therefore, we construct a visual inference-privacy (VIP) dataset, the first benchmark to evaluate the attribute inference capabilities of VLMs from seemingly innocuous images. An overview of our dataset collection pipeline is presented in Fig. 3. First,

we source all images from the popular pseudonymized social media site Reddit, where we select a set of subreddits that are likely to contain posts with images suitable for our evaluation task (listed in Appendix D). Next, we manually label all images, using the image as the only source of information (i.e., no other data from the posting profile), but without time or internet browsing restrictions. Note that for ethical considerations, in line with the practices established by Staab et al. [18] also working with Reddit data, we do not outsource the labeling task, instead, the labeling is fully conducted by the authors of the paper. To cover a wide range of attributes as required by criterion (iii), we collect the following private attributes: location of residence ( LOC ), place of image ( POI ), sex ( SEX ), age ( AGE ), occupation ( OCC ), income ( INC ), marital status ( MAR ), and education ( EDU ). Following Staab et al. [18], we also record a hardness score ranging from 1 to 5 for each label, corresponding to the difficulty for the labeler to extract/infer the label. Likewise we also adopt the scale used in [18], and rate from 1 to 3 for labels that require increasingly more complex reasoning but no online search. We assign hardness 4 and 5 to labels where the labeler required external knowledge tools, with hardness 5 indicating the additional need of advanced reasoning. As we only record the labels we could reliably extract from the image, we generally only obtain a label for a subset of the attributes per image. In a last step, to ensure that our recorded labels accurately reflect the profile of the posting author, we check the last 100 comments of the author, keeping only labels that are in line with the information contained in the comments. Note that we do not keep the comments for evaluation, as we aim to isolate the effect of privacy inferences from images, where the privacy leakage from text has already been explored in Staab et al. [18]. The distribution of the resulting labels for the main private attribute categories are shown in Table 1. For a detailed overview of the labeling procedure and instructions, we refer the reader to Appendix D.

0

100

200

300

400

500

554

Human labelled

GPT4-V

CogAgent-VQA

Idefics 80B

Gemini-Pro

LLaVa-NeXT 34B

InternVL-Chat-V1.2-Plus

LLaVa 1.5 13B

# Correct

SEX

POI

AGE

INC

LOC

EDU

OCC

MAR

5 Evaluation

In this section, we present the results of our experimental evaluation, which show how current frontier vision-language models enable privacy-infringing inferences from seemingly benign images. Additionally to the experiments presented in this section we include further results in Appendix B.

Experimental Setup We evaluate two proprietary, GPT4-V [3] and Gemini-Pro [4] (Gemini), and five open-source models, LLaVa 1.5 13B [34], LLaVa-NeXT 34B [44], Idefics 80B [45], CogAgentVQA [46], and InternVL-Chat-V1.2-Plus [47]. All models are run for every image-attribute pair in the VIP dataset, prompting the models to predict one private attribute at a time. To decrease the impact of randomness on our results, we use greedy sampling (temperature 0 ) across all our experiments. Unless mentioned explicitly, we use a single-round prompt with the models, not allowing for zooming, which we evaluate in a separate experiment. As described in Section 3, all proprietary models are aligned with safeguards. Therefore, we query these models via a gamified and CoT-extended prompt (later referred to as "Final" prompt) presented in Appendix E.3. We do so also for LLaVa-NeXT 34B and InternVL-Chat-V1.2-Plus. As CogAgent-VQA, Idefics 80B, and LLaVa 1.5 13B exhibit weaker language understanding capabilities and are mostly free from safeguards, we evaluate them with a simpler prompt (presented in Appendix E.5). Our prompting choices are motivated by avoiding the underreporting of the model's inference capabilities, and as such, potentially downplaying the posed privacy risk. We ablate the specific choice of prompts for all open-source models in Appendix B.3. Note that we do not evaluate other models than VLMs, as due to the complex nature of the VIP dataset, to the best of our knowledge, there is no supervised or other non-foundational machine learning method that generalizes to the challenging and diverse inference problem posed by VIP.

Calculating Inference Accuracy For the categorical attributes of SEX , INC , and EDU , we use a simple 0-1 accuracy in case the predicted category matches the label. For MAR , we report binary classification accuracy (has partner/no partner). Following the methodology of [18], for AGE , we let the model predict a probable interval for the subject's age. As our ground truth labels for AGE also consist of intervals, we count the model's guess as accurate if the two intervals have over 50% overlap. For the attributes LOC and POI , which have a high degree of freedom, we take a hierarchical approach: If the label contains city- or state-level information that is correctly predicted by VLM, we count that as a precise (P) correct prediction. When the model only predicts the country correctly, we still count it as a correct prediction for our main experiments but record that the inference has been less precise (LP) than the actual label. If the label only contains country-level information and the prediction contains the correct country information, we count the prediction as precise. For the last attribute, OCC , we take a semantic approach tolerating some minor precision loss, where, for instance, "Electronics Engineer" counts as a correct prediction for "Electrical Engineer". We evaluate this in a two-step approach, first prompting GPT-4 for a similarity judgement and afterwards manually verifying it. We give a more detailed overview of our evaluating procedure in Appendix A.3. Unless otherwise mentioned, we report the less precise accuracy in our experiments.

Main Results We show our combined results across all attributes and models in Fig. 4. Consistent with most benchmarks in the literature, we observe higher performance in proprietary models, with GPT4-V clearly outperforming all other models with a 77.6% accuracy. Remark-

ably, while GPT4-V is well-ahead of all models, CogAgent-VQA and Idefics 80B strongly outperform the proprietary model Gemini-Pro, with the best model reaching an accuracy of 66.4%. At the same time, other open-source models closely match Gemini-Pro in performance, with only LLaVa 1.5 13B lagging considerably behind with an inference accuracy of 53.3%. This result signifies that even if the safeguards of proprietary models were to be improved, there already exist open-source models that can make highly accurate privacy-infringing inferences.

Further, in line with [18], we observe that newer iterations of models exhibit a gradually increasing capability of inferring private attributes. In fact, looking at the MMMU [48] visual understanding and reasoning benchmark's leaderboard [49], we can see that the ranking of the models on VIP closely matches the ranking (of the included

models) on MMMU, indicating that privacy-inference and general capabilities are closely related. This result is concerning, as it shows that the inference-based privacy risk of VLMs will only increase with stronger models in the future, motivating a clear need for the development of targeted mitigations.

Accuracy over Attributes In Table 2, we show the per feature accuracy of GPT4-V, CogAgentVQA, and Gemini. Remarkably, GPT4-V exhibits a strong performance across most attributes, only struggling with inferring the income, where even the best model, CogAgent-VQA is only able to achieve 50% accuracy. Notably, GPT4-V achieves 94.5% accuracy on predicting SEX . At the same time, Gemini's performance is highly inconsistent across the examined attributes. While outperforming GPT4-V on POI , reaching 87% accuracy, on other non-location attributes, it performs considerably worse, with, for instance, SEX falling close to random guessing accuracy. By manual inspection of Gemini's outputs we observe that this is mostly due to the limited capabilities of the model, with it often claiming that no sex is inferrable in the absence of a human in the image.

Humans in the Image As we constructed our VIP dataset to emphasize the inference capabilities of models from nonperson-bound clues, only 9.7% of the collected labels came from images containing partial depictions of humans. Examples of these are depictions of hands, lower or full bodies, or reflections. To examine the impact of such depictions, we split our dataset into (1) images that contain parts of the human subject and (2) images that do not contain such depictions. In Table 3, we show our results for GPT4-V and the best open-

source model, CogAgent-VQA, on these splits. We can observe that both models exhibit a higher accuracy on the split containing humans, which we hypothesize is due to the fact that most labels contained in this split are usually directly inferable from human depictions, e.g., 31 out of 54 labels total in the split are for the features SEX and AGE . At the same time, the models still exhibit relatively strong performance on images with no human subjects, with GPT4-V achieving a remarkable 76.4% accuracy, signifying that VLMs enable private attribute inference from inconspicuous images that would not be otherwise considered under current HAR-privacy benchmarks. Additionally, the gap between the models is larger in absence of humans in the image, highlighting the advanced reasoning capabilities of GPT4-V when it comes to non-human sourced clues in inferring personal attributes.

Impact of Prompting We show the impact of our prompting techniques on the response refusal rate and accuracy in Table 4. Our baseline is a naive prompt directly asking the model for a given private attribute ("Naive"). As introduced in Section 3, in order to overcome the safety alignment of the models, we add adversarial prompting elements, such as the gamification of the inference task ("Extended"). Finally, we

further extend the prompt with chain-of-thought [43], reasoning guidelines, and generic reasoning hints to improve its performance ("Final"). All used prompts are included in Appendix E. Remark-

ably, our simple extension over the naive prompting achieves substantial improvements in terms of bypassing the safety alignment of GPT4-V, reducing the rejection rate from 54.5% to a mere 1.2%. This is concerning as it confirms that currently applied safeguards are incredibly brittle against even basic circumvention methods. Further, Table 4 shows the impact of the prompts on the overall accuracy of GPT4-V, showing that escaping the safeguard with a gamified prompt provides the largest improvement, and further extensions in our "Final" prompt lead to additional accuracy gain.

Automated Zooming We examine the impact of automated zooming on the location attributes ( LOC and POI ), as predictions on other attributes were largely not subject to resolution limitations. We show our results in Table 5, comparing our final prompt with an additional automated zooming exten-

Table 5: Precise (P) and less precise (LP) location ( LOC ) and place of image ( POI ) prediction accuracies [%] of GPT4-V on "Final" prompt vs. added zooming.

sion. We show the accuracy improvements made by GPT4-V on LOC and POI , distinguishing between precise (P) and less precise (LP) predictions. Notably, zooming provides the most accuracy improvements on precise predictions, enabling the model to make a more precise inference based on fine-grained clues in the images. As this process is automated, this result raises an important concern over the deployment of these models as autonomously acting inference adversaries.

Stability of Evaluation Results While our VIP dataset used for evaluating the inference-based privacy threat of VLMs directly reflects instances where such inferences pose a real-world privacy risk, it is limited in size. As such, while it allows for qualitative conclusions about VLM inferences it is not a priori clear how quantitatively stable our results are. For this reason, we conduct a closer examination of our results on GPT4-V. First, we calculate the 95% binomial confidence interval around the obtained inference accuracy of 77 . 6% , resulting in a range of 74 . 1% to 81 . 1% . Further, we estimate inference uncertainty by increasing the sampling temperature from 0 . 0 (as used in other results) to 0 . 2 and making three independent complete inferences over the whole dataset. This leads to accuracies of 76 . 2% , 77 . 1% , and 77 . 8% , i.e., to an average of 77% with a standard deviation of 0 . 62% -highly stable quantitative performance w.r.t. sampling. Finally, we compare the inference accuracy across varying temperatures of 0 . 0 , 0 . 2 , and 0 . 4 , obtaining accuracies of 77 . 6% , 77% (average from before), and 77 . 3% , respectively; showing stable performance across different temperatures. Overall, these experiments indicate that our results on VIP serve for both qualitative and quantitative statements about the real-world inference-based privacy risks of VLMs.

Inference Cost and Scalability As also highlighted by Staab et al. [18], a key concern of automated privacy-infringing inferences is their strong scalability compared to human annotators. The labeling of our VIP dataset took around 40 hours of human work, which, using the hourly rates of roughly $35 set by our institution, amounts to $1400 in labeling costs. In contrast, using the OpenAI API, processing the whole dataset cost only $12 and took 5 minutes for GPT4-V, with further parallelization of the inferences possible. As such, GPT4-V inferences are ∼ 117 × cheaper and 480 × faster than relying on human annotators. Crucially, both the scalability and the accuracy of VLM-based inferences are expected to increase in the future, as evidenced by recent trends in decreasing API pricing and our observation of newer and stronger models performing better also on private attribute inference. As such, VLM-based inferences pose a significant privacy concern for online imagery, where (a partial notion of) privacy was before primarily provided by the poor scalability of human annotation.

6 Discussion

Our empirical evaluation highlights several key privacy threats posed by VLMs, which are especially severe in the face of the wide adoption of these models: (1) Both proprietary and open-source models are capable of making accurate privacy-infringing inferences. (2) The safeguards of the better performing proprietary models such as GPT4-V are brittle and can be easily circumvented in practice, potentially providing a false sense of privacy. (3) As observed previously for text-only models, the capabilities of VLMs to infer personal attributes from images are directly correlated with their performance on other harmless and useful tasks. This is especially concerning, as it is to be expected that upcoming VLMs will only improve in general capabilities, and hence also on the results

we have shown in this work, making the threat to user privacy even more imminent. (4) Finally, the cost and time efficiency of such inferences means a categorical paradigm shift in privacy related to online imagery, with VLMs enabling privacy-infringing inferences at an unprecedented scale. Below, we further discuss the ethical implications of our study, potential mitigations, and limitations in detail.

Ethical Considerations Due to the sensitive nature of our study, we have taken several steps to ensure the no individual's privacy is compromised: (i) for constructing our VIP dataset, we have only taken images already included in prior public Reddit dataset dumps, (ii) we kept the labeled attributes to a set of ethically permissable ones, omitting more sensitive features such as mental health or race, (iii) we kept the labeling and the dataset on premise, providing access only to the authors, and (iv) we do not make our labeled dataset public and show only artifacts which do not compromise the privacy of any individual (e.g., aggregate results and modified examples). We note that these data handling practices have been cleared by our IRB. Further, on a broader perspective, even though we do not present an end-to-end solution to the uncovered threat here, we believe it is essential to publish our findings, and that the benefits of making the broader community aware of privacy threats posed by VLMs ultimately outweighs the potential short-term negative threat of adversaries replicating our attack. Especially, as it is not impossible that such inferences are already being conducted, based on precedence of similar misuses of foundation models on text [50].

Potential Mitigations While developing advanced defense methods against inference-based privacy attacks is beyond the scope of this paper, we strongly advocate for further action on improving both user-side and provider-side mitigations. On the provider side, we believe that our findings can be leveraged to strengthen the safety alignment of the models, training them to deny requests of potentially private attribute inference. However, as privacy-inference and general capabilities of the models are aligned, it can be challenging to balance a potential loss in utility with increased privacy protection. From the perspective of internet users that upload images, a potential direction for privacy protection could be an adaptation of the adversarial anonymization framework developed for text in [51]. Here, a VLM could be used to inform an image editing model about elements in the image that have to be obfuscated in order to remove the visual clues of private information.

Nonetheless, in our view, a crucial first step towards a more responsible use and deployment of VLMs is the wide-ranging awareness of the potential privacy risks across providers, regulators, and users alike. Providers have to be aware of such risks when enabling access to their models; regulators have to prepare sufficient legal instruments to protect users' rights for privacy; social media platform owners should make their users aware of inference-based privacy threats; and users have to be aware of the full extent of how their privacy may be compromised and adjust their online behavior accordingly. With this work we hope to take an important step into this direction.

Limitations This work aims to provide the first characterization and evaluation of the inferencebased privacy threat arising from recent frontier VLMs. This evaluation is enabled by a manually collected real-world image dataset alongside a wide selection of manually annotated personal attributes. Due to the sensitive nature of such datasets and in line with previous works as well as ethical concerns, we decided not to release the VIP dataset publicly. While VIP allowed us to make a qualitative assessment of the discussed risks, we believe that the field may benefit from future efforts in constructing larger-scale public benchmarks. As similar ethical concerns apply here, we see well-curated synthetic benchmarks as a promising remedy to evaluation data limitations.

7 Conclusion

In this work, we conducted the first investigation of the privacy risks emerging from the inference capabilities of frontier VLMs by tackling two key challenges: (1) To allow for a quantitative assessment, we constructed the first dataset for evaluating privacy-infringing inference from inconspicuous online images, and (2) we built a simple prompting scheme suitable for evaluating the full extent of potential private attribute inferences by enabling the evasion of current safeguards. Our evaluation shows that built-in safeguards of models are easily evaded, enabling the best model to achieve 77.6% overall accuracy. Our results indicate that large-scale, automated, and highly accurate inferences of private attributes from images posted online are already becoming feasible. With current defenses lacking, we, therefore, aim to raise awareness with our findings and appeal to the community for an increased focus on mitigating privacy threats from inferences with frontier VLMs.

References

OpenAI. Chatgpt, 2023. URL https://openai.com/blog/chatgpt .

Dan Milmo and agency. Chatgpt reaches 100 million users two months after launch. The Guardian , 2023. Accessed: 2024-02-01.

OpenAI. Gpt-4v(ision) system card, 2023. URL https://openai.com/research/ gpt-4v-system-card .

Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Team Gemini, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023.

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 , 2021.

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and Iason Gabriel. Ethical and social risks of harm from language models, 2021.

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674 , 2023.

European Union EU. General data protection regulation, 2016. URL https://gdpr-info. eu/ .

California CA. California consumer privacy act, 2018. URL https://leginfo. legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180AB375 .

European Union EU. Artificial intelligence act ai act, 2021. URL https://eur-lex.europa. eu/legal-content/EN/TXT/?uri=celex%3A52021PC0206 .

The White House WH. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence, 2023. URL https://tinyurl.com/bdhp4tsd .

Adam Satariano. Chatgpt is banned in italy over privacy concerns. The New York Times , 2023. Accessed: 2024-02-01.

Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview. net/forum?id=TatRHT_1cK .

Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual memorization in neural language models, 2023.

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035 , 2023.

Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The dawn of lmms: Preliminary explorations with gpt-4v(ision), 2023.

Robin Staab, Mark Vero, Mislav Balunovi´ c, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models, 2023.

OpenAI. Gpt-4 technical report, 2023.

Mary Meeker. Internet trends 2014. Kleiner Perkins Caufield Byers , 2014. Accessed: 2024-0201.

Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 , 2023.

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419 , 2023.

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119 , 2023.

Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Describing people: A poselet-based approach to attribute classification. In 2011 International Conference on Computer Vision , pages 1543-1550, 2011. doi: 10.1109/ICCV.2011.6126413.

Ehsan Yaghoubi, Farhad Khezeli, Diana Borza, SV Aruna Kumar, João Neves, and Hugo Proença. Human attribute recognition- a comprehensive survey. Applied Sciences , 10 (16), 2020. ISSN 2076-3417. doi: 10.3390/app10165608. URL https://www.mdpi.com/ 2076-3417/10/16/5608 .

Siddharth Ravi, Pau Climent-Pérez, and Francisco Florez-Revuelta. A review on visual privacy preservation techniques for active and assisted living. Multimedia Tools and Applications , 83(5): 14715-14755, 2024. doi: 10.1007/s11042-023-15775-2. URL https://doi.org/10.1007/ s11042-023-15775-2 .

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019.

Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and S Yu Philip. Multimodal large language models: A survey. In 2023 IEEE International Conference on Big Data (BigData) , pages 2247-2256. IEEE, 2023.

Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems , 34:200-212, 2021.

Constantin Eichenberg, Sidney Black, Samuel Weinbach, Letitia Parcalabescu, and Anette Frank. Magma-multimodal augmentation of generative models through adapter-based finetuning. arXiv preprint arXiv:2112.05253 , 2021.

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems , 35: 23716-23736, 2022.

Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, et al. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00598 , 2022.

Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, and Furu Wei. Language models are general-purpose interfaces. arXiv preprint arXiv:2206.06336 , 2022.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.08485 , 2023.

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2633-2650, 2021.

Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A Choquette-Choo, and Nicholas Carlini. Preventing verbatim memorization in language models gives a false sense of privacy. arXiv preprint arXiv:2210.17546 , 2022.

Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. Propile: Probing privacy leakage in large language models. arXiv preprint arXiv:2307.01881 , 2023.

Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago ZanellaBéguelin. Analyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539 , 2023.

Xiao Wang, Shaofei Zheng, Rui Yang, Aihua Zheng, Zhe Chen, Jin Tang, and Bin Luo. Pedestrian attribute recognition: A survey. Pattern Recognition , 121:108220, 2022. ISSN 0031-3203. doi: https://doi.org/10.1016/j.patcog.2021.108220. URL https://www.sciencedirect. com/science/article/pii/S0031320321004015 .

Xinhua Cheng, Mengxi Jia, Qian Wang, and Jian Zhang. A simple visual-textual baseline for pedestrian attribute recognition. IEEE Transactions on Circuits and Systems for Video Technology , 32(10):6994-7004, 2022. doi: 10.1109/TCSVT.2022.3178144.

Modesto Castrillón-Santana, Elena Sánchez-Nielsen, David Freire-Obregón, Oliverio J. Santana, Daniel Hernández-Sosa, and Javier Lorenzo-Navarro. Evaluation of a visual question answering architecture for pedestrian attribute recognition. In Nicolas Tsapatsoulis, Andreas Lanitis, Marios Pattichis, Constantinos Pattichis, Christos Kyrkou, Efthyvoulos Kyriacou, Zenonas Theodosiou, and Andreas Panayides, editors, Computer Analysis of Images and Patterns , pages 13-22, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-44237-7.

Xiao Wang, Jiandong Jin, Chenglong Li, Jin Tang, Cheng Zhang, and Wei Wang. Pedestrian attribute recognition via clip based prompt vision-language fusion. arXiv preprint arXiv:2312.10692 , 2023.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems , 35:24824-24837, 2022.

Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https: //llava-vl.github.io/blog/2024-01-30-llava-next/ .

Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M. Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh. Obelics: An open web-scale filtered dataset of interleaved image-text documents, 2023.

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:2312.08914 , 2023.

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238 , 2023.

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502 , 2023.

Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. Open llm leaderboard. https: //huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard , 2023.

Thomas Brewster. Chatgpt has been turned into a social media surveillance assistant. Forbes , 2023. Accessed: 2024-10-25.

Robin Staab, Mark Vero, Mislav Balunovi´ c, and Martin Vechev. Large language models are advanced anonymizers. arXiv preprint arXiv:2402.13846 , 2024.

Yubin Deng, Ping Luo, Chen Change Loy, and Xiaoou Tang. Pedestrian attribute recognition at far distance. In Proceedings of the 22nd ACM international conference on Multimedia , pages 789-792, 2014.

Dangwei Li, Zhang Zhang, Xiaotang Chen, and Kaiqi Huang. A richly annotated pedestrian dataset for person retrieval in real surveillance scenarios. IEEE transactions on image processing , 28(4):1575-1590, 2019.

Xihui Liu, Haiyu Zhao, Maoqing Tian, Lu Sheng, Jing Shao, Shuai Yi, Junjie Yan, and Xiaogang Wang. Hydraplus-net: Attentive deep features for pedestrian analysis. In Proceedings of the IEEE international conference on computer vision , pages 350-359, 2017.

Patrick Sudowe, Hannah Spitzer, and Bastian Leibe. Person attribute recognition with a jointlytrained holistic cnn model. In Proceedings of the IEEE International Conference on Computer Vision Workshops , pages 87-95, 2015.

Gaurav Sharma and Frederic Jurie. Learning discriminative spatial representation for image classification. In BMVC 2011-British Machine Vision Conference , pages 1-11. BMVA Press, 2011.

Appendix

A Further Experimental Details

In this section, we provide additional details about our experimental setting.

A.1 Prompts Used

For the results reported in Section 5, we run the models with the prompts specified in Table 6. Further, as GPT4-V and Gemini-Pro are capable enough to follow the output format required for later parsing of model responses (under the Appendix E.3 or the extended prompt in Appendix E.2), we do not use any additional post-processing on their output. For all other (open-source) models, we observe a much larger variance in the ability to follow the required output syntax. To address their incapability of outputting a structured output consistently, we utilize GPT-4 to restructure their responses into

a format we can easily parse without changing the inference result. For this, we use the restructuring prompt (see Appendix E.6). Similarly, we use a variation of the restructuring prompt (shown in Appendix E.6) to restructure the model responses of GPT4-V when we used the simple prompt from Appendix E.1 in our experiments to investigate impact of prompting to GPT4-V Table 4.

A.2 Model and Deployment Details

All closed-source models (i.e., GPT4-V and Gemini-Pro) were accessed through their respective APIs provided by OpenAI and Google. In particular, we used gpt-4-1106vision-preview for all experiments and gpt-4-1106-preview for output formatting and evaluation. For Gemini, we used gemini-pro-vision . All open-source models were run on a single Nvidia-H100 GPU instance. Experiments can be repeated in less than a day on similar hardware. We provide more detailed information about batch sizes and model quantizations in Table 7.

A.3 Further Details on Scoring

After successfully parsing all model outputs, we run a comparison script that evaluates whether a prediction made by the model is correct. For the free text attributes LOC , POI , and OCC , we use a semi-automated approach to classify predictions as correct or not correct. In a first step, we utilize GPT-4 to assess whether a given prediction-ground truth pair can be considered correct (P), correct but less precise (LP), or incorrect. For this purpose, we use a comparison prompt with in-context learning (Appendix E.7). We provide several examples of precise and less precise correct predictions in our in-context learning examples. Following this, we manually verify all decisions made by GPT-4 to ensure their alignment with human intuition and consistency across experiments. We only report the performance based on the resulting human-verified evaluations.

B Additional Results

In this section, we present additional results and ablations for the experiments shown in the main paper.

SEX

POI

AGE

INC

LOC

EDU

OCC

MAR

0.0

0.2

0.4

0.6

0.8

Accuracy [%]

Final Prompt

Extended Prompt

Naive Prompt

B.1 Accuracy Across Hardness Levels

As discussed in Section 4, we can generally divide the five hardness levels of the VIP dataset into two groups: (i) 1, 2, and 3, where the labelers required increasingly advanced reasoning for obtaining the label, but no external knowledge tools (such as internet search), and (ii) 4 and 5, where the labelers required external knowledge tools, with 5 indicating the need of extensive reasoning as well. We ablate the model performance of our most capable model,

GPT4-V, across all hardness levels in Table 8. For the hardness (1) to (3), which are purely based on reasoning, we make the same observation as Staab et al. [18], i.e., that model performance decreases with increasing inference complexity. However, somewhat counterintuitively, this trend does not carry over to hardness 4 and 5. We attest this to two potential reasons: (i) looking at Table 1, we note that for these levels, we consider almost exclusively location labels, which exhibit a higher baseline accuracy, and (ii) we, unfortunately, have only a small amount of labels for hardness 5. Nevertheless, these hardness categories are still knowledge-intensive, and the ∼ 80% accuracy GPT4-V achieves on them is remarkable, indicating that the model has been equipped with vast location knowledge in its pretraining.

B.2 Impact of Prompting on GPT4-V Across Attributes

Extending over the prompting choice impact experiment presented in the main paper, we further evaluate the impact of prompting choices on a per-attribute basis, providing results for GPT4-V in Fig. 5. We observe that the baseline prompt (Appendix E.1) fails to predict certain attributes (sex, age, education, marital status, and income) notably more than others (occupation, location, and place of the image) and suspect that this is a direct result of the specific alignment process of the model. Further, we notice how using the extended prompt already yields a significant improvement over this naive baseline. Our final prompt (Appendix E.3) improves on the results of the extended prompt even further on all but two attributes.

B.3 Prompt Choices for Open-Source Models

As our initial experiments have shown that weaker open-source models struggle at following specific, more complex prompt formats, we further ablate over prompt choices for open-source models. We show this in Table 9, where we highlight whether each model performs better when using the simple prompt in Appendix E.5 or the more complex final prompt in Appendix E.3. As LLaVa-NeXT and InternVL-Chat-V1.2-Plus use a larger language model as VLM-backbone and are more optimized to follow instructions, we observe that they can follow and benefit noticeably from more advanced prompts. In order not to underreport the capabilities of the open-source models we are using, we report the best performing prompt format when comparing models in Fig. 4.

C Dataset

C.1 Comparison of VIP with HAR and PAR Datasets

In this section, we list several popular HAR benchmarks, mostly centered around the task of pedestrian attribute recognition (PAR), and provide an overview of labeled features and image constituents, comparing them to our VIP dataset. We note that, unlike prior work, VIP is the first to focus on the inference of various personal attributes from images that do not center around depictions of natural humans. In particular, all listed datasets typically have only one or two labels for the attributes sex, age (intervals), or dress or posture details (e.g., whether a person wears a jacket). VIP, on the other hand, does not focus on depictions of humans but is aimed at providing a basis for investigating whether VLMs are capable of inferring attributes from small cues from shared pictures online. In addition to having binary and multi-class classification attributes, VIP also has several free-text attributes, such as occupation, location, and place of the image.

C.2 Further Dataset Statistics

As mentioned in Section 5, despite VIP's focus on non-human-depicting images, some image-attribute pairs ( 9 . 7% ) contain partial depictions of humans (even if they are not the primary focus of the image). In this subsection, we give a detailed overview of the parts of the dataset with and without human depictions. In particular, the tables Table 11a and Table 11b show how many image-attribute pairs VIP has over different hardness levels separately for each subset.

(b) Datapoints in VIP that do not contain (partial) human depictions.

Table 11: Label counts for each main personal attribute category across hardness levels in VIP.

D Labeling Instructions

In this section we give a detailed overview of our labeling process by presenting the detailed labeling instructions by which we labeled the Reddit images (indexed via RedCaps 2 ). To reduce the labeling effort, we only consider subreddits that are likely to contain images fit for evaluating the VLMs on the task of inferring personal attributes (for the list of included subreddits see Appendix D.4). We use similar labeling instructions as in [18].

D.1 Terminology

User refers to the Reddit account owner.

Human evaluator / evaluator refers to the person using the labeling App to infer attributes of the person the Reddit account belongs to.

Image refers to the image in the Reddit post the user posted on the social media platform.

Google Search refers to all sorts of methods the human evaluator can use to infer attributes of the image using reverse image search on parts or all of the image, text extraction from the image and using them in text search, or using google maps to find out a location.

Personal Attribute refers to the information we are interested about the user .

D.2 Filtering procedure

From a selected subset of subreddits in Appendix D.4 we first uniformly sample a subreddit and then uniformly sample an image from that subreddit.

D.3 Human Selection Overview

UI

Human Evaluators are presented with a single image randomly drawn from the dataset of images.

The Evaluator gets access to

-At the top left of the screen, the image once clicked, opens the image in fullscreen inside a new tab.

-At the bottom left of the screen, the row id of the datapoint and the posting time of the image

-The author, caption, subreddit of the image with a click of the 'More Information' button

-Google reverse image search results with a click of the 'Search on Google' button

-The Reddit post and comments with a click of the 'Reddit Post' button

-The author profile with a click of the author name once it is visible on the screen (after clicking the 'More Information' button).

-At the right side of the screen, several input fields (described below) in which the evaluator can enter whether the image or additional texts contain personal attribute information, rate how certain they are in their prediction, as well as hard it was to extract the personal attribute for them (rating defined below) and pick what was the information level available for them at the time of inferring the attribute.

-An 'Add Attribute' button to put extra attributes that the human evaluator cannot fit into a category.

-'Save', 'Next', 'Skip', 'Reset', 'Reset Time' buttons

-Time passed since the start of labeling the image

The goal of the evaluators is to curate a set of images and profiles containing personal attributes with varying degrees of extraction difficulty and information availability which later will be used to evaluate a VLM on the same task.

Guidelines

Personal Attributes We now outline what to consider when labeling each individual attribute:

Place of Image (Free-Text): Refers to the location of the image. Follow the same format as for 'Location'

Location (Free-Text): Refers to the location this person currently resides in. We are always looking for the most exact location information. E.g., When it is deducible that the image is taken in San Francisco / California / U.S.A. , it suffices to write San Francisco as state and country can be inferred. If there is even more precise information (the district / street / restaurant name), please also give the corresponding city. If a city name is insufficient for the unique identification, please add additional information (in the format 'city / state / country') to allow for unique identification.

Sex (2 Options): The sex of the post author selected from the two options male and female (in line with the ACS dataset format). If you believe a user does not fall into these categories, leave the field blank.

Age (Range): The user's age at the time of posting the image.You can specify a concrete age like 27 or a range 25-35 .

Occupation (Free-Text): The user's occupation. Give a brief descriptor such as 'Software Developer' or 'Teacher.' If it is clear from the image or comments that the person currently is not employed, indicate this by writing 'none.'

Place of Birth (Free-Text): The place where the user was born. Follow the same format as for 'Location.'

Marital Status (4 Options): The user's relationship status around the post time. No relation , (in a) Relation , Married , and Divorced .

Income (5 Options): The annual income in USD of the user at the time of posting the image selected from No , Low (1-30,000 USD), Medium (30,000-60,000 USD), High (60,000150,000 USD) and Very High (>150,000 USD).

EducationLevel (6 Options): The highest level of education achieved by the user No High School Diploma , In High School , High School Diploma , In College , College Degree , PhD .

Other attributes (Free-Text): Here you can add extra labeling components to add relevant information to the user that can be extracted from the image. Examples are:

Interests, Health/Medical, Political Orientation, Weight, Height, Hobbies, Lifestyle (Pet owner or not, owns kids or not), Religion, Diet, Behaviour etc.

In other attributes, the human evaluator should be more careful with assumptions. If you are providing more than one attribute such as a couple of 'interests' than separate them with comma 'music, rubic's cube, coffee' . Try to not make full sentences but rather simple word-level descriptions. Instead of 'The user plays the piano' use 'piano' . Also try to keep the same formulation and keep the usage consistent.

Rating scales You can rate each input with respect to Hardness and Certainty . We now give an overview of each scale

Hardness : We rate how hard it is to extract the personal attribute from 0 to 5

-0: Default value. You cannot extract the corresponding personal attribute.

-1: It is effortless to extract the personal attribute, i.e.,

It is explicitly written in the caption, e.g., 'My beard'

The sex of the image owner is explicitly visible

-2: The personal attribute is extractable in a straightforward manner without requiring strong deductive reasoning,

e.g., 'My wife and I are having our second child in April.'

e.g., You don't see the picture of the full person but you see some parts of their body.

-3: Extracting the personal attribute takes the same additional thinking such as combining pieces of information together

e.g., There are multiple people in the image of a room and based on the items, the occasion, pictures, deducing the marital status of the person.

-4: Extracting the personal attribute is challenging but achievable when having access to an online search page to look for specific information,

e.g., 'I love eating ice at stone rode' (Location Guelph / Ontario)

e.g., simple reverse image search shows where the location is

-5: Extracting the personal attribute is challenging and still requires considerable effort when accessing an online search page.

Examples here are mentions of specific intersections in cities, - e.g., 22nd and Flanders in Portland, for which one must consider different information in the text. E.g., for Portland, a reference to the legalization of weed in a specific timeframe.

reverse image search doesn't show direct results but give hints e.g. on the location and other images need to be searched and compared in detail.

information from the post, author profile and reverse image search needs to be combined.

additional google searches need to be conducted and the information from them need to be combined to resolve ambiguity

When you are evaluating hardness, pick the level that is applicable with the current information level you are using.

Certainty : You can rate your certainty w.r.t. the personal attribute extraction on a scale from 0 to 5

-0: Default value. You did not infer anything.

-1: You think that you extracted the personal attribute, but are very uncertain

-2: You think that you extracted the personal attribute correctly, but you could be mistaken

-3: You are quite certain you extracted the personal attribute correctly

-4: You are very certain that you extracted the personal attribute correctly

-5: You are absolutely certain that you extracted the personal attribute correctly

Information Level : You can select the information level that you have accessed to infer the personal attribute.

-No Information: Default value. You only used the image to infer the personal attribute.

-Post Information: You used the caption, author name, subreddit of the post that contains the image.

-+Reddit Post: You have in addition to the post information used the Reddit post of the image and its comments to extract the personal attribute.

-+Author Profile: You have used all the information available from the author profile (comments/posts)

Labeling Workflow

We now share detailled instructions on the workflow of labeling an image.

The human evaluator is presented with an image. Assuming the image belongs to the user, the human evaluator tries to infer as much information from the image as possible and if the evaluator can infer anything, they need to press the 'Save' button.

The human evaluator presses 'Search on Google' to conduct the reverse image search (mostly useful for location). In the reverse image search the human evaluator can crop different parts of the image to get a better understanding of the items in the image and do additional google searches, google maps searches but shouldn't use any LLMs. If any value changes from the previous step or new values are added with the new information the evaluator acquired then they need to press 'Save' again.

The human evaluator toggles the 'More information' button to open up a new set of information such as author, caption, subreddit. Based on this information, the human evaluator can infer a new personal attribute. If any value changes from the previous step or new values are added with the new information the evaluator acquired then they need to press 'Save' again.

The human evaluator opens the Reddit post by pressing the 'Reddit post' button. Based on this information the human evaluator can infer new personal attributes. If any value changes from the previous step or new values are added with the new information the evaluator acquired then they need to press 'Save' again.

The human evaluator presses the link in the author name and reads through the profile of the author both looking into comments and posts. Based on this information the human evaluator can infer new personal attributes. If any value changes from the previous step or new values are added with the new information the evaluator acquired then they need to press 'Save' again.

At the end of labeling a single image, the human evaluator presses the 'Next' button to repeat the process. If the human labeller inferred nothing, then they press the 'Skip' button.

D.4 List of Subreddits

'mealprepsunday', 'beerwithaview', 'interiordesign', 'pics', 'backpacking', 'homestead', 'houseplants', 'baking', 'motorcycleporn', 'bicycling', 'cityporn', 'healthyfood', 'designmyroom', 'coffeestations', 'desksetup', 'carpentry', 'mechanicalkeyboards', 'roomporn', 'amateurroomporn', 'streetphotography', 'itookapicture', 'urbanexploration', 'femalelivingspace', 'malelivingspace', 'battlestations', 'carporn', 'barista', 'gardening', 'amateurphotography', 'campingandhiking', 'houseporn', 'dogpictures', 'catpictures', 'ruralporn', 'vintage', 'cats', 'averagebattlestations', 'beerandpizza', 'bookporn', 'bookshelf', 'damnthatsinteresting', 'indoorgarden', 'infrastructureporn', 'interestingasfuck', 'outdoors', 'workbenches', 'doggos', 'workspaces'

E Prompt Templates

E.1 Naive Prompt

E.2 Extended Prompt

E.3 Final Prompt

E.4 Zoom Prompt

Zoom Prompt: Location

E.5 Open-Source Models Prompt

E.6 Output Structuring Prompts

Restructuring Prompt: Location

F i r s t output f o r t h e a t t r i b u t e s we l i s t .

Restructuring Prompt for Naive: Location

E.7 Automatic Comparison Prompt

Comparison Prompt: Location

Comparison Prompt: Ocupation

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims and contributions from the experiments have been clearly stated in the abstract and introduction (see Section 1).

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We list the limitations in Section 6

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: We do not have any theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We share all the necessary settings (see Section 5 and Appendix A.2) and the prompt templates (see Appendix E) to reproduce the model behavior.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We share the code for our methodology and experiments, detailed instructions to build a dataset (see Appendix D), and the labeling software. Although we are not able to share the exact dataset we have used, a dataset from the same distribution can be attained using the instructions and the labeling software. Running the code then should result in the expected behavior and experimental results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe the dataset we used for experiments in Section 4 and Appendix C. Furthermore all other details of the settings we used the models for inference can be found in Appendix A.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We include an experiment specifically studying the statistical significance and stability of our results in Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Information on compute resources can be found in Appendix A.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We are discussing the implications of our findings and the risks that come with it in Section 6. We don't share the exact dataset but describe our dataset in detail and provide the necessary tools to collect a dataset from the same distribution.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We share our discussion in Section 6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We do not release the dataset and we have completed a responsible disclosure.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safeguards.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite all the relevant material in Appendix D. We do not share any data and do the labeling ourselves to respect the licenses of our data sources.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: Yes we release the necessary code and prompts with the instructions to run it.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We explicitly do labeling ourselves and don't rely on crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We do not involve any human study or crowdsourcing that would require IRB approval.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a timely and important investigation into the privacy implications of Visual Language Models (VLMs) by evaluating their ability to predict sensitive personal information from images found online. The authors introduce a new dataset, which consists of:\n\n1. Images sourced from posts on selected subreddits.\n2. Eight categories of private attributes, including residence location, sex, age, and others.\n3. 554 manually-labeled attributes, each assigned a \"hardness score\" reflecting the degree of reasoning and online search required for annotation.\n\nThe authors assess several VLMs (black-box), with GPT4-V achieving the highest average accuracy at 77.6%. The authors further explore the impact of prompt engineering and implement automated zooming through prompting.\n\nSTRENGTHS:\n1. The paper studies the crucial and under-explored issue of privacy risks associated with VLMs.\n2. Unlike previous datasets that primarily rely on images of people, this dataset's annotated attributes only contain depictions of humans in 9.7% of cases. This allows for a more nuanced evaluation of VLMs' ability to infer sensitive information from subtler visual cues.\n3. The paper is well-written, with clear documentation of the annotation and evaluation procedures.\n\nWEAKNESSES:\n1. The ground truth labels are created by annotators, which may introduce subjectivity and may not always reflect real-world truths.\n2. The dataset is relatively small, and the absence of standard deviation reporting makes it difficult to assess the robustness of the results. If one labels another 500 attributes from another set of images collected using the same procedure, how would the accuracy change?\n3. It would be better if the authors could provide alignment rates between different annotators.\n\nQUESTIONS:\n1. What is the time spent on filtering subreddits and on annotating the images separately? This could help reproducibility.\n2. What are the estimated financial and time costs of human annotation compared to VLM-based annotation methods?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors focus on the privacy risks of multimodal LLMs by demonstrating the personal information can be extracted from publicly accessible images, and leveraged by LLMs to infer sensitive details.\n\nSTRENGTHS:\nThe intersection of privacy and multimodal AI is interesting, and the evaluation is strong, using both open and closed LLMs with vision capabilities. The paper is written well, and I appreciate the responsible disclosure.\n\nWEAKNESSES:\nI have two main issues with this paper:\n- The authors claim that inference of personal details from public images constitutes a privacy violation. Many privacy researchers would disagree with this. It could be claimed that a person voluntarily posting an image online implies that any content of this image and any subsequent inference is an acceptable risk to them. If the image is posted involuntarily, then it's not the LLM inference that constitutes the privacy violation, but the unauthorised posting of the image. In other words, the fact that VLMs can extract personal information from images, while potentially aggravating to users, is not straightforwardly a privacy violation per se. There is no particular discussion about this, or the potential regulatory implications or mitigations for the issue. \n- The fact that VLMs are capable of this type of inference is not particularly surprising to me, and the main takeaway is \"a human with some time on their hands, access to a search engine and some open-source intelligence skills could do the same, but it'd take a more time\". The only counter-argument here is scalability. There is no thorough discussion on potential incentives to perform this type of inference on an industrial scale.\n\nIn other words, I am hard-pressed to detect the strong, broad-interest contribution of this work beyond \"VLMs can do XYZ efficiently\", which is an addition to a long list of things that VLMs can do. Beyond this, the contributions are relatively low-impact: There is a some jailbreaking, and some engineering work to reveal details in the image by zooming automatically. For a NEURIPS paper, this is not a substantial enough contribution.\n\nBeyond this, the evaluation on a single dataset (sourced from the internet, see below) which is also not very large, and lack of a comparison against human observers, narrows the impact of the work.\n\nQUESTIONS:\n- What are the takeaways of your works in terms of concrete recommendations for society, platform owners, regulators, and privacy researchers? \n- Why was only a single dataset included? Can you certify that this dataset is representative of images encountered? I see that most of it is sourced from Reddit, are there any concerns of statistical bias from this method of selection?\n- Why is there no evaluation against human observers? I think this would be the most relevant and interesting type of comparison here: a systematic evaluation against humans with OSINT expertise. Perhaps parts of a labelling workflow on a blinded group of evaluators could be leveraged for this task?\n- What is the impact of hyperparameters? I see that the temperature was fixed. A more thorough evaluation could be useful here.\n- What are the costs of executing these inferences in your work?\n- It would be useful to have a bigger discussion around topics like e.g. uncertainty: The models can confidently hallucinate some personal property which misleads the \"attacker\" completely. How does this and similar phenomena affecting LLMs/VLMs interact with your findings?\n\nLIMITATIONS:\nPlease see the \"questions\" on some of the scientific limitations. \n\nI am a bit concerned about this study being executed without an institutional review board permission with the justification that \"human subjects were not involved\". Depending on jurisdiction, this can be contentious. You sourced images from Reddit, in my understanding, without asking for permission by the data owners, and conducted a scientific study on them. My understanding is that there was no statistical counselling about the representativeness of the dataset, or discussion with an ethics review board about the concerns arising from this study. In particular, the checklist states clearly that \"The answer NA means that the paper does not involve crowdsourcing\", and you answered \"NA\", but the dataset seems to be downloaded from Reddit, which (while not crowdsourcing in the strict sense) involves leveraging (in fact, attacking) human data without explicit consent, so I find the \"NA\" here problematic.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-05"}

comments: SUMMARY:
The paper performs a novel analysis privacy-leakage due to modern multimodal VLMs. Specifically, they show that modern VLMs, when correctly promoted, can infer sensitive information from seemingly innocuous images. They curate a dataset of images containing clues to sensitive information and query a few SOTA models using these [image, prompt] pairs, and show that the models are capable of inferring eight sensitive attributes from the images. The inference ability strengthens with the quality of models, which raises concerns about online privacy.

STRENGTHS:
- Very interesting, timely and practically relevant privacy analysis of VLMs
- The work raises appropriate concerns about privacy of online images
- Paper is very well-written and easy to follow

WEAKNESSES:
- The proposed attack although important seems quite easy to defend against
- The paper should evaluate at least a couple of  simplest of defenses, e.g., SFT.

QUESTIONS:
I really like the work due to its relevance to practice and shoutout to the authors for that! I have a few questions:
- The proposed attack seems super simple: it’s basically jailbreaking VLMs to elicit harmful responses. This is known type of an attack that is extensively studied, e..g, for LLMs. How will this attack work if a proprietary model provider does SFT using aligned data?

LIMITATIONS:
See questions above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
The paper proposes a new privacy attack with VLMs where the model is queried with an image and the goal is to predict the private attributes such as place and gender of the person not shown in the image.

STRENGTHS:
- Good experimental evaluations
- Clear problem formulation
- Practical automated attack using VLMs

WEAKNESSES:
Overall, I like the evaluation of the attack, although I feel it is missing a baseline comparison. For instance, how likely is it to predict the sensitive attributes like age, gender, etc. (without access to the VLM) using pure correlations in the data. If there is a pipeline for automatically annotating the important objects in the image, then a Bayesian model or a neural network can be trained to capture the correlations to the target attribute. This is also indicative from the experiments where the presence of partial human identifiers boosts the attack success. This type of correlation can be captured by a baseline model.

QUESTIONS:
Can you compare the effectivness of VLMs w.r.t. the baseline model that captures the general correlations between objects in the images?

LIMITATIONS:
The authors show a practical attack using modern VLMs where the adversary can infer sensitive attributes about an image by querying these models. While authors do not release their VIP dataset citing ethical concerns, I feel they need to acknowledge that an adversary can replicate their attack in practice, and should discuss more on the defenses against such attacks.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-12