Revisiting Model Inversion Evaluation: From Misleading Standards to Reliable Privacy Assessment

## Abstract

Abstract Model Inversion (MI) attacks aim to reconstruct information from private training data by exploiting access to machine learning models T 𝑇 T italic_T . To evaluate such attacks, the standard evaluation framework for such attacks relies on an evaluation model E 𝐸 E italic_E , trained under the same task design as T 𝑇 T italic_T . This framework has become the de facto standard for assessing progress in MI research, used across nearly all recent MI attacks and defenses without question. In this paper, we present the first in-depth study of this MI evaluation framework. In particular, we identify a critical issue of this standard MI evaluation framework: Type-I adversarial examples . These are reconstructions that do not capture the visual features of private training data, yet are still deemed successful by the target model T 𝑇 T italic_T and ultimately transferable to E 𝐸 E italic_E . Such false positives undermine the reliability of the standard MI evaluation framework. To address this issue, we introduce a new MI evaluation framework that replaces the evaluation model E 𝐸 E italic_E with advanced Multimodal Large Language Models (MLLMs). By leveraging their general-purpose visual understanding, our MLLM-based framework does not depend on training of shared task design as in T 𝑇 T italic_T , thus reducing Type-I transferability and providing more faithful assessments of reconstruction success. Using our MLLM-based evaluation framework,
we reevaluate 26 diverse MI attack setups and empirically reveal consistently high false positive rates under the standard evaluation framework. Importantly, we demonstrate that many state-of-the-art (SOTA) MI methods report inflated attack accuracy, indicating that actual privacy leakage is significantly lower than previously believed. By uncovering this critical issue and proposing a robust solution, our work enables a reassessment of progress in MI research and sets a new standard for reliable and robust evaluation. To support future work, we release our MLLM-based MI evaluation framework and benchmarking suite. 1 1 1 https://www.kaggle.com/datasets/hosytuyen/mi-reconstruction-collection

## 1 Introduction

Model Inversion (MI) attacks pose a significant privacy threat by attempting to reconstruct confidential information from sensitive training data through exploiting access to machine learning models. Recent state-of-the-art (SOTA) MI attacks Zhang et al. (2020); Chen et al. (2021); Wang et al. (2021a); Nguyen et al. (2023a, b); Qiu et al. (2024); Yuan et al. (2023) have shown considerable advancements, reporting attack success rates exceeding 90%. This vulnerability is particularly alarming for security-sensitive applications such as face recognition
Meng et al. (2021); Guo et al. (2020); Huang et al. (2020); Schroff et al. (2015), medical diagnosis Dufumier et al. (2021); Yang et al. (2022); Dippel et al. (2021), or speech recognition Chang et al. (2020); Krishna et al. (2019).

Figure: Figure 1: In this work, we present the first and in-depth study on the Model Inversion (MI) evaluation. Particularly, we investigate the most common MI evaluation framework $\mathcal{F}_{Curr}$ to measure MI Attack Accuracy (AttAcc). $\mathcal{F}_{Curr}$ is introduced in Zhang et al. (2020) and is utilized to assess almost all recent MI attacks/defenses. However, we find that $\mathcal{F}_{Curr}$ suffers from a significant number of false positives. These false positive MI reconstructed samples do not capture visual identity features of the target individual in the private training data, but they are still deemed successful attacks according to $\mathcal{F}_{Curr}$ with a high confidence (indicated in red text). Extensive visualization of false positives are included in the Supp.
Refer to caption: x1.png

Research gap. Recently, there are many studies on improving MI attacks Zhang et al. (2020); Fredrikson et al. (2014); An et al. (2022); Chen et al. (2021); Yuan et al. (2023); Qiu et al. (2024); Nguyen et al. (2023b, a); Kahla et al. (2022); Han et al. (2023) and MI defenses Wang et al. (2021b); Peng et al. (2022); Struppek et al. (2023); Ho et al. (2024); Koh et al. (2024). To assess the effectiveness of these MI attacks/defenses, MI Attack Accuracy (AttAcc) is a standard and the most important metric. To measure AttAcc, almost all recent MI studies adopt the evaluation framework introduced by Zhang et al. (2020), which we denote as $\mathcal{F}_{Curr}$. Under $\mathcal{F}_{Curr}$, an evaluation model $E$ is used to predict the identities of individuals based on the MI-reconstructed images. Model $E$ is trained on the same private dataset and follows the same training task design as in target model $T$.
In particular, consider the scenario where the adversary targets a specific identity $y$. From this point onward, we use
$y$ exclusively to denote the target class (label) in the MI attack, rather than a label predicted by any model.
The adversary produces reconstructed images $x^{r}_{y}$ of the target label $y$ by exploiting access to the target model $T$. To measure AttAcc, these $x^{r}_{y}$ samples are then passed through an evaluation model $E$. According to $\mathcal{F}_{Curr}$, the attack is deemed successful if $E$ classifies $x^{r}_{y}$ as $y$. Even though such MI evaluation framework becomes the de facto standard and the accuracy measured by $\mathcal{F}_{Curr}$ has been the most important metric in gauging MI research progress in almost all recent MI studies, there has not been any in-depth and comprehensive study to understand the accuracy of $\mathcal{F}_{Curr}$ and its limitations.

In this work, we conduct the first in-depth study on this standard MI evaluation framework $\mathcal{F}_{Curr}$. For an ideally successful attack, the reconstructed images $x^{r}_{y}$ should capture the visual identity features of $y$. However, we find that there exists a considerable number of MI reconstructed images that lack visual identity features of $y$, yet both target model $T$ and evaluation model $E$ under $\mathcal{F}_{Curr}$ still assign high probabilities to $y$, i.e., high values of $P_{E}(y|x^{r}_{y})$. Some examples are illustrated in Fig. [1](https://arxiv.org/html/2505.03519v3#S1.F1). These false positives potentially inflate the reported success rate of recent SOTA MI attacks.

To shed light on the causes of these false positives, we systematically discover the impact of Type I Adversarial Attacks Nguyen et al. (2015); Tang et al. (2019) in Model Inversion, highlighting a connection between two previously distinct research areas. Importantly, the optimization processes in MI attacks and Type I adversarial attacks are similar: both maximizing the likelihood with respect to (w.r.t.) the input under a fixed model. As a result, many false positives in MI may, in fact, be Type I adversarial examples. Moreover, due to the well-documented phenomenon of *adversarial transferability* Nguyen et al. (2015), these false positives can transfer to the MI evaluation model $E$. This transferability is especially pronounced when the evaluation model $E$ in $\mathcal{F}_{Curr}$ shares the same task design as the target model $T$ Liang et al. (2021); Papernot et al. (2016a). Ultimately, we demonstrate a fundamental issue in $\mathcal{F}_{Curr}$, potentially leading to unreliable assessment and inflated success rates reported for recent MI attacks under this evaluation framework.

To mitigate this issue, we propose a new MI evaluation framework $\mathcal{F}_{MLLM}$ that replaces the evaluation model $E$ with advanced Multimodal Large Language Models (MLLMs), leveraging their general-purpose visual understanding to determine whether a reconstruction constitutes a successful attack. While not relying on the same training task design used in $T$, $\mathcal{F}_{MLLM}$ minimizes Type-I transferability and offers a more faithful assessment of MI attacks. Using data annotated by $\mathcal{F}_{MLLM}$, we empirically benchmark the reliability of the common MI evaluation framework $\mathcal{F}_{Curr}$ and reveal consistently high false positive rates across 26 diverse MI setups. Our findings challenge the reliability of the standard MI evaluation framework and underscore the importance of adopting our MLLM-based approach for more reliable assessments.

We summarize our main contributions as below:

- •
We present the first in-depth study on the most common evaluation framework $\mathcal{F}_{Curr}$ to compute MI AttAcc. Our study identify the surprising effect of Type I Adversarial Features, as well as adversarial transferability Nguyen et al. (2015); Tang et al. (2019), on MI attacks, highlighting a relationship between two previously distinct research areas (see Sec. [3](https://arxiv.org/html/2505.03519v3#S3)). Our findings explain numerous false positives in $\mathcal{F}_{Curr}$.
- •
To mitigate this issue, we propose an MLLM-based MI evaluation framework $\mathcal{F}_{MLLM}$ (see Sec. [4](https://arxiv.org/html/2505.03519v3#S4)). Our framework leverages the powerful general-purpose visual understanding capabilities of advanced MLLMs to evaluate MI reconstructions, replacing the traditional evaluation model $E$. Without relying on the same training task design used in $T$, $\mathcal{F}_{MLLM}$ minimizes Type-I transferability and provides a more faithful assessment of MI attacks.
- •
Using data annotated by our evaluation framework, we empirically demonstrate that there are number type I adversarial examples under $\mathcal{F}_{\text{Curr}}$. Ultimately, our findings challenge the validity of this dominant evaluation framework and underscore the importance of adopting our proposed MI evaluation framework for more reliable assessments.

## 2 Related Work

Model Inversion (MI) aims to extract information about the training data given a trained model. Particularly, an adversary exploits a target model $T$ that was trained on a private dataset $\mathcal{D}_{priv}$. However, $\mathcal{D}_{priv}$ should not be disclosed. The main goal of MI attacks is to extract information about the private samples in $\mathcal{D}_{priv}$. The existing literature formulates MI attacks as a process of reconstructing an input $x^{r}_{y}$ that $T$ is likely to classify into the target class (label) $y$. For example, in facial recognition, MI attacks aim to reconstruct facial images that are likely to be identified as belonging to a particular person.

Model Inversion Attacks. One of the initial methods for MI is proposed by Fredrikson et al. Fredrikson et al. (2014), who discover that attackers could use a machine learning model to extract genomic and demographic information about patients. Their work is extended to facial recognition Fredrikson et al. (2015), demonstrating the potential to reconstruct identifiable facial images from model outputs. Advancing this concept, Yang et al. Yang et al. (2019) propose adversarial model inversion, treating the target model as an encoder and using a secondary network to reconstruct the original input data from the prediction vector. Recent advanced generative-based MI attack methods propose reducing the search space to the latent space by training a deep generator Zhang et al. (2020); Wang et al. (2021a); Chen et al. (2021); Yang et al. (2019); Yuan et al. (2023); Nguyen et al. (2023a); Struppek et al. (2022); Qiu et al. (2024), instead of directly performing MI attacks on high-dimensional space such as the image space. Specifically, GMI Zhang et al. (2020) and PPA Struppek et al. (2022) employ pretrained GAN models (e.g., WGAN Arjovsky et al. (2017) or StyleGAN Karras et al. (2019)) on an auxiliary dataset similar to private training data $\mathcal{D}_{priv}$. Inversion images are found through the latent vector of the generator. Recent efforts have aimed to enhance GAN-based MI methods from multiple perspectives. From the perspective of prior knowledge, KEDMI Chen et al. (2021) proposes training inversion-specific GANs using knowledge from the target model $T$. Similarly, Pseudo-Label Guided MI Yuan et al. (2023) utilizes pseudo-labels to guide conditional GAN training, while IF-GMI Qiu et al. (2024) leverages intermediate feature representations from pretrained GAN blocks. From MI objective perspective, max-margin loss Yuan et al. (2023) and logit loss Nguyen et al. (2023a) are introduced to address limitations in Cross-Entropy loss used in MI attacks. From MI overfitting perspective, LOMMA Nguyen et al. (2023a) employs augmented model concepts to improve generalizability of MI attacks. The Eq. [1](https://arxiv.org/html/2505.03519v3#S2.E1) represents the general step of existing SOTA MI attacks.

$$ $w^{*}=\arg\min_{w}(-\log P_{T}(y|G(w))+\lambda\mathcal{L}_{prior}(w))$ (1) $$

Here, $-\log\mathcal{P}_{T}(y|G(w))$ represents the identity loss, guiding the reconstruction of $x^{r}_{y}=G(w)$ that is most likely to be classified as target class $y$ by target classifier $T$. The $\mathcal{L}_{prior}$ is a prior loss, using public information to establish a distributional prior via GANs, thereby guiding the inversion process towards meaningful reconstructions.

Model Inversion Defenses. In contrast to MI attacks, MI defenses aim to minimize the disclosure of training samples during the MI optimization process. To protect against MI attacks, the objective is to create a method to train the target classifier $T$ on $\mathcal{D}_{priv}$ in such a way that $T$ reveals as little information about $\mathcal{D}_{priv}$ as possible, regarding specific labels, while still achieving satisfactory model performance. Efforts have been made to develop defenses against MI attacks. MID Wang et al. (2021b) adds a regularizer to the target classifier’s objective during training, penalizing the mutual information between inputs $x$ and outputs $T(x)$. BiDO Peng et al. (2022) introduces a bilateral regularizer that minimizes the information about inputs $x$ in feature representations $z$ while maximizing the information about labels $y$ in $z$. Beyond regularization-based defenses, TL-DMI Ho et al. (2024) improves MI robustness through transfer learning, while LS Struppek et al. (2023) employs Negative Label Smoothing to improve MI robustness. Recently, MI robustness is also explored from a architecture perspective Koh et al. (2024).

Model Inversion Evaluation Metrics. To evaluate MI attacks and defenses, almost all existing research primarily relies on the standard MI evaluation framework introduced in Zhang et al. (2020) to compute MI AttAcc. We denote this framework as $\mathcal{F}_{Curr}$. Let consider the scenario where the adversary targets a specific class $y$. The adversary generates reconstructed images $x^{r}_{y}$ by exploiting access to the target model $T$. These $x^{r}_{y}$ are then passed through an evaluation model $E$ (distinct from $T$). According to $\mathcal{F}_{Curr}$, an attack is deemed successful if $E$ classifies $x^{r}_{y}$ as $y$. Other metrics such as KNN distance, FID, and Knowledge Extraction Score are also utilized by some studies. The KNN distance metric, used in Zhang et al. (2020); Chen et al. (2021); Nguyen et al. (2023a); Ho et al. (2024); Yuan et al. (2023); Struppek et al. (2022), measures the shortest feature distance between the reconstructed image and actual images from the private dataset corresponding to the target class. Features for this comparison are extracted by the evaluation model $E$ or an external feature extractor. FID is used in other works Peng et al. (2022); Qiu et al. (2024); Struppek et al. (2022); Yuan et al. (2023) to assess the quality and realism of the reconstructions. Knowledge Extraction Score used in Struppek et al. (2023) measure the discriminative information extracted by an MI attack. Particularly, a surrogate classifier is trained on MI reconstructed images and measure its accuracy on the target model’s original training data. Despite some studies employing diverse metrics, the $\mathcal{F}_{Curr}$ framework to compute MI AttAcc remains the most commonly used metric in almost MI attack and defense studies.

Even though $\mathcal{F}_{Curr}$ is used prevalently in almost all recent MI research, and the accuracy measured by $\mathcal{F}_{Curr}$ has been the most important metric in gauging MI research progress, there has not been any in-depth and comprehensive study to understand the accuracy of $\mathcal{F}_{Curr}$ and its limitations. In this work, we take the first step toward a comprehensive investigation of this MI evaluation framework.

## 3 Common MI Evaluation Framework has issues: A Connection between MI Attacks and Type I Adversarial Attacks

We discover, for the first time, the strong connection between Type I Adversarial Attacks and MI Attacks. Due to this strong connection, the adversarial type I examples could be generated during MI attacks. Additionally, due to the well-documented phenomenon of adversarial transferability Nguyen et al. (2015), these adversarial type I examples can transfer to $E$ in $\mathcal{F}_{Curr}$. Ultimately, these phenomenons results in unreliable assessment for $\mathcal{F}_{Curr}$.

### 3.1 An Overview of Adversarial Attacks

An adversarial attack on machine learning models is an intentional manipulation of input data to cause incorrect predictions, highlighting potential vulnerabilities of the model. Adversarial attacks aim to create inputs that deceive machine learning classifiers into making errors while humans do not. There are two main types of adversarial attacks: Type I and Type II. Type II Adversarial Attacks Goodfellow et al. (2014); Szegedy et al. (2013); Kurakin et al. (2016); Papernot et al. (2016b); Carlini and Wagner (2017); Moosavi-Dezfooli et al. (2016); Shafahi et al. (2019) are commonly studied and aim to produce false negatives. In this attack, minor and imperceptible perturbations are added to the input data $x$ to generate an adversarial example $x^{advII}$, which is incorrectly classified by the model. Mathematically, this is represented as:

$$ $f(x^{\text{adv-II}})\neq f(x)\text{ and }f_{oracle}(x^{\text{adv-II}})=f_{ oracle}(x)$ (2) $$

Here, $f$ is the model under attack, $f_{oracle}$ is an oracle or hypothetical, idealized classifier. In addition to Type II attack, Type I Adversarial Attacks are designed in Nguyen et al. (2015); Tang et al. (2019) to generate false positives by creating examples that are significantly different from the original input but are still classified as the same class by the model. This involves producing an adversarial example $x^{\text{adv-I}}$ that, despite being significantly different from input $x$, the target model $f$ mis-classifies as the original class. Mathematically, Type I adversarial attack sample is represented as Nguyen et al. (2015); Tang et al. (2019):

$$ $f(x^{\text{adv-I}})=f(x)\text{ and }f_{oracle}(x^{\text{adv-I}})\neq f_{oracle }(x)$ (3) $$

One way to produce Type I adversarial examples involves optimizing the input by iteratively updating it to maximize the likelihood under a fixed target model classifier Nguyen et al. (2015). This process optimizes the input to remain within a targeted decision boundary, however, it is different from training data by human perception. This phenomenon can also be viewed as over-confidence phenomenon of machine learning models Wei et al. (2022); Guo et al. (2017).

### 3.2 The Strong Connection Between Model Inversion Attacks and Type I Adversarial Attacks

Table: Table 1: Connection between False Positive (FP) in MI (Eq. [4](https://arxiv.org/html/2505.03519v3#S3.E4)) and Type I adversarial attack (Eq. [3](https://arxiv.org/html/2505.03519v3#S3.E3)) Nguyen et al. (2015); Tang et al. (2019).

In the following, we analyze the connection between MI attacks and Type I adversarial attacks. The general inversion step used in SOTA MI attacks is described in Eq. [1](https://arxiv.org/html/2505.03519v3#S2.E1). As a result, reconstructed images often exhibit high likelihood under the target classifier $T$. However, not all reconstructed images successfully capture the visual identity features of the target individual from the private training data. Some examples are illustrated in Fig. [1](https://arxiv.org/html/2505.03519v3#S1.F1). We refer to these cases as false positives (under $T$) in MI.

Mathematically, false positives in MI can be interpreted similarly to Type I adversarial attack samples as represented in Eq. [3](https://arxiv.org/html/2505.03519v3#S3.E3). Importantly, both MI and Type I adversarial attacks exploit a fixed target model by maximizing the likelihood under the fixed target model w.r.t. the input. Specifically, with target classifier $T$ and $f_{oracle}$ denoting the oracle, false positive in MI attack is represented as:

$$ $\begin{split}T(x^{r}_{y})&=T(x_{y})=y\\ \text{ and }f_{oracle}(x^{r}_{y})&\neq f_{oracle}(x_{y})\end{split}$ (4) $$

Here, $x^{r}_{y}$ does not resemble the visual identity feature of $x_{y}$.
However, $T$ classifies $x^{r}_{y}$ as the target label $y$.
Hence, $x^{r}_{y}$ is a false positive in MI.
By comparing Eq. [4](https://arxiv.org/html/2505.03519v3#S3.E4) and Eq. [3](https://arxiv.org/html/2505.03519v3#S3.E3), the connection between false positives in MI and Type I adversarial can be noted, with the correspondences listed in Tab. [1](https://arxiv.org/html/2505.03519v3#S3.T1).

In Eq. [4](https://arxiv.org/html/2505.03519v3#S3.E4), an alternative interpretation is that the false positive $x^{r}_{y}$ is the example that can deceive $T$ to classify it as the target label $y$ while oracle classifier can not recognize $x^{r}_{y}$ as $y$.

### 3.3 Adversarial Transferability Can Lead to Critical Issues in the Common Model Inversion Evaluation Framework

In $\mathcal{F}_{\text{Curr}}$, an evaluation model $E$ is used to predict the identities of individuals based on MI-reconstructed images. The model $E$ is trained on the same private dataset and follows the same training task design (i.e., an $n$-way classification task). Prior work has shown that adversarial examples crafted for one model can often transfer and mislead other models, even those with different architectures, as long as they are trained on the similar dataset and task design Nguyen et al. (2015); Papernot et al. (2016a); Liang et al. (2021). Therefore, the similarity between $E$ and the target model $T$ may enable the transferability of Type I adversarial examples from $T$ to $E$.
This phenomenon can be expressed mathematically as:

$$ $\displaystyle T(x^{r}_{y})$ $\displaystyle=T(x_{y})=y$ (5) $\displaystyle E(x^{r}_{y})$ $\displaystyle=E(x_{y})=y$ $\displaystyle\text{ and }f_{oracle}(x^{r}_{y})$ $\displaystyle\neq f_{oracle}(x_{y})$ $$

In Sec. [4.2](https://arxiv.org/html/2505.03519v3#S4.SS2), we provide a quantitative results to support our analysis and demonstrate how of Type I adversarial perturbations inflate the false-positive rate of MI, causing an overestimation of the attack’s true effectiveness.

Figure: Figure 2: An example of evaluation query in our $\mathcal{F}_{\text{{MLLM}}}$. The task is to determine whether “Image A” depicts the same individual as those in “Image B”. We have two setups: (1) “Image A” and “Image B” consist of private images (Sec. [4.1](https://arxiv.org/html/2505.03519v3#S4.SS1)) and (2) “Image A” is an MI-reconstructed image $x^{r}_{y}$ of the target label $y$ while four real images of $y$ are randomly selected as “Image B” (Sec. [4.2](https://arxiv.org/html/2505.03519v3#S4.SS2)). Gemini is tasked with responding either “Yes” or “No” to indicate whether “Image A” matches the identity in “Image B”. The detailed prompt can be found in the Supp.
Refer to caption: x2.png

## 4 Our Proposed MLLM-Based Model Inversion Evaluation Framework

In this section, we introduce a novel evaluation framework, $\mathcal{F}_{\text{MLLM}}$. To mitigate the undesirable effects of adversarial transferability of Type I adversarial examples under the target model $T$, our key idea is to employ a model trained under a fundamentally different learning regime. Particularly, $\mathcal{F}_{\text{MLLM}}$ leverages a MLLM to determine attack success. MLLMs are ideal candidate because (1) they are trained on broad, general-purpose tasks using data and optimization pipelines that are entirely distinct from those of the target model; (2) they have demonstrated strong capabilities in general-purpose visual understanding; and (3) they are increasingly used for automated data labeling, offering both robustness and scalability in both academy and industry Community (2025); Smith and Chen (2024); Lee and Patel (2024); Zhou et al. (2024). Particularly, we employ Gemini DeepMind (2023) for $\mathcal{F}_{\text{MLLM}}$.

In the following section, we first demonstrate that Gemini is a reliable evaluators for our MI setups in Sec. [4.1](https://arxiv.org/html/2505.03519v3#S4.SS1). Then, we re-evaluate MI attack accuracy using $\mathcal{F}_{\text{MLLM}}$ in Sec. [4.2](https://arxiv.org/html/2505.03519v3#S4.SS2). Finally, we provide quantitative evidence supporting our analysis in Sec. [3](https://arxiv.org/html/2505.03519v3#S3) on the effect of Type I adversarial attacks on MI.

### 4.1 Gemini as a Reliable Evaluator

In $\mathcal{F}_{MLLM}$, we employ Gemini to assess the success of MI attacks given a query shown in Fig. [2](https://arxiv.org/html/2505.03519v3#S3.F2). Gemini is pretrained on a large and diverse corpus that includes both images and text across a wide range of domains, demonstrating strong capabilities in understanding interleaved image and text information Pramanick et al. (2024). In this section, we show that Gemini can also serve as an effective evaluator for our MI attack setups.

Setup. An effective evaluator is a model that can accurately recognize samples from a private dataset Zhang et al. (2020); Struppek et al. (2022). To assess our $\mathcal{F_{MLLM}}$ on this, Gemini is shown queries similar to Fig. [2](https://arxiv.org/html/2505.03519v3#S3.F2). Specifically, we utilize the Gemini-2.0-Flash model in our framework.
We design two test sets: $\bullet$ Positive pairs: “Image A” is a real image of the matching individual from “Images B”; the expected answer is “Yes”. $\bullet$ Negative pairs: “Image A” shows a real image of a different individual presented in “Images B”; the expected answer is “No”. Gemini is considered a reliable evaluator if it consistently returns high “Yes” rates for Positive pairs and high “No” rates for Negative pairs. We conduct experiments on the two common MI attacks datasets, FaceScrubNg and Winkler (2014) and CelebALiu et al. (2015), which are used in our reassess evaluation in Sec. [4.2](https://arxiv.org/html/2505.03519v3#S4.SS2).

Table: Table 2: We conduct an experiment to demonstrate Gemini’s effectiveness in recognizing samples from the private dataset. This results establish that Gemini can serve as a reliable evaluator in MI attack setups.

Results. We report the results in Tab. [2](https://arxiv.org/html/2505.03519v3#S4.T2). We observe consistently high “Yes” rates for Positive pairs and high “No” rates for Negative pairs across datasets. This suggests effectiveness of Gemini in recognizing samples from the private dataset in MI attack setups. Importantly, since Gemini is trained under a fundamentally different learning regime, the adversarial transferability is minimized. Ultimately, all these support that $\mathcal{F}_{MLLM}$ powered by Gemini is a reliable evaluator, especially in our MI attack setups.

Our $\mathcal{F}_{\text{MLLM}}$ powered by Gemini is also reliable from a randomness perspective. In the Supp., we provide results from three evaluation runs and show that the outcomes are consistent across runs.

### 4.2 Reassessing MI Attack Accuracy Using Our MLLM-Based Evaluation Framework

We empirically reassess SOTA MI attacks using our proposed evaluation framework $\mathcal{F}_{\text{MLLM}}$. Importantly, we quantitatively show that there are many Type I adversarial examples, which are classified as successful by $\mathcal{F}_{\text{Curr}}$ but do not to capture true visual identity. The false positive rates is consistently high and up to 99% (See Tab. [3](https://arxiv.org/html/2505.03519v3#S4.T3)). This demonstrates that many SOTA MI methods report inflated attack accuracy, indicating that actual privacy leakage is significantly lower than previously believed.

Experimental Setups. Using our evaluation framework $\mathcal{F}_{\text{MLLM}}$, we reassess a diverse set of 26 SOTA MI attacks spanning 5 attacks, 4 defenses, 3 private datasets, 4 public datasets, and 8 target model $T$. We follow the attack setups in their original papers. The detailed experimental setup can be found in the Supp. We make this collection of MI-reconstructed images public for future study(^2^22https://www.kaggle.com/datasets/hosytuyen/mi-reconstruction-collection).

Significant False Positive by $\mathcal{F}_{Curr}$ are Type I adversarial examples. An ideally successful attack, $x^{r}_{y}$ should capture the visual identity of $y$. However, for a successful attack as according to $\mathcal{F}_{Curr}$, $x^{r}_{y}$ only needs to be classified as $y$ by an evaluation model $E$. As shown in Fig. [1](https://arxiv.org/html/2505.03519v3#S1.F1), we observe that within MI-reconstructed images $x^{r}_{y}$, there are cases where the visual identity to $y$ is minimal. Nevertheless, $E$ assigns very high probabilities to $y$ for these examples, i.e., high values of $P_{E}(y|x^{r}_{y})$. We refer to these cases as false positives under the $\mathcal{F}_{Curr}$ framework. To better understand the extent of this false positive rate, we compare the ground truth success rate (established using our $\mathcal{F}_{MLLM}$) to the success rate as measured by $\mathcal{F}_{Curr}$ framework. Particularly, given the MLLM-annotated labels and the prediction via $\mathcal{F}_{Curr}$, we compute the False Positives (FP) rate, False Negatives (FN) rate, True Positives (TP) rate, and True Negatives (TP) rate for each MI setup. The AttAcc via $\mathcal{F}_{Curr}$ is computed as:

$$ $\text{AttAcc}_{\mathcal{F}_{Curr}}=\frac{FP+TP}{FN+TP+FP+TN}$ (6) $$

Table: Table 3: Our investigation on MI evaluation framework using our comprehensive dataset of MI attack samples. Our investigation are based standard 26 MI setups focusing on SOTA and well established MI attacks/defense including PPA Struppek et al. (2022), LOMMA Qiu et al. (2024), KEDMI Chen et al. (2021), PLGMI Yuan et al. (2023), IFGMI Qiu et al. (2024), TL Ho et al. (2024), TTS Koh et al. (2024), RoLSS Koh et al. (2024), and LS Struppek et al. (2024). These 26 MI setups span $8$ target classifier $T$, $3$ private dataset $\mathcal{D}_{priv}$, $4$ public dataset $\mathcal{D}_{pub}$. Detailed descriptions of the experimental setups and defense results are provided in the Supp. Overall, under $\mathcal{F}_{Curr}$, the results show that the number of false positives (FP) rates is significant. As a results, the threat of MI attacks has been overestimated in SOTA MI attacks, and the extent of leaked information is significantly less than previously assumed. E.g. SOTA MI attacks such as IFGMI Qiu et al. (2024), LOMMA Nguyen et al. (2023a), PLGMI Yuan et al. (2023), or PPA Struppek et al. (2022) claim over 90% to nearly 100% AttAcc in certain setups. However, the actual privacy leakage of these attacks remains below 80% across all setups and some attacks with lower than 60% accuracy is reported.

The results in Tab. [3](https://arxiv.org/html/2505.03519v3#S4.T3) consistently show that the FP rates are significant high across MI setups. In other words, there are numerous MI reconstructed images that do not capture visual identity of, yet they are deemed success by $\mathcal{F}_{Curr}$. Such high FP rate contributes to the significant inflation in reported AttAcc via $\mathcal{F}_{Curr}$ of latest SOTA MI attack such as PPA Struppek et al. (2022), PLGMI Yuan et al. (2023), IFGMI Qiu et al. (2024), or LOMMA Nguyen et al. (2023a). Notably, in their reported AttAcc using $\mathcal{F}_{Curr}$, these recent attacks report AttAcc values exceeding 90%, or even nearly 100% for certain setups. However, across a wide range of MI setups, the actual success rates never reach 80%. While we focus on high FP rates, FN rates also reveal limitations in the $\mathcal{F}_{Curr}$. Across MI setups, FN rates are consistently lower than FP rates. The FN rates depend on the classification accuracy and generalization capability of $E$. For example, under the PLGMI attack, when $E=$ FaceNet112 is trained on CelebA and evaluated with MI reconstructed images also from CelebA prior ($\mathcal{D}_{pub}=$ CelebA), FN rates are lower. In contrast, if this $E$ is evaluated with MI reconstructed images from FFHQ prior ($\mathcal{D}_{pub}$ = FFHQ), FN rates increase due to distribution shifts.

Furthermore, in certain MI setups, we find that $\mathcal{F}_{Curr}$ does not align well with $\mathcal{F}_{MLLM}$ in evaluating MI attacks. For example, in the setup of MaxViT as $T$ under the PPA attack, the AttAcc measured by $\mathcal{F}_{Curr}$ is 11.91% lower than the setup for ResNet18 as $T$ under the same attack. However, the MaxViT as $T$ setup shows a 1.82% higher AttAcc measured by $\mathcal{F}_{MLLM}$ than the ResNet18 as $T$ setup. This suggests that, although less effective, the common MI evaluation framework $\mathcal{F}_{Curr}$ could rate the attack as more successful than it actually is.

In conclusion, our analysis shows that the common MI evaluation framework $\mathcal{F}_{Curr}$ is suffered from very high FP rate, significantly affecting the reported results of contemporary MI studies based on $\mathcal{F}_{Curr}$.

Table: Table 4: Our controlled experiment to show the effect of Type I adversarial attacks in MI on false positive rates. We compare false positive rate of negative $x^{r}_{y}$ and negative $x^{natural}_{y}$. Many $x^{r}_{y}$ are affected by Type I adversarial attack during MI, whereas Neg $x^{natural}_{y}$ is free from Type I adversarial feature (see main text for creation of Neg $x^{natural}_{y}$). Due to Type I adversarial attack, $x^{r}_{y}$ has a significantly high false positive rate. We conduct this experiment on a comprehensive setting with all evaluation models $E$ used in previous MI studies (i.e., InceptionNetV3 for PPA on and FaceNet112 for PLGMI) and an additional architecture of $E$ (i.e., MaxViT for PPA). Further results on this experiments can be found in the Supp.

The effect of Type I adversarial features on MI in producing false positives. Due to the similarity nature of Type I adversarial attacks and MI, i.e. maximizing the likelihood with respect to (w.r.t) input under a fixed model, we hypothesize that a significant number of MI reconstructed samples $x^{r}_{y}$ carrying Type I adversarial features. As a result, they are mis-classified by $T$ and $E$ into the target label $y$ although these $x^{r}_{y}$ do not resemble $x_{y}$
(Eq. [5](https://arxiv.org/html/2505.03519v3#S3.E5)).
We conduct experiments to validate our hypothesis.
We take the two MI setups: $\bullet$ Setup 1: $T$= ResNet18, $\mathcal{D}_{priv}$= FaceScrub, attack = PPA, $E$= InceptionV3/ MaxVIT $\bullet$ Setup 2: $T$= VGG16, $\mathcal{D}_{priv}$= CelebA, attack = PLGMI, $E$= FaceNet112.
We conduct this experiment on a comprehensive setting with all evaluation models $E$ used in previous MI studies (i.e., InceptionNetV3 for PPA on and FaceNet112 for PLGMI) and an additional architecture of $E$ (i.e., MaxViT for PPA). We first perform MI attacks to obtain MI-generated samples $x^{r}_{y}$.
We then identified all MI-generated negative samples, denoted as Neg $x^{r}_{y}$, through MLLM annotation, i.e., $f_{MLLM}($Neg $x^{r}_{y})\neq f_{MLLM}(x_{y})$. Let $n$ denote the number of MI-generated negative samples $|$Neg $x^{r}_{y}|=n$.

We create another dataset of natural negative samples, denoted as Neg $x^{natural}_{y}$, which is free from Type I adversarial attack, for controlled experiments. We do so by randomly selecting $n$ FFHQ images (no class overlapping with FaceScrub), and
intentionally mis-label these FFHQ images with randomly selected FaceScrub identities, obtaining $n$ Neg $x^{natural}_{y}$
($f_{MLLM}($Neg $x^{natural}_{y})\neq f_{MLLM}(x_{y})$).
Importantly, unlike Neg $x^{r}_{y}$,
these Neg $x^{natural}_{y}$ are randomly selected from FFHQ and therefore they are free from Type I adversarial attack.
Neg $x^{natural}_{y}$ is our controlled dataset.

We pass Neg $x^{r}_{y}$ and Neg $x^{natural}_{y}$ into $E$, and count the number of false positive, i.e., Neg $x^{r}_{y}$/ Neg $x^{natural}_{y}$ being mis-classified into ${y}$.
The false positive rate of Neg $x^{r}_{y}$ and Neg $x^{natural}_{y}$ are compared in Tab. [4](https://arxiv.org/html/2505.03519v3#S4.T4). MI-generated negatives Neg $x^{r}_{y}$, with Type I adversarial feature learned during MI, have a high FP rate, whereas natural negatives Neg $x^{natural}_{y}$, which are free from Type I adversarial feature, have a low FP rate. For example, in Setup 1, FP rates of Neg $x^{r}_{y}$ is 90.09% while FP rates of Neg $x^{natural}_{y}$ is 0.94%. This experiment further demonstrates the effect of Type I Adversarial features in MI evaluation resulting in a significant number of false positives.

## 5 Conclusion

This work identifies a critical issue in the standard evaluation framework for Model Inversion attacks: the inflation of attack success due to Type I adversarial examples that do not capture true visual identity. To address this, we propose a reliable MLLM-based MI evaluation framework that minimize the impact of Type I adversarial transferability. Our extensive empirical analysis across 26 MI attack setups demonstrates that false positive rates under the standard evaluation framework can reach up to 99%, severely overstating actual privacy leakage. With our proposed evaluation framework, we offer a more accurate and robust way to measure MI attack success, setting a new standard for evaluating privacy risks in machine learning systems. We release our MLLM-based MI evaluation framework and benchmarking suite for future MI study.

## References

- An et al. [2022]
Shengwei An, Guanhong Tao, Qiuling Xu, Yingqi Liu, Guangyu Shen, Yuan Yao, Jingwei Xu, and Xiangyu Zhang.
Mirror: Model inversion for deep learning network with high fidelity.
In *Proceedings of the 29th Network and Distributed System Security Symposium*, 2022.
- Arjovsky et al. [2017]
Martin Arjovsky, Soumith Chintala, and Léon Bottou.
Wasserstein generative adversarial networks.
In *International conference on machine learning*, pages 214–223. PMLR, 2017.
- Carlini and Wagner [2017]
Nicholas Carlini and David Wagner.
Towards evaluating the robustness of neural networks.
In *2017 ieee symposium on security and privacy (sp)*, pages 39–57. Ieee, 2017.
- Chang et al. [2020]
Xuankai Chang, Wangyou Zhang, Yanmin Qian, Jonathan Le Roux, and Shinji Watanabe.
End-to-end multi-speaker speech recognition with transformer.
In *ICASSP 2020*, pages 6134–6138. IEEE, 2020.
- Chen et al. [2021]
Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi.
Knowledge-enriched distributional model inversion attacks.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 16178–16187, 2021.
- Community [2025]
Google Cloud Community.
Simplifying data preparation for gen ai with google’s gemini.
Google Cloud Community, 2025.
https://www.googlecloudcommunity.com/gc/Cloud-Product-Articles/Simplifying-Data-Preparation-for-Gen-AI-with-Google-s-Gemini/ta-p/841925.
- DeepMind [2023]
Google DeepMind.
Gemini: A family of highly capable multimodal models.
*arXiv preprint arXiv:2312.11805*, 2023.
URL [https://arxiv.org/abs/2312.11805](https://arxiv.org/abs/2312.11805).
- Dippel et al. [2021]
Jonas Dippel, Steffen Vogler, and Johannes Höhne.
Towards fine-grained visual representations by combining contrastive learning with image reconstruction and attention-weighted pooling.
*arXiv preprint arXiv:2104.04323*, 2021.
- Dufumier et al. [2021]
Benoit Dufumier, Pietro Gori, Julie Victor, Antoine Grigis, Michele Wessa, Paolo Brambilla, Pauline Favre, Mircea Polosan, Colm Mcdonald, Camille Marie Piguet, et al.
Contrastive learning with continuous proxy meta-data for 3d mri classification.
In *MICCAI 2021*. Springer, 2021.
- Fredrikson et al. [2015]
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart.
Model inversion attacks that exploit confidence information and basic countermeasures.
In *Proceedings of the 22nd ACM SIGSAC conference on computer and communications security*, pages 1322–1333, 2015.
- Fredrikson et al. [2014]
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart.
Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing.
In *23rd USENIX Security Symposium (USENIX Security 14)*, pages 17–32, 2014.
- Goodfellow et al. [2014]
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.
Explaining and harnessing adversarial examples.
*arXiv preprint arXiv:1412.6572*, 2014.
- Guo et al. [2017]
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.
On calibration of modern neural networks.
In *ICML*, pages 1321–1330. PMLR, 2017.
- Guo et al. [2020]
Jianzhu Guo, Xiangyu Zhu, Chenxu Zhao, Dong Cao, Zhen Lei, and Stan Z Li.
Learning meta face recognition in unseen domains.
In *CVPR*, pages 6163–6172, 2020.
- Han et al. [2023]
Gyojin Han, Jaehyun Choi, Haeil Lee, and Junmo Kim.
Reinforcement learning-based black-box model inversion attacks.
In *CVPR*, pages 20504–20513, 2023.
- Ho et al. [2024]
Sy-Tuyen Ho, Koh Jun Hao, Keshigeyan Chandrasegaran, Ngoc-Bao Nguyen, and Ngai-Man Cheung.
Model inversion robustness: Can transfer learning help?
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 12183–12193, 2024.
- Huang et al. [2020]
Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang.
Curricularface: adaptive curriculum learning loss for deep face recognition.
In *proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 5901–5910, 2020.
- Kahla et al. [2022]
Mostafa Kahla, Si Chen, Hoang Anh Just, and Ruoxi Jia.
Label-only model inversion attacks via boundary repulsion.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 15045–15053, 2022.
- Karras et al. [2019]
Tero Karras, Samuli Laine, and Timo Aila.
A style-based generator architecture for generative adversarial networks.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 4401–4410, 2019.
- Koh et al. [2024]
Jun Hao Koh, Sy-Tuyen Ho, Ngoc-Bao Nguyen, and Ngai-man Cheung.
On the vulnerability of skip connections to model inversion attacks.
*arXiv preprint arXiv:2409.01696*, 2024.
- Krishna et al. [2019]
Gautam Krishna, Co Tran, Jianguo Yu, and Ahmed H Tewfik.
Speech recognition with no speech or with noisy speech.
In *ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 1090–1094. IEEE, 2019.
- Kurakin et al. [2016]
Alexey Kurakin, Ian Goodfellow, and Samy Bengio.
Adversarial machine learning at scale.
*arXiv preprint arXiv:1611.01236*, 2016.
- Lee and Patel [2024]
Sang Lee and Anisha Patel.
Gemini-assisted deep learning classification model for automated medical image annotation.
*Journal of Digital Imaging*, 37:123–135, 2024.
- Liang et al. [2021]
Kaizhao Liang, Jacky Y Zhang, Boxin Wang, Zhuolin Yang, Sanmi Koyejo, and Bo Li.
Uncovering the connections between adversarial transferability and knowledge transferability.
In *International Conference on Machine Learning*, pages 6577–6587. PMLR, 2021.
- Liu et al. [2015]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.
Deep learning face attributes in the wild.
In *Proceedings of the IEEE international conference on computer vision*, pages 3730–3738, 2015.
- Meng et al. [2021]
Qiang Meng, Shichao Zhao, Zhida Huang, and Feng Zhou.
Magface: A universal representation for face recognition and quality assessment.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 14225–14234, 2021.
- Moosavi-Dezfooli et al. [2016]
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard.
Deepfool: a simple and accurate method to fool deep neural networks.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 2574–2582, 2016.
- Ng and Winkler [2014]
Hong-Wei Ng and Stefan Winkler.
A data-driven approach to cleaning large face datasets.
In *2014 IEEE international conference on image processing (ICIP)*, pages 343–347. IEEE, 2014.
- Nguyen et al. [2015]
Anh Nguyen, Jason Yosinski, and Jeff Clune.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 427–436, 2015.
- Nguyen et al. [2023a]
Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung.
Re-thinking model inversion attacks against deep neural networks.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2023a.
- Nguyen et al. [2023b]
Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai man Cheung.
Label-only model inversion attacks via knowledge transfer.
In *Thirty-seventh Conference on Neural Information Processing Systems*, 2023b.
URL [https://openreview.net/forum?id=NuoIThPPag](https://openreview.net/forum?id=NuoIThPPag).
- Papernot et al. [2016a]
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow.
Transferability in machine learning: from phenomena to black-box attacks using adversarial samples.
*arXiv preprint arXiv:1605.07277*, 2016a.
- Papernot et al. [2016b]
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami.
The limitations of deep learning in adversarial settings.
In *2016 IEEE European symposium on security and privacy (EuroS&P)*, pages 372–387. IEEE, 2016b.
- Peng et al. [2022]
Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han.
Bilateral dependency optimization: Defending against model-inversion attacks.
In *KDD*, 2022.
- Pramanick et al. [2024]
Shraman Pramanick, Rama Chellappa, and Subhashini Venugopalan.
SPIQA: A dataset for multimodal question answering on scientific papers.
In *The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track*, 2024.
URL [https://openreview.net/forum?id=h3lddsY5nf](https://openreview.net/forum?id=h3lddsY5nf).
- Qiu et al. [2024]
Yixiang Qiu, Hao Fang, Hongyao Yu, Bin Chen, MeiKang Qiu, and Shu-Tao Xia.
A closer look at gan priors: Exploiting intermediate features for enhanced model inversion attacks.
*arXiv preprint arXiv:2407.13863*, 2024.
- Schroff et al. [2015]
Florian Schroff, Dmitry Kalenichenko, and James Philbin.
Facenet: A unified embedding for face recognition and clustering.
In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 815–823, 2015.
- Shafahi et al. [2019]
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein.
Adversarial training for free!
*Advances in neural information processing systems*, 32, 2019.
- Smith and Chen [2024]
Laura Smith and Wei Chen.
Capabilities of gemini models in medicine.
*arXiv preprint arXiv:2404.18416*, 2024.
- Struppek et al. [2022]
Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting.
Plug & play attacks: Towards robust and flexible model inversion attacks.
*arXiv preprint arXiv:2201.12179*, 2022.
- Struppek et al. [2023]
Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting.
Be careful what you smooth for: Label smoothing can be a privacy shield but also a catalyst for model inversion attacks.
*arXiv preprint arXiv:2310.06549*, 2023.
- Struppek et al. [2024]
Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting.
Be careful what you smooth for: Label smoothing can be a privacy shield but also a catalyst for model inversion attacks.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=1SbkubNdbW](https://openreview.net/forum?id=1SbkubNdbW).
- Szegedy et al. [2013]
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.
Intriguing properties of neural networks.
*arXiv preprint arXiv:1312.6199*, 2013.
- Tang et al. [2019]
Sanli Tang, Xiaolin Huang, Mingjian Chen, Chengjin Sun, and Jie Yang.
Adversarial attack type i: Cheat classifiers by significant changes.
*IEEE transactions on pattern analysis and machine intelligence*, 43(3):1100–1109, 2019.
- Wang et al. [2021a]
Kuan-Chieh Wang, Yan Fu, Ke Li, Ashish Khisti, Richard Zemel, and Alireza Makhzani.
Variational model inversion attacks.
*Advances in Neural Information Processing Systems*, 34:9706–9719, 2021a.
- Wang et al. [2021b]
Tianhao Wang, Yuheng Zhang, and Ruoxi Jia.
Improving robustness to model inversion attacks via mutual information regularization.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, pages 11666–11673, 2021b.
- Wei et al. [2022]
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li.
Mitigating neural network overconfidence with logit normalization.
In *International conference on machine learning*, pages 23631–23644. PMLR, 2022.
- Yang et al. [2022]
Jiawei Yang, Hanbo Chen, Jiangpeng Yan, Xiaoyu Chen, and Jianhua Yao.
Towards better understanding and better generalization of few-shot classification in histology images with contrastive learning.
2022.
- Yang et al. [2019]
Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang.
Neural network inversion in adversarial setting via background knowledge alignment.
In *Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security*, pages 225–240, 2019.
- Yuan et al. [2023]
Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang.
Pseudo label-guided model inversion attack via conditional generative adversarial network.
*Thirty Seventh AAAI Conference on Artificial Intelligence (AAAI 23)*, 2023.
- Zhang et al. [2020]
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song.
The secret revealer: Generative model-inversion attacks against deep neural networks.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 253–261, 2020.
- Zhou et al. [2024]
Yi Zhou, Cheng Li, and Manish Gupta.
A semi-automated approach for crafting outputs with gemini pro.
*Advanced Engineering Informatics*, 61:101432, 2024.
doi: 10.1007/s40192-024-00356-4.

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 4
presentation: 3
questions: See weaknesses
soundness: 3
strengths: - Well Structured: The paper is well-structured and easy to follow.
- Interesting Insights: The paper is the first to investigate the MI evaluation framework and proposes some new insights.
- Generality of MLLMs: MLLMs do not depend on task-specific training, making it applicable to diverse MI scenarios.
summary: This paper focuses on the evaluation framework of Model Inversion Attacks (MIA). Specifically, the paper replaces the traditional evaluator with state-of-the-art Multimodal Large Language Models (MLLMs) and build a MLLM-annotated dataset for MI attack samples. Based on the new dataset, the paper points out there are significant false positives in current MI attack samples and analyzes that most of them are actually Type-I adversarial examples. Experiments results serve as evidence to support the analyses.
weaknesses: - The study merely focuses on the white-box MI attacks. More evaluation on the black-box attacks and label-only attacks are expected.
- The experiments mainly focus on facial recognition datasets. As a benchmark, the paper should also validate whether the impact of Type-I adversarial features generalize to other non-facial domains.
- The paper lacks a comparison of the latest methods, such as [1] and [2].

[1] Peng X, Han B, Liu F, et al. Pseudo-private data guided model inversion attacks[J]. Advances in Neural Information Processing Systems, 2024, 37: 33338-33375.

[2] Li Z, Zhang H, Wang J, et al. From Head to Tail: Efficient Black-box Model Inversion Attack via Long-tailed Learning[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 29288-29298.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 3
questions: 1) What do "64" and "112" mean in "FaceNet64" and "FaceNet112"?
1) How is the correlation between $F_\text{MLLM}$ and $F_\text{Curr}$? Could you illustrate it using Pearson coefficient or a scatter plot?
soundness: 3
strengths: 1) They discover the equivalence between false positive in model inversion and Type I adversarial attack.
1) They manually evaluate some MLLMs, indicating that Gemini-2.0 is a reliable one.
1) They re-evaluate model inversion attacks and defenses, clarifying some long-standing misunderstandings about attack accuracy.
summary: This paper points out the limitation of attack accuracy, a current evaluation indicator for model inversion, and proposes using a multimodal large model to replace it.
weaknesses: 1) This paper claims to revisit model inversion evaluation, yet in reality, it only revisits attack accuracy. There are many other metrics for model inversion evaluation. In addition to the KNN distance, FID, and Knowledge Extraction Score mentioned in their related work, there are two feature distances, $\delta_{eval}$ and $\delta_{face}$ [1]. This paper lacks discussion of these metrics.
1) The core method is using a multimodal large model to evaluate attack accuracy with a simple prompt, which lacks novelty and appears more like a technical report.
1) Model inversion attack aims to reveal the information in the training data. If the reconstructed samples contain transferable features, then the privacy information has been leaked and the attack can be considered effective. For example, in their Figure 1, the reconstructed samples reveal the true skin color and eye shape. It is not necessary for them to be the same individual. 

Minor revisions:
1) In Line 387, a $D_{pub}$ should be $D_{pri}$
1) $F_\text{MLLM}$ is sometimes written as $F_{MLLM}$, and $F_\text{Curr}$ is sometimes written as $F_{Curr}$

[1] A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Can you re-score a representative subset with at least one additional strong judge (open and closed), randomize candidate order, and report inter-judge agreement plus any position-bias checks and prompt/version details?

2. Can you include a stress-test on at least one expert or out-of-distribution domain, and report refusal rates and common error modes of the judge on those images?

3. Can you add one non-face MI setting and show whether FCurr inflation and the judge's corrections persist, including per-domain deltas?
soundness: 3
strengths: **Sharp diagnosis of evaluator coupling.** The manuscript clearly argues that evaluating MI reconstructions with the same (or closely related) classifier can count adversarially transferrable images as "success", not genuine identity recovery. The transferability phenomenon is well-documented across vision models, so flagging it as a core confounder is compelling.

**Broad, controlled re-evaluation.** Re-running diverse MI attacks across multiple targets and reporting error decompositions (not just a single accuracy) is the right methodology and should recalibrate how the field interprets prior headline numbers. A recent survey underscores how heterogeneous MI setups are, making this breadth especially valuable.

**Practical, scalable judging.** Using a capable M/VLM as an automatic judge is inexpensive and aligns with emerging practice where model-based evaluators can approximate human preferences in visual tasks (e.g., text-to-3D evaluation) while avoiding the logistics of large human studies.

**Likely field impact.** By decoupling the attack target from the evaluator, the work promotes a more defensible standard for reporting privacy leakage, consistent with lessons from adversarial robustness that who evaluates you matters.
summary: The paper argues that the standard evaluation protocol for model inversion (MI) attacks, measuring success with the same (or closely related) classifier that was attacked, systematically overestimates privacy leakage because MI often produces Type-I adversarial images that trigger the target classifier without actually capturing the true visual identity. The authors (i) diagnose this failure mode; (ii) propose an alternative evaluator, FMLLM, which uses a multimodal LLM (MLLM) to decide whether a reconstructed image matches the ground-truth identity using interleaved image–text prompts; and (iii) re-evaluate a large suite of recent MI attacks. Across 27 MI setups (9 target classifiers × multiple datasets/attacks), they report very high false-positive rates under FCurr (up to 99%), and show that headline attack accuracies above 90-100% in prior work drop below ~80% (sometimes <60%) when re-evaluated with FMLLM. They further provide controlled experiments contrasting "MI negatives" vs. "natural negatives", showing that the inflated success largely comes from adversarial transfer to the evaluation classifier, not genuine identity recovery. Finally, they propose criteria for selecting a reliable MLLM evaluator and recommend "Gemini 2.0" based on a small validation showing high "Yes" on positive pairs, high "No" on negatives, and low refusal rate.
weaknesses: **Single-model judge risk, closed, drifting, and biased.** The manuscript relies on a single proprietary MLLM as the evaluator [1]. Independent studies document position and order biases in LLM-as-a-judge settings, along with verdict instability under mirrored orderings and small prompt changes [2][3]. API models can also drift over time, which undermines reproducibility unless versions and prompts are frozen and multiple judges are reported [9]. Recommended mitigation: report agreement across two or more judges, randomize candidate order, and pre-register prompts.

**Limited human grounding of the evaluator.** The paper's claims hinge on an automatic judge; although it builds a human-annotated MI dataset, the evidence that the judge tracks human decisions at scale is limited [1]. Meta-evaluations show LLM judges can exhibit systematic judgment biases, which argues for a larger human study with inter-rater agreement and targeted error analysis before treating an MLLM as a gold-standard referee for identity matching [4].

**Evaluator reliability varies by domain.** Vision-language models perform unevenly on expert image interpretation. Peer-reviewed studies report strong results on text-only exam items but marked drops on image-based questions across radiology boards and related evaluations, with additional reports of misinterpretation and refusal behavior [6][7][8][5]. Treating a general-purpose MLLM as an identity verifier should therefore include domain-specific checks and refusal-rate reporting.

**Scope mostly faces, uncertain generalization.** Model inversion spans multiple modalities and threat models beyond faces. Recent surveys detail image, text, graph, and federated learning variants, which means conclusions drawn from face-recognition targets alone may not transfer without additional evidence [10][11]. A non-face case study would strengthen external validity.

References:

[1] Ho S-T, Jun Hao K, Nguyen N-B, Binder A, Cheung N-M. Uncovering the Limitations of Model Inversion Evaluation: Benchmarks and Connection to Type-I Adversarial Attacks. arXiv:2505.03519, 2025. 

[2] Lu Y et al. Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge. arXiv:2406.07791, 2024. 

[3] Chen X et al. Diagnosing Bias and Instability in LLM Evaluation: A Scalable Pairwise Meta-Evaluator. Information 16(8):652, 2025. 

[4] Chen G H et al. Humans or LLMs as the Judge? A Study on Judgement Bias. EMNLP 2024. 

[5] RSNA press summary of Radiology study: Vision-based ChatGPT Shows Deficits Interpreting Radiologic Images. 2024. 

[6] Oura T et al. Diagnostic accuracy of vision-language models on Japanese radiology, nuclear medicine, and interventional radiology board exams. Japanese Journal of Radiology, 2024. 

[7] Vision-language model performance on the Japanese Nuclear Medicine Board Examination: high accuracy in text but challenges with image interpretation. Annals of Nuclear Medicine, 2025. 

[8] Wei B. Performance Evaluation and Implications of Large Language Models in Radiology Board Exams. JMIR Medical Education, 2025. 

[9] Chen L, Zaharia M, Zou J. How is ChatGPT’s behavior changing over time. arXiv:2307.09009, 2023. 

[10] Yang W et al. Deep learning model inversion attacks and defenses: a comprehensive survey. Artificial Intelligence Review, 2025. 

[11] Zhou Z et al. Model Inversion Attacks: A Survey of Approaches and Countermeasures. arXiv:2411.10023, 2024.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: See "weakness"
soundness: 2
strengths: + The perspective of evaluation bias is novel.
+ The connection between MI and adversarial type-I attack sounds interesting.
summary: The paper revisits the evaluation metric of model inversion attacks (MIAs) and points out the weakness of *Attack Accuracy* using another classification model as the evaluation model. This paper connects the Type-I error of adversarial attack and the False Positive in model inversion attacks. Then, evaluation by MLLM is proposed.
weaknesses: 1. Prompt has a significant impact on the accuracy of MLLM evaluation. However, this paper only provides the results with the same prompt. The influence of prompt has not been researched.
2. Existing MIA methods do not deliberately conduct adversarial attacks, but the article lacks specific explanations. There is a lack of theoretical analysis of existing attack methods to improve Type-I errors.
3.  Lack of comparison of the type-I error between $F_{curr}$ and $F_{MLLM}$ .
4. While  $F_{MLLM}$ arguably provides a more faithful perceptual evaluation, it is unclear how its judgment correlates with real-world privacy leakage. MLLMs may focus on semantic coherence rather than sensitive visual features.
5. Will this approach raise type-II error?
6. How do various attack and defense algorithms compare under the new evaluation? The comparison is not shown clearly in the paper.